From cfbolz at codespeak.net Sun Jul 1 13:33:00 2007 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 1 Jul 2007 13:33:00 +0200 (CEST) Subject: [pypy-svn] r44647 - pypy/extradoc/talk/badhonnef2007 Message-ID: <20070701113300.549C480C0@code0.codespeak.net> Author: cfbolz Date: Sun Jul 1 13:32:58 2007 New Revision: 44647 Added: pypy/extradoc/talk/badhonnef2007/Makefile pypy/extradoc/talk/badhonnef2007/badhonnef.tex pypy/extradoc/talk/badhonnef2007/dyla.bib - copied unchanged from r44646, pypy/extradoc/talk/dyla2007/dyla.bib pypy/extradoc/talk/badhonnef2007/llncs.cls - copied unchanged from r44646, pypy/extradoc/talk/dyla2007/llncs.cls Log: beginnig of the bad honnef paper Added: pypy/extradoc/talk/badhonnef2007/Makefile ============================================================================== --- (empty file) +++ pypy/extradoc/talk/badhonnef2007/Makefile Sun Jul 1 13:32:58 2007 @@ -0,0 +1,15 @@ +badhonnef.pdf: badhonnef.tex badhonnef.bib + pdflatex badhonnef + bibtex badhonnef + pdflatex badhonnef + pdflatex badhonnef + +%.pdf: %.eps + epstopdf $< + +view: badhonnef.pdf + xpdf badhonnef.pdf & + +clean: + rm badhonnef.pdf + Added: pypy/extradoc/talk/badhonnef2007/badhonnef.tex ============================================================================== --- (empty file) +++ pypy/extradoc/talk/badhonnef2007/badhonnef.tex Sun Jul 1 13:32:58 2007 @@ -0,0 +1,84 @@ +\documentclass{llncs} + +\usepackage{makeidx} +\usepackage{graphicx} + +\begin{document} + +\pagestyle{headings} + +\title{A Flexible Prolog Interpreter in Python} + +%\titlerunning{XXX} % abbreviated title (for running head) +% also used for the TOC unless +% \toctitle is used + +\author{Carl Friedrich Bolz} + +\authorrunning{Bolz} % abbreviated author list (for running head) + +%%%% modified list of authors for the TOC (add the affiliations) +\tocauthor{Bolz (D\"usseldorf)} + +\institute{ Lehrstuhl Softwaretechnik und Programmiersprachen\\ +Institut f\"{u}r Informatik, Universit\"at D\"usseldorf, Germany\\ + \email{ cfbolz at gmx.de}} + +\maketitle + +\begin{abstract} + +Efficient virtual machines for the Prolog programming language (and also for +other dynamic highlevel languages) are usually large and intricate pieces of +software implemented in a low-level language such as C that are extensively +optimized over many years and are therefore difficult to change in many +respects. Due to their inherent complexity it is hard to develop them further, +to maintain them or to use them to experiment with language extensions or new +built-ins. + +Early implementation decisions, such as choice of garbage collector, are +extremely hard to change afterwards. In addition, it is also much harder to +perform advanced optimizations or transformations (such as partial evaluation) +on the virtual machine itself due to the low level of the implementation +language. Furthermore it is (by definition) impossible to port them to a non-C +platform such as the Java Virtual Machine or the .NET framework Therefore the +numerous implementations of a Prolog virtual machine integrated into one of +these environments are all new implementations which have to be maintained +themselves and therefore all have different levels of maturity and slightly +different sets of built-ins available. + +Using a higher-level language than C addresses many of these problems. It +allows the language implementor to work at a higher level of abstraction, which +eases implementation, maintenance and extensions of the virtual machine and +reduces the size of the code base. Furthermore, advanced optimizations and +transformations can be performed. To regain usable performance the +implementation needs then to be translated to a low-level language again. + +The goal of the PyPy project is to support the implementation of interpreters +in the high-level language Python. Python is an object oriented dynamic +language which is becoming more and more popular for scripting use as well as +for ?real? applications. PyPy provides a translation toolchain that helps to +translate interpreters written in Python to a low-level language such as C and +the .NET Intermediate Language. To make this translation work, these +interpreters have to be written in RPython, which is a proper subset of Python +chosen in such a way that it is amenable to analysis. + +In this talk we give an quick overview of the PyPy project and present a Prolog +interpreter written in RPython, exploring the possibilities of implementing a +well-known declarative language in a high-level language (as opposed to C). One +of the main goals of the implementation was to keep it as simple and as +extensible as possible (even at the cost of performance). The PyPy tool suite +is used to reach a reasonable level of performance and to ensure portability to +various platforms.\footnote{This research was partially supported by the EU funded + project: IST 004779 PyPy (PyPy: Implementing Python in Python).} \\ + +\end{abstract} + + +% ---- Bibliography ---- +%\begin{small} +\bibliographystyle{abbrv} +\bibliography{badhonnef} +%\end{small} + +\end{document} From cfbolz at codespeak.net Mon Jul 2 01:02:48 2007 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 2 Jul 2007 01:02:48 +0200 (CEST) Subject: [pypy-svn] r44657 - pypy/extradoc/talk/badhonnef2007 Message-ID: <20070701230248.ABC5880A9@code0.codespeak.net> Author: cfbolz Date: Mon Jul 2 01:02:47 2007 New Revision: 44657 Added: pypy/extradoc/talk/badhonnef2007/badhonnef.bib - copied unchanged from r44655, pypy/extradoc/talk/badhonnef2007/dyla.bib Removed: pypy/extradoc/talk/badhonnef2007/dyla.bib Log: rename bib file From bea at codespeak.net Mon Jul 2 17:59:12 2007 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 2 Jul 2007 17:59:12 +0200 (CEST) Subject: [pypy-svn] r44674 - pypy/extradoc/sprintinfo/post-ep2007 Message-ID: <20070702155912.8A08680A5@code0.codespeak.net> Author: bea Date: Mon Jul 2 17:59:11 2007 New Revision: 44674 Modified: pypy/extradoc/sprintinfo/post-ep2007/people.txt Log: my dates for ep2007 Modified: pypy/extradoc/sprintinfo/post-ep2007/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/post-ep2007/people.txt (original) +++ pypy/extradoc/sprintinfo/post-ep2007/people.txt Mon Jul 2 17:59:11 2007 @@ -20,6 +20,7 @@ Christian Tismer 8/16th Ratonda Centrum Hotels Simon Burton 8/15th Armin Rigo 8/16th Comfort Vilnius +Bea D?ring 8/13th Comfort Vilnius ==================== ============== ======================= People on the following list were present at previous sprints: From jlg at codespeak.net Tue Jul 3 11:23:47 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 11:23:47 +0200 (CEST) Subject: [pypy-svn] r44689 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070703092347.3484980BA@code0.codespeak.net> Author: jlg Date: Tue Jul 3 11:23:46 2007 New Revision: 44689 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py pypy/dist/pypy/lang/scheme/test/test_object.py pypy/dist/pypy/lang/scheme/test/test_parser.py Log: procedure identifier evals to W_Procedure; W_procedure refactoring Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 3 11:23:46 2007 @@ -33,7 +33,7 @@ w_obj = ctx.get(self.name) if w_obj is not None: - return w_obj.eval(ctx) + return w_obj #.eval(ctx) else: #reference to undefined identifier #unbound @@ -117,7 +117,7 @@ def eval(self, ctx): oper = self.car.eval(ctx) - return oper(ctx, self.cdr) + return oper.eval(ctx, self.cdr) class W_Nil(W_Root): def to_string(self): @@ -129,7 +129,6 @@ #it its very similar to operation.add ############################# class W_Procedure(W_Root): - def __init__(self, pname=""): self.pname = pname @@ -137,19 +136,10 @@ return "#" % (self.pname,) def eval(self, ctx, lst=None): - raise NotImplementedError - -def add_lst(ctx, lst): - def adder(x, y): - return x + y - - return apply_lst(ctx, adder, lst) - -def mul_lst(ctx, lst): - def multiplier(x, y): - return x * y + return self.oper(ctx, lst) - return apply_lst(ctx, multiplier, lst) + def oper(self, ctx, lst): + raise NotImplementedError def apply_lst(ctx, fun, lst): acc = None @@ -172,67 +162,59 @@ return W_Float(acc) class Add(W_Procedure): - def eval(self, ctx): - return add_lst + def adder(self, x, y): + return x + y + + def oper(self, ctx, lst): + return apply_lst(ctx, self.adder, lst) class Mul(W_Procedure): - def eval(self, ctx): - return mul_lst + def multiplier(self, x, y): + return x * y -def define(ctx, lst): - w_identifier = lst.car - assert isinstance(w_identifier, W_Identifier) - - w_val = lst.cdr.car.eval(ctx) - ctx.put(w_identifier.name, w_val) - return w_val + def oper(self, ctx, lst): + return apply_lst(ctx, self.multiplier, lst) class Define(W_Procedure): - def eval(self, ctx): - return define - -def macro_if(ctx, lst): - w_condition = lst.car - w_then = lst.cdr.car - if isinstance(lst.cdr.cdr, W_Nil): - w_else = W_Boolean(False) - else: - w_else = lst.cdr.cdr.car - - w_cond_val = w_condition.eval(ctx) - if w_cond_val.to_boolean() is True: - return w_then.eval(ctx) - else: - return w_else.eval(ctx) + def oper(self, ctx, lst): + w_identifier = lst.car + assert isinstance(w_identifier, W_Identifier) + + w_val = lst.cdr.car.eval(ctx) + ctx.put(w_identifier.name, w_val) + return w_val class MacroIf(W_Procedure): - def eval(self, ctx): - return macro_if + def oper(self, ctx, lst): + w_condition = lst.car + w_then = lst.cdr.car + if isinstance(lst.cdr.cdr, W_Nil): + w_else = W_Boolean(False) + else: + w_else = lst.cdr.cdr.car -def cons(ctx, lst): - w_car = lst.car.eval(ctx) - w_cdr = lst.cdr.car.eval(ctx) - return W_Pair(w_car, w_cdr) + w_cond_val = w_condition.eval(ctx) + if w_cond_val.to_boolean() is True: + return w_then.eval(ctx) + else: + return w_else.eval(ctx) class Cons(W_Procedure): - def eval(self, ctx): - return cons - -def car(ctx, lst): - w_pair = lst.car.eval(ctx) - return w_pair.car + def oper(self, ctx, lst): + w_car = lst.car.eval(ctx) + w_cdr = lst.cdr.car.eval(ctx) + #cons is always creating a new pair + return W_Pair(w_car, w_cdr) class Car(W_Procedure): - def eval(self, ctx): - return car - -def cdr(ctx, lst): - w_pair = lst.car.eval(ctx) - return w_pair.cdr + def oper(self, ctx, lst): + w_pair = lst.car.eval(ctx) + return w_pair.car class Cdr(W_Procedure): - def eval(self, ctx): - return cdr + def oper(self, ctx, lst): + w_pair = lst.car.eval(ctx) + return w_pair.cdr ###################################### # dict mapping operations to callables Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 3 11:23:46 2007 @@ -2,6 +2,7 @@ from pypy.lang.scheme.ssparser import parse from pypy.lang.scheme.object import W_Boolean, W_Fixnum, W_Float, W_String from pypy.lang.scheme.object import W_Nil, W_Pair, W_Symbol, W_Identifier +from pypy.lang.scheme.object import W_Procedure from pypy.lang.scheme.object import ExecutionContext from pypy.lang.scheme.operation import mul, add @@ -85,6 +86,11 @@ w_num = eval_expr(ctx, "(+ 1 v1 v2)") assert w_num.to_number() == 46.1 +def test_func(): + ctx = ExecutionContext() + w_func = eval_expr(ctx, "+") + assert isinstance(w_func, W_Procedure) + def test_if_simple(): ctx = ExecutionContext() w_t = eval_expr(ctx, "(if #t #t #f)") Modified: pypy/dist/pypy/lang/scheme/test/test_object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_object.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_object.py Tue Jul 3 11:23:46 2007 @@ -67,3 +67,26 @@ assert w_fnum is ctx.get("v1") assert ctx.get("no_such_key") is None +def test_location(): + w_fnum = W_Fixnum(42) + loc = Location(w_fnum) + assert isinstance(loc, Location) + assert loc.obj is w_fnum + +def test_ctx_sets(): + w_fnum = W_Fixnum(42) + w_fnum2 = W_Fixnum(43) + w_fnum3 = W_Fixnum(44) + + ctx = ExecutionContext({}) + ctx.put("v1", w_fnum) + + ctx2 = ctx.copy() + assert w_fnum is ctx2.get("v1") + ctx.set("v1", w_fnum2) + assert w_fnum2 is ctx2.get("v1") + assert w_fnum2 is ctx.get("v1") + + ctx2.put("v1", w_fnum3) + assert w_fnum3 is ctx2.get("v1") + Modified: pypy/dist/pypy/lang/scheme/test/test_parser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_parser.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_parser.py Tue Jul 3 11:23:46 2007 @@ -54,6 +54,14 @@ assert unwrap(w_float) == -123456.1234 def test_sexpr(): + w_list = parse('( 1 )') + assert isinstance(w_list, W_Pair) + assert isinstance(w_list.car, W_Fixnum) + assert isinstance(w_list.cdr, W_Nil) + + #w_list = parse('()') + #assert isinstance(w_list, W_Nil) + w_list = parse('(+ 1 2)') assert isinstance(w_list, W_Pair) assert isinstance(w_list.car, W_Identifier) From jlg at codespeak.net Tue Jul 3 11:24:44 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 11:24:44 +0200 (CEST) Subject: [pypy-svn] r44690 - pypy/dist/pypy/lang/scheme Message-ID: <20070703092444.6510B80B6@code0.codespeak.net> Author: jlg Date: Tue Jul 3 11:24:41 2007 New Revision: 44690 Modified: pypy/dist/pypy/lang/scheme/object.py Log: Location() object; context uses Location() Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 3 11:24:41 2007 @@ -123,11 +123,9 @@ def to_string(self): return "()" -############################ +## # operations -#not sure though any operations should exist here -#it its very similar to operation.add -############################# +## class W_Procedure(W_Root): def __init__(self, pname=""): self.pname = pname @@ -216,23 +214,34 @@ w_pair = lst.car.eval(ctx) return w_pair.cdr -###################################### -# dict mapping operations to callables +## +# Location() +## +class Location(object): + def __init__(self, w_obj): + self.obj = w_obj + +## +# dict mapping operations to W_Xxx objects # callables must have 2 arguments # - ctx = execution context # - lst = list of arguments -###################################### -OPERATION_MAP = \ +## +OMAP = \ { - '+': Add("+"), - '*': Mul("*"), - 'define': Define("define"), - 'if': MacroIf("if"), - 'cons': Cons("cons"), - 'car': Car("car"), - 'cdr': Cdr("cdr"), + '+': Add, + '*': Mul, + 'define': Define, + 'if': MacroIf, + 'cons': Cons, + 'car': Car, + 'cdr': Cdr, } +OPERATION_MAP = {} +for name, cls in OMAP.items(): + OPERATION_MAP[name] = Location(cls(name)) + class ExecutionContext(object): """Execution context implemented as a dict. @@ -242,9 +251,28 @@ assert scope is not None self.scope = scope + def copy(self): + return ExecutionContext(dict(self.scope)) + def get(self, name): - return self.scope.get(name, None) + loc = self.scope.get(name, None) + + if loc is not None: + return loc.obj + else: + return None + + def set(self, name, obj): + loc = self.scope.get(name, None) + + if loc is not None: + loc.obj = obj + else: + self.put(name, obj) def put(self, name, obj): - self.scope[name] = obj + """Overwrites existing variable location + (new location is created) + """ + self.scope[name] = Location(obj) From jlg at codespeak.net Tue Jul 3 14:19:23 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 14:19:23 +0200 (CEST) Subject: [pypy-svn] r44694 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070703121923.11BED80B1@code0.codespeak.net> Author: jlg Date: Tue Jul 3 14:19:22 2007 New Revision: 44694 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: W_Macro != W_Procedure; lambda prototype added Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 3 14:19:22 2007 @@ -123,6 +123,11 @@ def to_string(self): return "()" +class W_Lambda(W_Root): + def __init__(self, args, body): + self.args = args + self.body = body + ## # operations ## @@ -139,6 +144,16 @@ def oper(self, ctx, lst): raise NotImplementedError +class W_Macro(W_Root): + def __init__(self, pname=""): + self.pname = pname + + def to_string(self): + return "#" % (self.pname,) + + def eval(self, ctx, lst=None): + raise NotImplementedError + def apply_lst(ctx, fun, lst): acc = None @@ -173,17 +188,17 @@ def oper(self, ctx, lst): return apply_lst(ctx, self.multiplier, lst) -class Define(W_Procedure): - def oper(self, ctx, lst): +class Define(W_Macro): + def eval(self, ctx, lst): w_identifier = lst.car assert isinstance(w_identifier, W_Identifier) w_val = lst.cdr.car.eval(ctx) - ctx.put(w_identifier.name, w_val) + ctx.set(w_identifier.name, w_val) return w_val -class MacroIf(W_Procedure): - def oper(self, ctx, lst): +class MacroIf(W_Macro): + def eval(self, ctx, lst): w_condition = lst.car w_then = lst.cdr.car if isinstance(lst.cdr.cdr, W_Nil): @@ -214,6 +229,12 @@ w_pair = lst.car.eval(ctx) return w_pair.cdr +class Lambda(W_Macro): + def eval(self, ctx, lst): + w_args = lst.car + w_body = lst.cdr.car + return W_Lambda(w_args, w_body) + ## # Location() ## @@ -236,6 +257,7 @@ 'cons': Cons, 'car': Car, 'cdr': Cdr, + 'lambda': Lambda, } OPERATION_MAP = {} Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 3 14:19:22 2007 @@ -2,7 +2,7 @@ from pypy.lang.scheme.ssparser import parse from pypy.lang.scheme.object import W_Boolean, W_Fixnum, W_Float, W_String from pypy.lang.scheme.object import W_Nil, W_Pair, W_Symbol, W_Identifier -from pypy.lang.scheme.object import W_Procedure +from pypy.lang.scheme.object import W_Procedure, W_Lambda from pypy.lang.scheme.object import ExecutionContext from pypy.lang.scheme.operation import mul, add @@ -144,3 +144,8 @@ w_cddr = eval_noctx("(cdr (cdr (cons 1 (cons 2 3))))") assert w_cddr.to_number() == 3 +def test_lambda_definition(): + ctx = ExecutionContext() + w_lambda = eval_expr(ctx, "(lambda () 12)") + assert isinstance(w_lambda, W_Lambda) + From jlg at codespeak.net Tue Jul 3 14:41:44 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 14:41:44 +0200 (CEST) Subject: [pypy-svn] r44696 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070703124144.F270180B6@code0.codespeak.net> Author: jlg Date: Tue Jul 3 14:41:43 2007 New Revision: 44696 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: trivial lambdas with arguments works Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 3 14:41:43 2007 @@ -128,6 +128,20 @@ self.args = args self.body = body + def eval(self, ctx, lst): + name = self.args + val = lst + my_ctx = ctx.copy() + while not isinstance(name, W_Nil): + assert isinstance(name.car, W_Identifier) + w_val = val.car.eval(ctx) + my_ctx.put(name.car.to_string(), w_val) + + val = val.cdr + name = name.cdr + + return self.body.eval(my_ctx) + ## # operations ## Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 3 14:41:43 2007 @@ -144,8 +144,21 @@ w_cddr = eval_noctx("(cdr (cdr (cons 1 (cons 2 3))))") assert w_cddr.to_number() == 3 -def test_lambda_definition(): +def test_lambda_noargs(): ctx = ExecutionContext() w_lambda = eval_expr(ctx, "(lambda () 12)") assert isinstance(w_lambda, W_Lambda) + ctx.put("f1", w_lambda) + w_result = eval_expr(ctx, "(f1)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 12 + +def test_lambda_args(): + ctx = ExecutionContext() + eval_expr(ctx, "(define f1 (lambda (n) n))") + + w_result = eval_expr(ctx, "(f1 42)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 42 + From jlg at codespeak.net Tue Jul 3 14:55:01 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 14:55:01 +0200 (CEST) Subject: [pypy-svn] r44697 - pypy/dist/pypy/lang/scheme/test Message-ID: <20070703125501.1622480BA@code0.codespeak.net> Author: jlg Date: Tue Jul 3 14:55:00 2007 New Revision: 44697 Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py Log: test lambdas (context lookup) Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 3 14:55:00 2007 @@ -162,3 +162,20 @@ assert isinstance(w_result, W_Fixnum) assert w_result.to_number() == 42 + w_result = eval_expr(ctx, "((lambda (n m) (+ n m)) 42 -42)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 0 + +def test_lambda_top_ctx(): + ctx = ExecutionContext() + eval_expr(ctx, "(define n 42)") + eval_expr(ctx, "(define f1 (lambda (m) (+ n m)))") + w_result = eval_expr(ctx, "(f1 -42)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 0 + + eval_expr(ctx, "(define n 84)") + w_result = eval_expr(ctx, "(f1 -42)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 42 + From jlg at codespeak.net Tue Jul 3 17:08:01 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 17:08:01 +0200 (CEST) Subject: [pypy-svn] r44704 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070703150801.5D15180A9@code0.codespeak.net> Author: jlg Date: Tue Jul 3 17:08:00 2007 New Revision: 44704 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: procedures refactoring; Sub procedure added Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 3 17:08:00 2007 @@ -123,10 +123,37 @@ def to_string(self): return "()" -class W_Lambda(W_Root): - def __init__(self, args, body): +class W_Procedure(W_Root): + def __init__(self, pname=""): + self.pname = pname + + def to_string(self): + return "#" % (self.pname,) + + def eval(self, ctx, lst): + return self.procedure(ctx, lst) + + def procedure(self, ctx, lst): + raise NotImplementedError + +class W_Macro(W_Root): + def __init__(self, pname=""): + self.pname = pname + + def to_string(self): + return "#" % (self.pname,) + + def eval(self, ctx, lst=None): + raise NotImplementedError + +class W_Lambda(W_Procedure): + def __init__(self, args, body, pname="#f"): self.args = args self.body = body + self.pname = pname + + def to_string(self): + return "#" % (self.pname,) def eval(self, ctx, lst): name = self.args @@ -145,29 +172,6 @@ ## # operations ## -class W_Procedure(W_Root): - def __init__(self, pname=""): - self.pname = pname - - def to_string(self): - return "#" % (self.pname,) - - def eval(self, ctx, lst=None): - return self.oper(ctx, lst) - - def oper(self, ctx, lst): - raise NotImplementedError - -class W_Macro(W_Root): - def __init__(self, pname=""): - self.pname = pname - - def to_string(self): - return "#" % (self.pname,) - - def eval(self, ctx, lst=None): - raise NotImplementedError - def apply_lst(ctx, fun, lst): acc = None @@ -188,19 +192,27 @@ else: return W_Float(acc) -class Add(W_Procedure): - def adder(self, x, y): +class ListOper(W_Procedure): + def procedure(self, ctx, lst): + return apply_lst(ctx, self.oper, lst) + +class Add(ListOper): + def oper(self, x, y): return x + y - def oper(self, ctx, lst): - return apply_lst(ctx, self.adder, lst) +class Sub(ListOper): + def procedure(self, ctx, lst): + if isinstance(lst.cdr, W_Nil): + return apply_lst(ctx, self.oper, W_Pair( W_Fixnum(0), lst)) + else: + return apply_lst(ctx, self.oper, lst) -class Mul(W_Procedure): - def multiplier(self, x, y): - return x * y + def oper(self, x, y): + return x - y - def oper(self, ctx, lst): - return apply_lst(ctx, self.multiplier, lst) +class Mul(ListOper): + def oper(self, x, y): + return x * y class Define(W_Macro): def eval(self, ctx, lst): @@ -227,19 +239,19 @@ return w_else.eval(ctx) class Cons(W_Procedure): - def oper(self, ctx, lst): + def procedure(self, ctx, lst): w_car = lst.car.eval(ctx) w_cdr = lst.cdr.car.eval(ctx) #cons is always creating a new pair return W_Pair(w_car, w_cdr) class Car(W_Procedure): - def oper(self, ctx, lst): + def procedure(self, ctx, lst): w_pair = lst.car.eval(ctx) return w_pair.car class Cdr(W_Procedure): - def oper(self, ctx, lst): + def procedure(self, ctx, lst): w_pair = lst.car.eval(ctx) return w_pair.cdr @@ -265,6 +277,7 @@ OMAP = \ { '+': Add, + '-': Sub, '*': Mul, 'define': Define, 'if': MacroIf, Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 3 17:08:00 2007 @@ -53,6 +53,13 @@ w_num = eval_noctx("(* 4 -5 6.1)") assert w_num.to_number() == (4 * -5 * 6.1) + w_num = eval_noctx("(- 4)") + assert w_num.to_number() == -4 + w_num = eval_noctx("(- 4 5)") + assert w_num.to_number() == -1 + w_num = eval_noctx("(- 4 -5 6.1)") + assert w_num.to_number() == 4 - (-5) - 6.1 + def test_numerical_nested(): w_num = eval_noctx("(+ 4 (* (+ 5) 6) (+ 1 2))") assert w_num.to_number() == 37 From jlg at codespeak.net Tue Jul 3 17:28:13 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 3 Jul 2007 17:28:13 +0200 (CEST) Subject: [pypy-svn] r44705 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070703152813.0BEC680AB@code0.codespeak.net> Author: jlg Date: Tue Jul 3 17:28:13 2007 New Revision: 44705 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: number comparison: Equal; recursive factorial lambda test Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 3 17:28:13 2007 @@ -90,6 +90,9 @@ def to_float(self): return float(self.fixnumval) + def equal(self, w_obj): + return self.fixnumval == w_obj.to_number() + class W_Float(W_Root): def __init__(self, val): self.floatval = float(val) @@ -106,6 +109,9 @@ def to_float(self): return self.floatval + def equal(self, w_obj): + return self.floatval == w_obj.to_number() + class W_Pair(W_Root): def __init__(self, car, cdr): self.car = car @@ -255,6 +261,12 @@ w_pair = lst.car.eval(ctx) return w_pair.cdr +class Equal(W_Procedure): + def procedure(self, ctx, lst): + w_first = lst.car.eval(ctx) + w_second = lst.cdr.car.eval(ctx) + return W_Boolean(w_first.equal(w_second)) + class Lambda(W_Macro): def eval(self, ctx, lst): w_args = lst.car @@ -276,14 +288,19 @@ ## OMAP = \ { + #arithmetic operations '+': Add, '-': Sub, '*': Mul, - 'define': Define, - 'if': MacroIf, + #list operations 'cons': Cons, 'car': Car, 'cdr': Cdr, + #comparisons + '=': Equal, + #macros + 'define': Define, + 'if': MacroIf, 'lambda': Lambda, } Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 3 17:28:13 2007 @@ -151,9 +151,30 @@ w_cddr = eval_noctx("(cdr (cdr (cons 1 (cons 2 3))))") assert w_cddr.to_number() == 3 +def test_comparison_homonums(): + w_bool = eval_noctx("(= 1 2)") + assert w_bool.to_boolean() is False + + w_bool = eval_noctx("(= 2 2)") + assert w_bool.to_boolean() is True + + w_bool = eval_noctx("(= 2.1 1.2)") + assert w_bool.to_boolean() is False + + w_bool = eval_noctx("(= 2.1 2.1)") + assert w_bool.to_boolean() is True + +def test_comparison_heteronums(): + w_bool = eval_noctx("(= 1 2.2)") + assert w_bool.to_boolean() is False + + w_bool = eval_noctx("(= 2.0 2)") + assert w_bool.to_boolean() is True + def test_lambda_noargs(): ctx = ExecutionContext() w_lambda = eval_expr(ctx, "(lambda () 12)") + assert isinstance(w_lambda, W_Procedure) assert isinstance(w_lambda, W_Lambda) ctx.put("f1", w_lambda) @@ -186,3 +207,18 @@ assert isinstance(w_result, W_Fixnum) assert w_result.to_number() == 42 +def test_lambda_fac(): + ctx = ExecutionContext() + eval_expr(ctx, """ + (define fac + (lambda (n) + (if (= n 1) + n + (* (fac (- n 1)) n))))""") + assert isinstance(ctx.get("fac"), W_Lambda) + w_result = eval_expr(ctx, "(fac 4)") + assert w_result.to_number() == 24 + + w_result = eval_expr(ctx, "(fac 5)") + assert w_result.to_number() == 120 + From jlg at codespeak.net Wed Jul 4 14:38:13 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 14:38:13 +0200 (CEST) Subject: [pypy-svn] r44721 - pypy/dist/pypy/lang/scheme Message-ID: <20070704123813.50622815E@code0.codespeak.net> Author: jlg Date: Wed Jul 4 14:38:11 2007 New Revision: 44721 Modified: pypy/dist/pypy/lang/scheme/object.py Log: procedures refactoring, arguments evaluated in W_Procedure to list() Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 4 14:38:11 2007 @@ -137,7 +137,14 @@ return "#" % (self.pname,) def eval(self, ctx, lst): - return self.procedure(ctx, lst) + #evaluate all arguments into list + arg_lst = [] + arg = lst + while not isinstance(arg, W_Nil): + arg_lst.append(arg.car.eval(ctx)) + arg = arg.cdr + + return self.procedure(ctx, arg_lst) def procedure(self, ctx, lst): raise NotImplementedError @@ -178,29 +185,20 @@ ## # operations ## -def apply_lst(ctx, fun, lst): - acc = None - - if not isinstance(lst, W_Pair): - #raise argument error - raise - - arg = lst - while not isinstance(arg, W_Nil): - if acc is None: - acc = arg.car.eval(ctx).to_number() - else: - acc = fun(acc, arg.car.eval(ctx).to_number()) - arg = arg.cdr - - if isinstance(acc, int): - return W_Fixnum(acc) - else: - return W_Float(acc) class ListOper(W_Procedure): def procedure(self, ctx, lst): - return apply_lst(ctx, self.oper, lst) + acc = None + for arg in lst: + if acc is None: + acc = arg.eval(ctx).to_number() + else: + acc = self.oper(acc, arg.eval(ctx).to_number()) + + if isinstance(acc, int): + return W_Fixnum(acc) + else: + return W_Float(acc) class Add(ListOper): def oper(self, x, y): @@ -208,10 +206,10 @@ class Sub(ListOper): def procedure(self, ctx, lst): - if isinstance(lst.cdr, W_Nil): - return apply_lst(ctx, self.oper, W_Pair( W_Fixnum(0), lst)) + if len(lst) == 1: + return ListOper.procedure(self, ctx, [W_Fixnum(0), lst[0]]) else: - return apply_lst(ctx, self.oper, lst) + return ListOper.procedure(self, ctx, lst) def oper(self, x, y): return x - y @@ -246,25 +244,25 @@ class Cons(W_Procedure): def procedure(self, ctx, lst): - w_car = lst.car.eval(ctx) - w_cdr = lst.cdr.car.eval(ctx) + w_car = lst[0] + w_cdr = lst[1] #cons is always creating a new pair return W_Pair(w_car, w_cdr) class Car(W_Procedure): def procedure(self, ctx, lst): - w_pair = lst.car.eval(ctx) + w_pair = lst[0] return w_pair.car class Cdr(W_Procedure): def procedure(self, ctx, lst): - w_pair = lst.car.eval(ctx) + w_pair = lst[0] return w_pair.cdr class Equal(W_Procedure): def procedure(self, ctx, lst): - w_first = lst.car.eval(ctx) - w_second = lst.cdr.car.eval(ctx) + w_first = lst[0] + w_second = lst[1] return W_Boolean(w_first.equal(w_second)) class Lambda(W_Macro): From jlg at codespeak.net Wed Jul 4 14:40:12 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 14:40:12 +0200 (CEST) Subject: [pypy-svn] r44722 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070704124012.69CBE8166@code0.codespeak.net> Author: jlg Date: Wed Jul 4 14:40:12 2007 New Revision: 44722 Removed: pypy/dist/pypy/lang/scheme/operation.py Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py Log: operation.py is no more needed Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Wed Jul 4 14:40:12 2007 @@ -4,28 +4,6 @@ from pypy.lang.scheme.object import W_Nil, W_Pair, W_Symbol, W_Identifier from pypy.lang.scheme.object import W_Procedure, W_Lambda from pypy.lang.scheme.object import ExecutionContext -from pypy.lang.scheme.operation import mul, add - -def test_operations_simple(): - w_num1 = W_Fixnum(4) - w_num2 = W_Fixnum(5) - w_num3 = W_Float(6.1) - - w_num = mul(None, [w_num1]) - assert w_num.to_number() == w_num1.to_number() - w_num = mul(None, [w_num1, w_num2]) - assert w_num.to_number() == w_num1.to_number() * w_num2.to_number() - w_num = mul(None, [w_num1, w_num2, w_num3]) - assert w_num.to_number() == (w_num1.to_number() * w_num2.to_number() - * w_num3.to_number()) - - w_num = add(None, [w_num1]) - assert w_num.to_number() == w_num1.to_number() - w_num = add(None, [w_num1, w_num2]) - assert w_num.to_number() == w_num1.to_number() + w_num2.to_number() - w_num = add(None, [w_num1, w_num2, w_num3]) - assert w_num.to_number() == (w_num1.to_number() + w_num2.to_number() - + w_num3.to_number()) def test_eval_obj(): w_num = W_Pair(W_Identifier("+"), From jlg at codespeak.net Wed Jul 4 15:11:05 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 15:11:05 +0200 (CEST) Subject: [pypy-svn] r44723 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070704131105.085C5815E@code0.codespeak.net> Author: jlg Date: Wed Jul 4 15:11:05 2007 New Revision: 44723 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_object.py Log: lambda is more like W_Procedure now Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 4 15:11:05 2007 @@ -161,31 +161,35 @@ class W_Lambda(W_Procedure): def __init__(self, args, body, pname="#f"): - self.args = args + self.args = [] + arg = args + while not isinstance(arg, W_Nil): + #list of argumen names, not evaluated + self.args.append(arg.car) + arg = arg.cdr + self.body = body self.pname = pname def to_string(self): return "#" % (self.pname,) - def eval(self, ctx, lst): - name = self.args - val = lst - my_ctx = ctx.copy() - while not isinstance(name, W_Nil): - assert isinstance(name.car, W_Identifier) - w_val = val.car.eval(ctx) - my_ctx.put(name.car.to_string(), w_val) - - val = val.cdr - name = name.cdr + def procedure(self, ctx, lst): + if len(lst) != len(self.args): + #wrong argument count + raise + + local_ctx = ctx.copy() + vars = zip(self.args, lst) + for name_val in vars: + assert isinstance(name_val[0], W_Identifier) + local_ctx.put(name_val[0].to_string(), name_val[1]) - return self.body.eval(my_ctx) + return self.body.eval(local_ctx) ## # operations ## - class ListOper(W_Procedure): def procedure(self, ctx, lst): acc = None Modified: pypy/dist/pypy/lang/scheme/test/test_object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_object.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_object.py Wed Jul 4 15:11:05 2007 @@ -1,6 +1,5 @@ import py from pypy.lang.scheme.object import * -from pypy.lang.scheme.operation import * def test_false(): w_false = W_Boolean(False) From jlg at codespeak.net Wed Jul 4 15:22:15 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 15:22:15 +0200 (CEST) Subject: [pypy-svn] r44724 - pypy/dist/pypy/lang/scheme Message-ID: <20070704132215.CBDBA815E@code0.codespeak.net> Author: jlg Date: Wed Jul 4 15:22:15 2007 New Revision: 44724 Modified: pypy/dist/pypy/lang/scheme/object.py Log: agrument W_Identfier -> str in W_Lambda init Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 4 15:22:15 2007 @@ -164,8 +164,9 @@ self.args = [] arg = args while not isinstance(arg, W_Nil): - #list of argumen names, not evaluated - self.args.append(arg.car) + assert isinstance(arg.car, W_Identifier) + #list of argument names, not evaluated + self.args.append(arg.car.to_string()) arg = arg.cdr self.body = body @@ -181,9 +182,8 @@ local_ctx = ctx.copy() vars = zip(self.args, lst) - for name_val in vars: - assert isinstance(name_val[0], W_Identifier) - local_ctx.put(name_val[0].to_string(), name_val[1]) + for (name, val) in vars: + local_ctx.put(name, val) return self.body.eval(local_ctx) From jlg at codespeak.net Wed Jul 4 16:56:18 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 16:56:18 +0200 (CEST) Subject: [pypy-svn] r44725 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070704145618.276CB815E@code0.codespeak.net> Author: jlg Date: Wed Jul 4 16:56:17 2007 New Revision: 44725 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: lambda in lambda cousure; Colusure(Location) Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 4 16:56:17 2007 @@ -33,11 +33,11 @@ w_obj = ctx.get(self.name) if w_obj is not None: - return w_obj #.eval(ctx) + return w_obj else: #reference to undefined identifier #unbound - raise NotImplementedError + raise "Unbound variable: %s" % (self.name, ) class W_Symbol(W_Root): def __init__(self, val): @@ -160,7 +160,7 @@ raise NotImplementedError class W_Lambda(W_Procedure): - def __init__(self, args, body, pname="#f"): + def __init__(self, args, body, clousure, pname="#f"): self.args = [] arg = args while not isinstance(arg, W_Nil): @@ -171,19 +171,23 @@ self.body = body self.pname = pname + self.clousure = clousure def to_string(self): return "#" % (self.pname,) def procedure(self, ctx, lst): if len(lst) != len(self.args): - #wrong argument count - raise + raise "Wrong argument count" local_ctx = ctx.copy() + #ugly hack + for (name, val) in self.clousure.scope.items(): + local_ctx.lput(name, val.obj) + vars = zip(self.args, lst) for (name, val) in vars: - local_ctx.put(name, val) + local_ctx.lput(name, val) return self.body.eval(local_ctx) @@ -273,7 +277,7 @@ def eval(self, ctx, lst): w_args = lst.car w_body = lst.cdr.car - return W_Lambda(w_args, w_body) + return W_Lambda(w_args, w_body, ctx.lcopy()) ## # Location() @@ -282,6 +286,10 @@ def __init__(self, w_obj): self.obj = w_obj +class Clousure(Location): + def __init__(self, w_obj): + self.obj = w_obj + ## # dict mapping operations to W_Xxx objects # callables must have 2 arguments @@ -313,7 +321,7 @@ class ExecutionContext(object): """Execution context implemented as a dict. - { "IDENTIFIER": W_Root } + { "IDENTIFIER": Location(W_Root()) } """ def __init__(self, scope=OPERATION_MAP): assert scope is not None @@ -322,6 +330,14 @@ def copy(self): return ExecutionContext(dict(self.scope)) + def lcopy(self): + lscope = {} + for (name, loc) in self.scope.items(): + if isinstance(loc, Clousure): + lscope[name] = loc + + return ExecutionContext(lscope) + def get(self, name): loc = self.scope.get(name, None) @@ -331,6 +347,7 @@ return None def set(self, name, obj): + """update existing location or create new location new""" loc = self.scope.get(name, None) if loc is not None: @@ -338,9 +355,21 @@ else: self.put(name, obj) + def seta(self, name, obj): + """update existing location""" + loc = self.scope.get(name, None) + + if loc is not None: + loc.obj = obj + else: + #unbound var + raise "Unbound" + def put(self, name, obj): - """Overwrites existing variable location - (new location is created) - """ + """create new location""" self.scope[name] = Location(obj) + def lput(self, name, obj): + """create new location""" + self.scope[name] = Clousure(obj) + Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Wed Jul 4 16:56:17 2007 @@ -200,3 +200,18 @@ w_result = eval_expr(ctx, "(fac 5)") assert w_result.to_number() == 120 +def test_lambda2(): + ctx = ExecutionContext() + eval_expr(ctx, """(define adder (lambda (x) (lambda (y) (+ x y))))""") + w_lambda = eval_expr(ctx, "(adder 6)") + assert isinstance(w_lambda, W_Lambda) + + eval_expr(ctx, """(define add6 (adder 6))""") + w_result = eval_expr(ctx, "(add6 5)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 11 + + w_result = eval_expr(ctx, "((adder 6) 5)") + assert isinstance(w_result, W_Fixnum) + assert w_result.to_number() == 11 + From jlg at codespeak.net Wed Jul 4 17:00:59 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 17:00:59 +0200 (CEST) Subject: [pypy-svn] r44726 - pypy/dist/pypy/lang/scheme Message-ID: <20070704150059.C5CDE8181@code0.codespeak.net> Author: jlg Date: Wed Jul 4 17:00:59 2007 New Revision: 44726 Modified: pypy/dist/pypy/lang/scheme/object.py Log: spelling, s/clousre/slosure/ Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 4 17:00:59 2007 @@ -160,7 +160,7 @@ raise NotImplementedError class W_Lambda(W_Procedure): - def __init__(self, args, body, clousure, pname="#f"): + def __init__(self, args, body, closure, pname="#f"): self.args = [] arg = args while not isinstance(arg, W_Nil): @@ -171,7 +171,7 @@ self.body = body self.pname = pname - self.clousure = clousure + self.closure = closure def to_string(self): return "#" % (self.pname,) @@ -182,7 +182,7 @@ local_ctx = ctx.copy() #ugly hack - for (name, val) in self.clousure.scope.items(): + for (name, val) in self.closure.scope.items(): local_ctx.lput(name, val.obj) vars = zip(self.args, lst) @@ -286,7 +286,7 @@ def __init__(self, w_obj): self.obj = w_obj -class Clousure(Location): +class Closure(Location): def __init__(self, w_obj): self.obj = w_obj @@ -333,7 +333,7 @@ def lcopy(self): lscope = {} for (name, loc) in self.scope.items(): - if isinstance(loc, Clousure): + if isinstance(loc, Closure): lscope[name] = loc return ExecutionContext(lscope) @@ -371,5 +371,5 @@ def lput(self, name, obj): """create new location""" - self.scope[name] = Clousure(obj) + self.scope[name] = Closure(obj) From jlg at codespeak.net Wed Jul 4 17:56:46 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 4 Jul 2007 17:56:46 +0200 (CEST) Subject: [pypy-svn] r44727 - pypy/dist/pypy/lang/scheme Message-ID: <20070704155646.2BDA98113@code0.codespeak.net> Author: jlg Date: Wed Jul 4 17:56:45 2007 New Revision: 44727 Modified: pypy/dist/pypy/lang/scheme/object.py Log: ExecutionContext now has globalscope and scope, no more Closure(), lambda evaluation ignores caller ctx Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 4 17:56:45 2007 @@ -177,17 +177,16 @@ return "#" % (self.pname,) def procedure(self, ctx, lst): + #ctx is a caller context, which is hoyfully ignored if len(lst) != len(self.args): raise "Wrong argument count" - local_ctx = ctx.copy() - #ugly hack - for (name, val) in self.closure.scope.items(): - local_ctx.lput(name, val.obj) + local_ctx = self.closure.copy() + #set lambda arguments vars = zip(self.args, lst) for (name, val) in vars: - local_ctx.lput(name, val) + local_ctx.put(name, val) return self.body.eval(local_ctx) @@ -232,7 +231,7 @@ assert isinstance(w_identifier, W_Identifier) w_val = lst.cdr.car.eval(ctx) - ctx.set(w_identifier.name, w_val) + ctx.gset(w_identifier.name, w_val) return w_val class MacroIf(W_Macro): @@ -277,7 +276,7 @@ def eval(self, ctx, lst): w_args = lst.car w_body = lst.cdr.car - return W_Lambda(w_args, w_body, ctx.lcopy()) + return W_Lambda(w_args, w_body, ctx.copy()) ## # Location() @@ -286,10 +285,6 @@ def __init__(self, w_obj): self.obj = w_obj -class Closure(Location): - def __init__(self, w_obj): - self.obj = w_obj - ## # dict mapping operations to W_Xxx objects # callables must have 2 arguments @@ -323,28 +318,31 @@ { "IDENTIFIER": Location(W_Root()) } """ - def __init__(self, scope=OPERATION_MAP): - assert scope is not None - self.scope = scope - - def copy(self): - return ExecutionContext(dict(self.scope)) + def __init__(self, globalscope=None, scope=None): + if globalscope is None: + self.globalscope = dict(OPERATION_MAP) + else: + self.globalscope = globalscope - def lcopy(self): - lscope = {} - for (name, loc) in self.scope.items(): - if isinstance(loc, Closure): - lscope[name] = loc + if scope is None: + self.scope = {} + else: + self.scope = scope - return ExecutionContext(lscope) + def copy(self): + return ExecutionContext(self.globalscope, dict(self.scope)) def get(self, name): loc = self.scope.get(name, None) if loc is not None: return loc.obj - else: - return None + + loc = self.globalscope.get(name, None) + if loc is not None: + return loc.obj + + return None def set(self, name, obj): """update existing location or create new location new""" @@ -355,21 +353,20 @@ else: self.put(name, obj) - def seta(self, name, obj): - """update existing location""" - loc = self.scope.get(name, None) + def gset(self, name, obj): + """update existing location or create new location new""" + loc = self.globalscope.get(name, None) if loc is not None: loc.obj = obj else: - #unbound var - raise "Unbound" + self.gput(name, obj) - def put(self, name, obj): + def gput(self, name, obj): """create new location""" - self.scope[name] = Location(obj) + self.globalscope[name] = Location(obj) - def lput(self, name, obj): + def put(self, name, obj): """create new location""" - self.scope[name] = Closure(obj) + self.scope[name] = Location(obj) From antocuni at codespeak.net Wed Jul 4 19:06:44 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 4 Jul 2007 19:06:44 +0200 (CEST) Subject: [pypy-svn] r44728 - pypy/dist/pypy/translator Message-ID: <20070704170644.72E5B8141@code0.codespeak.net> Author: antocuni Date: Wed Jul 4 19:06:43 2007 New Revision: 44728 Modified: pypy/dist/pypy/translator/driver.py Log: at the moment main.exe is needed to import the generated dll. Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Wed Jul 4 19:06:43 2007 @@ -694,6 +694,12 @@ pypylib_dll = os.path.join(usession_path, 'pypylib.dll') shutil.copy(dllname, '.') shutil.copy(pypylib_dll, '.') + + # main.exe is a stub but is needed right now because it's + # referenced by pypylib.dll. Will be removed in the future + translator_path, _ = os.path.split(__file__) + main_exe = os.path.join(translator_path, 'cli/src/main.exe') + shutil.copy(main_exe, '.') def task_run_cli(self): pass From antocuni at codespeak.net Wed Jul 4 19:38:06 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 4 Jul 2007 19:38:06 +0200 (CEST) Subject: [pypy-svn] r44729 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20070704173806.AC4FD814B@code0.codespeak.net> Author: antocuni Date: Wed Jul 4 19:38:05 2007 New Revision: 44729 Modified: pypy/dist/pypy/translator/cli/carbonpython.py pypy/dist/pypy/translator/cli/test/test_carbonpython.py Log: add the possibility to specify the name of the target dll Modified: pypy/dist/pypy/translator/cli/carbonpython.py ============================================================================== --- pypy/dist/pypy/translator/cli/carbonpython.py (original) +++ pypy/dist/pypy/translator/cli/carbonpython.py Wed Jul 4 19:38:05 2007 @@ -116,27 +116,41 @@ return mydict[name] -def compile_dll(filename): - _, name = os.path.split(filename) - dllname, _ = os.path.splitext(name) +def compile_dll(filename, dllname=None, copy_dll=True): + dirname, name = os.path.split(filename) + if dllname is None: + dllname, _ = os.path.splitext(name) + elif dllname.endswith('.dll'): + dllname, _ = os.path.splitext(dllname) module = new.module(dllname) namespace = module.__dict__.get('_namespace_', dllname) + sys.path.insert(0, dirname) execfile(filename, module.__dict__) + sys.path.pop(0) dll = DllDef(dllname, namespace) dll.functions = collect_entrypoints(module.__dict__) dll.compile() - dll.driver.copy_cli_dll() + if copy_dll: + dll.driver.copy_cli_dll() def main(argv): - if len(argv) != 2: + if len(argv) == 2: + filename = argv[1] + dllname = None + elif len(argv) == 3: + filename = argv[1] + dllname = argv[2] + else: print >> sys.stderr, __doc__ sys.exit(2) - filename = argv[1] + + if not filename.endswith('.py'): + filename += '.py' if not os.path.exists(filename): print >> sys.stderr, "Cannot find file %s" % filename sys.exit(1) - compile_dll(filename) + compile_dll(filename, dllname) if __name__ == '__main__': main(sys.argv) Modified: pypy/dist/pypy/translator/cli/test/test_carbonpython.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_carbonpython.py (original) +++ pypy/dist/pypy/translator/cli/test/test_carbonpython.py Wed Jul 4 19:38:05 2007 @@ -1,7 +1,9 @@ +import os +import os.path from pypy.tool import udir from pypy.translator.cli.rte import Target from pypy.translator.cli.carbonpython import DllDef, export, collect_entrypoints,\ - collect_class_entrypoints + collect_class_entrypoints, compile_dll from pypy.translator.cli.test.runtest import CliFunctionWrapper, CliTest TEMPLATE = """ @@ -140,3 +142,21 @@ Console.WriteLine(Test.getitem(obj, 0)); """) assert res == 42 + + def test_compile_dll(self): + cwd, _ = os.path.split(__file__) + mylib_py = os.path.join(cwd, 'mylib.py') + compile_dll(mylib_py, copy_dll=False) + res = self._csharp('mylib', """ + Console.WriteLine(mylib.sum(20, 22)); + """) + assert res == 42 + + def test_compile_dll_alternative_name(self): + cwd, _ = os.path.split(__file__) + mylib_py = os.path.join(cwd, 'mylib.py') + compile_dll(mylib_py, 'mylibxxx.dll', copy_dll=False) + res = self._csharp('mylibxxx', """ + Console.WriteLine(mylibxxx.sum(20, 22)); + """) + assert res == 42 From antocuni at codespeak.net Wed Jul 4 19:38:45 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 4 Jul 2007 19:38:45 +0200 (CEST) Subject: [pypy-svn] r44730 - pypy/dist/pypy/translator/cli Message-ID: <20070704173845.94508814B@code0.codespeak.net> Author: antocuni Date: Wed Jul 4 19:38:45 2007 New Revision: 44730 Modified: pypy/dist/pypy/translator/cli/carbonpython.py Log: sorry, forgot to update the synopsis Modified: pypy/dist/pypy/translator/cli/carbonpython.py ============================================================================== --- pypy/dist/pypy/translator/cli/carbonpython.py (original) +++ pypy/dist/pypy/translator/cli/carbonpython.py Wed Jul 4 19:38:45 2007 @@ -1,6 +1,6 @@ #! /usr/bin/env python """ -Usage: carbonpython.py .py +Usage: carbonpython.py [dll-name] Compiles an RPython module into a .NET dll. """ From antocuni at codespeak.net Wed Jul 4 21:04:19 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 4 Jul 2007 21:04:19 +0200 (CEST) Subject: [pypy-svn] r44731 - pypy/dist/pypy/translator Message-ID: <20070704190419.5DDE580B9@code0.codespeak.net> Author: antocuni Date: Wed Jul 4 21:04:18 2007 New Revision: 44731 Modified: pypy/dist/pypy/translator/driver.py Log: add an information message Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Wed Jul 4 21:04:18 2007 @@ -700,6 +700,7 @@ translator_path, _ = os.path.split(__file__) main_exe = os.path.join(translator_path, 'cli/src/main.exe') shutil.copy(main_exe, '.') + self.log.info("Copied to %s" % os.path.join(os.getcwd(), dllname)) def task_run_cli(self): pass From antocuni at codespeak.net Wed Jul 4 21:13:16 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 4 Jul 2007 21:13:16 +0200 (CEST) Subject: [pypy-svn] r44732 - in pypy/extradoc/talk/ep2007: . rpython-demo Message-ID: <20070704191316.77E3780C4@code0.codespeak.net> Author: antocuni Date: Wed Jul 4 21:13:15 2007 New Revision: 44732 Added: pypy/extradoc/talk/ep2007/rpython-demo/ pypy/extradoc/talk/ep2007/rpython-demo/carbonpython.py (contents, props changed) pypy/extradoc/talk/ep2007/rpython-demo/demo.txt (contents, props changed) pypy/extradoc/talk/ep2007/rpython-demo/func.py (contents, props changed) Modified: pypy/extradoc/talk/ep2007/rpython.txt Log: some slides about gencli/carbonpython and the demo Added: pypy/extradoc/talk/ep2007/rpython-demo/carbonpython.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/rpython-demo/carbonpython.py Wed Jul 4 21:13:15 2007 @@ -0,0 +1,13 @@ +import sys + +if sys.platform == 'cli': + # on IronPython, export does nothing + def export(*args, **kwds): + if len(args) == 1 and isinstance(args[0], type(export)): + return args[0] + else: + return lambda fn: fn +else: + # import carbonpython's export + from pypy.translator.cli.carbonpython import export + Added: pypy/extradoc/talk/ep2007/rpython-demo/demo.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/rpython-demo/demo.txt Wed Jul 4 21:13:15 2007 @@ -0,0 +1,39 @@ +# CarbonPython live demo + +$ cat func.py +from carbonpython import export +import time + + at export(int) +def fn(N): + t1 = time.clock() + z = 0 + for index in xrange(N): + x = 2.34+index + y = 3.45+index + z = (x*x)+(y*y)+index + t2 = time.clock() + print t2-t1, 'seconds' + return z + +$ carbonpython.py func.py mylib.dll +... + +$ ipy +IronPython 1.0 (1.0) on .NET 2.0.50727.42 +Copyright (c) Microsoft Corporation. All rights reserved. +>>> import clr +>>> clr.AddReferenceToFile('mylib.dll') +>>> +>>> import func +>>> import mylib +>>> +>>> N = 10**7 +>>> print func.fn(N) +7.770177 seconds +2.000000858e+014 +>>> print mylib.fn(N) +0.25608 seconds +2.000000858e+014 +>>> 7.770177 / 0.25608 +30.3427717901 Added: pypy/extradoc/talk/ep2007/rpython-demo/func.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/rpython-demo/func.py Wed Jul 4 21:13:15 2007 @@ -0,0 +1,14 @@ +from carbonpython import export +import time + + at export(int) +def fn(N): + t1 = time.clock() + z = 0 + for index in xrange(N): + x = 2.34+index + y = 3.45+index + z = (x*x)+(y*y)+index + t2 = time.clock() + print t2-t1, 'seconds' + return z Modified: pypy/extradoc/talk/ep2007/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython.txt Wed Jul 4 21:13:15 2007 @@ -3,7 +3,7 @@ RPython: need for speed aka C and C# considered harmful ======================================================= -:Authors: Antonio Cuni (xxx), Maciej Fijalkowski (merlinux GmbH) +:Authors: Antonio Cuni (DISI), Maciej Fijalkowski (merlinux GmbH) :Place: Europython 2007 :Date: XXX @@ -108,12 +108,40 @@ The CLI backend --------------- -XXX anto, feel free to fill +* Produces assemblies for .NET (both Mono and CLR) + +* (Experimental) access to .NET libraries + +* As fast as C# + +* From 4 to 100 times faster than IronPython + +CarbonPython +------------ + +* **Highly** experimental + +* RPython --> .NET DLL + +* DLLs can be imported by C#, IronPython, ... + +* CarbonPython + IronPython = SteelPython :-) + +* Explicit definition of multiple entrypoints + +* Live demo + The JVM backend --------------- -XXX anto, feel free to fill +* Produces classes for the JVM + +* Shares a lot of code with the CLI backend + +* Not as mature as gencli + + The JS backend -------------- From jlg at codespeak.net Thu Jul 5 13:59:35 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 13:59:35 +0200 (CEST) Subject: [pypy-svn] r44738 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705115935.97AF580E4@code0.codespeak.net> Author: jlg Date: Thu Jul 5 13:59:34 2007 New Revision: 44738 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: lambdas body can be multiple expressions; (set! ...) added Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 5 13:59:34 2007 @@ -188,7 +188,13 @@ for (name, val) in vars: local_ctx.put(name, val) - return self.body.eval(local_ctx) + body_expression = self.body + body_result = None + while not isinstance(body_expression, W_Nil): + body_result = body_expression.car.eval(local_ctx) + body_expression = body_expression.cdr + + return body_result # self.body.eval(local_ctx) ## # operations @@ -234,6 +240,15 @@ ctx.gset(w_identifier.name, w_val) return w_val +class Sete(W_Macro): + def eval(self, ctx, lst): + w_identifier = lst.car + assert isinstance(w_identifier, W_Identifier) + + w_val = lst.cdr.car.eval(ctx) + ctx.sete(w_identifier.name, w_val) + return w_val + class MacroIf(W_Macro): def eval(self, ctx, lst): w_condition = lst.car @@ -275,7 +290,7 @@ class Lambda(W_Macro): def eval(self, ctx, lst): w_args = lst.car - w_body = lst.cdr.car + w_body = lst.cdr #.car return W_Lambda(w_args, w_body, ctx.copy()) ## @@ -305,6 +320,7 @@ '=': Equal, #macros 'define': Define, + 'set!': Sete, 'if': MacroIf, 'lambda': Lambda, } @@ -353,6 +369,21 @@ else: self.put(name, obj) + def sete(self, name, obj): + """update existing location or raise""" + loc = self.scope.get(name, None) + if loc is not None: + loc.obj = obj + return obj + + loc = self.globalscope.get(name, None) + if loc is not None: + loc.obj = obj + return obj + + raise "Unbound" + + def gset(self, name, obj): """update existing location or create new location new""" loc = self.globalscope.get(name, None) Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 5 13:59:34 2007 @@ -71,6 +71,13 @@ w_num = eval_expr(ctx, "(+ 1 v1 v2)") assert w_num.to_number() == 46.1 +def test_sete(): + ctx = ExecutionContext() + eval_expr(ctx, "(define x 42)") + eval_expr(ctx, "(set! x 43)") + assert ctx.get("x").to_number() == 43 + py.test.raises("Unbound", eval_expr, ctx, "(set! y 42)") + def test_func(): ctx = ExecutionContext() w_func = eval_expr(ctx, "+") @@ -215,3 +222,9 @@ assert isinstance(w_result, W_Fixnum) assert w_result.to_number() == 11 +def test_lambda_long_body(): + ctx = ExecutionContext() + eval_expr(ctx, """(define long_body (lambda () (define x 42) (+ x 1)))""") + w_result = eval_expr(ctx, "(long_body)") + assert w_result.to_number() == 43 + #assert ctx.get("x") is None From jlg at codespeak.net Thu Jul 5 14:14:58 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 14:14:58 +0200 (CEST) Subject: [pypy-svn] r44739 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705121458.B70D880C1@code0.codespeak.net> Author: jlg Date: Thu Jul 5 14:14:56 2007 New Revision: 44739 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: after copy contex cannot create new locations in globalscope Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 5 14:14:56 2007 @@ -237,7 +237,7 @@ assert isinstance(w_identifier, W_Identifier) w_val = lst.cdr.car.eval(ctx) - ctx.gset(w_identifier.name, w_val) + ctx.set(w_identifier.name, w_val) return w_val class Sete(W_Macro): @@ -334,7 +334,7 @@ { "IDENTIFIER": Location(W_Root()) } """ - def __init__(self, globalscope=None, scope=None): + def __init__(self, globalscope=None, scope=None, closure=False): if globalscope is None: self.globalscope = dict(OPERATION_MAP) else: @@ -345,12 +345,13 @@ else: self.scope = scope + self.closure = closure + def copy(self): - return ExecutionContext(self.globalscope, dict(self.scope)) + return ExecutionContext(self.globalscope, dict(self.scope), True) def get(self, name): loc = self.scope.get(name, None) - if loc is not None: return loc.obj @@ -360,17 +361,10 @@ return None - def set(self, name, obj): - """update existing location or create new location new""" - loc = self.scope.get(name, None) - - if loc is not None: - loc.obj = obj - else: - self.put(name, obj) - def sete(self, name, obj): - """update existing location or raise""" + """update existing location or raise + directly used by (set! ) macro + """ loc = self.scope.get(name, None) if loc is not None: loc.obj = obj @@ -383,21 +377,36 @@ raise "Unbound" - - def gset(self, name, obj): + def set(self, name, obj): """update existing location or create new location new""" - loc = self.globalscope.get(name, None) + if self.closure: + loc = self.scope.get(name, None) + else: + loc = self.globalscope.get(name, None) if loc is not None: loc.obj = obj else: - self.gput(name, obj) - - def gput(self, name, obj): - """create new location""" - self.globalscope[name] = Location(obj) + self.put(name, obj) def put(self, name, obj): """create new location""" - self.scope[name] = Location(obj) + if self.closure: + self.scope[name] = Location(obj) + else: + self.globalscope[name] = Location(obj) + + def get_location(self, name): + """internal/test use only + returns location bound to variable + """ + loc = self.scope.get(name, None) + if loc is not None: + return loc + + loc = self.globalscope.get(name, None) + if loc is not None: + return loc + + return None Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 5 14:14:56 2007 @@ -74,8 +74,11 @@ def test_sete(): ctx = ExecutionContext() eval_expr(ctx, "(define x 42)") + loc1 = ctx.get_location("x") eval_expr(ctx, "(set! x 43)") + loc2 = ctx.get_location("x") assert ctx.get("x").to_number() == 43 + assert loc1 is loc2 py.test.raises("Unbound", eval_expr, ctx, "(set! y 42)") def test_func(): @@ -227,4 +230,5 @@ eval_expr(ctx, """(define long_body (lambda () (define x 42) (+ x 1)))""") w_result = eval_expr(ctx, "(long_body)") assert w_result.to_number() == 43 - #assert ctx.get("x") is None + assert ctx.get("x") is None + From jlg at codespeak.net Thu Jul 5 16:02:01 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 16:02:01 +0200 (CEST) Subject: [pypy-svn] r44740 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705140201.544FD80F4@code0.codespeak.net> Author: jlg Date: Thu Jul 5 16:02:00 2007 New Revision: 44740 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: (lambda x ) takes any number of arguments, x is newly allocated list of arguments Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 5 16:02:00 2007 @@ -164,10 +164,14 @@ self.args = [] arg = args while not isinstance(arg, W_Nil): - assert isinstance(arg.car, W_Identifier) - #list of argument names, not evaluated - self.args.append(arg.car.to_string()) - arg = arg.cdr + if isinstance(arg, W_Identifier): + self.args.append([arg.to_string()]) + break + else: + assert isinstance(arg.car, W_Identifier) + #list of argument names, not evaluated + self.args.append(arg.car.to_string()) + arg = arg.cdr self.body = body self.pname = pname @@ -177,16 +181,24 @@ return "#" % (self.pname,) def procedure(self, ctx, lst): - #ctx is a caller context, which is hoyfully ignored - if len(lst) != len(self.args): - raise "Wrong argument count" + #ctx is a caller context, which is joyfully ignored + + #if len(lst) != len(self.args): + # raise "Wrong argument count" local_ctx = self.closure.copy() #set lambda arguments - vars = zip(self.args, lst) - for (name, val) in vars: - local_ctx.put(name, val) + for idx in range(len(self.args)): + name = self.args[idx] + if isinstance(name, list): + local_ctx.put(name[0], plst2lst(lst[idx:])) + else: + local_ctx.put(name, lst[idx]) + + #vars = zip(self.args, lst) + #for (name, val) in vars: + # local_ctx.put(name, val) body_expression = self.body body_result = None @@ -196,6 +208,15 @@ return body_result # self.body.eval(local_ctx) +def plst2lst(plst): + """coverts python list() of W_Root into W_Pair scheme list""" + w_cdr = W_Nil() + plst.reverse() + for w_obj in plst: + w_cdr = W_Pair(w_obj, w_cdr) + + return w_cdr + ## # operations ## @@ -378,7 +399,7 @@ raise "Unbound" def set(self, name, obj): - """update existing location or create new location new""" + """update existing location or create new location""" if self.closure: loc = self.scope.get(name, None) else: Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 5 16:02:00 2007 @@ -232,3 +232,11 @@ assert w_result.to_number() == 43 assert ctx.get("x") is None +def test_lambda_lstarg(): + ctx = ExecutionContext() + w_result = eval_expr(ctx, """((lambda x x) 1 2 3)""") + assert isinstance(w_result, W_Pair) + assert w_result.car.to_number() == 1 + assert w_result.cdr.car.to_number() == 2 + assert w_result.cdr.cdr.car.to_number() == 3 + From jlg at codespeak.net Thu Jul 5 16:26:08 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 16:26:08 +0200 (CEST) Subject: [pypy-svn] r44741 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705142608.BB36480F4@code0.codespeak.net> Author: jlg Date: Thu Jul 5 16:26:07 2007 New Revision: 44741 Modified: pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_parser.py Log: dotted list parsing Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 5 16:26:07 2007 @@ -47,7 +47,8 @@ return {s}; sexpr: - list + dotted + | list | FLOAT | FIXNUM | BOOLEAN @@ -62,6 +63,17 @@ IGNORE* return {p}; + dotted: + '(' + IGNORE* + car = sexpr + '.' + IGNORE* + cdr = sexpr + ')' + IGNORE* + return {W_Pair(car, cdr)}; + pair: car = sexpr cdr = pair Modified: pypy/dist/pypy/lang/scheme/test/test_parser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_parser.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_parser.py Thu Jul 5 16:26:07 2007 @@ -100,3 +100,9 @@ t = parse("#t") assert unwrap(t) == True +def test_dotted(): + t = parse("(1 . 2)") + assert isinstance(t, W_Pair) + assert unwrap(t.car) == 1 + assert unwrap(t.cdr) == 2 + From jlg at codespeak.net Thu Jul 5 16:56:56 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 16:56:56 +0200 (CEST) Subject: [pypy-svn] r44742 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705145656.4C85580F1@code0.codespeak.net> Author: jlg Date: Thu Jul 5 16:56:56 2007 New Revision: 44742 Modified: pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_parser.py Log: dotted list parsing take2 Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 5 16:56:56 2007 @@ -47,8 +47,7 @@ return {s}; sexpr: - dotted - | list + list | FLOAT | FIXNUM | BOOLEAN @@ -63,19 +62,13 @@ IGNORE* return {p}; - dotted: - '(' - IGNORE* + pair: car = sexpr '.' IGNORE* cdr = sexpr - ')' - IGNORE* - return {W_Pair(car, cdr)}; - - pair: - car = sexpr + return {W_Pair(car, cdr)} + | car = sexpr cdr = pair return {W_Pair(car, cdr)} | return {W_Nil()}; Modified: pypy/dist/pypy/lang/scheme/test/test_parser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_parser.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_parser.py Thu Jul 5 16:56:56 2007 @@ -100,9 +100,26 @@ t = parse("#t") assert unwrap(t) == True -def test_dotted(): +def test_list_dotted(): t = parse("(1 . 2)") assert isinstance(t, W_Pair) assert unwrap(t.car) == 1 assert unwrap(t.cdr) == 2 + t = parse("(1 . (2 . 3))") + assert unwrap(t.car) == 1 + assert unwrap(t.cdr.car) == 2 + assert unwrap(t.cdr.cdr) == 3 + + t = parse("(1 . (2 . (3 . ())))") + assert unwrap(t) == [1, 2, 3] + +def test_list_mixed(): + t = parse("(1 2 . 3)") + assert unwrap(t.car) == 1 + assert unwrap(t.cdr.car) == 2 + assert unwrap(t.cdr.cdr) == 3 + + t = parse("(1 2 . (3 4))") + assert unwrap(t) == [1, 2, 3, 4] + From jlg at codespeak.net Thu Jul 5 17:50:50 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 17:50:50 +0200 (CEST) Subject: [pypy-svn] r44744 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705155050.3301880F2@code0.codespeak.net> Author: jlg Date: Thu Jul 5 17:50:49 2007 New Revision: 44744 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: (quote ...); W_Identifier.to_symbol() method Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 5 17:50:49 2007 @@ -26,6 +26,9 @@ def __repr__(self): return "" + def to_symbol(self): + return W_Symbol(self.name) + def eval(self, ctx): if ctx is None: @@ -314,6 +317,24 @@ w_body = lst.cdr #.car return W_Lambda(w_args, w_body, ctx.copy()) +class Quote(W_Macro): + def symbolize(self, lst): + + if isinstance(lst, W_Pair): + arg = lst + while not isinstance(arg, W_Nil): + arg.car = self.symbolize(arg.car) + arg = arg.cdr + + if isinstance(lst, W_Identifier): + lst = lst.to_symbol() + + return lst + + def eval(self, ctx, lst): + w_obj = self.symbolize(lst.car) + return w_obj + ## # Location() ## @@ -344,6 +365,7 @@ 'set!': Sete, 'if': MacroIf, 'lambda': Lambda, + 'quote': Quote, } OPERATION_MAP = {} Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 5 17:50:49 2007 @@ -240,3 +240,31 @@ assert w_result.cdr.car.to_number() == 2 assert w_result.cdr.cdr.car.to_number() == 3 +def test_quote(): + w_fnum = eval_noctx("(quote 42)") + assert isinstance(w_fnum, W_Fixnum) + assert w_fnum.to_number() == 42 + + w_sym = eval_noctx("(quote symbol)") + assert isinstance(w_sym, W_Symbol) + assert w_sym.to_string() == "symbol" + + w_lst = eval_noctx("(quote (1 2 3))") + assert isinstance(w_lst, W_Pair) + assert w_lst.car.to_number() == 1 + assert w_lst.cdr.car.to_number() == 2 + assert w_lst.cdr.cdr.car.to_number() == 3 + + w_lst = eval_noctx("(quote (a (x y) c))") + assert isinstance(w_lst, W_Pair) + assert isinstance(w_lst.car, W_Symbol) + assert w_lst.car.to_string() == "a" + w_pair = w_lst.cdr.car + assert isinstance(w_lst.cdr.cdr.car, W_Symbol) + assert w_lst.cdr.cdr.car.to_string() == "c" + + assert isinstance(w_pair.car, W_Symbol) + assert w_pair.car.to_string() == "x" + assert isinstance(w_pair.cdr.car, W_Symbol) + assert w_pair.cdr.car.to_string() == "y" + From arigo at codespeak.net Thu Jul 5 18:00:10 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 5 Jul 2007 18:00:10 +0200 (CEST) Subject: [pypy-svn] r44745 - pypy/extradoc/sprintinfo/post-ep2007 Message-ID: <20070705160010.5FE4880F2@code0.codespeak.net> Author: arigo Date: Thu Jul 5 18:00:09 2007 New Revision: 44745 Modified: pypy/extradoc/sprintinfo/post-ep2007/people.txt Log: My room still has a 2nd bed free if someone didn't make hotel arrangements yet. Modified: pypy/extradoc/sprintinfo/post-ep2007/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/post-ep2007/people.txt (original) +++ pypy/extradoc/sprintinfo/post-ep2007/people.txt Thu Jul 5 18:00:09 2007 @@ -8,9 +8,9 @@ column are known to be coming but there are no details available yet from them. -==================== ============== ======================= +==================== ============== ================================= Name Arrive/Depart Accomodation -==================== ============== ======================= +==================== ============== ================================= Maciej Fijalkowski 5/14th Some hostel Antonio Cuni 8/15th Same as Maciek :-) Samuele Pedroni 7/14th Ratonda Centrum Hotels @@ -19,9 +19,9 @@ Jakub Gustak 8/15th Hospitality Club :) Christian Tismer 8/16th Ratonda Centrum Hotels Simon Burton 8/15th -Armin Rigo 8/16th Comfort Vilnius +Armin Rigo 8/16th Comfort Vilnius (room to share!) Bea D?ring 8/13th Comfort Vilnius -==================== ============== ======================= +==================== ============== ================================= People on the following list were present at previous sprints: From jlg at codespeak.net Thu Jul 5 18:32:05 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 18:32:05 +0200 (CEST) Subject: [pypy-svn] r44747 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070705163205.A180E80F6@code0.codespeak.net> Author: jlg Date: Thu Jul 5 18:32:05 2007 New Revision: 44747 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: quotation parsing Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 5 18:32:05 2007 @@ -16,6 +16,16 @@ def eval(self, ctx): return self +class W_Symbol(W_Root): + def __init__(self, val): + self.name = val + + def to_string(self): + return self.name + + def __repr__(self): + return "" + class W_Identifier(W_Root): def __init__(self, val): self.name = val @@ -40,17 +50,7 @@ else: #reference to undefined identifier #unbound - raise "Unbound variable: %s" % (self.name, ) - -class W_Symbol(W_Root): - def __init__(self, val): - self.name = val - - def to_string(self): - return self.name - - def __repr__(self): - return "" + raise Exception("Unbound variable: %s" % (self.name, )) class W_Boolean(W_Root): def __init__(self, val): @@ -317,9 +317,11 @@ w_body = lst.cdr #.car return W_Lambda(w_args, w_body, ctx.copy()) +def Literal(sexpr): + return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) + class Quote(W_Macro): def symbolize(self, lst): - if isinstance(lst, W_Pair): arg = lst while not isinstance(arg, W_Nil): @@ -418,7 +420,7 @@ loc.obj = obj return obj - raise "Unbound" + raise Exception("Unbound variable: %s" % (name, )) def set(self, name, obj): """update existing location or create new location""" Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 5 18:32:05 2007 @@ -2,7 +2,7 @@ from pypy.rlib.parsing.pypackrat import PackratParser from pypy.rlib.parsing.makepackrat import BacktrackException, Status from pypy.lang.scheme.object import W_Pair, W_Fixnum, W_String, W_Identifier -from pypy.lang.scheme.object import W_Nil, W_Boolean, W_Float +from pypy.lang.scheme.object import W_Nil, W_Boolean, W_Float, Literal, W_Symbol def unquote(s): return s.replace('\\"', '"') @@ -46,8 +46,14 @@ EOF return {s}; + literal: + `'` + s = sexpr + return {Literal(s)}; + sexpr: list + | literal | FLOAT | FIXNUM | BOOLEAN Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 5 18:32:05 2007 @@ -79,7 +79,7 @@ loc2 = ctx.get_location("x") assert ctx.get("x").to_number() == 43 assert loc1 is loc2 - py.test.raises("Unbound", eval_expr, ctx, "(set! y 42)") + py.test.raises(Exception, eval_expr, ctx, "(set! y 42)") def test_func(): ctx = ExecutionContext() @@ -268,3 +268,31 @@ assert isinstance(w_pair.cdr.car, W_Symbol) assert w_pair.cdr.car.to_string() == "y" +def test_quote_parse(): + w_fnum = eval_noctx("'42") + assert isinstance(w_fnum, W_Fixnum) + assert w_fnum.to_number() == 42 + + w_sym = eval_noctx("'symbol") + assert isinstance(w_sym, W_Symbol) + assert w_sym.to_string() == "symbol" + + w_lst = eval_noctx("'(1 2 3)") + assert isinstance(w_lst, W_Pair) + assert w_lst.car.to_number() == 1 + assert w_lst.cdr.car.to_number() == 2 + assert w_lst.cdr.cdr.car.to_number() == 3 + + w_lst = eval_noctx("'(a (x y) c)") + assert isinstance(w_lst, W_Pair) + assert isinstance(w_lst.car, W_Symbol) + assert w_lst.car.to_string() == "a" + w_pair = w_lst.cdr.car + assert isinstance(w_lst.cdr.cdr.car, W_Symbol) + assert w_lst.cdr.cdr.car.to_string() == "c" + + assert isinstance(w_pair.car, W_Symbol) + assert w_pair.car.to_string() == "x" + assert isinstance(w_pair.cdr.car, W_Symbol) + assert w_pair.cdr.car.to_string() == "y" + From jlg at codespeak.net Thu Jul 5 18:57:24 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 18:57:24 +0200 (CEST) Subject: [pypy-svn] r44750 - pypy/dist/pypy/lang/scheme Message-ID: <20070705165724.D8A5A80F4@code0.codespeak.net> Author: jlg Date: Thu Jul 5 18:57:24 2007 New Revision: 44750 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py Log: (quote ) does nothing Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 5 18:57:24 2007 @@ -26,7 +26,7 @@ def __repr__(self): return "" -class W_Identifier(W_Root): +class W_Identifier(W_Symbol): def __init__(self, val): self.name = val @@ -36,9 +36,6 @@ def __repr__(self): return "" - def to_symbol(self): - return W_Symbol(self.name) - def eval(self, ctx): if ctx is None: @@ -321,21 +318,8 @@ return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) class Quote(W_Macro): - def symbolize(self, lst): - if isinstance(lst, W_Pair): - arg = lst - while not isinstance(arg, W_Nil): - arg.car = self.symbolize(arg.car) - arg = arg.cdr - - if isinstance(lst, W_Identifier): - lst = lst.to_symbol() - - return lst - def eval(self, ctx, lst): - w_obj = self.symbolize(lst.car) - return w_obj + return lst.car ## # Location() Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 5 18:57:24 2007 @@ -2,7 +2,7 @@ from pypy.rlib.parsing.pypackrat import PackratParser from pypy.rlib.parsing.makepackrat import BacktrackException, Status from pypy.lang.scheme.object import W_Pair, W_Fixnum, W_String, W_Identifier -from pypy.lang.scheme.object import W_Nil, W_Boolean, W_Float, Literal, W_Symbol +from pypy.lang.scheme.object import W_Nil, W_Boolean, W_Float, Literal def unquote(s): return s.replace('\\"', '"') From jlg at codespeak.net Thu Jul 5 19:11:25 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 5 Jul 2007 19:11:25 +0200 (CEST) Subject: [pypy-svn] r44751 - pypy/dist/pypy/lang/scheme/test Message-ID: <20070705171125.C5A9E80F3@code0.codespeak.net> Author: jlg Date: Thu Jul 5 19:11:25 2007 New Revision: 44751 Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py Log: lambda formals with last cdr not nil Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 5 19:11:25 2007 @@ -240,6 +240,17 @@ assert w_result.cdr.car.to_number() == 2 assert w_result.cdr.cdr.car.to_number() == 3 +def test_lambda_dotted_lstarg(): + ctx = ExecutionContext() + w_result = eval_expr(ctx, """((lambda (x y . z) z) 3 4)""") + assert isinstance(w_result, W_Nil) + + w_result = eval_expr(ctx, """((lambda (x y . z) z) 3 4 5 6)""") + assert isinstance(w_result, W_Pair) + assert w_result.car.to_number() == 5 + assert w_result.cdr.car.to_number() == 6 + assert isinstance(w_result.cdr.cdr, W_Nil) + def test_quote(): w_fnum = eval_noctx("(quote 42)") assert isinstance(w_fnum, W_Fixnum) From pedronis at codespeak.net Fri Jul 6 13:21:12 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 13:21:12 +0200 (CEST) Subject: [pypy-svn] r44771 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706112112.191BE8156@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 13:21:11 2007 New Revision: 44771 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ Log: directory for the long talk From pedronis at codespeak.net Fri Jul 6 13:46:45 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 13:46:45 +0200 (CEST) Subject: [pypy-svn] r44775 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706114645.538758167@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 13:46:44 2007 New Revision: 44775 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui (contents, props changed) Log: start drafting the talk Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 13:46:44 2007 @@ -0,0 +1,53 @@ +.. include:: + +========================================================== +PyPy 1.0 xxxx +========================================================== + +:Authors: Armin Rigo,Samuele Pedroni + +What is PyPy? +============== + +PyPy 1.0 +================ + + +Paradigm +==================== + +.. raw:: html + +
+ +.. image:: arch-overview.png + :align: center + + +Translation framework +================================ + +.. raw:: html + +

+ +.. image:: arch-framework.png + :align: center + + +Compatibility Matrix +========================= + +... + +... starts of JIT stuff + + +.. xxx extra slide arch-translation diagram + + +.. |bullet| unicode:: U+02022 +.. footer:: Samuele Pedroni, Armin Rigo + + + Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui Fri Jul 6 13:46:44 2007 @@ -0,0 +1 @@ +link ../ui \ No newline at end of file From pedronis at codespeak.net Fri Jul 6 15:17:55 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:17:55 +0200 (CEST) Subject: [pypy-svn] r44781 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706131755.AE9378165@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:17:55 2007 New Revision: 44781 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/arch-overview.png - copied unchanged from r44678, pypy/eu-tracking/talk/review2/translation_opts/arch-overview.png Log: copy this From pedronis at codespeak.net Fri Jul 6 15:18:30 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:18:30 +0200 (CEST) Subject: [pypy-svn] r44782 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706131830.34B508168@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:18:29 2007 New Revision: 44782 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/arch-framework.png - copied unchanged from r44678, pypy/eu-tracking/talk/review2/translation_opts/arch-framework.png Log: copy this png - From pedronis at codespeak.net Fri Jul 6 15:20:00 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:20:00 +0200 (CEST) Subject: [pypy-svn] r44783 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706132000.141F28169@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:19:59 2007 New Revision: 44783 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/arch-translation.png - copied unchanged from r44678, pypy/eu-tracking/talk/review2/translation_opts/arch-translation.png Log: copy this From pedronis at codespeak.net Fri Jul 6 15:20:33 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:20:33 +0200 (CEST) Subject: [pypy-svn] r44784 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706132033.56C32816B@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:20:32 2007 New Revision: 44784 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/compat-matrix.png - copied unchanged from r44678, pypy/eu-tracking/talk/review2/test_integration/compat-matrix.png Log: copy this From pedronis at codespeak.net Fri Jul 6 15:25:43 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:25:43 +0200 (CEST) Subject: [pypy-svn] r44785 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706132543.1B253817D@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:25:42 2007 New Revision: 44785 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: some work on the intro part Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 15:25:42 2007 @@ -9,9 +9,16 @@ What is PyPy? ============== -PyPy 1.0 -================ +PyPy is: + +* A fun and challenging open source project with many contributions + +* An open source implementation of Python (written in Python) + +* A tool-chain with a *new paradigm* for building interpreters +From Dec 2004 to March 2006 PyPy was a research project, +funded by the EU and a consortium of companies. Paradigm ==================== @@ -23,6 +30,32 @@ .. image:: arch-overview.png :align: center +Motivation +============= + +Conventional approaches to interpreters face hard trade-offs: +speed, maintainability, flexibility. + +A lot of decisions are hard-coded pervasively, improvements +and evolution are hampered (e.g. psyco, stackless require quite some efforts +and cannot bring their full potential). + +Lots of duplication of effort to cover many platforms. + +*It should be possible to do better.* + +PyPy 1.0 +================ + +released: 27th March 2007 + +- *JIT compiler generator working on the Python interpreter* +- more optimizations +- CLI backend produces a PyPy.NET which can interact with CLR libraries +- taint object space and transparent proxies + +*Not completely mature yet but our original vision is fully +materialized* Translation framework ================================ @@ -34,17 +67,20 @@ .. image:: arch-framework.png :align: center +.. JIT stuff Compatibility Matrix ========================= -... - -... starts of JIT stuff - +.. image:: compat-matrix.png + :align: center -.. xxx extra slide arch-translation diagram +Translation Diagram +===================== +.. image:: arch-translation.png + :align: center + :scale: 90 .. |bullet| unicode:: U+02022 .. footer:: Samuele Pedroni, Armin Rigo From pedronis at codespeak.net Fri Jul 6 15:26:33 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:26:33 +0200 (CEST) Subject: [pypy-svn] r44787 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706132633.8DF6B8183@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:26:33 2007 New Revision: 44787 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: we need a conclusion part Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 15:26:33 2007 @@ -69,6 +69,8 @@ .. JIT stuff +.. Conclusion??? + Compatibility Matrix ========================= From arigo at codespeak.net Fri Jul 6 15:26:56 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 6 Jul 2007 15:26:56 +0200 (CEST) Subject: [pypy-svn] r44788 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706132656.75FEA8183@code0.codespeak.net> Author: arigo Date: Fri Jul 6 15:26:56 2007 New Revision: 44788 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui Log: Fix the symlink. Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui Fri Jul 6 15:26:56 2007 @@ -1 +1 @@ -link ../ui \ No newline at end of file +link ../../ui \ No newline at end of file From pedronis at codespeak.net Fri Jul 6 15:42:01 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:42:01 +0200 (CEST) Subject: [pypy-svn] r44789 - pypy/extradoc/talk/ep2007/ui Message-ID: <20070706134201.B9BA8816F@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:42:01 2007 New Revision: 44789 Added: pypy/extradoc/talk/ep2007/ui/ - copied from r44678, pypy/extradoc/talk/dls2006/ui/ Log: use this one From pedronis at codespeak.net Fri Jul 6 15:43:10 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 15:43:10 +0200 (CEST) Subject: [pypy-svn] r44790 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706134310.CBEBC818C@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 15:43:10 2007 New Revision: 44790 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui Log: change back, I forgot to checkin the ui one level up It seems we have different versions of it around Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui Fri Jul 6 15:43:10 2007 @@ -0,0 +1 @@ +link ../ui \ No newline at end of file From pedronis at codespeak.net Fri Jul 6 16:23:27 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 16:23:27 +0200 (CEST) Subject: [pypy-svn] r44791 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706142327.0FE92813A@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 16:23:26 2007 New Revision: 44791 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/overview2.png - copied unchanged from r44678, pypy/eu-tracking/talk/review2/dynamic_opts/overview2.png Log: jit approach overview From pedronis at codespeak.net Fri Jul 6 16:25:03 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 16:25:03 +0200 (CEST) Subject: [pypy-svn] r44792 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706142503.2EB23813A@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 16:25:01 2007 New Revision: 44792 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: paste in our the material from the review Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 16:25:01 2007 @@ -67,9 +67,180 @@ .. image:: arch-framework.png :align: center -.. JIT stuff +Flexibility vs. Performance +============================= + +* interpreters are easy to write and evolve + +* for high performance dynamic compilation is required + +Traditional dynamic compilers +=============================== + +* huge resource investment +* the richer the semantics, the harder to write +* poor encoding of language semantics +* hard to evolve + +Need for novel approaches! + +PyPy Approach: Goal +============================= + +.. raw:: html + +
+ +.. image:: overview2.png + :scale: 55 + + +PyPy Approach (2) +======================= + +* use partial evaluation techniques to generate a dynamic compiler + from an interpreter +* inspiration: Psyco +* our translation tool-chain was designed for trying this + +Futamura +===================== + +* *Partial evalution of computation process - + an approach to a compiler-compiler*, '71 + +* Generating compilers from interpreters, + automatic specialization + +* Relatively little practical impact so far + +Challenges +====================== + +* Effective dynamic compilation requires feedback of runtime + information into compile-time + +* A shortcoming of PE is that in many cases not much can be really + assumed constant at compile-time: poor results + +* For a dynamic language: types are a primary example + +Solution: Promotion +==================== + +* enhance PE with the ability + to "promote" run-time values + to compile-time + +* leverage the dynamic setting + + +Overall ingredients +===================== + +The pieces to enable effective dynamic compiler generation +in PyPy: + +- implementer-supplied hint based approach to fixing + what should be assumed compile-time constant +- *promotion* +- direct support for lazy + allocation only on escape + ("virtuals") +- use stack and registers for language-level frame contents + +.. ("virtualizables") + +Language-agnostic +==================== + +* The dynamic generation process and primitives are language-agnostic. + +* The language implementations should be able to evolve up to + maintaining the hints. + +* By construction all interpreter/language features are supported + +pypy-c-jit +====================== + +PyPy 1.0 contains both the dynamic compiler generator and the start of +its application to PyPy's Python intepreter. + +* included are backends for IA32 and PPC +* integer aritmethic operations are optimized +* for things for which hints were provided + we are in the speed range of gcc -O0 +* demo (63x faster than CPython) + +.. demo f1 + +The transformation: Timeshifting +================================== + +The generation process is implemented as a *binding-time ("color")* +directed transformation of the low-level graphs. + +Coloring +================= + +* green: compile-time value +* red: runtime value + +we reuse the type inference +framework to propagate colors +respecting the constraints from +the hints + +Timeshifting Basics +==================== + +* green operations: unchanged, executed at compile-time + +* red operations: converted into corresponding code emitting code + +Timeshifting Control Flow +=========================== + +- red split points: schedule multiple compilation states +- merge points: merge logic to reuse code for equivalent states + +Promotion +================= + +Promotion is implemented generating a switch that grows to cover the +seen runtime values + +* First compilation stops at a promotion point and the switch with + only a default case is generated. The default will call back into + the compiler with runtime values. + +* On callback the compiler add one more case to the switch and + generate more code assuming the received value. + +.. need to save state in a compact form: paths + +Virtuals + Promotion +===================== + +* example + +.. tlc example results + +Conclusion +============= + +xxx we need some more general status/conclusion slides + +Effective dynamic compiler generation make flexibility and ease of +evolution mostly **orthogonal to the performance question**. + +Implementers are free to implement languages as **understandable interpreters**. + +PyPy proves this a viable approach worth of further exploration. + + -.. Conclusion??? Compatibility Matrix ========================= From arigo at codespeak.net Fri Jul 6 16:56:53 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 6 Jul 2007 16:56:53 +0200 (CEST) Subject: [pypy-svn] r44793 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706145653.136558132@code0.codespeak.net> Author: arigo Date: Fri Jul 6 16:56:52 2007 New Revision: 44793 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui (contents, props changed) Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: Reorder a few slides; add a bit of content. The intro is still light but getting better IMHO. Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 16:56:52 2007 @@ -20,6 +20,20 @@ From Dec 2004 to March 2006 PyPy was a research project, funded by the EU and a consortium of companies. +Motivation +============= + +Conventional approaches to interpreters face hard trade-offs: +speed, maintainability, flexibility. + +A lot of decisions are hard-coded pervasively, improvements +and evolution are hampered (e.g. psyco, stackless require quite some efforts +and cannot bring their full potential). + +Lots of duplication of effort to cover many platforms (C, Java, .NET). + +*It should be possible to do better.* + Paradigm ==================== @@ -30,24 +44,23 @@ .. image:: arch-overview.png :align: center -Motivation -============= - -Conventional approaches to interpreters face hard trade-offs: -speed, maintainability, flexibility. +PyPy 1.0 +================ -A lot of decisions are hard-coded pervasively, improvements -and evolution are hampered (e.g. psyco, stackless require quite some efforts -and cannot bring their full potential). +Released: 27th March 2007 -Lots of duplication of effort to cover many platforms. +Contains, as before: -*It should be possible to do better.* +- a full compliant Python interpreter +- a translation tool-chain that produces reasonably efficient interpreters: + PyPy-C, PyPy-LLVM, PyPy.NET PyPy 1.0 ================ -released: 27th March 2007 +Released: 27th March 2007 + +What's new: - *JIT compiler generator working on the Python interpreter* - more optimizations @@ -57,15 +70,22 @@ *Not completely mature yet but our original vision is fully materialized* -Translation framework -================================ +Compatibility Matrix +========================= -.. raw:: html +.. image:: compat-matrix.png + :align: center -

+Status +========= -.. image:: arch-framework.png - :align: center +xxx a status slide could fit here + + +JIT compiler generator +====================== + +.. (empty transition slide to say "in this talk we're mainly focusing on...") Flexibility vs. Performance ============================= @@ -107,7 +127,7 @@ ===================== * *Partial evalution of computation process - - an approach to a compiler-compiler*, '71 + an approach to a compiler-compiler*, 1971 * Generating compilers from interpreters, automatic specialization @@ -199,12 +219,16 @@ * red operations: converted into corresponding code emitting code +.. demo? + Timeshifting Control Flow =========================== - red split points: schedule multiple compilation states - merge points: merge logic to reuse code for equivalent states +.. demo? + Promotion ================= @@ -220,6 +244,8 @@ .. need to save state in a compact form: paths +.. demo? + Virtuals + Promotion ===================== @@ -227,10 +253,8 @@ .. tlc example results -Conclusion -============= - -xxx we need some more general status/conclusion slides +Conclusion (JIT) +================ Effective dynamic compiler generation make flexibility and ease of evolution mostly **orthogonal to the performance question**. @@ -239,13 +263,23 @@ PyPy proves this a viable approach worth of further exploration. +Conclusion +========== +xxx we need some more general status/conclusion slides -Compatibility Matrix -========================= -.. image:: compat-matrix.png +.. the end + +Translation framework +================================ + +.. raw:: html + +

+ +.. image:: arch-framework.png :align: center Translation Diagram Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/ui Fri Jul 6 16:56:52 2007 @@ -0,0 +1 @@ +link ../ui \ No newline at end of file From hpk at codespeak.net Fri Jul 6 18:50:00 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 6 Jul 2007 18:50:00 +0200 (CEST) Subject: [pypy-svn] r44795 - in pypy/extradoc/talk/ep2007: . interpreter Message-ID: <20070706165000.B7AAB8170@code0.codespeak.net> Author: hpk Date: Fri Jul 6 18:49:59 2007 New Revision: 44795 Added: pypy/extradoc/talk/ep2007/interpreter/ pypy/extradoc/talk/ep2007/interpreter/dynlang.png - copied unchanged from r44794, pypy/extradoc/talk/dzug2007/dynlang.png pypy/extradoc/talk/ep2007/interpreter/interpreter.txt - copied, changed from r44794, pypy/extradoc/talk/ep2007/interpreter.txt Removed: pypy/extradoc/talk/ep2007/interpreter.txt Log: shifting interpreter talk to its own directory, amending it with a few more slides (need to cross-check for redundancy ...) from dzug2007 From hpk at codespeak.net Fri Jul 6 18:50:35 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 6 Jul 2007 18:50:35 +0200 (CEST) Subject: [pypy-svn] r44796 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070706165035.BBBC78170@code0.codespeak.net> Author: hpk Date: Fri Jul 6 18:50:34 2007 New Revision: 44796 Added: pypy/extradoc/talk/ep2007/interpreter/ui (contents, props changed) Log: adding ui link Added: pypy/extradoc/talk/ep2007/interpreter/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/interpreter/ui Fri Jul 6 18:50:34 2007 @@ -0,0 +1 @@ +link ../ui \ No newline at end of file From arigo at codespeak.net Fri Jul 6 18:54:53 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 6 Jul 2007 18:54:53 +0200 (CEST) Subject: [pypy-svn] r44797 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706165453.00D50816F@code0.codespeak.net> Author: arigo Date: Fri Jul 6 18:54:53 2007 New Revision: 44797 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: Status and a Performance slide. Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 18:54:53 2007 @@ -79,13 +79,35 @@ Status ========= -xxx a status slide could fit here - +* still rough edges that need polishing +* still missing a few important extension modules +* right now, useful parts: special languag features; RPython for high-perf + +Performance +============= + +Current interpreter performance is not bad, but lower than CPython: + ++----------+-------------------+---------------+ +| | CPython 2.4.4 | pypy-llvm | ++==========+===================+===============+ +| richards | 1.00 | 1.17 | ++----------+-------------------+---------------+ +| pystone | 1.00 | 1.55 | ++----------+-------------------+---------------+ +| templess | 1.00 | 5.41 | ++----------+-------------------+---------------+ +| gadfly2 | 1.00 | 6.38 | ++----------+-------------------+---------------+ +| mako | 1.00 | 4.46 | ++----------+-------------------+---------------+ JIT compiler generator ====================== -.. (empty transition slide to say "in this talk we're mainly focusing on...") +* our trump card to run plain Python programs fast + +* still work in progress! Flexibility vs. Performance ============================= From fijal at codespeak.net Fri Jul 6 20:01:27 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 6 Jul 2007 20:01:27 +0200 (CEST) Subject: [pypy-svn] r44802 - in pypy/extradoc/talk/ep2007: . rpython rpython-demo Message-ID: <20070706180127.0DA2C816F@code0.codespeak.net> Author: fijal Date: Fri Jul 6 20:01:27 2007 New Revision: 44802 Added: pypy/extradoc/talk/ep2007/rpython/ - copied from r44798, pypy/extradoc/talk/ep2007/rpython-demo/ pypy/extradoc/talk/ep2007/rpython/rpython.txt - copied unchanged from r44798, pypy/extradoc/talk/ep2007/rpython.txt Removed: pypy/extradoc/talk/ep2007/rpython-demo/ pypy/extradoc/talk/ep2007/rpython.txt Log: Make direcory structure a bit more the same everywhere From fijal at codespeak.net Fri Jul 6 20:02:10 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 6 Jul 2007 20:02:10 +0200 (CEST) Subject: [pypy-svn] r44804 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070706180210.1CA598159@code0.codespeak.net> Author: fijal Date: Fri Jul 6 20:02:09 2007 New Revision: 44804 Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt Log: Add a date Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython/rpython.txt Fri Jul 6 20:02:09 2007 @@ -5,7 +5,7 @@ :Authors: Antonio Cuni (DISI), Maciej Fijalkowski (merlinux GmbH) :Place: Europython 2007 -:Date: XXX +:Date: 10 July 2007 Why invent new language? ------------------------ From arigo at codespeak.net Fri Jul 6 20:33:35 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 6 Jul 2007 20:33:35 +0200 (CEST) Subject: [pypy-svn] r44806 - in pypy/extradoc/talk/ep2007: interpreter pypy_10_and_jit Message-ID: <20070706183335.90D968173@code0.codespeak.net> Author: arigo Date: Fri Jul 6 20:33:33 2007 New Revision: 44806 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: The current benchmarks (it's the latest pypy-llvm that builds). Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Fri Jul 6 20:33:33 2007 @@ -151,21 +151,21 @@ Relative speeds to CPython ------------------------------ +(2007-06-14) + +----------+-------------------+---------------+ | | CPython 2.4.4 | pypy-llvm | +==========+===================+===============+ -| richards | 1.00 | 1.17 | +| richards | 1.00 | 1.44 | +----------+-------------------+---------------+ -| pystone | 1.00 | 1.55 | -+----------+-------------------+---------------+ -| templess | 1.00 | 5.41 | +| pystone | 1.00 | 1.28 | +----------+-------------------+---------------+ -| gadfly | 1.00 | 6.38 | +| templess | 1.00 | 3.81 | +----------+-------------------+---------------+ -| mako | 1.00 | 7.65 | +| gadfly2 | 1.00 | 6.22 | ++----------+-------------------+---------------+ +| mako | 1.00 | 3.11 | +----------+-------------------+---------------+ - -(March 2007, improvements afterwards) Python Interpreter Status (1.0) --------------------------------- Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 20:33:33 2007 @@ -86,28 +86,33 @@ Performance ============= -Current interpreter performance is not bad, but lower than CPython: +Current interpreter performance (2007-06-14) is not bad, but lower than +CPython: +----------+-------------------+---------------+ | | CPython 2.4.4 | pypy-llvm | +==========+===================+===============+ -| richards | 1.00 | 1.17 | +| richards | 1.00 | 1.44 | +----------+-------------------+---------------+ -| pystone | 1.00 | 1.55 | +| pystone | 1.00 | 1.28 | +----------+-------------------+---------------+ -| templess | 1.00 | 5.41 | +| templess | 1.00 | 3.81 | +----------+-------------------+---------------+ -| gadfly2 | 1.00 | 6.38 | +| gadfly2 | 1.00 | 6.22 | +----------+-------------------+---------------+ -| mako | 1.00 | 4.46 | +| mako | 1.00 | 3.11 | +----------+-------------------+---------------+ JIT compiler generator ====================== -* our trump card to run plain Python programs fast +* a lot of time is spent in the Garbage Collector, + so we have room for improvement there -* still work in progress! +* but our real trump card to run plain Python programs fast + is the JIT compiler generator + +* works - but still work in progress before it's really useful! Flexibility vs. Performance ============================= From pedronis at codespeak.net Fri Jul 6 20:34:14 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 20:34:14 +0200 (CEST) Subject: [pypy-svn] r44807 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706183414.A2F758179@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 20:34:14 2007 New Revision: 44807 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: typo Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 20:34:14 2007 @@ -81,7 +81,7 @@ * still rough edges that need polishing * still missing a few important extension modules -* right now, useful parts: special languag features; RPython for high-perf +* right now, useful parts: special language features; RPython for high-perf Performance ============= From antocuni at codespeak.net Fri Jul 6 20:49:24 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 6 Jul 2007 20:49:24 +0200 (CEST) Subject: [pypy-svn] r44809 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070706184924.F36908183@code0.codespeak.net> Author: antocuni Date: Fri Jul 6 20:49:24 2007 New Revision: 44809 Added: pypy/extradoc/talk/ep2007/rpython/demo.py (contents, props changed) Log: automatic execution of the demo Added: pypy/extradoc/talk/ep2007/rpython/demo.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/rpython/demo.py Fri Jul 6 20:49:24 2007 @@ -0,0 +1,9 @@ +import clr +clr.AddReferenceToFile('mylib.dll') + +import func +import mylib + +N = 10**7 +print func.fn(N) +print mylib.fn(N) From antocuni at codespeak.net Fri Jul 6 20:50:34 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 6 Jul 2007 20:50:34 +0200 (CEST) Subject: [pypy-svn] r44810 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070706185034.4C1D78181@code0.codespeak.net> Author: antocuni Date: Fri Jul 6 20:50:33 2007 New Revision: 44810 Added: pypy/extradoc/talk/ep2007/rpython/ui (contents, props changed) Log: add a link to ../ui Added: pypy/extradoc/talk/ep2007/rpython/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/rpython/ui Fri Jul 6 20:50:33 2007 @@ -0,0 +1 @@ +link ../ui/ \ No newline at end of file From antocuni at codespeak.net Fri Jul 6 22:03:01 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 6 Jul 2007 22:03:01 +0200 (CEST) Subject: [pypy-svn] r44811 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070706200301.70C15818C@code0.codespeak.net> Author: antocuni Date: Fri Jul 6 22:02:59 2007 New Revision: 44811 Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt Log: some edits and some new slides Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython/rpython.txt Fri Jul 6 22:02:59 2007 @@ -1,16 +1,21 @@ ======================================================= -RPython: need for speed aka C and C# considered harmful +RPython: Need for speed ======================================================= -:Authors: Antonio Cuni (DISI), Maciej Fijalkowski (merlinux GmbH) +aka: C and C# considered harmful +================================ + +:Authors: Antonio Cuni (DISI - University of Genoa), Maciej Fijalkowski (merlinux GmbH) :Place: Europython 2007 :Date: 10 July 2007 Why invent new language? ------------------------ -* Grown as a side effect of the pypy project +XXX antocuni: I think this slide makes more sense after the next + +* Grown as a side effect of the PyPy project * Python is too slow for some purposes @@ -21,55 +26,72 @@ What is RPython? ---------------- -* Python (every RPython program has the same meaning - when run on top of CPython) +* RPython programs run unmodified on top of CPython * Restricted subset * Constructed from live objects - initialization is done in Python -* XXX more? +* Python is the meta-programming language for RPython + Restricted, what do you mean? ----------------------------- -* Full type inference - you cannot mix incompatible types +* Static type system -* Polymorphic collections (ie list of ints) +* Full type inference - you cannot mix incompatible types -* No dynamic stuff: switch class, add/remove methods, etc. +* Polymorphic collections (i.e. list of ints) * Java-like object model +* No dynamic stuff: switch class, add/remove methods, etc. + * No support for special methods (only __init__ and __del__ works) -* No frames and such stuff +* No introspection Example of use case ------------------- * Write code -* test/debug on top of CPython +* Test/debug on top of CPython -* compile +* Compile -* graph viewer +* Graph viewer Type inference -------------- XXXX should we really mention that? +:: + + def foo(): + return bar(1, 2) + bar(3, 4.5) + + def bar(a, b): + return a+1 + +* a: Signed U Siged --> Signed + +* b: Signed U Float --> Float + +* result of bar: Signed + 1 --> Signed + + Modules restrictions -------------------- -* few modules work +* Few modules work -* they have different, more static API (ie open()) +* They have different, more static API (ie open()) -* all modules might be used when initializing +* All modules might be used when initializing Few more words about file API ----------------------------- @@ -94,10 +116,21 @@ * no more pros... -Writing standalone C programs ------------------------------ +Writing standalone C/.NET/JVM programs +-------------------------------------- + +:: -XXX explain writing targets + def target(*args): + return entry_point, None + + def entry_point(argv): + print 'Hello World!' + return 0 + +* see also translator/goal/targetnopstandalone.py + +* ./translate.py [--backend={c|cli|jvm}] targetXXX.py Writing CPython extensions --------------------------- @@ -131,6 +164,8 @@ * Live demo +* http://codespeak.net/pypy/dist/pypy/doc/carbonpython.html + The JVM backend --------------- @@ -141,6 +176,7 @@ * Not as mature as gencli +* Only executables at the moment The JS backend @@ -148,3 +184,16 @@ XXX fijal, feel free to fill in + +Conclusions +----------- + +* RPython has lot of rough edges + +* ...but it might be useful in some contexts + +* More convenient than C/C++/C#/Java + +* Much faster than CPython/IronPython/Jython + +* Stay tuned From pedronis at codespeak.net Fri Jul 6 23:11:24 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 6 Jul 2007 23:11:24 +0200 (CEST) Subject: [pypy-svn] r44812 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070706211124.A05BE818E@code0.codespeak.net> Author: pedronis Date: Fri Jul 6 23:11:22 2007 New Revision: 44812 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: tentative conclusion slide Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Fri Jul 6 23:11:22 2007 @@ -293,8 +293,17 @@ Conclusion ========== -xxx we need some more general status/conclusion slides +We have achieved a malleable interpreter and interpreter-making +process which can only further improve over time, proving our vision +sound. +This is a solid base looking forward: + +- the rough edges are what we need to tackle next + +- development is going to continue in open source fashion + +- there will be further sprints: one is organized after the conference! .. the end From fijal at codespeak.net Sat Jul 7 11:14:20 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 7 Jul 2007 11:14:20 +0200 (CEST) Subject: [pypy-svn] r44819 - pypy/extradoc/talk/rupy2007/demos Message-ID: <20070707091420.E9535818D@code0.codespeak.net> Author: fijal Date: Sat Jul 7 11:14:19 2007 New Revision: 44819 Modified: pypy/extradoc/talk/rupy2007/demos/ (props changed) pypy/extradoc/talk/rupy2007/demos/f.py (props changed) pypy/extradoc/talk/rupy2007/demos/listserver.py (props changed) pypy/extradoc/talk/rupy2007/demos/local.py (contents, props changed) pypy/extradoc/talk/rupy2007/demos/myproxy.py (props changed) pypy/extradoc/talk/rupy2007/demos/nasty.py (props changed) pypy/extradoc/talk/rupy2007/demos/remote.py (contents, props changed) pypy/extradoc/talk/rupy2007/demos/serv.py (props changed) pypy/extradoc/talk/rupy2007/demos/test_serv.py (props changed) pypy/extradoc/talk/rupy2007/demos/thunk.py (props changed) Log: fixeol Modified: pypy/extradoc/talk/rupy2007/demos/local.py ============================================================================== --- pypy/extradoc/talk/rupy2007/demos/local.py (original) +++ pypy/extradoc/talk/rupy2007/demos/local.py Sat Jul 7 11:14:19 2007 @@ -4,7 +4,7 @@ import sys import pdb -PORT = 12121 +PORT = 12122 def f(): print "Calling f" Modified: pypy/extradoc/talk/rupy2007/demos/remote.py ============================================================================== --- pypy/extradoc/talk/rupy2007/demos/remote.py (original) +++ pypy/extradoc/talk/rupy2007/demos/remote.py Sat Jul 7 11:14:19 2007 @@ -3,7 +3,7 @@ from distributed.socklayer import Finished, socket_listener, socket_connecter import sys -PORT = 12121 +PORT = 12122 class X: def __init__(self): @@ -15,6 +15,9 @@ x = X() +def f(name): + return open(name) + if __name__ == '__main__': send, receive = socket_listener(address=('', PORT)) try: From fijal at codespeak.net Sat Jul 7 11:47:18 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 7 Jul 2007 11:47:18 +0200 (CEST) Subject: [pypy-svn] r44820 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070707094718.7C040819B@code0.codespeak.net> Author: fijal Date: Sat Jul 7 11:47:17 2007 New Revision: 44820 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: Few updates Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Sat Jul 7 11:47:17 2007 @@ -15,8 +15,6 @@ * they give new dimensions to how one can use python * pick the feature set of your choice... -XXX crap - PyPy Overview -------------- @@ -47,6 +45,8 @@ - static RPython-level bindings - on top dynamic bindings using reflection (clr module) +- more on tomorrow's talk + Multiple object implementations ------------------------------- @@ -56,6 +56,8 @@ * example - multidicts +XXX multidict demo + Python Interpreter Prototypes ------------------------------------- @@ -83,8 +85,6 @@ * see also .NET TransparentProxy -XXX list-over-http demo - Lazy ad-hoc distribution ------------------------ @@ -93,8 +93,6 @@ - remote objects are presented through transparent proxies - access to remote tracebacks, frames, etc. work as if local! -XXX show - Orthogonal Persistence -------------------------- @@ -109,6 +107,8 @@ Threading in PyPy ----------------------- +XXX will be in the stackless talk, shall we mention it? + * principal choice of os-threading models (GIL, no, ...) * stackless *transform* / micro-threads: @@ -186,4 +186,4 @@ * lack of extension modules -XXX more? +* os-level threads not working very well From fijal at codespeak.net Sat Jul 7 11:47:32 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 7 Jul 2007 11:47:32 +0200 (CEST) Subject: [pypy-svn] r44821 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070707094732.7208E8197@code0.codespeak.net> Author: fijal Date: Sat Jul 7 11:47:31 2007 New Revision: 44821 Added: pypy/extradoc/talk/ep2007/interpreter/listserver.py - copied unchanged from r44819, pypy/extradoc/talk/rupy2007/demos/listserver.py pypy/extradoc/talk/ep2007/interpreter/local.py - copied unchanged from r44819, pypy/extradoc/talk/rupy2007/demos/local.py pypy/extradoc/talk/ep2007/interpreter/myproxy.py - copied unchanged from r44819, pypy/extradoc/talk/rupy2007/demos/myproxy.py pypy/extradoc/talk/ep2007/interpreter/persistance.py (contents, props changed) pypy/extradoc/talk/ep2007/interpreter/remote.py - copied unchanged from r44819, pypy/extradoc/talk/rupy2007/demos/remote.py pypy/extradoc/talk/ep2007/interpreter/rope.py - copied unchanged from r44819, pypy/extradoc/talk/rupy2007/demos/nasty.py Log: Add some demos (mostly copied from the rupy talk) Added: pypy/extradoc/talk/ep2007/interpreter/persistance.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/interpreter/persistance.py Sat Jul 7 11:47:31 2007 @@ -0,0 +1,17 @@ +import sys +import pickle +import tputil + +class X: + def __init__(self, x): + self.x = x + self.caller = sys._getframe(1) + +def proxy_controller(oper): + result = oper.delegate() + f = open("data", "w") + f.write(pickle.dumps(oper.proxyobj)) + f.close() # explicit close + return result + +XXX # finish From fijal at codespeak.net Sat Jul 7 12:03:25 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 7 Jul 2007 12:03:25 +0200 (CEST) Subject: [pypy-svn] r44822 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070707100325.C393C8197@code0.codespeak.net> Author: fijal Date: Sat Jul 7 12:03:25 2007 New Revision: 44822 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: Few slides shuffles, few addons Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Sat Jul 7 12:03:25 2007 @@ -10,10 +10,12 @@ -------------- * New Interpreter features through implementation + * We don't aim at changing python syntax nor semantics -* avoiding religious war... + +* Keeping implementation/language design separate + * they give new dimensions to how one can use python -* pick the feature set of your choice... PyPy Overview -------------- @@ -27,6 +29,9 @@ Backends / Runtime integration ----------------------------------- +XXX fijal: we're talking about pypy interpreter +XXX features, those slides belong to rpython + - single source program translates to: - Main target: C/Posix @@ -47,17 +52,6 @@ - more on tomorrow's talk -Multiple object implementations -------------------------------- - -* same types - different implementations - -* example - ropes - -* example - multidicts - -XXX multidict demo - Python Interpreter Prototypes ------------------------------------- @@ -70,6 +64,16 @@ - Transparent Distribution - Orthogonal Persistence +Multiple object implementations +------------------------------- + +* same types - different implementations + +* example - ropes + +* example - multidicts + +XXX multidict demo Transparent proxy ----------------- From arigo at codespeak.net Sat Jul 7 18:35:29 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 7 Jul 2007 18:35:29 +0200 (CEST) Subject: [pypy-svn] r44826 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070707163529.389A481C7@code0.codespeak.net> Author: arigo Date: Sat Jul 7 18:35:28 2007 New Revision: 44826 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/compat-matrix.png pypy/extradoc/talk/ep2007/pypy_10_and_jit/overview2.png pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: Resizes and rewordings to fit into the pages of my web browser. Added examples with red and green variables. Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/compat-matrix.png ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/overview2.png ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Sat Jul 7 18:35:28 2007 @@ -17,18 +17,17 @@ * A tool-chain with a *new paradigm* for building interpreters -From Dec 2004 to March 2006 PyPy was a research project, -funded by the EU and a consortium of companies. +* Dec 2004 - March 2006: a research project funded by the EU + and companies. Motivation ============= -Conventional approaches to interpreters face hard trade-offs: -speed, maintainability, flexibility. +Conventional interpreters face hard trade-offs: +*speed - maintainability - flexibility* A lot of decisions are hard-coded pervasively, improvements -and evolution are hampered (e.g. psyco, stackless require quite some efforts -and cannot bring their full potential). +and evolution are hampered (e.g. psyco, stackless) Lots of duplication of effort to cover many platforms (C, Java, .NET). @@ -53,13 +52,16 @@ - a full compliant Python interpreter - a translation tool-chain that produces reasonably efficient interpreters: - PyPy-C, PyPy-LLVM, PyPy.NET + + *PyPy-C* + + *PyPy-LLVM* + + *PyPy.NET* PyPy 1.0 ================ -Released: 27th March 2007 - What's new: - *JIT compiler generator working on the Python interpreter* @@ -67,8 +69,13 @@ - CLI backend produces a PyPy.NET which can interact with CLR libraries - taint object space and transparent proxies -*Not completely mature yet but our original vision is fully -materialized* +PyPy 1.0 +================ + +Status: + + *Not completely mature yet but our original vision is fully + materialized* Compatibility Matrix ========================= @@ -81,13 +88,18 @@ * still rough edges that need polishing * still missing a few important extension modules -* right now, useful parts: special language features; RPython for high-perf + +Right now, some parts are directly useful: + +- *special language features* + +- *RPython for high-perf* Performance ============= -Current interpreter performance (2007-06-14) is not bad, but lower than -CPython: +Current interpreter performance (2007-06-14) +not bad, but lower than CPython: +----------+-------------------+---------------+ | | CPython 2.4.4 | pypy-llvm | @@ -103,8 +115,8 @@ | mako | 1.00 | 3.11 | +----------+-------------------+---------------+ -JIT compiler generator -====================== +How to improve +========================== * a lot of time is spent in the Garbage Collector, so we have room for improvement there @@ -121,7 +133,7 @@ * for high performance dynamic compilation is required -Traditional dynamic compilers +Traditional JIT compilers =============================== * huge resource investment @@ -139,10 +151,10 @@
.. image:: overview2.png - :scale: 55 + :align: center -PyPy Approach (2) +Basics ======================= * use partial evaluation techniques to generate a dynamic compiler @@ -156,11 +168,20 @@ * *Partial evalution of computation process - an approach to a compiler-compiler*, 1971 -* Generating compilers from interpreters, - automatic specialization +* Generating compilers from interpreters + with automatic specialization * Relatively little practical impact so far +General idea +================ + +Partial evaluation (PE): + +* Assume the Python bytecode to be constant, and + constant-propagate it into the Python + interpreter. + Challenges ====================== @@ -188,13 +209,12 @@ The pieces to enable effective dynamic compiler generation in PyPy: -- implementer-supplied hint based approach to fixing - what should be assumed compile-time constant +- a few hints in the Python interpreter to guide the JIT + generator - *promotion* -- direct support for lazy - allocation only on escape +- lazy allocation of objects - only when they escape ("virtuals") -- use stack and registers for language-level frame contents +- use CPU stack and registers for the contents of the Python frame .. ("virtualizables") @@ -215,18 +235,22 @@ its application to PyPy's Python intepreter. * included are backends for IA32 and PPC -* integer aritmethic operations are optimized -* for things for which hints were provided - we are in the speed range of gcc -O0 +* integer arithmetic operations are optimized +* for these, we are in the speed range of ``gcc -O0`` * demo (63x faster than CPython) .. demo f1 -The transformation: Timeshifting +The transformation ================================== -The generation process is implemented as a *binding-time ("color")* -directed transformation of the low-level graphs. +* The generation process is implemented as a + transformation of the low-level control flow graphs + of the interpreter + +* guided by a binding time analysis ("color" of the graphs) + +*"timeshifting"* Coloring ================= @@ -234,10 +258,11 @@ * green: compile-time value * red: runtime value +* the hints give constraints from which the colors + of all values are derived + we reuse the type inference framework to propagate colors -respecting the constraints from -the hints Timeshifting Basics ==================== @@ -246,7 +271,12 @@ * red operations: converted into corresponding code emitting code -.. demo? ++-----------------------------------------------+-----------------------------------------+----------------------------------------------+ +| | ``def f(`` :green:`x`, :red:`y` ``):`` | | *(case x=3)* | | *(case x=10)* | +| | :green:`x2` = :green:`x` ``*`` :green:`x` | | ``def f_3(y):`` | | ``def f_10(y):`` | +| | :red:`y2` = :red:`y` ``*`` :red:`y` | | ``y2 = y * y`` | | ``y2 = y * y`` | +| | ``return`` :green:`x2` ``+`` :red:`y2` | | ``return 9 + y2`` | | ``return 100 + y2`` | ++-----------------------------------------------+-----------------------------------------+----------------------------------------------+ Timeshifting Control Flow =========================== @@ -254,7 +284,12 @@ - red split points: schedule multiple compilation states - merge points: merge logic to reuse code for equivalent states -.. demo? ++-----------------------------+----------------------------+ +| | ``if`` :red:`x`: | | :green:`(case y != 0)` | +| | ``print "x is true"`` | | ``if x:`` | +| | ``if`` :green:`y`: | | ``print "x is true"`` | +| | ``print "y is true"`` | | ``print "y is true"`` | ++-----------------------------+----------------------------+ Promotion ================= @@ -262,21 +297,58 @@ Promotion is implemented generating a switch that grows to cover the seen runtime values -* First compilation stops at a promotion point and the switch with - only a default case is generated. The default will call back into +* First compilation stops at a promotion point and generates a switch + with only a default case. The default will call back into the compiler with runtime values. -* On callback the compiler add one more case to the switch and +* On callback the compiler adds one more case to the switch and generate more code assuming the received value. .. need to save state in a compact form: paths .. demo? +Promotion (example) +======================== + ++----------------------------------------------------------------------------+---------------------------------------------------------------+ +| | ``def f(`` :red:`x`, :red:`y` ``):`` | | +| | :green:`x1` = ``hint(``:red:`x`, ``promote=True)`` | | ``def f_(x, y):`` | +| | ``return`` :green:`x1` ``*`` :green:`x1` ``+`` :red:`y` ``*`` :red:`y` | | ``switch x:`` | +| | | ``pass`` | +| | | ``default:`` | +| | | ``compile_more(value=x)`` | +| | | | +| +---------------------------------------------------------------+ +| | | ``def f_(x, y):`` | +| | | ``switch x:`` | +| | | *case 3:* | +| | | *return 9 + y* | +| | | ``default:`` | +| | | ``compile_more(value=x)`` | +| | | | ++----------------------------------------------------------------------------+---------------------------------------------------------------+ + +Virtuals + Promotion +===================== + +* example from PyPy (simplified!): + ++----------------------------------------------------------------------------------------+ +| | ``def add_python_objects(``:red:`obj1`, :red:`obj2` ``):`` | +| | :green:`obj1cls` = ``hint(``:red:`obj1`. ``__class__, promote=True)`` | +| | :green:`obj2cls` = ``hint(``:red:`obj2`. ``__class__, promote=True)`` | +| | ``if`` :green:`obj1cls` ``is IntObject and`` :green:`obj2cls` ``is IntObject:`` | +| | :red:`x` = :red:`obj1`. ``intval`` | +| | :red:`y` = :red:`obj2`. ``intval`` | +| | :red:`z` = :red:`x` ``+`` :red:`y` | +| | ``return IntObject(intval=``:red:`z` ``)`` | ++----------------------------------------------------------------------------------------+ + Virtuals + Promotion ===================== -* example +* example from the Toy Language interpreter .. tlc example results @@ -294,16 +366,17 @@ ========== We have achieved a malleable interpreter and interpreter-making -process which can only further improve over time, proving our vision +process which can only further improve over time - proving our vision sound. -This is a solid base looking forward: +.. This is a solid base looking forward: -- the rough edges are what we need to tackle next +- rough edges are what we'll tackle next -- development is going to continue in open source fashion +- development goes on open source fashion at + http://codespeak.net/pypy/ -- there will be further sprints: one is organized after the conference! +- next PyPy sprint after the conference! .. the end From santagada at codespeak.net Sat Jul 7 21:31:42 2007 From: santagada at codespeak.net (santagada at codespeak.net) Date: Sat, 7 Jul 2007 21:31:42 +0200 (CEST) Subject: [pypy-svn] r44833 - in pypy/dist/pypy/lang/js: . test Message-ID: <20070707193142.0623681D4@code0.codespeak.net> Author: santagada Date: Sat Jul 7 21:31:40 2007 New Revision: 44833 Modified: pypy/dist/pypy/lang/js/astbuilder.py pypy/dist/pypy/lang/js/js_interactive.py pypy/dist/pypy/lang/js/test/test_astbuilder.py pypy/dist/pypy/lang/js/test/test_interactive.py Log: after a long hiatus, fixed a problem discovered by arigo with multiple calls happening one after the other and a new js_interactive contributed by Nilton Volpato Modified: pypy/dist/pypy/lang/js/astbuilder.py ============================================================================== --- pypy/dist/pypy/lang/js/astbuilder.py (original) +++ pypy/dist/pypy/lang/js/astbuilder.py Sat Jul 7 21:31:40 2007 @@ -260,8 +260,11 @@ def visit_callexpression(self, node): pos = self.get_pos(node) left = self.dispatch(node.children[0]) - right = self.dispatch(node.children[1]) - return operations.Call(pos, left, right) + for rightnode in node.children[1:]: + right = self.dispatch(rightnode) + left = operations.Call(pos, left, right) + + return left def visit_assignmentexpression(self, node): pos = self.get_pos(node) Modified: pypy/dist/pypy/lang/js/js_interactive.py ============================================================================== --- pypy/dist/pypy/lang/js/js_interactive.py (original) +++ pypy/dist/pypy/lang/js/js_interactive.py Sat Jul 7 21:31:40 2007 @@ -7,11 +7,10 @@ import autopath import sys import getopt -from pypy.lang.js.interpreter import * +from pypy.lang.js.interpreter import load_source, Interpreter, load_file +from pypy.lang.js.jsparser import parse, ParseError from pypy.lang.js.jsobj import W_Builtin, W_String, ThrowException, w_Undefined -from pypy.lang.js import jsparser -import os -import cmd +from pypy.rlib.streamio import open_file_as_stream help_message = """ PyPy's JavaScript Interpreter: @@ -20,10 +19,12 @@ -h show this help message -d jump to a pdb in case of failure """ +import code +sys.ps1 = 'js> ' +sys.ps2 = '... ' -interactive = True - -def setup_readline(): +try: + # Setup Readline import readline import os histfile = os.path.join(os.environ["HOME"], ".jspypyhist") @@ -34,14 +35,13 @@ pass import atexit atexit.register(readline.write_history_file, histfile) +except ImportError: + pass -class Usage(Exception): - def __init__(self, msg): - self.msg = msg def loadjs(ctx, args, this): - filename = args[0] - t = load_file(filename.ToString()) + filename = args[0].ToString() + t = load_file(filename) return t.execute(ctx) def tracejs(ctx, args, this): @@ -52,124 +52,99 @@ def quitjs(ctx, args, this): sys.exit(0) - -def main(argv=None): - # XXX: note. This will not work when translated, because - # globals cannot be modified (ie. interactive is always True). - # so I'm adding support which will not be translated, probably - # for further consideration - global interactive - debug = False - if argv is None: - argv = sys.argv - try: +class JSInterpreter(code.InteractiveConsole): + def __init__(self, locals=None, filename=""): + code.InteractiveConsole.__init__(self, locals, filename) + self.interpreter = Interpreter() + self.interpreter.w_Global.Put('quit', W_Builtin(quitjs)) + self.interpreter.w_Global.Put('load', W_Builtin(loadjs)) + self.interpreter.w_Global.Put('trace', W_Builtin(tracejs)) + + + def runcodefromfile(self, filename): + f = open_file_as_stream(filename) + self.runsource(f.readall()) + f.close() + + def runcode(self, ast): + """Run the javascript code in the AST. All exceptions raised + by javascript code must be caught and handled here. When an + exception occurs, self.showtraceback() is called to display a + traceback. + """ try: - opts, args = getopt.getopt(argv[1:], "hdnf:", ["help",]) - except getopt.error, msg: - raise Usage(msg) - - # option processing - filenames = [] - for option, value in opts: - if option == "-f": - filenames.append(value) - if option == "-n": - interactive = False - if option in ("-h", "--help"): - raise Usage(help_message) - if option == '-d': - debug = True - - except Usage, err: - print >> sys.stderr, sys.argv[0].split("/")[-1] + ": " + str(err.msg) - print >> sys.stderr, "\t for help use --help" - return 2 - - interp = Interpreter() - - interp.w_Global.Put('quit', W_Builtin(quitjs)) - interp.w_Global.Put('load', W_Builtin(loadjs)) - interp.w_Global.Put('trace', W_Builtin(tracejs)) - for filename in filenames: + res = self.interpreter.run(ast) + if res not in (None, w_Undefined): + try: + print res.GetValue().ToString(self.interpreter.w_Global) + except ThrowException, exc: + print exc.exception.ToString(self.interpreter.w_Global) + except SystemExit: + raise + except ThrowException, exc: + self.showtraceback(exc) + else: + if code.softspace(sys.stdout, 0): + print + + def runsource(self, source, filename=""): + """Parse and run source in the interpreter. + + One of these cases can happen: + 1) The input is incorrect. Prints a nice syntax error message. + 2) The input in incomplete. More input is required. Returns None. + 3) The input is complete. Executes the source code. + """ try: - loadjs(interp.global_context, [W_String(filename)], None) - # XXX we should catch more stuff here, like not implemented - # and such - except (jsparser.ParseError, ThrowException), e: - if isinstance(e, jsparser.ParseError): - print "\nSyntax error!" - raise - elif isinstance(e, ThrowException): - print "\nJS Exception thrown!" - return - - #while interactive: - # res = interp.run(load_source(raw_input("js-pypy> "))) - # if res is not None: - # print res - if interactive: - MyCmd(interp, debug).cmdloop() - -class MyCmd(cmd.Cmd): - prompt = "js-pypy> " - def __init__(self, interp, debug): - cmd.Cmd.__init__(self) - setup_readline() - self.debug = debug - self.interp = interp - self.reset() - - def reset(self): - self.prompt = self.__class__.prompt - self.lines = [] - self.level = 0 - - def emptyline(self): - pass - - def default(self, line): - # let's count lines and continue till matching proper nr of { - # XXX: '{' will count as well - # we can avoid this by using our own's tokeniser, when we possess one - if line == 'EOF': - print "\nQuitting" - sys.exit() - opens = line.count('{') - closes = line.count('}') - self.level += opens - closes - self.lines.append(line) - if self.level > 0: - self.prompt = ' ... ' - return - elif self.level < 0: - print "\nError!!! Too many closing braces" - self.level = 0 - return - try: - try: - res = self.interp.run(load_source("\n".join(self.lines))) - # XXX we should catch more stuff here, like not implemented - # and such - except (jsparser.ParseError, ThrowException), e: - e_info = sys.exc_info() - if self.debug: - import pdb - pdb.post_mortem(e_info[2]) - else: - if isinstance(e, jsparser.ParseError): - print "\nSyntax error!" - elif isinstance(e, ThrowException): - print e.exception.ToString() - return - finally: - self.reset() - if (res is not None) and (res is not w_Undefined): - try: - print res.GetValue().ToString(self.interp.w_Global) - except ThrowException, e: - print e.exception.ToString(self.interp.w_Global) - -if __name__ == "__main__": - import py - py.test.config.parse([]) - sys.exit(main()) + ast = load_source(source) + except ParseError, exc: + if exc.source_pos.i == len(source): + # Case 2 + return True # True means that more input is needed + else: + # Case 1 + self.showsyntaxerror(filename, exc) + return False + + # Case 3 + self.runcode(ast) + return False + + def showtraceback(self, exc): + # XXX format exceptions nicier + print exc.exception.ToString() + + def showsyntaxerror(self, filename, exc): + # XXX format syntax errors nicier + print ' '*4 + \ + ' '*exc.source_pos.columnno + \ + '^' + print 'Syntax Error' + + def interact(self, banner=None): + if banner is None: + banner = 'PyPy JavaScript Interpreter' + code.InteractiveConsole.interact(self, banner) + +def main(inspect=False, filename=None, args=[]): + jsi = JSInterpreter() + if filename is not None: + jsi.runcodefromfile(filename) + if (filename is None) or inspect: + jsi.interact() + +if __name__ == '__main__': + from optparse import OptionParser + parser = OptionParser(usage='%prog [options] [file] [arg] ...', + description='PyPy JavaScript Interpreter') + parser.add_option('-i', dest='inspect', + action='store_true', default=False, + help='inspect interactively after running script') + # ... (add other options) + opts, args = parser.parse_args() + + if args: + main(inspect=opts.inspect, filename=args[0], args=args[1:]) + else: + main(inspect=opts.inspect) + sys.exit(0) Modified: pypy/dist/pypy/lang/js/test/test_astbuilder.py ============================================================================== --- pypy/dist/pypy/lang/js/test/test_astbuilder.py (original) +++ pypy/dist/pypy/lang/js/test/test_astbuilder.py Sat Jul 7 21:31:40 2007 @@ -1,12 +1,13 @@ from pypy.lang.js.jsparser import parse from pypy.lang.js.astbuilder import ASTBuilder from pypy.lang.js import operations +from pypy.lang.js.operations import Call def to_ast(s): print s tp = parse(s) print tp - ASTBuilder().dispatch(tp) + return ASTBuilder().dispatch(tp) def test_simple(): yield to_ast, "1;" @@ -25,3 +26,8 @@ def test_funcvarfinder(): pos = operations.Position() +def test_callcall(): + p = to_ast('x()()') + c1 = p.body.nodes[0] + assert isinstance(c1, Call) + assert isinstance(c1.left, Call) Modified: pypy/dist/pypy/lang/js/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/js/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/js/test/test_interactive.py Sat Jul 7 21:31:40 2007 @@ -1,6 +1,7 @@ import py import sys +py.test.skip("problems to run those on pypy test server") class TestInteraction: """ These tests require pexpect (UNIX-only). From fijal at codespeak.net Sun Jul 8 10:04:35 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 10:04:35 +0200 (CEST) Subject: [pypy-svn] r44839 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070708080435.AA19081E2@code0.codespeak.net> Author: fijal Date: Sun Jul 8 10:04:35 2007 New Revision: 44839 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: Update slides, add one, shorten titles a bit to fit in my browser Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Sun Jul 8 10:04:35 2007 @@ -26,23 +26,22 @@ .. image:: dynlang.png -Backends / Runtime integration +Backends integration ----------------------------------- -XXX fijal: we're talking about pypy interpreter -XXX features, those slides belong to rpython - -- single source program translates to: +- single source interpreter translates to: - Main target: C/Posix - Main new target: .NET/CLI/CLR - Work in progress: JVM - - Special target: Javascript for Web applications +XXX external libraries bindings are not single-source CLR/CLI Backend ---------------- +XXX fijal: kill this one? + - full python interpreter generated for .NET - support for CLR integration: @@ -52,7 +51,7 @@ - more on tomorrow's talk -Python Interpreter Prototypes +Interpreter Prototypes ------------------------------------- - features are independent of backend @@ -73,7 +72,16 @@ * example - multidicts -XXX multidict demo +* rope demo + +Object space +------------ + +* Class that implements all objects operations + +* Single place to change object behaviour + +* Allows easy interpreter prototype implementation Transparent proxy ----------------- @@ -93,8 +101,11 @@ ------------------------ - transparent lazy access to remote objects + - internally uses RPC-like protocol + - remote objects are presented through transparent proxies + - access to remote tracebacks, frames, etc. work as if local! Orthogonal Persistence @@ -106,13 +117,9 @@ * implement your own custom persistence scheme (e.g. using ZODB) -XXX some demo - Threading in PyPy ----------------------- -XXX will be in the stackless talk, shall we mention it? - * principal choice of os-threading models (GIL, no, ...) * stackless *transform* / micro-threads: @@ -171,7 +178,7 @@ | mako | 1.00 | 3.11 | +----------+-------------------+---------------+ -Python Interpreter Status (1.0) +Python Interpreter Status --------------------------------- - compliant, 340 KLOC / 85 test KLOC From fijal at codespeak.net Sun Jul 8 10:04:48 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 10:04:48 +0200 (CEST) Subject: [pypy-svn] r44840 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070708080448.E921581F9@code0.codespeak.net> Author: fijal Date: Sun Jul 8 10:04:48 2007 New Revision: 44840 Modified: pypy/extradoc/talk/ep2007/interpreter/local.py Log: Make it a bit more convinient Modified: pypy/extradoc/talk/ep2007/interpreter/local.py ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/local.py (original) +++ pypy/extradoc/talk/ep2007/interpreter/local.py Sun Jul 8 10:04:48 2007 @@ -14,5 +14,6 @@ if __name__ == '__main__': send, receive = socket_connecter(('localhost', PORT)) r = RemoteProtocol(send, receive) - - + x = r.get_remote('x') + F = r.get_remote('f') + From fijal at codespeak.net Sun Jul 8 10:44:24 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 10:44:24 +0200 (CEST) Subject: [pypy-svn] r44841 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070708084424.559118203@code0.codespeak.net> Author: fijal Date: Sun Jul 8 10:44:22 2007 New Revision: 44841 Added: pypy/extradoc/talk/ep2007/interpreter/interpreterarch.png - copied unchanged from r44818, pypy/extradoc/talk/dzug2007/interpreterarch.png Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: Add interprerarch.png, fix typo Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Sun Jul 8 10:44:22 2007 @@ -183,7 +183,7 @@ - compliant, 340 KLOC / 85 test KLOC - single source for all platforms -- flexibel, fast, well-tested (11805 tests) +- flexible, fast, well-tested (11805 tests) - new middleware features - need more extension modules! - better GCs and more JITting will even improve speed! From fijal at codespeak.net Sun Jul 8 11:03:13 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 11:03:13 +0200 (CEST) Subject: [pypy-svn] r44842 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070708090313.EA4308216@code0.codespeak.net> Author: fijal Date: Sun Jul 8 11:03:13 2007 New Revision: 44842 Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt Log: Shuffle slides a bit Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython/rpython.txt Sun Jul 8 11:03:13 2007 @@ -10,19 +10,6 @@ :Place: Europython 2007 :Date: 10 July 2007 -Why invent new language? ------------------------- - -XXX antocuni: I think this slide makes more sense after the next - -* Grown as a side effect of the PyPy project - -* Python is too slow for some purposes - -* Pyrex/C are ugly - -* We want to play as it were Python as much as possible - What is RPython? ---------------- @@ -35,6 +22,16 @@ * Python is the meta-programming language for RPython +Why invent new language? +------------------------ + +* Grown as a side effect of the PyPy project + +* Python is too slow for some purposes + +* Pyrex/C are ugly + +* We want to play as it were Python as much as possible Restricted, what do you mean? ----------------------------- @@ -182,8 +179,11 @@ The JS backend -------------- -XXX fijal, feel free to fill in +* Can produce complex, standalone executables + +* Can bind to ie. mochikit +* Demos are online at http://play1.codespeak.net Conclusions ----------- From fijal at codespeak.net Sun Jul 8 11:04:12 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 11:04:12 +0200 (CEST) Subject: [pypy-svn] r44843 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070708090412.C91D78215@code0.codespeak.net> Author: fijal Date: Sun Jul 8 11:04:12 2007 New Revision: 44843 Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt Log: Add a missing slide Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython/rpython.txt Sun Jul 8 11:04:12 2007 @@ -132,8 +132,11 @@ Writing CPython extensions --------------------------- -XXX disclaimer -XXX using extcompiler +* create a mixed module (there is _demo) + +* compile it as CPython extension + +* quite incomplete The CLI backend --------------- From fijal at codespeak.net Sun Jul 8 16:26:11 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 16:26:11 +0200 (CEST) Subject: [pypy-svn] r44847 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070708142611.E5BF2826F@code0.codespeak.net> Author: fijal Date: Sun Jul 8 16:26:10 2007 New Revision: 44847 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: Move threading slide later on Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Sun Jul 8 16:26:10 2007 @@ -35,13 +35,9 @@ - Main new target: .NET/CLI/CLR - Work in progress: JVM -XXX external libraries bindings are not single-source - CLR/CLI Backend ---------------- -XXX fijal: kill this one? - - full python interpreter generated for .NET - support for CLR integration: @@ -117,17 +113,6 @@ * implement your own custom persistence scheme (e.g. using ZODB) -Threading in PyPy ------------------------ - -* principal choice of os-threading models (GIL, no, ...) -* stackless *transform* / micro-threads: - - * suspending/resuming computations - * pickling/migration of computations - * unlimited recursion - * *composable* greenlets, tasklets, co-routines - Taint Space -------------------- @@ -159,6 +144,17 @@ * easy hack (200 loc, including demo) +Threading in PyPy +----------------------- + +* principal choice of os-threading models (GIL, no, ...) +* stackless *transform* / micro-threads: + + * suspending/resuming computations + * pickling/migration of computations + * unlimited recursion + * *composable* greenlets, tasklets, co-routines + Relative speeds to CPython ------------------------------ From fijal at codespeak.net Sun Jul 8 16:29:55 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 16:29:55 +0200 (CEST) Subject: [pypy-svn] r44848 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070708142955.4B121826C@code0.codespeak.net> Author: fijal Date: Sun Jul 8 16:29:54 2007 New Revision: 44848 Modified: pypy/extradoc/talk/ep2007/interpreter/listserver.py Log: Eat KeyboardInterrupt, not to surprise listeners with unexpected traceback Modified: pypy/extradoc/talk/ep2007/interpreter/listserver.py ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/listserver.py (original) +++ pypy/extradoc/talk/ep2007/interpreter/listserver.py Sun Jul 8 16:29:54 2007 @@ -27,5 +27,9 @@ addr = ('', 8010) httpd = server.create_server(server_address=addr, handler=Handler, server=GreenHTTPServer) - httpd.serve_forever() + try: + httpd.serve_forever() + except KeyboardInterrupt: + pass + From fijal at codespeak.net Sun Jul 8 17:39:46 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 8 Jul 2007 17:39:46 +0200 (CEST) Subject: [pypy-svn] r44849 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070708153946.D46628284@code0.codespeak.net> Author: fijal Date: Sun Jul 8 17:39:45 2007 New Revision: 44849 Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt Log: Kill XXX Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython/rpython.txt Sun Jul 8 17:39:45 2007 @@ -64,8 +64,6 @@ Type inference -------------- -XXXX should we really mention that? - :: def foo(): @@ -90,11 +88,6 @@ * All modules might be used when initializing -Few more words about file API ------------------------------ - -XXX really? - RPython - why you wouldn't like to use it ----------------------------------------- From bea at codespeak.net Mon Jul 9 11:12:52 2007 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 9 Jul 2007 11:12:52 +0200 (CEST) Subject: [pypy-svn] r44865 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070709091252.EFD22820C@code0.codespeak.net> Author: bea Date: Mon Jul 9 11:12:52 2007 New Revision: 44865 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: bigger is better (capital letters) Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Mon Jul 9 11:12:52 2007 @@ -15,7 +15,7 @@ * Keeping implementation/language design separate -* they give new dimensions to how one can use python +* They give new dimensions to how one can use python PyPy Overview -------------- @@ -29,7 +29,7 @@ Backends integration ----------------------------------- -- single source interpreter translates to: +- Single source interpreter translates to: - Main target: C/Posix - Main new target: .NET/CLI/CLR @@ -38,19 +38,19 @@ CLR/CLI Backend ---------------- -- full python interpreter generated for .NET +- Full python interpreter generated for .NET -- support for CLR integration: +- Support for CLR integration: - static RPython-level bindings - on top dynamic bindings using reflection (clr module) -- more on tomorrow's talk +- More on tomorrow's talk Interpreter Prototypes ------------------------------------- -- features are independent of backend +- Features are independent of backend - Security / Taint Space @@ -62,13 +62,13 @@ Multiple object implementations ------------------------------- -* same types - different implementations +* Same types - different implementations -* example - ropes +* Example - ropes -* example - multidicts +* Example - multidicts -* rope demo +* Rope demo Object space ------------ @@ -82,47 +82,47 @@ Transparent proxy ----------------- -* proxy: intercept any operation on an builtin object +* Proxy: intercept any operation on an builtin object -* transparent: don't change object interface +* Transparent: don't change object interface -* useful for implementing application level functionality +* Useful for implementing application level functionality orthogonally to usage of objects -* is the mechanism for distribution and persistence prototypes +* Is the mechanism for distribution and persistence prototypes -* see also .NET TransparentProxy +* See also .NET TransparentProxy Lazy ad-hoc distribution ------------------------ -- transparent lazy access to remote objects +- Transparent lazy access to remote objects -- internally uses RPC-like protocol +- Internally uses RPC-like protocol -- remote objects are presented through transparent proxies +- Remote objects are presented through transparent proxies -- access to remote tracebacks, frames, etc. work as if local! +- Access to remote tracebacks, frames, etc. work as if local! Orthogonal Persistence -------------------------- -* persist Python objects "invisibly" +* Persist Python objects "invisibly" -* interpose interception of changes to objects +* Interpose interception of changes to objects -* implement your own custom persistence scheme (e.g. using ZODB) +* Implement your own custom persistence scheme (e.g. using ZODB) Taint Space -------------------- -* control of information data flow: +* Control of information data flow: - label sensitive data - avoid sensitive information leaks - explicit primitive to declassify -* easily implemented as an object space around +* Easily implemented as an object space around the standard one securing all operations on objects @@ -135,20 +135,20 @@ .. image:: interpreterarch.png -* interactive prompt demo +* Interactive prompt demo Thunk object space ------------------ -* lazy computations +* Lazy computations -* easy hack (200 loc, including demo) +* Easy hack (200 loc, including demo) Threading in PyPy ----------------------- -* principal choice of os-threading models (GIL, no, ...) -* stackless *transform* / micro-threads: +* Principal choice of os-threading models (GIL, no, ...) +* Stackless *transform* / micro-threads: * suspending/resuming computations * pickling/migration of computations @@ -177,20 +177,20 @@ Python Interpreter Status --------------------------------- -- compliant, 340 KLOC / 85 test KLOC -- single source for all platforms -- flexible, fast, well-tested (11805 tests) -- new middleware features -- need more extension modules! -- better GCs and more JITting will even improve speed! +- Compliant, 340 KLOC / 85 test KLOC +- Single source for all platforms +- Flexible, fast, well-tested (11805 tests) +- New middleware features +- Need more extension modules! +- Better GCs and more JITting will even improve speed! Disclaimer ---------- -* pypy features are not ready for production use +* PyPy features are not ready for production use -* although not because they're buggy +* Although not because they're buggy -* lack of extension modules +* Lack of extension modules -* os-level threads not working very well +* OS-level threads not working very well From bea at codespeak.net Mon Jul 9 11:18:58 2007 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 9 Jul 2007 11:18:58 +0200 (CEST) Subject: [pypy-svn] r44867 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070709091858.872768211@code0.codespeak.net> Author: bea Date: Mon Jul 9 11:18:58 2007 New Revision: 44867 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: bigger is better -again Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Mon Jul 9 11:18:58 2007 @@ -86,8 +86,8 @@ Status ========= -* still rough edges that need polishing -* still missing a few important extension modules +* Still rough edges that need polishing +* Still missing a few important extension modules Right now, some parts are directly useful: @@ -118,28 +118,28 @@ How to improve ========================== -* a lot of time is spent in the Garbage Collector, +* A lot of time is spent in the Garbage Collector, so we have room for improvement there -* but our real trump card to run plain Python programs fast +* But our real trump card to run plain Python programs fast is the JIT compiler generator -* works - but still work in progress before it's really useful! +* Works - but still work in progress before it's really useful! Flexibility vs. Performance ============================= -* interpreters are easy to write and evolve +* Interpreters are easy to write and evolve -* for high performance dynamic compilation is required +* For high performance dynamic compilation is required Traditional JIT compilers =============================== -* huge resource investment -* the richer the semantics, the harder to write -* poor encoding of language semantics -* hard to evolve +* Huge resource investment +* The richer the semantics, the harder to write +* Poor encoding of language semantics +* Hard to evolve Need for novel approaches! @@ -157,10 +157,10 @@ Basics ======================= -* use partial evaluation techniques to generate a dynamic compiler +* Use partial evaluation techniques to generate a dynamic compiler from an interpreter -* inspiration: Psyco -* our translation tool-chain was designed for trying this +* Inspiration: Psyco +* Our translation tool-chain was designed for trying this Futamura ===================== @@ -196,11 +196,11 @@ Solution: Promotion ==================== -* enhance PE with the ability +* Enhance PE with the ability to "promote" run-time values to compile-time -* leverage the dynamic setting +* Leverage the dynamic setting Overall ingredients @@ -248,28 +248,28 @@ transformation of the low-level control flow graphs of the interpreter -* guided by a binding time analysis ("color" of the graphs) +* Guided by a binding time analysis ("color" of the graphs) *"timeshifting"* Coloring ================= -* green: compile-time value -* red: runtime value +* Green: compile-time value +* Red: runtime value -* the hints give constraints from which the colors +* The hints give constraints from which the colors of all values are derived -we reuse the type inference +We reuse the type inference framework to propagate colors Timeshifting Basics ==================== -* green operations: unchanged, executed at compile-time +* Green operations: unchanged, executed at compile-time -* red operations: converted into corresponding code emitting code +* Red operations: converted into corresponding code emitting code +-----------------------------------------------+-----------------------------------------+----------------------------------------------+ | | ``def f(`` :green:`x`, :red:`y` ``):`` | | *(case x=3)* | | *(case x=10)* | @@ -332,7 +332,7 @@ Virtuals + Promotion ===================== -* example from PyPy (simplified!): +* Example from PyPy (simplified!): +----------------------------------------------------------------------------------------+ | | ``def add_python_objects(``:red:`obj1`, :red:`obj2` ``):`` | @@ -348,7 +348,7 @@ Virtuals + Promotion ===================== -* example from the Toy Language interpreter +* Example from the Toy Language interpreter .. tlc example results From hpk at codespeak.net Mon Jul 9 12:21:51 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 12:21:51 +0200 (CEST) Subject: [pypy-svn] r44868 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070709102151.0F666820E@code0.codespeak.net> Author: hpk Date: Mon Jul 9 12:21:50 2007 New Revision: 44868 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: marks for talk distribtion Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Mon Jul 9 12:21:50 2007 @@ -59,6 +59,8 @@ - Transparent Distribution - Orthogonal Persistence +.. now Maciej + Multiple object implementations ------------------------------- @@ -144,6 +146,8 @@ * Easy hack (200 loc, including demo) +.. holger + Threading in PyPy ----------------------- From hpk at codespeak.net Mon Jul 9 12:22:27 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 12:22:27 +0200 (CEST) Subject: [pypy-svn] r44869 - in pypy/extradoc/talk/ep2007: howtoeu pytest Message-ID: <20070709102227.309D78210@code0.codespeak.net> Author: hpk Date: Mon Jul 9 12:22:26 2007 New Revision: 44869 Added: pypy/extradoc/talk/ep2007/howtoeu/ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt pypy/extradoc/talk/ep2007/howtoeu/ui (contents, props changed) pypy/extradoc/talk/ep2007/pytest/ pypy/extradoc/talk/ep2007/pytest/pytest.txt pypy/extradoc/talk/ep2007/pytest/ui (contents, props changed) Log: commit py test talk and howtoeu talk here to make it more easily sharable. Added: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Mon Jul 9 12:22:26 2007 @@ -0,0 +1,195 @@ + +.. include:: + + +================================================== +How to do an open source EU research project +================================================== + + +:authors: Lene Wagner, Holger Krekel (merlinux GmbH) +:event: 10.7.2007, EuroPython 2007, Vilnius + + +Intro +=================== + +- have participated and contributed to the PyPy EU project +- experiences +- discuss *open source* research projects + (cut out IPR issues more or less completely) + +EU Framework Research Programme +==================================== + +- IST = Information Society and Technology +- PyPy was a STREP +- FP7 spends some 20.000.000.000 Euro in 7 years +- regularly publishes Work Programmes and Topics +- puts out CALLS for Proposals +- we mainly try to discuss FP7 (a few changes from FP6) + +Getting started +===================== + +- vision / idea? +- read Work programmes / CALL topics +- finding people to collaborate +- finding a set of organisations +- drafting a proposal ... + +(Research) Objectives? +============================== + +- core research objective(s) + (state of the art & how the project aims beyond) +- core technical objectives? +- core other objectives? (methodological/social) + +can you summarize it on one page? + +The Proposal +=============== + +- High level Objectives +- Impact on industry and society +- management structures +- work plan in long form +- detailed description of work packages +- budget + +Work Packages +=================== + +- each has a specific objective +- Person Months allocated from Partners +- Lead Partner +- Deliverables (Reports, Releases, Papers) + +consortium +============================ + +- looking for interested parties +- consortium consists of organisations +- is collective entity towards EU +- requires co-ordinator/contact person for EU +- receives and distributes money + + +applying to the EU +========================= + +- answer to CALLS within stated deadline +- co-ordinator registers project and proposal +- proposals gets reviewed by *independent* experts +- proposals receive points (PyPy had 26 out of 30) +- Invitation to "negotiation" +- Project duration? + +Negotiation +============== + +- discuss criticism of reviewers, change proposal +- Prepare paperwork (CPF = Contract Preparation Forms) +- sign CONTRACT with the European Union + +Funding Schemes +===================== + +XXX (lene) no more cost models, will check furthermore +- Universities receive (basically) 100% funding +- Small/Medium Companies (SME) receive 75% funding of costs +- FCF = 20% Flat administrative overhead added to costs +- FC = actual indirect costs + + +Signing the contract? +======================== + + +Infrastructure +================== +- ensure close collaboration of non co-located developers +- ensure openness for external contributors +- ensure consistency of codebase/project +- PyPy: IRC, svn, mailing lists + + +Management structures +============================ + +- administrative team (preparing consortium decisions) +- Technical Board (discussing problems, deciding roadmap) +- developer body (everybody can participate) +- Consortium + +XXX insert the graph from the EU talks that shows +the overlap - seems important to me in the open +source context to recommend close and overlapping +structures + + +Meetings and Sprints +============================ + +- sprints are the rhythm of the project +- plan for core and for newcomer-friendly sprints +- integrate a model for funding contributors! + + +Contractual and actual developments +===================================== +- Contract implies a 'plan' for the development +- Project may evolve into other directions +- checkpoints for assessing the work package status + +Planning for changes +=========================== + +- be able to shift work among partners +- if necessary: amend work plan, change the contract with the EU +- but: don't touch the objectives! + + +Reporting / Deliverables +============================ + +- interim reviews each 18 months +- contractual and technical deliverable +- contractual deliverables + +Recommendations +==================== + +- sprint-driven model of development; +- share some funding with upcoming contributors; +- balance contractual with community interests +- consider changes to the contractual work plan if + it fits the objectives better; +- implement a transparent and flat communication and decision + culture, be ready to adjust roles and not stick to formal issues too much. + +Roles and Responsbilities +============================ + +- assign responsibilities to people + + - per aspect (administration, technical, research) + - per work package + +- plan for changing responsibilities + +Changes from FP6 to FP7 +============================= + +- less collective financial responsibility? +- 75% funding for (small/medium) companies +- XXX + + +What remains? +================================ + +- build the project on a shared vision and shared interest +- don't build it on money (too much) + Added: pypy/extradoc/talk/ep2007/howtoeu/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/howtoeu/ui Mon Jul 9 12:22:26 2007 @@ -0,0 +1 @@ +link ../ui \ No newline at end of file Added: pypy/extradoc/talk/ep2007/pytest/pytest.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pytest/pytest.txt Mon Jul 9 12:22:26 2007 @@ -0,0 +1,157 @@ +.. include:: + +================================================================= +py.test: towards interactive, distributed and rapid testing +================================================================= + +:authors: Holger Krekel, Maciej Fijalkowski (merlinux GmbH) +:event: 9.7.2007, EuroPython 2007, Vilnius + + +Intro +=================== + +- py.test released 0.9 as part of py lib, feb 2007 +- cross-project external testing tool +- many mature and experimental features +- minimal boilerplate approach +- developed partly for purposes of PyPy +- this talk will assess status and discuss future + +recap features +====================== + +- assertion via "assert", nice tracebacks +- automatic customizable collection of tests +- test state management (module,class,function) +- capture stdout/stderr per-test +- select tests by keyword +- generative tests, doctests, ReST-tests, ... + +ad-hoc distribution of tests +=================================== + +``py.test --dist`` sends tests to remote places +or multiple processors + +- for each host: + + - setup (ssh) connection + - sync local source code to remote place + - trigger running of (isolated) tests + +ad-hoc distribution of tests (2) +===================================== + +- synchronise source code "1 -> N" simultanously +- uses py.execnet: + + - can connect e.g. via SSH + - expects plain Python executable on remote side + - no need to pre-install other software remotely + +- generated AJAX application async displays test results + +- demo + +cross-platform testing +============================= + +- test run on linux, test execution on windows +- uses py.execnet (same technique as "--dist") +- works well with "--looponfailing" + +status cross-platform/distributed testing +============================================== + +- should basically work for any project +- gives full tracebacks, most options work + +todo: + +- make introspection/pdb work +- use ``screen`` for access to failed tests +- accellerate py.execnet setup +- more unification with other testing modes + +collect info about functions (apigen) +============================================= + +- collect function signatures (via settrace) +- track input values/types, stacktraces, return values +- web page with rich automatically produced information +- TODO: decoupling of collecting info and generating html + +doctests +============== + +- basically work (automatically collected from text files) +- uses 2.5's doctest module +- needs more fine-grained integration + +Unifying Test Reporting +================================= + +- py.test contains two approaches for processing test results: + + - "old style": reporting methods invoked on session object + - "new style": generate reporting events -> Reporter object(s) + +next: + + - settle on reporting event architecture + - refactor and document reporting events + - use Reporter object(s) + +py.test extensions (conftest) +================================= + +- all conftest.py files are considered in traversed directories +- can extend/modify testing process in arbitrary ways +- however: + + - often requires too much knowledge of internal architecture + - no convention for organising shared test support code + +- ergo: introduce mechanism to share test support code / plugins + + +Platform support +====================== + +- py lib works on linux, freebsd, windows and OSX +- works on python 2.3, 2.4, 2.5 +- py lib provides doctest/optparse/... unified compat modules +- todo: improve and automate packaging/installation + +test networks +================== + +- currently model for 1:1 process-to-process +- "A <-> B and B <-> C" connections do not imply "A <-> C" +- fix that! +- then: manage (large) network of test hosts +- select set of test deployment hosts by platform, load +- have modes for testing on several platforms at once + +difference to "buildbot" ... +================================== + +- py.test generally works from developers WC +- no need to first commit and wait for later (nightly) test run +- interactive developer driven choice of test setting + +summary development topics +============================= + +- refactor/unify reporting mechanisms +- introduce shared test support code (plugins) +- improve py.execnet (towards networks) +- interactive debugging / introspection everywhere +- systematically persist test results + +Goal +============= + +Maximize fun and efficiency of (test-driven) development + Added: pypy/extradoc/talk/ep2007/pytest/ui ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pytest/ui Mon Jul 9 12:22:26 2007 @@ -0,0 +1 @@ +link ../ui \ No newline at end of file From hpk at codespeak.net Mon Jul 9 12:25:22 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 12:25:22 +0200 (CEST) Subject: [pypy-svn] r44870 - pypy/extradoc/talk/ep2007/pytest Message-ID: <20070709102522.76400820E@code0.codespeak.net> Author: hpk Date: Mon Jul 9 12:25:21 2007 New Revision: 44870 Modified: pypy/extradoc/talk/ep2007/pytest/pytest.txt Log: small things Modified: pypy/extradoc/talk/ep2007/pytest/pytest.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pytest/pytest.txt (original) +++ pypy/extradoc/talk/ep2007/pytest/pytest.txt Mon Jul 9 12:25:21 2007 @@ -21,12 +21,7 @@ recap features ====================== -- assertion via "assert", nice tracebacks -- automatic customizable collection of tests -- test state management (module,class,function) -- capture stdout/stderr per-test -- select tests by keyword -- generative tests, doctests, ReST-tests, ... +XXX from website ad-hoc distribution of tests =================================== @@ -72,7 +67,7 @@ - make introspection/pdb work - use ``screen`` for access to failed tests - accellerate py.execnet setup -- more unification with other testing modes +- more unification among testing modes collect info about functions (apigen) ============================================= @@ -89,19 +84,23 @@ - uses 2.5's doctest module - needs more fine-grained integration +html Test Reporting +========================= + +XXX htmlconftest (daily runs) + Unifying Test Reporting ================================= - py.test contains two approaches for processing test results: - - "old style": reporting methods invoked on session object - - "new style": generate reporting events -> Reporter object(s) + - "old style": methods invoked on session object + - "new style": reporting events are sent to Reporter object next: - - settle on reporting event architecture - - refactor and document reporting events - - use Reporter object(s) + - use reporting event architecture pervasively + - refactor existing reporting extensions py.test extensions (conftest) ================================= @@ -119,7 +118,7 @@ Platform support ====================== -- py lib works on linux, freebsd, windows and OSX +- py lib works on linux, freebsd, windows and OSX, ... - works on python 2.3, 2.4, 2.5 - py lib provides doctest/optparse/... unified compat modules - todo: improve and automate packaging/installation @@ -139,7 +138,8 @@ - py.test generally works from developers WC - no need to first commit and wait for later (nightly) test run -- interactive developer driven choice of test setting +- developer's choice of test setting +- more interactivity summary development topics ============================= From fijal at codespeak.net Mon Jul 9 12:29:25 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 9 Jul 2007 12:29:25 +0200 (CEST) Subject: [pypy-svn] r44871 - pypy/extradoc/talk/ep2007/rpython Message-ID: <20070709102925.F2BEF8111@code0.codespeak.net> Author: fijal Date: Mon Jul 9 12:29:25 2007 New Revision: 44871 Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt Log: Update js slide Modified: pypy/extradoc/talk/ep2007/rpython/rpython.txt ============================================================================== --- pypy/extradoc/talk/ep2007/rpython/rpython.txt (original) +++ pypy/extradoc/talk/ep2007/rpython/rpython.txt Mon Jul 9 12:29:25 2007 @@ -177,10 +177,14 @@ * Can produce complex, standalone executables +* AJAX by method calls + * Can bind to ie. mochikit * Demos are online at http://play1.codespeak.net +* Can be used with any possible web framework + Conclusions ----------- From hpk at codespeak.net Mon Jul 9 12:30:54 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 12:30:54 +0200 (CEST) Subject: [pypy-svn] r44872 - pypy/extradoc/talk/ep2007/pytest Message-ID: <20070709103054.325268111@code0.codespeak.net> Author: hpk Date: Mon Jul 9 12:30:53 2007 New Revision: 44872 Modified: pypy/extradoc/talk/ep2007/pytest/pytest.txt Log: second try at resolving XXX Modified: pypy/extradoc/talk/ep2007/pytest/pytest.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pytest/pytest.txt (original) +++ pypy/extradoc/talk/ep2007/pytest/pytest.txt Mon Jul 9 12:30:53 2007 @@ -18,10 +18,14 @@ - developed partly for purposes of PyPy - this talk will assess status and discuss future -recap features +recap: main features ====================== -XXX from website +- assertions by assert statement, nice debugging +- unittests, generative tests, doctests, rest-tests +- automatic customaize collection of tests +- select tests by keyword +- capture stdout/stderr per-test ad-hoc distribution of tests =================================== @@ -84,23 +88,18 @@ - uses 2.5's doctest module - needs more fine-grained integration -html Test Reporting -========================= - -XXX htmlconftest (daily runs) - Unifying Test Reporting ================================= -- py.test contains two approaches for processing test results: +py.test contains two approaches for processing test results: - "old style": methods invoked on session object - "new style": reporting events are sent to Reporter object next: - - use reporting event architecture pervasively - - refactor existing reporting extensions + - use reporting event architecture pervasively + - refactor existing reporting extensions py.test extensions (conftest) ================================= From hpk at codespeak.net Mon Jul 9 12:38:01 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 12:38:01 +0200 (CEST) Subject: [pypy-svn] r44873 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070709103801.0762581C0@code0.codespeak.net> Author: hpk Date: Mon Jul 9 12:38:00 2007 New Revision: 44873 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: (maciej, hpk) skipping some things Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Mon Jul 9 12:38:00 2007 @@ -13,7 +13,7 @@ * We don't aim at changing python syntax nor semantics -* Keeping implementation/language design separate +* Keeping implementation and language design separate * They give new dimensions to how one can use python @@ -45,7 +45,7 @@ - static RPython-level bindings - on top dynamic bindings using reflection (clr module) -- More on tomorrow's talk +- More on tomorrow's talk (Tuesday, 9am) Interpreter Prototypes ------------------------------------- @@ -139,12 +139,6 @@ * Interactive prompt demo -Thunk object space ------------------- - -* Lazy computations - -* Easy hack (200 loc, including demo) .. holger From arigo at codespeak.net Mon Jul 9 12:38:33 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 9 Jul 2007 12:38:33 +0200 (CEST) Subject: [pypy-svn] r44874 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070709103833.639578166@code0.codespeak.net> Author: arigo Date: Mon Jul 9 12:38:32 2007 New Revision: 44874 Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/tlc1.dot pypy/extradoc/talk/ep2007/pypy_10_and_jit/tlc2.dot Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: - add the source of the factorial-in-TLC - check in graphs for the TLC results Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Mon Jul 9 12:38:32 2007 @@ -348,7 +348,22 @@ Virtuals + Promotion ===================== -* Example from the Toy Language interpreter +| *The factorial for the Toy Language interpreter:* +| ``PUSH 1 # accumulator`` +| ``PUSHARG`` +| ``start:`` +| ``PICK 0`` +| ``PUSH 1`` +| ``LE`` +| ``BR_COND exit`` +| ``SWAP`` +| ``PICK 1`` +| ``MUL`` +| ``SWAP`` +| ``PUSH 1`` +| ``SUB`` +| ``PUSH 1`` +| ``BR_COND start`` .. tlc example results Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/tlc1.dot ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/tlc1.dot Mon Jul 9 12:38:32 2007 @@ -0,0 +1,19 @@ +digraph _generated__graph { +subgraph _generated__ { +_generated__ [shape="box", label="generated", color="black", fillcolor="#a5e6f0", style="filled", width="0.75"]; +edge [label="startblock", style="dashed", color="black", dir="forward", weight="5"]; +_generated__ -> _generated____1 +_generated____1 [shape="box", label="generated__1\ninputargs: v6024\n\nv6025 = cast_pointer(v6024)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v6025", style="solid", color="black", dir="forward", weight="5"]; +_generated____1 -> _generated____2 +_generated____2 [shape="box", label="generated__2\ninputargs: v6026\n\nv6027 = getfield(v6026, ('typeptr'))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v6026 v6027", style="solid", color="black", dir="forward", weight="5"]; +_generated____2 -> _generated____3 +_generated____3 [shape="box", label="generated__3\ninputargs: v6030 v6028\n\nv6029 = cast_ptr_to_int(v6028)\l\lexitswitch: v6029", color="black", fillcolor="#e5ffff", style="filled", width="0.75"]; +edge [label="'default': v6028 v6030", style="dotted", color="red", dir="forward", weight="5"]; +_generated____3 -> _generated____4 +_generated____4 [shape="box", label="generated__4\ninputargs: v6035 v6039\n\nv6031 = getfield((<* struct ExcData { ex...=... }>), ('exc_type'))\lv6032 = getfield((<* struct ExcData { ex...=... }>), ('exc_value'))\lv6033 = setfield((<* struct ExcData { ex...=... }>), ('exc_type'), (<* None>))\lv6034 = setfield((<* struct ExcData { ex...=... }>), ('exc_value'), (<* None>))\lv6036 = direct_call((<* fn ll_continue_comp...blePtr>), (<* struct object { typ...=... }>), v6035)\lv6037 = setfield((<* struct ExcData { ex...=... }>), ('exc_type'), v6031)\lv6038 = setfield((<* struct ExcData { ex...=... }>), ('exc_value'), v6032)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v6039 v6035", style="solid", color="black", dir="forward", weight="5"]; +_generated____4 -> _generated____3 +} +} \ No newline at end of file Added: pypy/extradoc/talk/ep2007/pypy_10_and_jit/tlc2.dot ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/tlc2.dot Mon Jul 9 12:38:32 2007 @@ -0,0 +1,90 @@ +digraph _generated__graph { +subgraph _generated__ { +_generated__ [shape="box", label="generated", color="black", fillcolor="#a5e6f0", style="filled", width="0.75"]; +edge [label="startblock", style="dashed", color="black", dir="forward", weight="5"]; +_generated__ -> _generated____1 +_generated____1 [shape="box", label="generated__1\ninputargs: v6024\n\nv6025 = cast_pointer(v6024)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v6025", style="solid", color="black", dir="forward", weight="5"]; +_generated____1 -> _generated____2 +_generated____2 [shape="box", label="generated__2\ninputargs: v6026\n\nv6027 = getfield(v6026, ('typeptr'))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v6026 v6027", style="solid", color="black", dir="forward", weight="5"]; +_generated____2 -> _generated____3 +_generated____3 [shape="octagon", label="generated__3\ninputargs: v6030 v6028\n\nv6029 = cast_ptr_to_int(v6028)\l\lexitswitch: v6029", color="red", fillcolor="#e5ffff", style="filled", width="0.75"]; +edge [label="-1219596420: v6028 v6030", style="dotted", color="red", dir="forward", weight="5"]; +_generated____3 -> _generated____4 +edge [label="'default': v6028 v6030", style="dotted", color="red", dir="forward", weight="5"]; +_generated____3 -> _generated____5 +_generated____4 [shape="box", label="generated__4\ninputargs: v10226 v10223\n\nv10224 = cast_pointer(v10223)\lv10225 = getfield(v10224, ('inst_value'))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10225 v10223", style="solid", color="black", dir="forward", weight="5"]; +_generated____4 -> _generated____6 +_generated____6 [shape="box", label="generated__6\ninputargs: v10227 v10229\n\nv10228 = int_lt((1), v10227)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10228 v10229", style="solid", color="black", dir="forward", weight="5"]; +_generated____6 -> _generated____7 +_generated____7 [shape="octagon", label="generated__7\ninputargs: v10230 v10231\n\n\lexitswitch: v10230", color="red", fillcolor="white", style="filled", width="0.75"]; +edge [label="False: (1)", style="dotted", color="red", dir="forward", weight="5"]; +_generated____7 -> _generated____8 +edge [label="True: v10231", style="dotted", color="red", dir="forward", weight="5"]; +_generated____7 -> _generated____9 +_generated____8 [shape="box", label="generated__8\ninputargs: v10236\n\nv10232 = malloc((GcStruct pypy.jit.tl.tlc.IntObj))\lv10233 = cast_pointer(v10232)\lv10234 = setfield(v10233, ('typeptr'), (<* struct object_vtabl...=... }>))\lv10235 = cast_pointer(v10233)\lv10237 = setfield(v10235, ('inst_value'), v10236)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10233", style="solid", color="black", dir="forward", weight="5"]; +_generated____8 -> _generated____10 +_generated____10 [shape="box", label="generated__10\ninputargs: v10238\n\nv10239 = cast_pointer(v10238)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10239", style="solid", color="black", dir="forward", weight="5"]; +_generated____10 -> _generated____11 +_generated____11 [shape="box", label="generated__11\ninputargs: v10240\n\nreturn v10240", color="black", fillcolor="green", style="filled", width="0.75"]; +_generated____9 [shape="box", label="generated__9\ninputargs: v10242\n\nv10241 = same_as((0))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10242 v10241", style="solid", color="black", dir="forward", weight="5"]; +_generated____9 -> _generated____12 +_generated____12 [shape="box", label="generated__12\ninputargs: v10245 v10243\n\nv10244 = int_is_true(v10243)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10244 v10245", style="solid", color="black", dir="forward", weight="5"]; +_generated____12 -> _generated____13 +_generated____13 [shape="octagon", label="generated__13\ninputargs: v10246 v10247\n\n\lexitswitch: v10246", color="red", fillcolor="white", style="filled", width="0.75"]; +edge [label="False: v10247", style="dotted", color="red", dir="forward", weight="5"]; +_generated____13 -> _generated____14 +edge [label="True: v10247", style="dotted", color="red", dir="forward", weight="5"]; +_generated____13 -> _generated____15 +_generated____14 [shape="box", label="generated__14\ninputargs: v10248\n\nv10249 = cast_pointer(v10248)\lv10250 = getfield(v10249, ('inst_value'))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10250 v10248", style="solid", color="black", dir="forward", weight="5"]; +_generated____14 -> _generated____16 +_generated____16 [shape="box", label="generated__16\ninputargs: v10251 v10253\n\nv10252 = int_mul((1), v10251)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10253 v10252", style="solid", color="black", dir="forward", weight="5"]; +_generated____16 -> _generated____17 +_generated____17 [shape="box", label="generated__17\ninputargs: v10254 v10257\n\nv10255 = cast_pointer(v10254)\lv10256 = getfield(v10255, ('inst_value'))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10256 v10257 v10254", style="solid", color="black", dir="forward", weight="5"]; +_generated____17 -> _generated____18 +_generated____18 [shape="box", label="generated__18\ninputargs: v10258 v10260 v10261\n\nv10259 = int_sub(v10258, (1))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10259 v10260 v10261", style="solid", color="black", dir="forward", weight="5"]; +_generated____18 -> _generated____19 +_generated____19 [shape="box", label="generated__19\ninputargs: v10262 v10264 v10265\n\nv10263 = int_lt((1), v10262)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10263 v10264 v10262 v10265", style="solid", color="black", dir="forward", weight="5"]; +_generated____19 -> _generated____20 +_generated____20 [shape="octagon", label="generated__20\ninputargs: v10266 v10267 v10268 v10269\n\n\lexitswitch: v10266", color="red", fillcolor="white", style="filled", width="0.75"]; +edge [label="False: v10267", style="dotted", color="red", dir="forward", weight="5"]; +_generated____20 -> _generated____8 +edge [label="True: v10267 v10269 v10268", style="dotted", color="red", dir="forward", weight="5"]; +_generated____20 -> _generated____21 +_generated____21 [shape="box", label="generated__21\ninputargs: v10271 v10272 v10273\n\nv10270 = same_as((0))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10272 v10271 v10273 v10270", style="solid", color="black", dir="forward", weight="5"]; +_generated____21 -> _generated____22 +_generated____22 [shape="box", label="generated__22\ninputargs: v10276 v10277 v10278 v10274\n\nv10275 = int_is_true(v10274)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10275 v10277 v10278 v10276", style="solid", color="black", dir="forward", weight="5"]; +_generated____22 -> _generated____23 +_generated____23 [shape="octagon", label="generated__23\ninputargs: v10279 v10280 v10281 v10282\n\n\lexitswitch: v10279", color="red", fillcolor="white", style="filled", width="0.75"]; +edge [label="False: v10281 v10280 v10282", style="dotted", color="red", dir="forward", weight="5"]; +_generated____23 -> _generated____24 +edge [label="True: v10280", style="dotted", color="red", dir="forward", weight="5"]; +_generated____23 -> _generated____8 +_generated____24 [shape="box", label="generated__24\ninputargs: v10284 v10283 v10286\n\nv10285 = int_mul(v10283, v10284)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10284 v10285 v10286", style="solid", color="black", dir="forward", weight="5"]; +_generated____24 -> _generated____25 +_generated____25 [shape="box", label="generated__25\ninputargs: v10287 v10289 v10290\n\nv10288 = int_sub(v10287, (1))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10288 v10289 v10290", style="solid", color="black", dir="forward", weight="5"]; +_generated____25 -> _generated____19 +_generated____15 [shape="box", label="generated__15\ninputargs: v10292\n\nv10291 = same_as((1))\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v10291", style="solid", color="black", dir="forward", weight="5"]; +_generated____15 -> _generated____8 +_generated____5 [shape="box", label="generated__5\ninputargs: v6035 v6039\n\nv6031 = getfield((<* struct ExcData { ex...=... }>), ('exc_type'))\lv6032 = getfield((<* struct ExcData { ex...=... }>), ('exc_value'))\lv6033 = setfield((<* struct ExcData { ex...=... }>), ('exc_type'), (<* None>))\lv6034 = setfield((<* struct ExcData { ex...=... }>), ('exc_value'), (<* None>))\lv6036 = direct_call((<* fn ll_continue_comp...blePtr>), (<* struct object { typ...=... }>), v6035)\lv6037 = setfield((<* struct ExcData { ex...=... }>), ('exc_type'), v6031)\lv6038 = setfield((<* struct ExcData { ex...=... }>), ('exc_value'), v6032)\l", color="black", fillcolor="white", style="filled", width="0.75"]; +edge [label="v6039 v6035", style="solid", color="black", dir="forward", weight="5"]; +_generated____5 -> _generated____3 +} +} \ No newline at end of file From fijal at codespeak.net Mon Jul 9 12:38:49 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 9 Jul 2007 12:38:49 +0200 (CEST) Subject: [pypy-svn] r44875 - pypy/extradoc/talk/ep2007/pytest Message-ID: <20070709103849.12F4D820E@code0.codespeak.net> Author: fijal Date: Mon Jul 9 12:38:48 2007 New Revision: 44875 Modified: pypy/extradoc/talk/ep2007/pytest/pytest.txt Log: I hate rest Modified: pypy/extradoc/talk/ep2007/pytest/pytest.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pytest/pytest.txt (original) +++ pypy/extradoc/talk/ep2007/pytest/pytest.txt Mon Jul 9 12:38:48 2007 @@ -43,11 +43,12 @@ ===================================== - synchronise source code "1 -> N" simultanously + - uses py.execnet: - - can connect e.g. via SSH - - expects plain Python executable on remote side - - no need to pre-install other software remotely + - can connect e.g. via SSH + - expects plain Python executable on remote side + - no need to pre-install other software remotely - generated AJAX application async displays test results @@ -93,13 +94,13 @@ py.test contains two approaches for processing test results: - - "old style": methods invoked on session object - - "new style": reporting events are sent to Reporter object +- "old style": methods invoked on session object +- "new style": reporting events are sent to Reporter object next: - - use reporting event architecture pervasively - - refactor existing reporting extensions +- use reporting event architecture pervasively +- refactor existing reporting extensions py.test extensions (conftest) ================================= From hpk at codespeak.net Mon Jul 9 12:40:06 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 12:40:06 +0200 (CEST) Subject: [pypy-svn] r44876 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070709104006.D067D8166@code0.codespeak.net> Author: hpk Date: Mon Jul 9 12:40:05 2007 New Revision: 44876 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: add reference to christian's talk tomorrow Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Mon Jul 9 12:40:05 2007 @@ -153,6 +153,8 @@ * unlimited recursion * *composable* greenlets, tasklets, co-routines +* see also Christian's talk Tuesday, 10am + Relative speeds to CPython ------------------------------ From pedronis at codespeak.net Mon Jul 9 12:55:44 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 9 Jul 2007 12:55:44 +0200 (CEST) Subject: [pypy-svn] r44877 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070709105544.C3F1C8209@code0.codespeak.net> Author: pedronis Date: Mon Jul 9 12:55:44 2007 New Revision: 44877 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: tweaks Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Mon Jul 9 12:55:44 2007 @@ -1,7 +1,7 @@ .. include:: ========================================================== -PyPy 1.0 xxxx +PyPy 1.0 ... ========================================================== :Authors: Armin Rigo,Samuele Pedroni From hpk at codespeak.net Mon Jul 9 13:06:37 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 9 Jul 2007 13:06:37 +0200 (CEST) Subject: [pypy-svn] r44878 - pypy/extradoc/talk/ep2007/interpreter Message-ID: <20070709110637.41742820E@code0.codespeak.net> Author: hpk Date: Mon Jul 9 13:06:35 2007 New Revision: 44878 Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Log: explicit high-level ness Modified: pypy/extradoc/talk/ep2007/interpreter/interpreter.txt ============================================================================== --- pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (original) +++ pypy/extradoc/talk/ep2007/interpreter/interpreter.txt Mon Jul 9 13:06:35 2007 @@ -11,12 +11,16 @@ * New Interpreter features through implementation +* Python Interpreter implemented at high level + * We don't aim at changing python syntax nor semantics * Keeping implementation and language design separate * They give new dimensions to how one can use python + + PyPy Overview -------------- From lene at codespeak.net Mon Jul 9 17:05:19 2007 From: lene at codespeak.net (lene at codespeak.net) Date: Mon, 9 Jul 2007 17:05:19 +0200 (CEST) Subject: [pypy-svn] r44880 - pypy/extradoc/talk/ep2007/howtoeu Message-ID: <20070709150519.83DA58221@code0.codespeak.net> Author: lene Date: Mon Jul 9 17:05:18 2007 New Revision: 44880 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Log: some amendments, two small XXX note that cost models have been eliminated completely in FP7 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Mon Jul 9 17:05:18 2007 @@ -17,7 +17,7 @@ - have participated and contributed to the PyPy EU project - experiences - discuss *open source* research projects - (cut out IPR issues more or less completely) + (skip IPR issues more or less completely) EU Framework Research Programme ==================================== @@ -33,7 +33,7 @@ ===================== - vision / idea? -- read Work programmes / CALL topics +- reading Work programmes / CALL topics - finding people to collaborate - finding a set of organisations - drafting a proposal ... @@ -66,7 +66,7 @@ - Lead Partner - Deliverables (Reports, Releases, Papers) -consortium +Consortium ============================ - looking for interested parties @@ -76,11 +76,14 @@ - receives and distributes money -applying to the EU +Applying to the EU ========================= - answer to CALLS within stated deadline - co-ordinator registers project and proposal + +XXX (lene) co-ordinator or EU-officer? + - proposals gets reviewed by *independent* experts - proposals receive points (PyPy had 26 out of 30) - Invitation to "negotiation" @@ -96,16 +99,14 @@ Funding Schemes ===================== -XXX (lene) no more cost models, will check furthermore -- Universities receive (basically) 100% funding +- Universities receive (basically) 75% funding - Small/Medium Companies (SME) receive 75% funding of costs -- FCF = 20% Flat administrative overhead added to costs -- FC = actual indirect costs +- actual or 20% flat administrative overheads accountable Signing the contract? ======================== - +XXX (lene) i do not understand this slide Infrastructure ================== @@ -118,15 +119,10 @@ Management structures ============================ -- administrative team (preparing consortium decisions) +- Administrative team (preparing Consortium decisions) - Technical Board (discussing problems, deciding roadmap) - developer body (everybody can participate) -- Consortium - -XXX insert the graph from the EU talks that shows -the overlap - seems important to me in the open -source context to recommend close and overlapping -structures +- Consortium and Co-ordinator Meetings and Sprints @@ -154,9 +150,10 @@ Reporting / Deliverables ============================ -- interim reviews each 18 months -- contractual and technical deliverable -- contractual deliverables +- interim reviews each 12-18 months +- Deliverables: Reports, Releases etc. +- Deadlines +- official submission of Reports/Documentation to EU Recommendations ==================== @@ -168,6 +165,8 @@ it fits the objectives better; - implement a transparent and flat communication and decision culture, be ready to adjust roles and not stick to formal issues too much. +- dual-use Deliverables for for Community/Industry and EU contract purposes + Roles and Responsbilities ============================ @@ -181,13 +180,21 @@ Changes from FP6 to FP7 ============================= +XXX (lene) not sure if this slide is necessary, most +changes are only interesting for those who +participated in FP6, and that are not many of the +audience. keeping the statements vague, as i only looked up +a summary page: + +http://www.finance-helpdesk.org/front/ShowCategory.aspx?ItemID=883 -- less collective financial responsibility? +- apparently FP7 improves some regulations that PyPy found burdonsome +- less collective financial responsibility - 75% funding for (small/medium) companies -- XXX +- subcontracting relaxed (important for international companies) -What remains? +What Remains? ================================ - build the project on a shared vision and shared interest From tismer at codespeak.net Tue Jul 10 00:36:35 2007 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 10 Jul 2007 00:36:35 +0200 (CEST) Subject: [pypy-svn] r44887 - in pypy/extradoc/talk/ep2007: . demo Message-ID: <20070709223635.30874822D@code0.codespeak.net> Author: tismer Date: Tue Jul 10 00:36:34 2007 New Revision: 44887 Added: pypy/extradoc/talk/ep2007/The Essentials Of Stackless Python V2.ppt (contents, props changed) pypy/extradoc/talk/ep2007/demo/ pypy/extradoc/talk/ep2007/demo/pickledtasklet.py Log: reworked/extended slides Added: pypy/extradoc/talk/ep2007/The Essentials Of Stackless Python V2.ppt ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/ep2007/demo/pickledtasklet.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/demo/pickledtasklet.py Tue Jul 10 00:36:34 2007 @@ -0,0 +1,27 @@ +import pickle, sys +import stackless + +ch = stackless.channel() + +def recurs(depth, level=1): + print 'enter level %s%d' % (level*' ', level) + if level >= depth: + ch.send('hi') + if level < depth: + recurs(depth, level+1) + print 'leave level %s%d' % (level*' ', level) + +def demo(depth): + t = stackless.tasklet(recurs)(depth) + print ch.receive() + pickle.dump(t, file('tasklet.pickle', 'wb')) + +if __name__ == '__main__': + if len(sys.argv) > 1: + t = pickle.load(file(sys.argv[1], 'rb')) + t.insert() + else: + t = stackless.tasklet(demo)(14) + stackless.run() + +# remark: think of fixing cells etc. on the sprint From lene at codespeak.net Tue Jul 10 00:38:10 2007 From: lene at codespeak.net (lene at codespeak.net) Date: Tue, 10 Jul 2007 00:38:10 +0200 (CEST) Subject: [pypy-svn] r44888 - pypy/extradoc/talk/ep2007/howtoeu Message-ID: <20070709223810.33C58822D@code0.codespeak.net> Author: lene Date: Tue Jul 10 00:38:09 2007 New Revision: 44888 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Log: split the recommendations slide in two Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Tue Jul 10 00:38:09 2007 @@ -163,9 +163,12 @@ - balance contractual with community interests - consider changes to the contractual work plan if it fits the objectives better; -- implement a transparent and flat communication and decision - culture, be ready to adjust roles and not stick to formal issues too much. -- dual-use Deliverables for for Community/Industry and EU contract purposes + +More Recommendations +======================== +- transparent and flat communication and decision culture +- be ready to adjust roles and not stick to formal issues too much. +- dual-use Deliverables for Community/Industry and EU contract purposes Roles and Responsbilities From bea at codespeak.net Tue Jul 10 07:51:03 2007 From: bea at codespeak.net (bea at codespeak.net) Date: Tue, 10 Jul 2007 07:51:03 +0200 (CEST) Subject: [pypy-svn] r44889 - pypy/extradoc/talk/ep2007 Message-ID: <20070710055103.DA4418240@code0.codespeak.net> Author: bea Date: Tue Jul 10 07:51:02 2007 New Revision: 44889 Added: pypy/extradoc/talk/ep2007/pypy_work.txt Log: my talk Added: pypy/extradoc/talk/ep2007/pypy_work.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/pypy_work.txt Tue Jul 10 07:51:02 2007 @@ -0,0 +1,118 @@ +======================================= +PyPy: Why and how dit it (not) work? +======================================= + +:Authors: Bea D?ring (Change Maker) & Holger Krekel (merlinux GmbH) +:Place: Europython 2007, Vilnius +:Date: 10th July 2007 + + +Introduction: Bea?s 2cents +---------------------------- + +- Mostly my personal views - so who am I? +- Old school gamer (non-profit work), high school + teacher, education and software development + project management,instructor, newbie agilista + +The PyPy Long Island Ice Tea +-------------------------------- + +- If PyPy was a drink - what would it be? +- Long Island Ice Tea - why? +- Why have one alcohol base when you can have seven? +- And you do get drunk - and confused ;-) + +Recommendations: +-------------------- +- What worked for us +- Mostly parts already used by other projects +- Unique for PyPy was the blend of OSS/agile practices + within a EU contractual environment + +Automated Test-Driven Development +----------------------------------------- +- py.test --> supports rapid prototyping + in a single code base +- Combined with efficient version control (svn) +- Version control and test on both code and documentation +- Automated commot mailinglists for both +---> supports open and transparent workstyle + +Mobile Sprints +---------------------------------------- +- Sprints: one week (in our case) co-located + coding sessions in different locations +- "Open and closed" sprints +- In the EU-funded project: 19 sprints +- Combined with Summer of Code/PyPy +---> supports open and transparent workstyle + +Communication +--------------------------------------- +- IRC: #pypy on freenode.org (+others) +- Mailinglists: manage, technical board, dev, sprints +- Automated mailinglists (code and documentation) +---> supports open and transparent workstyle + +Information/documentation +--------------------------------------- +- Sprint reports +- EU-reports written with a dual use purpose + (EU, industry, community) +- Video documentation (tutorials, talks, interviews, sprints) +- All consortium involved persons could access _all_ data +---> supports open and transparent workstyle + +Synchronization +---------------------------------------- +- Sync-meetings: weekly dev-IRC meetings with a fixed date +- Fixed agenda: last, next, blockers +- Not more than 30 minutes +- Inspired by Canonical +---> supports open and transparent workstyle + +Conceptual integrity +----------------------------------------- +- How to maintain the core developers OSS mandate + within EU-contract? +- Technical board driving the technical development +- GUI between EU-level and OSS community level +- Also driving management ;-) +---> supports open and transparent workstyle + +Learning by burnings: +----------------------- +- Learning by doing - learning by reflection +- In japanese: learning by burning + +Open and transparent is costly +------------------------------------------ +- Formal roles existed but PyPy is a "beyond the call of duty" project +- Cost in time, energy and resources to keep everyone involved +-->Main cost though was making this fit into the EU-structure +-->It was worth it ("excellent technical results"), but the price was high + +Increased contribution is costly +------------------------------------------ +- Open sprints, IRC-mentoring and Summer of Code/PyPy + meant new core developers +- Cost in time, energy and resources to mentor and support +-->Main cost though was making this fit into the EU-structure +-->It was worth it ("excellent technical results") and new friends! + +Overlapping realities is costly +------------------------------------------ +- PyPy - a practical EU OSS research project +- Is that an oxymoron? +- Goals too high and too wide (novel approaches)? +-->Main cost though was making this fit into the EU-structure + while evolving the community +-->It was worth it ("excellent technical results") - really cool stuff ;-) + +What now? +------------------------------------------ +- We continue as an OSS project: http://codespeak.net/pypy +- We sprint: welcome to the EP2007 sprint on Thursday! +- We have an Open Space session inviting PyPy users and would-be + users to to discuss their needs on Wednesday - join us! \ No newline at end of file From hpk at codespeak.net Tue Jul 10 07:56:35 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 10 Jul 2007 07:56:35 +0200 (CEST) Subject: [pypy-svn] r44890 - pypy/extradoc/talk/ep2007/howtoeu Message-ID: <20070710055635.6AF558248@code0.codespeak.net> Author: hpk Date: Tue Jul 10 07:56:33 2007 New Revision: 44890 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Log: refinements, resolving XXX Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Tue Jul 10 07:56:33 2007 @@ -17,7 +17,7 @@ - have participated and contributed to the PyPy EU project - experiences - discuss *open source* research projects - (skip IPR issues more or less completely) +- skip "Intellectucal Property" issues EU Framework Research Programme ==================================== @@ -27,12 +27,12 @@ - FP7 spends some 20.000.000.000 Euro in 7 years - regularly publishes Work Programmes and Topics - puts out CALLS for Proposals -- we mainly try to discuss FP7 (a few changes from FP6) +- we mainly discuss FP7 (only few changes from FP6) Getting started ===================== -- vision / idea? +- visions / ideas? - reading Work programmes / CALL topics - finding people to collaborate - finding a set of organisations @@ -41,9 +41,9 @@ (Research) Objectives? ============================== -- core research objective(s) +- formulate core research objective(s) (state of the art & how the project aims beyond) -- core technical objectives? +- formulate core technical objectives? - core other objectives? (methodological/social) can you summarize it on one page? @@ -56,7 +56,7 @@ - management structures - work plan in long form - detailed description of work packages -- budget +- Project duration, budget? Work Packages =================== @@ -81,13 +81,9 @@ - answer to CALLS within stated deadline - co-ordinator registers project and proposal - -XXX (lene) co-ordinator or EU-officer? - - proposals gets reviewed by *independent* experts - proposals receive points (PyPy had 26 out of 30) - Invitation to "negotiation" -- Project duration? Negotiation ============== @@ -103,10 +99,14 @@ - Small/Medium Companies (SME) receive 75% funding of costs - actual or 20% flat administrative overheads accountable - Signing the contract? ======================== -XXX (lene) i do not understand this slide + +signature of the (negotiated) contract: + +- is legally binding for all contractors and the EU +- afterwards the EU does pre-payments (40-80%) +- the consortium can ask for amendments later Infrastructure ================== @@ -128,16 +128,16 @@ Meetings and Sprints ============================ -- sprints are the rhythm of the project -- plan for core and for newcomer-friendly sprints -- integrate a model for funding contributors! +- sprints provide the rhythm of the project +- plan for core for newcomer-friendly sprints +- integrate a model for funding new contributors! Contractual and actual developments ===================================== - Contract implies a 'plan' for the development - Project may evolve into other directions -- checkpoints for assessing the work package status +- do checkpoints for assessing the work package status Planning for changes =========================== @@ -151,7 +151,7 @@ ============================ - interim reviews each 12-18 months -- Deliverables: Reports, Releases etc. +- Deliverables are: Reports, Releases etc. - Deadlines - official submission of Reports/Documentation to EU @@ -183,13 +183,6 @@ Changes from FP6 to FP7 ============================= -XXX (lene) not sure if this slide is necessary, most -changes are only interesting for those who -participated in FP6, and that are not many of the -audience. keeping the statements vague, as i only looked up -a summary page: - -http://www.finance-helpdesk.org/front/ShowCategory.aspx?ItemID=883 - apparently FP7 improves some regulations that PyPy found burdonsome - less collective financial responsibility @@ -202,4 +195,8 @@ - build the project on a shared vision and shared interest - don't build it on money (too much) +- get into contact with national support offices! +- use the national and EU help infrastructure! + + From arigo at codespeak.net Tue Jul 10 07:59:24 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jul 2007 07:59:24 +0200 (CEST) Subject: [pypy-svn] r44891 - pypy/extradoc/talk/ep2007 Message-ID: <20070710055924.9AB288254@code0.codespeak.net> Author: arigo Date: Tue Jul 10 07:59:24 2007 New Revision: 44891 Modified: pypy/extradoc/talk/ep2007/pypy_work.txt Log: - remove windows end of lines - ReSTify Modified: pypy/extradoc/talk/ep2007/pypy_work.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_work.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_work.txt Tue Jul 10 07:59:24 2007 @@ -1,118 +1,130 @@ -======================================= -PyPy: Why and how dit it (not) work? -======================================= - -:Authors: Bea D?ring (Change Maker) & Holger Krekel (merlinux GmbH) -:Place: Europython 2007, Vilnius -:Date: 10th July 2007 - - -Introduction: Bea?s 2cents ----------------------------- - -- Mostly my personal views - so who am I? -- Old school gamer (non-profit work), high school - teacher, education and software development - project management,instructor, newbie agilista - -The PyPy Long Island Ice Tea --------------------------------- - -- If PyPy was a drink - what would it be? -- Long Island Ice Tea - why? -- Why have one alcohol base when you can have seven? -- And you do get drunk - and confused ;-) - -Recommendations: --------------------- -- What worked for us -- Mostly parts already used by other projects -- Unique for PyPy was the blend of OSS/agile practices - within a EU contractual environment - -Automated Test-Driven Development ------------------------------------------ -- py.test --> supports rapid prototyping - in a single code base -- Combined with efficient version control (svn) -- Version control and test on both code and documentation -- Automated commot mailinglists for both ----> supports open and transparent workstyle - -Mobile Sprints ----------------------------------------- -- Sprints: one week (in our case) co-located - coding sessions in different locations -- "Open and closed" sprints -- In the EU-funded project: 19 sprints -- Combined with Summer of Code/PyPy ----> supports open and transparent workstyle - -Communication ---------------------------------------- -- IRC: #pypy on freenode.org (+others) -- Mailinglists: manage, technical board, dev, sprints -- Automated mailinglists (code and documentation) ----> supports open and transparent workstyle - -Information/documentation ---------------------------------------- -- Sprint reports -- EU-reports written with a dual use purpose - (EU, industry, community) -- Video documentation (tutorials, talks, interviews, sprints) -- All consortium involved persons could access _all_ data ----> supports open and transparent workstyle - -Synchronization ----------------------------------------- -- Sync-meetings: weekly dev-IRC meetings with a fixed date -- Fixed agenda: last, next, blockers -- Not more than 30 minutes -- Inspired by Canonical ----> supports open and transparent workstyle - -Conceptual integrity ------------------------------------------ -- How to maintain the core developers OSS mandate - within EU-contract? -- Technical board driving the technical development -- GUI between EU-level and OSS community level -- Also driving management ;-) ----> supports open and transparent workstyle - -Learning by burnings: ------------------------ -- Learning by doing - learning by reflection -- In japanese: learning by burning - -Open and transparent is costly ------------------------------------------- -- Formal roles existed but PyPy is a "beyond the call of duty" project -- Cost in time, energy and resources to keep everyone involved --->Main cost though was making this fit into the EU-structure --->It was worth it ("excellent technical results"), but the price was high - -Increased contribution is costly ------------------------------------------- -- Open sprints, IRC-mentoring and Summer of Code/PyPy - meant new core developers -- Cost in time, energy and resources to mentor and support --->Main cost though was making this fit into the EU-structure --->It was worth it ("excellent technical results") and new friends! - -Overlapping realities is costly ------------------------------------------- -- PyPy - a practical EU OSS research project -- Is that an oxymoron? -- Goals too high and too wide (novel approaches)? --->Main cost though was making this fit into the EU-structure - while evolving the community --->It was worth it ("excellent technical results") - really cool stuff ;-) - -What now? ------------------------------------------- -- We continue as an OSS project: http://codespeak.net/pypy -- We sprint: welcome to the EP2007 sprint on Thursday! -- We have an Open Space session inviting PyPy users and would-be - users to to discuss their needs on Wednesday - join us! \ No newline at end of file +======================================= +PyPy: Why and how dit it (not) work? +======================================= + +:Authors: Bea D?ring (Change Maker) & Holger Krekel (merlinux GmbH) +:Place: EuroPython 2007, Vilnius +:Date: 10th July 2007 + + +Introduction: Bea?s 2cents +---------------------------- + +- Mostly my personal views - so who am I? +- Old school gamer (non-profit work), high school + teacher, education and software development + project management, instructor, newbie agilista + +The PyPy Long Island Ice Tea +-------------------------------- + +- If PyPy was a drink - what would it be? +- Long Island Ice Tea - why? +- Why have one alcohol base when you can have seven? +- And you do get drunk - and confused ;-) + +Recommendations: +-------------------- +- What worked for us +- Mostly parts already used by other projects +- Unique for PyPy was the blend of OSS/agile practices + within a EU contractual environment + +Automated Test-Driven Devel. +----------------------------------------- +- py.test --> supports rapid prototyping + in a single code base +- Combined with efficient version control (svn) +- Version control and test on both code and documentation +- Automated commit mailinglists for both + +---> supports open and transparent workstyle + +Mobile Sprints +---------------------------------------- +- Sprints: one week (in our case) co-located + coding sessions in different locations +- "Open and closed" sprints +- In the EU-funded project: 19 sprints +- Combined with Summer of Code/PyPy + +---> supports open and transparent workstyle + +Communication +--------------------------------------- +- IRC: #pypy on freenode.org (+others) +- Mailinglists: manage, technical board, dev, sprints +- Automated mailinglists (code and documentation) + +---> supports open and transparent workstyle + +Information/documentation +--------------------------------------- +- Sprint reports +- EU-reports written with a dual use purpose + (EU, industry, community) +- Video documentation (tutorials, talks, interviews, sprints) +- All consortium involved persons could access *all* data + +---> supports open and transparent workstyle + +Synchronization +---------------------------------------- +- Sync-meetings: weekly dev-IRC meetings with a fixed date +- Fixed agenda: last, next, blockers +- Not more than 30 minutes +- Inspired by Canonical + +---> supports open and transparent workstyle + +Conceptual integrity +----------------------------------------- +- How to maintain the core developers OSS mandate + within EU-contract? +- Technical board driving the technical development +- GUI between EU-level and OSS community level +- Also driving management ;-) + +---> supports open and transparent workstyle + +Learning by burnings: +----------------------- +- Learning by doing - learning by reflection +- In japanese: learning by burning + +Open and transparent is costly +------------------------------------------ +- Formal roles existed but PyPy is a "beyond the call of duty" project +- Cost in time, energy and resources to keep everyone involved + +--> Main cost though was making this fit into the EU-structure + +--> It was worth it ("excellent technical results"), but the price was high + +Increased contribution is costly +------------------------------------------ +- Open sprints, IRC-mentoring and Summer of Code/PyPy + meant new core developers +- Cost in time, energy and resources to mentor and support + +--> Main cost though was making this fit into the EU-structure + +--> It was worth it ("excellent technical results") and new friends! + +Overlapping realities is costly +------------------------------------------ +- PyPy - a practical EU OSS research project +- Is that an oxymoron? +- Goals too high and too wide (novel approaches)? + +--> Main cost though was making this fit into the EU-structure +while evolving the community + +--> It was worth it ("excellent technical results") - really cool stuff ;-) + +What now? +------------------------------------------ +- We continue as an OSS project: http://codespeak.net/pypy +- We sprint: welcome to the EP2007 sprint on Thursday! +- We have an Open Space session inviting PyPy users and would-be + users to to discuss their needs on Wednesday - join us! From arigo at codespeak.net Tue Jul 10 07:59:51 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jul 2007 07:59:51 +0200 (CEST) Subject: [pypy-svn] r44892 - pypy/extradoc/talk/ep2007 Message-ID: <20070710055951.8CA418258@code0.codespeak.net> Author: arigo Date: Tue Jul 10 07:59:51 2007 New Revision: 44892 Modified: pypy/extradoc/talk/ep2007/pypy_work.txt (props changed) Log: fixeol From hpk at codespeak.net Tue Jul 10 08:05:55 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 10 Jul 2007 08:05:55 +0200 (CEST) Subject: [pypy-svn] r44893 - pypy/extradoc/talk/ep2007/howtoeu Message-ID: <20070710060555.3DC588261@code0.codespeak.net> Author: hpk Date: Tue Jul 10 08:05:54 2007 New Revision: 44893 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Log: tell a bit about background Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Tue Jul 10 08:05:54 2007 @@ -14,9 +14,10 @@ Intro =================== -- have participated and contributed to the PyPy EU project -- experiences -- discuss *open source* research projects +- we have participated in the PyPy project +- both on technical and management levels +- founded merlinux GmbH in 2004, doing various projects +- we discuss *open source* research projects - skip "Intellectucal Property" issues EU Framework Research Programme From pedronis at codespeak.net Tue Jul 10 09:23:46 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 10 Jul 2007 09:23:46 +0200 (CEST) Subject: [pypy-svn] r44894 - pypy/extradoc/talk/ep2007/pypy_10_and_jit Message-ID: <20070710072346.210C2827F@code0.codespeak.net> Author: pedronis Date: Tue Jul 10 09:23:44 2007 New Revision: 44894 Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Log: fix year Modified: pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt Tue Jul 10 09:23:44 2007 @@ -17,7 +17,7 @@ * A tool-chain with a *new paradigm* for building interpreters -* Dec 2004 - March 2006: a research project funded by the EU +* Dec 2004 - March 2007: a research project funded by the EU and companies. Motivation From lene at codespeak.net Tue Jul 10 09:26:23 2007 From: lene at codespeak.net (lene at codespeak.net) Date: Tue, 10 Jul 2007 09:26:23 +0200 (CEST) Subject: [pypy-svn] r44895 - pypy/extradoc/talk/ep2007/howtoeu Message-ID: <20070710072623.EFEB4827F@code0.codespeak.net> Author: lene Date: Tue Jul 10 09:26:23 2007 New Revision: 44895 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Log: inserted official EU links for further information. Not sure if http://pypy.org makes sense in that context Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Tue Jul 10 09:26:23 2007 @@ -200,4 +200,20 @@ - use the national and EU help infrastructure! +Further information +========================== + +- official EU FP7 website: + http://cordis.europa.eu/fp7/home_en.html + +- information leaflet on FP7: + http://ec.europa.eu/research/fp7/pdf/fp7-inbrief_en.pdf + +- Information and Communication Technology department: + http://cordis.europa.eu/fp7/ict/ + +- http://pypy.org + + + From hpk at codespeak.net Tue Jul 10 09:46:58 2007 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 10 Jul 2007 09:46:58 +0200 (CEST) Subject: [pypy-svn] r44896 - in pypy/extradoc/talk/ep2007: . howtoeu Message-ID: <20070710074658.682718291@code0.codespeak.net> Author: hpk Date: Tue Jul 10 09:46:57 2007 New Revision: 44896 Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt pypy/extradoc/talk/ep2007/pypy_work.txt Log: small adjustments Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Tue Jul 10 09:46:57 2007 @@ -212,7 +212,7 @@ - Information and Communication Technology department: http://cordis.europa.eu/fp7/ict/ -- http://pypy.org +- PyPy project http://pypy.org Modified: pypy/extradoc/talk/ep2007/pypy_work.txt ============================================================================== --- pypy/extradoc/talk/ep2007/pypy_work.txt (original) +++ pypy/extradoc/talk/ep2007/pypy_work.txt Tue Jul 10 09:46:57 2007 @@ -1,5 +1,5 @@ ======================================= -PyPy: Why and how dit it (not) work? +PyPy: Why and how did it (not) work? ======================================= :Authors: Bea D?ring (Change Maker) & Holger Krekel (merlinux GmbH) From jlg at codespeak.net Tue Jul 10 12:44:38 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Tue, 10 Jul 2007 12:44:38 +0200 (CEST) Subject: [pypy-svn] r44897 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070710104438.F3AD582A4@code0.codespeak.net> Author: jlg Date: Tue Jul 10 12:44:38 2007 New Revision: 44897 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: list operation added Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Tue Jul 10 12:44:38 2007 @@ -1,21 +1,21 @@ Do now ------ -- variables and execution context -- list operations: cons car cdr list -- comparison: = < > eq? eqv? -- lambdas! +- RPythonize (during EP sprint) Do next ------- -- symbols (should be done with next point) +- definitions: let, let*, letrec + +- symbols vs identifier, which name is better - global dict for symbols _obarray_ -- (quote ) and ', not only symbols - implement key funcions (apply, reduce, mapcar and so on) +- comparison: < > eq? eqv? + Do in some future ----------------- @@ -23,6 +23,7 @@ - macros - proper tail-recursion + - mutulally recursive procedures (r5rs p.12) - delayed evaluation - continuations Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Tue Jul 10 12:44:38 2007 @@ -252,6 +252,10 @@ def oper(self, x, y): return x * y +class List(W_Procedure): + def procedure(self, ctx, lst): + return plst2lst(lst) + class Define(W_Macro): def eval(self, ctx, lst): w_identifier = lst.car @@ -344,6 +348,7 @@ 'cons': Cons, 'car': Car, 'cdr': Cdr, + 'list': List, #comparisons '=': Equal, #macros Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Tue Jul 10 12:44:38 2007 @@ -307,3 +307,14 @@ assert isinstance(w_pair.cdr.car, W_Symbol) assert w_pair.cdr.car.to_string() == "y" +def test_list(): + ctx = ExecutionContext() + ctx.put("var", W_Fixnum(42)) + w_lst = eval_expr(ctx, "(list 1 var (+ 2 1) 'a)") + assert isinstance(w_lst, W_Pair) + assert w_lst.car.to_number() == 1 + assert w_lst.cdr.car.to_number() == 42 + assert w_lst.cdr.cdr.car.to_number() == 3 + assert w_lst.cdr.cdr.cdr.car.to_string() == "a" + assert isinstance(w_lst.cdr.cdr.cdr.cdr, W_Nil) + From santagada at codespeak.net Tue Jul 10 17:58:39 2007 From: santagada at codespeak.net (santagada at codespeak.net) Date: Tue, 10 Jul 2007 17:58:39 +0200 (CEST) Subject: [pypy-svn] r44912 - pypy/dist/pypy/rlib/parsing Message-ID: <20070710155839.B48F481CC@code0.codespeak.net> Author: santagada Date: Tue Jul 10 17:58:39 2007 New Revision: 44912 Modified: pypy/dist/pypy/rlib/parsing/deterministic.py Log: remove a print statement Modified: pypy/dist/pypy/rlib/parsing/deterministic.py ============================================================================== --- pypy/dist/pypy/rlib/parsing/deterministic.py (original) +++ pypy/dist/pypy/rlib/parsing/deterministic.py Tue Jul 10 17:58:39 2007 @@ -340,7 +340,7 @@ result = result.get_code() while "\n\n" in result: result = result.replace("\n\n", "\n") - print result + #print result exec py.code.Source(result).compile() return recognize From santagada at codespeak.net Tue Jul 10 18:00:10 2007 From: santagada at codespeak.net (santagada at codespeak.net) Date: Tue, 10 Jul 2007 18:00:10 +0200 (CEST) Subject: [pypy-svn] r44913 - in pypy/dist/pypy/lang/js: . test Message-ID: <20070710160010.B50EE81FD@code0.codespeak.net> Author: santagada Date: Tue Jul 10 18:00:10 2007 New Revision: 44913 Modified: pypy/dist/pypy/lang/js/js_interactive.py pypy/dist/pypy/lang/js/test/test_interactive.py Log: Modified: pypy/dist/pypy/lang/js/js_interactive.py ============================================================================== --- pypy/dist/pypy/lang/js/js_interactive.py (original) +++ pypy/dist/pypy/lang/js/js_interactive.py Tue Jul 10 18:00:10 2007 @@ -12,13 +12,6 @@ from pypy.lang.js.jsobj import W_Builtin, W_String, ThrowException, w_Undefined from pypy.rlib.streamio import open_file_as_stream -help_message = """ -PyPy's JavaScript Interpreter: - -f filename - Loads a file - -n to not be interactive - -h show this help message - -d jump to a pdb in case of failure -""" import code sys.ps1 = 'js> ' sys.ps2 = '... ' @@ -126,25 +119,26 @@ banner = 'PyPy JavaScript Interpreter' code.InteractiveConsole.interact(self, banner) -def main(inspect=False, filename=None, args=[]): +def main(inspect=False, files=[]): jsi = JSInterpreter() - if filename is not None: + for filename in files: jsi.runcodefromfile(filename) if (filename is None) or inspect: jsi.interact() if __name__ == '__main__': from optparse import OptionParser - parser = OptionParser(usage='%prog [options] [file] [arg] ...', + parser = OptionParser(usage='%prog [options] [files] ...', description='PyPy JavaScript Interpreter') parser.add_option('-i', dest='inspect', - action='store_true', default=False, - help='inspect interactively after running script') + action='store_true', default=False, + help='inspect interactively after running script') + # ... (add other options) opts, args = parser.parse_args() if args: - main(inspect=opts.inspect, filename=args[0], args=args[1:]) + main(inspect=opts.inspect, files=args) else: main(inspect=opts.inspect) sys.exit(0) Modified: pypy/dist/pypy/lang/js/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/js/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/js/test/test_interactive.py Tue Jul 10 18:00:10 2007 @@ -23,7 +23,7 @@ return self._spawn(str(py.magic.autopath().dirpath().dirpath().join('js_interactive.py')), argv) def prompt_send(self, message): - self.child.expect('js-pypy>') + self.child.expect('js>') self.child.sendline(message) def expect(self, message): @@ -33,7 +33,7 @@ self.child.sendline(message) def continue_send(self, message): - self.child.expect(' ... ') + self.child.expect('...') self.child.sendline(message) def setup_method(cls, method): From jlg at codespeak.net Wed Jul 11 09:22:46 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 11 Jul 2007 09:22:46 +0200 (CEST) Subject: [pypy-svn] r44915 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711072246.05AFB8188@code0.codespeak.net> Author: jlg Date: Wed Jul 11 09:22:45 2007 New Revision: 44915 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: let macro added Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Wed Jul 11 09:22:45 2007 @@ -1,18 +1,20 @@ Do now ------ +- lambda called with wrong number of args issue - RPythonize (during EP sprint) Do next ------- -- definitions: let, let*, letrec +- definitions: let*, letrec - symbols vs identifier, which name is better - global dict for symbols _obarray_ - implement key funcions (apply, reduce, mapcar and so on) + # reduce will not make into python 3.0 ;) - comparison: < > eq? eqv? @@ -22,8 +24,9 @@ Here starts the real fun! - macros -- proper tail-recursion - - mutulally recursive procedures (r5rs p.12) - delayed evaluation - continuations +- proper tail-recursion + - mutulally recursive procedures (r5rs p.12) +- switch to bytecode enberation + eval istead of evaluating AST Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 11 09:22:45 2007 @@ -182,9 +182,6 @@ def procedure(self, ctx, lst): #ctx is a caller context, which is joyfully ignored - - #if len(lst) != len(self.args): - # raise "Wrong argument count" local_ctx = self.closure.copy() @@ -196,10 +193,6 @@ else: local_ctx.put(name, lst[idx]) - #vars = zip(self.args, lst) - #for (name, val) in vars: - # local_ctx.put(name, val) - body_expression = self.body body_result = None while not isinstance(body_expression, W_Nil): @@ -318,6 +311,26 @@ w_body = lst.cdr #.car return W_Lambda(w_args, w_body, ctx.copy()) +class Let(W_Macro): + def eval(slef, ctx, lst): + local_ctx = ctx.copy() + w_formal = lst.car + while not isinstance(w_formal, W_Nil): + name = w_formal.car.car.to_string() + #evaluate the values in caller ctx + val = w_formal.car.cdr.car.eval(ctx) + local_ctx.put(name, val) + w_formal = w_formal.cdr + + body_expression = lst.cdr + body_result = None + while not isinstance(body_expression, W_Nil): + body_result = body_expression.car.eval(local_ctx) + body_expression = body_expression.cdr + + return body_result + + def Literal(sexpr): return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) @@ -356,6 +369,7 @@ 'set!': Sete, 'if': MacroIf, 'lambda': Lambda, + 'let': Let, 'quote': Quote, } Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Wed Jul 11 09:22:45 2007 @@ -318,3 +318,11 @@ assert w_lst.cdr.cdr.cdr.car.to_string() == "a" assert isinstance(w_lst.cdr.cdr.cdr.cdr, W_Nil) +def test_let(): + ctx = ExecutionContext() + w_global = W_Fixnum(0) + ctx.put("var", w_global) + w_result = eval_expr(ctx, "(let ((var 42) (x (+ 2 var))) (+ var x))") + assert w_result.to_number() == 44 + assert ctx.get("var") is w_global + From fijal at codespeak.net Wed Jul 11 09:49:06 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 09:49:06 +0200 (CEST) Subject: [pypy-svn] r44916 - pypy/dist/pypy/lang/js Message-ID: <20070711074906.4F8F081A8@code0.codespeak.net> Author: fijal Date: Wed Jul 11 09:49:05 2007 New Revision: 44916 Modified: pypy/dist/pypy/lang/js/js_interactive.py Log: Fix stupid bug Modified: pypy/dist/pypy/lang/js/js_interactive.py ============================================================================== --- pypy/dist/pypy/lang/js/js_interactive.py (original) +++ pypy/dist/pypy/lang/js/js_interactive.py Wed Jul 11 09:49:05 2007 @@ -123,7 +123,7 @@ jsi = JSInterpreter() for filename in files: jsi.runcodefromfile(filename) - if (filename is None) or inspect: + if (not files) or inspect: jsi.interact() if __name__ == '__main__': From fijal at codespeak.net Wed Jul 11 10:07:48 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 10:07:48 +0200 (CEST) Subject: [pypy-svn] r44917 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711080748.8755F81AD@code0.codespeak.net> Author: fijal Date: Wed Jul 11 10:07:47 2007 New Revision: 44917 Added: pypy/dist/pypy/lang/scheme/interactive.py (contents, props changed) pypy/dist/pypy/lang/scheme/test/test_interactive.py (contents, props changed) Log: Add a simple scheme interactive interpreter Added: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 10:07:47 2007 @@ -0,0 +1,33 @@ +#!/usr/bin/env python +""" Interactive (untranslatable) version of the pypy +scheme interpreter +""" + +from pypy.lang.scheme.object import ExecutionContext +from pypy.lang.scheme.ssparser import parse +import os, sys + +def check_parens(s): + return s.count("(") == s.count(")") + +def interactive(): + print "PyPy Scheme interpreter" + ctx = ExecutionContext() + to_exec = "" + cont = False + while 1: + if cont: + ps = '.. ' + else: + ps = '-> ' + sys.stdout.write(ps) + to_exec += sys.stdin.readline() + if check_parens(to_exec): + print parse(to_exec).eval(ctx) + to_exec = "" + cont = False + else: + cont = True + +if __name__ == '__main__': + interactive() Added: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 10:07:47 2007 @@ -0,0 +1,32 @@ + +from pypy.lang.scheme.interactive import check_parens +import py +import re +import sys + +def test_paren(): + assert check_parens("((( ))())") + assert not check_parens("(x()x") + +class TestInteractive: + def _spawn(self, *args, **kwds): + try: + import pexpect + except ImportError, e: + py.test.skip(str(e)) + kwds.setdefault('timeout', 10) + print 'SPAWN:', args, kwds + child = pexpect.spawn(*args, **kwds) + child.logfile = sys.stdout + return child + + def spawn(self, argv=[]): + path = py.magic.autopath()/".."/".."/"interactive.py" + return self._spawn(str(path), argv) + + def test_interactive(self): + child = self.spawn() + child.expect("->") + child.sendline("(+ 1 2)") + child.expect("3W") + From fijal at codespeak.net Wed Jul 11 10:08:49 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 10:08:49 +0200 (CEST) Subject: [pypy-svn] r44918 - pypy/dist/pypy/lang/scheme/test Message-ID: <20070711080849.4017C81AD@code0.codespeak.net> Author: fijal Date: Wed Jul 11 10:08:48 2007 New Revision: 44918 Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py Log: Add another test (just because) Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 10:08:48 2007 @@ -30,3 +30,10 @@ child.sendline("(+ 1 2)") child.expect("3W") + def test_multiline(self): + child = self.spawn() + child.expect("-> ") + child.sendline("(+ 1") + child.expect(".. ") + child.sendline(" 2)") + child.expect("3W") From fijal at codespeak.net Wed Jul 11 10:20:44 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 10:20:44 +0200 (CEST) Subject: [pypy-svn] r44920 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711082044.B067681AD@code0.codespeak.net> Author: fijal Date: Wed Jul 11 10:20:44 2007 New Revision: 44920 Modified: pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py pypy/dist/pypy/lang/scheme/test/test_interactive.py Log: (jlg, fijal) * Improve interactive interpreter to show exceptions * Move UnboundVariable to new exception class Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 10:20:44 2007 @@ -3,7 +3,7 @@ scheme interpreter """ -from pypy.lang.scheme.object import ExecutionContext +from pypy.lang.scheme.object import ExecutionContext, SchemeException from pypy.lang.scheme.ssparser import parse import os, sys @@ -23,7 +23,10 @@ sys.stdout.write(ps) to_exec += sys.stdin.readline() if check_parens(to_exec): - print parse(to_exec).eval(ctx) + try: + print parse(to_exec).eval(ctx) + except SchemeException, e: + print "error: %s" % e to_exec = "" cont = False else: Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 11 10:20:44 2007 @@ -1,5 +1,12 @@ import autopath +class SchemeException(Exception): + pass + +class UnboundVariable(SchemeException): + def __str__(self): + return "Unbound variable %s" % self.args[0] + class W_Root(object): def to_string(self): return '' @@ -47,7 +54,7 @@ else: #reference to undefined identifier #unbound - raise Exception("Unbound variable: %s" % (self.name, )) + raise UnboundVariable(self.name) class W_Boolean(W_Root): def __init__(self, val): @@ -423,7 +430,7 @@ loc.obj = obj return obj - raise Exception("Unbound variable: %s" % (name, )) + raise UnboundVariable(name) def set(self, name, obj): """update existing location or create new location""" Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Wed Jul 11 10:20:44 2007 @@ -1,9 +1,6 @@ import py from pypy.lang.scheme.ssparser import parse -from pypy.lang.scheme.object import W_Boolean, W_Fixnum, W_Float, W_String -from pypy.lang.scheme.object import W_Nil, W_Pair, W_Symbol, W_Identifier -from pypy.lang.scheme.object import W_Procedure, W_Lambda -from pypy.lang.scheme.object import ExecutionContext +from pypy.lang.scheme.object import * def test_eval_obj(): w_num = W_Pair(W_Identifier("+"), @@ -79,7 +76,7 @@ loc2 = ctx.get_location("x") assert ctx.get("x").to_number() == 43 assert loc1 is loc2 - py.test.raises(Exception, eval_expr, ctx, "(set! y 42)") + py.test.raises(UnboundVariable, eval_expr, ctx, "(set! y 42)") def test_func(): ctx = ExecutionContext() Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 10:20:44 2007 @@ -37,3 +37,10 @@ child.expect(".. ") child.sendline(" 2)") child.expect("3W") + + def test_unbound_variable(self): + child = self.spawn() + child.expect("-> ") + child.sendline("x") + child.expect("Unbound variable x") + child.expect("-> ") From jlg at codespeak.net Wed Jul 11 10:26:23 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 11 Jul 2007 10:26:23 +0200 (CEST) Subject: [pypy-svn] r44921 - pypy/dist/pypy/lang/scheme Message-ID: <20070711082623.5E66C81AD@code0.codespeak.net> Author: jlg Date: Wed Jul 11 10:26:22 2007 New Revision: 44921 Modified: pypy/dist/pypy/lang/scheme/interactive.py Log: autopath to interactive added Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 10:26:22 2007 @@ -2,7 +2,7 @@ """ Interactive (untranslatable) version of the pypy scheme interpreter """ - +import autopath from pypy.lang.scheme.object import ExecutionContext, SchemeException from pypy.lang.scheme.ssparser import parse import os, sys From jlg at codespeak.net Wed Jul 11 12:35:36 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 11 Jul 2007 12:35:36 +0200 (CEST) Subject: [pypy-svn] r44922 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711103536.0D9208182@code0.codespeak.net> Author: jlg Date: Wed Jul 11 12:35:35 2007 New Revision: 44922 Modified: pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py pypy/dist/pypy/lang/scheme/test/test_interactive.py Log: Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 12:35:35 2007 @@ -5,6 +5,7 @@ import autopath from pypy.lang.scheme.object import ExecutionContext, SchemeException from pypy.lang.scheme.ssparser import parse +from pypy.rlib.parsing.makepackrat import BacktrackException import os, sys def check_parens(s): @@ -27,6 +28,9 @@ print parse(to_exec).eval(ctx) except SchemeException, e: print "error: %s" % e + except BacktrackException, e: + print "syntax error" + to_exec = "" cont = False else: Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 11 12:35:35 2007 @@ -6,7 +6,7 @@ class UnboundVariable(SchemeException): def __str__(self): return "Unbound variable %s" % self.args[0] - + class W_Root(object): def to_string(self): return '' @@ -15,7 +15,7 @@ return True def __str__(self): - return self.to_string() + "W" + return self.to_string() def __repr__(self): return "" @@ -319,7 +319,7 @@ return W_Lambda(w_args, w_body, ctx.copy()) class Let(W_Macro): - def eval(slef, ctx, lst): + def eval(self, ctx, lst): local_ctx = ctx.copy() w_formal = lst.car while not isinstance(w_formal, W_Nil): @@ -337,6 +337,32 @@ return body_result +class Letrec(W_Macro): + def eval(self, ctx, lst): + local_ctx = ctx.copy() + + #bound variables + w_formal = lst.car + while not isinstance(w_formal, W_Nil): + name = w_formal.car.car.to_string() + local_ctx.put(name, W_Nil()) + w_formal = w_formal.cdr + + #eval in local_ctx and assign values + w_formal = lst.car + while not isinstance(w_formal, W_Nil): + name = w_formal.car.car.to_string() + val = w_formal.car.cdr.car.eval(local_ctx) + local_ctx.set(name, val) + w_formal = w_formal.cdr + + body_expression = lst.cdr + body_result = None + while not isinstance(body_expression, W_Nil): + body_result = body_expression.car.eval(local_ctx) + body_expression = body_expression.cdr + + return body_result def Literal(sexpr): return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) @@ -377,6 +403,7 @@ 'if': MacroIf, 'lambda': Lambda, 'let': Let, + 'letrec': Letrec, 'quote': Quote, } Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Wed Jul 11 12:35:35 2007 @@ -68,6 +68,10 @@ w_num = eval_expr(ctx, "(+ 1 v1 v2)") assert w_num.to_number() == 46.1 +def text_unbound(): + ctx = ExecutionContext() + py.test.raises(UnboundVariable, eval_expr, ctx, "y") + def test_sete(): ctx = ExecutionContext() eval_expr(ctx, "(define x 42)") @@ -323,3 +327,18 @@ assert w_result.to_number() == 44 assert ctx.get("var") is w_global +def test_letrec(): + ctx = ExecutionContext() + w_result = eval_expr(ctx, """ + (letrec ((even? + (lambda (n) + (if (= n 0) + #t + (odd? (- n 1))))) + (odd? + (lambda (n) + (if (= n 0) + #f + (even? (- n 1)))))) + (even? 12))""") + assert w_result.to_boolean() is True Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 12:35:35 2007 @@ -1,4 +1,3 @@ - from pypy.lang.scheme.interactive import check_parens import py import re @@ -44,3 +43,11 @@ child.sendline("x") child.expect("Unbound variable x") child.expect("-> ") + + def test_syntax_error(self): + clild = self.spawn() + child.expect("-> ") + child.sendline("))((") + child.expect("syntax error") + child.expect("-> ") + From cfbolz at codespeak.net Wed Jul 11 12:49:30 2007 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 11 Jul 2007 12:49:30 +0200 (CEST) Subject: [pypy-svn] r44923 - pypy/dist/pypy/lang/scheme/test Message-ID: <20070711104930.4B3208178@code0.codespeak.net> Author: cfbolz Date: Wed Jul 11 12:49:29 2007 New Revision: 44923 Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py Log: fix obvious typo. what are those Ws doing there, are they intentional? they make the tests fail. Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 12:49:29 2007 @@ -27,7 +27,7 @@ child = self.spawn() child.expect("->") child.sendline("(+ 1 2)") - child.expect("3W") + child.expect("3") def test_multiline(self): child = self.spawn() @@ -35,7 +35,7 @@ child.sendline("(+ 1") child.expect(".. ") child.sendline(" 2)") - child.expect("3W") + child.expect("3") def test_unbound_variable(self): child = self.spawn() @@ -45,7 +45,7 @@ child.expect("-> ") def test_syntax_error(self): - clild = self.spawn() + child = self.spawn() child.expect("-> ") child.sendline("))((") child.expect("syntax error") From jlg at codespeak.net Wed Jul 11 12:55:40 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 11 Jul 2007 12:55:40 +0200 (CEST) Subject: [pypy-svn] r44924 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711105540.65E8E81B2@code0.codespeak.net> Author: jlg Date: Wed Jul 11 12:55:39 2007 New Revision: 44924 Modified: pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: quit procedure (raises SchemeQuit); intercept the parser errors and reraise as SchemeParsingError Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 12:55:39 2007 @@ -3,9 +3,9 @@ scheme interpreter """ import autopath -from pypy.lang.scheme.object import ExecutionContext, SchemeException -from pypy.lang.scheme.ssparser import parse -from pypy.rlib.parsing.makepackrat import BacktrackException +from pypy.lang.scheme.object import ExecutionContext, SchemeException, \ + SchemeQuit +from pypy.lang.scheme.ssparser import parse, SchemeParsingError import os, sys def check_parens(s): @@ -26,9 +26,11 @@ if check_parens(to_exec): try: print parse(to_exec).eval(ctx) + except SchemeQuit, e: + break except SchemeException, e: print "error: %s" % e - except BacktrackException, e: + except SchemeParsingError, e: print "syntax error" to_exec = "" Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Wed Jul 11 12:55:39 2007 @@ -7,6 +7,9 @@ def __str__(self): return "Unbound variable %s" % self.args[0] +class SchemeQuit(SchemeException): + pass + class W_Root(object): def to_string(self): return '' @@ -312,6 +315,10 @@ w_second = lst[1] return W_Boolean(w_first.equal(w_second)) +class Quit(W_Procedure): + def procedure(self, ctx, lst): + raise SchemeQuit + class Lambda(W_Macro): def eval(self, ctx, lst): w_args = lst.car @@ -395,6 +402,7 @@ 'car': Car, 'cdr': Cdr, 'list': List, + 'quit': Quit, #comparisons '=': Equal, #macros Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Wed Jul 11 12:55:39 2007 @@ -1,8 +1,8 @@ import autopath from pypy.rlib.parsing.pypackrat import PackratParser from pypy.rlib.parsing.makepackrat import BacktrackException, Status -from pypy.lang.scheme.object import W_Pair, W_Fixnum, W_String, W_Identifier -from pypy.lang.scheme.object import W_Nil, W_Boolean, W_Float, Literal +from pypy.lang.scheme.object import W_Pair, W_Fixnum, W_String, W_Identifier, \ + W_Nil, W_Boolean, W_Float, Literal, SchemeException def unquote(s): return s.replace('\\"', '"') @@ -80,6 +80,14 @@ | return {W_Nil()}; ''' +class SchemeParsingError(SchemeException): + pass + def parse(code): - p = SchemeParser(code) + try: + p = SchemeParser(code) + except BacktrackException, e: + raise SchemeParsingError + return p.file() + Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Wed Jul 11 12:55:39 2007 @@ -342,3 +342,7 @@ (even? (- n 1)))))) (even? 12))""") assert w_result.to_boolean() is True + +def test_quit(): + py.test.raises(SchemeQuit, eval_noctx, "(quit)") + From jlg at codespeak.net Wed Jul 11 13:22:20 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 11 Jul 2007 13:22:20 +0200 (CEST) Subject: [pypy-svn] r44925 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711112220.F0D858076@code0.codespeak.net> Author: jlg Date: Wed Jul 11 13:22:19 2007 New Revision: 44925 Modified: pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_interactive.py Log: test_interactive passes now Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 13:22:19 2007 @@ -5,7 +5,8 @@ import autopath from pypy.lang.scheme.object import ExecutionContext, SchemeException, \ SchemeQuit -from pypy.lang.scheme.ssparser import parse, SchemeParsingError +from pypy.lang.scheme.ssparser import parse +from pypy.rlib.parsing.makepackrat import BacktrackException import os, sys def check_parens(s): @@ -28,10 +29,10 @@ print parse(to_exec).eval(ctx) except SchemeQuit, e: break + except BacktrackException, e: + print "syntax error" except SchemeException, e: print "error: %s" % e - except SchemeParsingError, e: - print "syntax error" to_exec = "" cont = False Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Wed Jul 11 13:22:19 2007 @@ -2,7 +2,7 @@ from pypy.rlib.parsing.pypackrat import PackratParser from pypy.rlib.parsing.makepackrat import BacktrackException, Status from pypy.lang.scheme.object import W_Pair, W_Fixnum, W_String, W_Identifier, \ - W_Nil, W_Boolean, W_Float, Literal, SchemeException + W_Nil, W_Boolean, W_Float, Literal def unquote(s): return s.replace('\\"', '"') @@ -80,14 +80,7 @@ | return {W_Nil()}; ''' -class SchemeParsingError(SchemeException): - pass - def parse(code): - try: - p = SchemeParser(code) - except BacktrackException, e: - raise SchemeParsingError - + p = SchemeParser(code) return p.file() Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 13:22:19 2007 @@ -47,7 +47,7 @@ def test_syntax_error(self): child = self.spawn() child.expect("-> ") - child.sendline("))((") + child.sendline(")(") child.expect("syntax error") child.expect("-> ") From fijal at codespeak.net Wed Jul 11 13:29:37 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 13:29:37 +0200 (CEST) Subject: [pypy-svn] r44926 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070711112937.F28AD809E@code0.codespeak.net> Author: fijal Date: Wed Jul 11 13:29:37 2007 New Revision: 44926 Modified: pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/test/test_interactive.py Log: Improve a bit multiline handling Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Wed Jul 11 13:29:37 2007 @@ -24,8 +24,13 @@ ps = '-> ' sys.stdout.write(ps) to_exec += sys.stdin.readline() - if check_parens(to_exec): + if to_exec == "\n": + to_exec = "" + elif check_parens(to_exec): try: + if to_exec == "": + print + raise SchemeQuit print parse(to_exec).eval(ctx) except SchemeQuit, e: break Modified: pypy/dist/pypy/lang/scheme/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_interactive.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_interactive.py Wed Jul 11 13:29:37 2007 @@ -51,3 +51,14 @@ child.expect("syntax error") child.expect("-> ") + def test_multiline_enter(self): + child = self.spawn() + child.expect("-> ") + child.sendline("") + child.expect("-> ") + child.sendline("") + child.expect("-> ") + child.sendline("") + child.expect("-> ") + child.sendline("") + # we cannot sendeof, because pexpect is confused :-( From fijal at codespeak.net Wed Jul 11 14:06:48 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 14:06:48 +0200 (CEST) Subject: [pypy-svn] r44927 - pypy/extradoc/talk/ep2007 Message-ID: <20070711120648.366E4815A@code0.codespeak.net> Author: fijal Date: Wed Jul 11 14:06:45 2007 New Revision: 44927 Added: pypy/extradoc/talk/ep2007/blob.dia (contents, props changed) pypy/extradoc/talk/ep2007/blob.png (contents, props changed) Log: Add my lightning talk Added: pypy/extradoc/talk/ep2007/blob.dia ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/ep2007/blob.png ============================================================================== Binary file. No diff available. From fijal at codespeak.net Wed Jul 11 14:34:53 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 14:34:53 +0200 (CEST) Subject: [pypy-svn] r44928 - pypy/dist/pypy/lang/scheme Message-ID: <20070711123453.38C7F8141@code0.codespeak.net> Author: fijal Date: Wed Jul 11 14:34:51 2007 New Revision: 44928 Modified: pypy/dist/pypy/lang/scheme/TODO.txt Log: Typo? Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Wed Jul 11 14:34:51 2007 @@ -28,5 +28,5 @@ - continuations - proper tail-recursion - mutulally recursive procedures (r5rs p.12) -- switch to bytecode enberation + eval istead of evaluating AST +- switch to bytecode generation + eval istead of evaluating AST From pdg at codespeak.net Wed Jul 11 14:36:32 2007 From: pdg at codespeak.net (pdg at codespeak.net) Date: Wed, 11 Jul 2007 14:36:32 +0200 (CEST) Subject: [pypy-svn] r44929 - in pypy/dist/pypy/translator/jvm: . src/pypy test Message-ID: <20070711123632.AB7AA8141@code0.codespeak.net> Author: pdg Date: Wed Jul 11 14:36:31 2007 New Revision: 44929 Modified: pypy/dist/pypy/translator/jvm/generator.py pypy/dist/pypy/translator/jvm/opcodes.py pypy/dist/pypy/translator/jvm/prebuiltnodes.py pypy/dist/pypy/translator/jvm/src/pypy/Interlink.java pypy/dist/pypy/translator/jvm/src/pypy/PyPy.java pypy/dist/pypy/translator/jvm/test/test_float.py pypy/dist/pypy/translator/jvm/test/test_int.py Log: translator/jvm - added overflow checking, long casting, new simple opcodes, and a new exception (pdg) Modified: pypy/dist/pypy/translator/jvm/generator.py ============================================================================== --- pypy/dist/pypy/translator/jvm/generator.py (original) +++ pypy/dist/pypy/translator/jvm/generator.py Wed Jul 11 14:36:31 2007 @@ -52,10 +52,11 @@ class Opcode(object): def __init__(self, jvmstr): """ - flags is a set of flags (see above) that describe opcode + flags is a set of flags (see above) that describe opcode #UPDATE jvmstr is the name for jasmin printouts """ self.jvmstr = jvmstr + self.flags = None #Should flags be added to args? def __repr__(self): return "" % (self.jvmstr, self.flags) @@ -363,9 +364,26 @@ DOUBLETOSTRINGD = Method.s(jDoubleClass, 'toString', (jDouble,), jString) CHARTOSTRINGC = Method.s(jCharClass, 'toString', (jChar,), jString) MATHIABS = Method.s(jMath, 'abs', (jInt,), jInt) +IABSOVF = Method.s(jPyPy, 'abs_ovf', (jInt,), jInt) MATHLABS = Method.s(jMath, 'abs', (jLong,), jLong) +LABSOVF = Method.s(jPyPy, 'abs_ovf', (jLong,), jLong) MATHDABS = Method.s(jMath, 'abs', (jDouble,), jDouble) +INEGOVF = Method.s(jPyPy, 'negate_ovf', (jInt,), jInt) +LNEGOVF = Method.s(jPyPy, 'negate_ovf', (jInt,), jInt) +IADDOVF = Method.s(jPyPy, 'add_ovf', (jInt, jInt), jInt) +LADDOVF = Method.s(jPyPy, 'add_ovf', (jLong, jLong), jLong) +ISUBOVF = Method.s(jPyPy, 'sub_ovf', (jInt, jInt), jInt) +LSUBOVF = Method.s(jPyPy, 'sub_ovf', (jLong, jLong), jLong) +IMULOVF = Method.s(jPyPy, 'multiply_ovf', (jInt, jInt), jInt) +LMULOVF = Method.s(jPyPy, 'multiply_ovf', (jLong, jLong), jLong) MATHFLOOR = Method.s(jMath, 'floor', (jDouble,), jDouble) +IFLOORDIVOVF = Method.s(jPyPy, 'floordiv_ovf', (jInt, jInt), jInt) +LFLOORDIVOVF = Method.s(jPyPy, 'floordiv_ovf', (jLong, jLong), jLong) +IFLOORDIVZEROVF = Method.s(jPyPy, 'floordiv_zer_ovf', (jInt, jInt), jInt) +LFLOORDIVZEROVF = Method.s(jPyPy, 'floordiv_zer_ovf', (jLong, jLong), jLong) +IREMOVF = Method.s(jPyPy, 'mod_ovf', (jInt, jInt), jInt) +LREMOVF = Method.s(jPyPy, 'mod_ovf', (jLong, jLong), jLong) +MATHDPOW = Method.s(jMath, 'pow', (jDouble, jDouble), jDouble) PRINTSTREAMPRINTSTR = Method.v(jPrintStream, 'print', (jString,), jVoid) CLASSFORNAME = Method.s(jClass, 'forName', (jString,), jClass) CLASSISASSIGNABLEFROM = Method.v(jClass, 'isAssignableFrom', (jClass,), jBool) @@ -375,7 +393,7 @@ PYPYULONGCMP = Method.s(jPyPy, 'ulong_cmp', (jLong,jLong), jInt) PYPYUINTTODOUBLE = Method.s(jPyPy, 'uint_to_double', (jInt,), jDouble) PYPYDOUBLETOUINT = Method.s(jPyPy, 'double_to_uint', (jDouble,), jInt) -#PYPYDOUBLETOLONG = Method.s(jPyPy, 'double_to_long', (jDouble,), jLong) #PAUL +PYPYDOUBLETOLONG = Method.s(jPyPy, 'double_to_long', (jDouble,), jLong) #PAUL PYPYLONGBITWISENEGATE = Method.s(jPyPy, 'long_bitwise_negate', (jLong,), jLong) PYPYSTRTOINT = Method.s(jPyPy, 'str_to_int', (jString,), jInt) PYPYSTRTOUINT = Method.s(jPyPy, 'str_to_uint', (jString,), jInt) Modified: pypy/dist/pypy/translator/jvm/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/jvm/opcodes.py (original) +++ pypy/dist/pypy/translator/jvm/opcodes.py Wed Jul 11 14:36:31 2007 @@ -32,20 +32,8 @@ _proc(op))] def _check_ovf(op): - # TODO return op -def _check_unary_ovf(op): - # TODO We should just use rlib's overflow dtection - # Assume LLONG_MIN = (- LLONG_MAX-1) -# if op.operand == LLONG_MIN: -# return [TranslateException( -# jvmtype.jArithmeticException, -# 'throwOverflowError', -# _proc(op))] -# else: -# return op - return op # This table maps the opcodes to micro-ops for processing them. # It is post-processed by _proc. @@ -96,9 +84,9 @@ 'int_is_true': 'not_equals_zero', 'int_neg': jvmgen.INEG, - 'int_neg_ovf': None, # How to handle overflow? + 'int_neg_ovf': jvmgen.INEGOVF, 'int_abs': 'iabs', - 'int_abs_ovf': _check_ovf('iabs'), + 'int_abs_ovf': jvmgen.IABSOVF, 'int_invert': 'bitwise_negate', 'int_add': jvmgen.IADD, @@ -118,11 +106,12 @@ 'int_lshift': jvmgen.ISHL, 'int_rshift': jvmgen.ISHR, 'int_xor': jvmgen.IXOR, - 'int_add_ovf': _check_ovf(jvmgen.IADD), - 'int_sub_ovf': _check_ovf(jvmgen.ISUB), - 'int_mul_ovf': _check_ovf(jvmgen.IMUL), + 'int_add_ovf': jvmgen.IADDOVF, + 'int_sub_ovf': jvmgen.ISUBOVF, + 'int_mul_ovf': jvmgen.IMULOVF, 'int_floordiv_ovf': jvmgen.IDIV, # these can't overflow! - 'int_mod_ovf': jvmgen.IREM, + 'int_mod_zer': _check_zer(jvmgen.IREM), + 'int_mod_ovf': jvmgen.IREMOVF, 'int_lt_ovf': 'less_than', 'int_le_ovf': 'less_equals', 'int_eq_ovf': 'equals', @@ -138,7 +127,7 @@ 'int_rshift_ovf': jvmgen.ISHR, # these can't overflow! 'int_xor_ovf': jvmgen.IXOR, 'int_floordiv_ovf_zer': _check_zer(jvmgen.IDIV), - 'int_mod_ovf_zer': _check_zer(jvmgen.IREM), + 'int_mod_ovf_zer': _check_zer(jvmgen.IREMOVF), 'uint_is_true': 'not_equals_zero', 'uint_invert': 'bitwise_negate', @@ -171,7 +160,8 @@ 'float_add': jvmgen.DADD, 'float_sub': jvmgen.DSUB, 'float_mul': jvmgen.DMUL, - 'float_truediv': jvmgen.DDIV, + 'float_truediv': jvmgen.DDIV, + 'float_pow': jvmgen.MATHDPOW, 'float_lt': 'dbl_less_than', 'float_le': 'dbl_less_equals', 'float_eq': 'dbl_equals', @@ -183,8 +173,9 @@ jvmgen.LCONST_0, 'long_not_equals'], 'llong_neg': jvmgen.LNEG, - 'llong_neg_ovf': _check_ovf(jvmgen.LNEG), + 'llong_neg_ovf': jvmgen.LNEGOVF, 'llong_abs': jvmgen.MATHLABS, + 'llong_abs_ovf': jvmgen.LABSOVF, 'llong_invert': jvmgen.PYPYLONGBITWISENEGATE, 'llong_add': jvmgen.LADD, @@ -193,7 +184,9 @@ 'llong_div': jvmgen.LDIV, 'llong_truediv': None, # TODO 'llong_floordiv': jvmgen.LDIV, + 'llong_floordiv_zer': _check_zer(jvmgen.LDIV), 'llong_mod': jvmgen.LREM, + 'llong_mod_zer': _check_zer(jvmgen.LREM), 'llong_lt': 'long_less_than', 'llong_le': 'long_less_equals', 'llong_eq': 'long_equals', @@ -205,6 +198,8 @@ 'llong_lshift': jvmgen.LSHL, 'llong_rshift': jvmgen.LSHR, 'llong_xor': jvmgen.LXOR, + 'llong_floordiv_ovf': jvmgen.LDIV, # these can't overflow! + 'llong_mod_ovf': jvmgen.LREMOVF, 'ullong_is_true': [PushAllArgs, jvmgen.LCONST_0, @@ -218,6 +213,7 @@ 'ullong_truediv': None, # TODO 'ullong_floordiv': jvmgen.LDIV, # valid? 'ullong_mod': jvmgen.LREM, # valid? + 'ullong_mod_zer': _check_zer(jvmgen.LREM), 'ullong_lt': 'ulong_less_than', 'ullong_le': 'ulong_less_equals', 'ullong_eq': 'ulong_equals', @@ -244,7 +240,7 @@ 'cast_uint_to_float': jvmgen.PYPYUINTTODOUBLE, 'cast_float_to_int': jvmgen.D2I, #'cast_float_to_longlong': jvmgen.D2L, #PAUL - #'cast_float_to_longlong': jvmgen.PYPYDOUBLETOLONG, #PAUL + 'cast_float_to_longlong': jvmgen.PYPYDOUBLETOLONG, #PAUL 'cast_float_to_uint': jvmgen.PYPYDOUBLETOUINT, 'truncate_longlong_to_int': jvmgen.L2I, 'cast_longlong_to_float': jvmgen.L2D, @@ -255,3 +251,4 @@ for opc, val in _opcodes.items(): opcodes[opc] = _proc(val) del _opcodes + Modified: pypy/dist/pypy/translator/jvm/prebuiltnodes.py ============================================================================== --- pypy/dist/pypy/translator/jvm/prebuiltnodes.py (original) +++ pypy/dist/pypy/translator/jvm/prebuiltnodes.py Wed Jul 11 14:36:31 2007 @@ -8,6 +8,9 @@ def throwIndexError(): raise IndexError +def throwOverflowError(): + raise OverflowError + # ___________________________________________________________________________ def create_interlink_node(db): Modified: pypy/dist/pypy/translator/jvm/src/pypy/Interlink.java ============================================================================== --- pypy/dist/pypy/translator/jvm/src/pypy/Interlink.java (original) +++ pypy/dist/pypy/translator/jvm/src/pypy/Interlink.java Wed Jul 11 14:36:31 2007 @@ -12,4 +12,5 @@ public interface Interlink { public void throwZeroDivisionError(); public void throwIndexError(); -} \ No newline at end of file + public void throwOverflowError(); +} Modified: pypy/dist/pypy/translator/jvm/src/pypy/PyPy.java ============================================================================== --- pypy/dist/pypy/translator/jvm/src/pypy/PyPy.java (original) +++ pypy/dist/pypy/translator/jvm/src/pypy/PyPy.java Wed Jul 11 14:36:31 2007 @@ -17,6 +17,12 @@ public static Interlink interlink; + public static final long LONG_MAX = Long.MAX_VALUE; + public static final long LONG_MIN = Long.MIN_VALUE; + public static final int INT_MAX = Integer.MAX_VALUE; + public static final int INT_MIN = Integer.MIN_VALUE; + + /** * Compares two unsigned integers (value1 and value2) and returns * a value greater than, equal to, or less than zero if value 1 is @@ -116,6 +122,15 @@ return (hiword << 16) + loword; } + public static long double_to_long(double value) + { + //if (value <= LONG_MAX) + //{ + return (long)value; + //} + //TODO: Add some logic here, but I don't think we'll need it + } + public static long long_bitwise_negate(long value) { return ~value; } @@ -259,6 +274,202 @@ } // ---------------------------------------------------------------------- + // Checked Arithmetic - Overflow protection + public static int negate_ovf(int x) + { + if (x == INT_MIN) + { + throwOverflowError(); + } + return -x; + } + + public static long negate_ovf(long x) + { + if (x == LONG_MIN) + { + throwOverflowError(); + } + return -x; + } + + public static int abs_ovf(int x) + { + if (x == INT_MIN) + { + throwOverflowError(); + } + return Math.abs(x); + } + + public static long abs_ovf(long x) + { + if (x == LONG_MIN) + { + throwOverflowError(); + } + return Math.abs(x); + } + + public static int add_ovf(int x, int y) + { + int result = x+y; + if (!(((result^x) >=0) || ((result^y) >=0))) + { + throwOverflowError(); + } + return result; + } + + public static int subtract_ovf(int x, int y) + { + int result = x-y; + if (!(((result^x) >=0) || ((result^(~y)) >=0))) + { + throwOverflowError(); + } + return result; + } + + private static boolean int_multiply(int x, int y) + { + double dprod = (double)x * (double)y; + long longprod = x * y; + double dlongprod = (double)longprod; + double diff = dlongprod - dprod; + double absdiff = Math.abs(diff); + double absprod = Math.abs(dprod); + + if (dlongprod == dprod) //if diff == 0 + return true; + else if (32.0 * absdiff <= absprod) //if we lost some information, are we at least 5 good bits? + return true; + else + return false; + } + public static int multiply_ovf(int x, int y) + { + if (!(int_multiply(x, y))) + { + throwOverflowError(); + } + return x * y; + } + + public static long add_ovf(long x, long y) + { + long result = x+y; + if (!(((result^x) >=0) || ((result^y) >=0))) + { + throwOverflowError(); + } + return result; + } + + public static long subtract_ovf(long x, long y) + { + long result = x-y; + if (!(((result^x) >=0) || ((result^(~y)) >=0))) + { + throwOverflowError(); + } + return result; + } + + private static boolean long_multiply(long x, long y) + { + double dprod = (double)x * (double)y; + long longprod = x * y; + double dlongprod = (double)longprod; + double diff = dlongprod - dprod; + double absdiff = Math.abs(diff); + double absprod = Math.abs(dprod); + + if (dlongprod == dprod) //if diff == 0 + return true; + else if (32.0 * absdiff <= absprod) //if we lost some information, are we at least 5 good bits? + return true; + else + return false; + } + public static long multiply_ovf(long x, long y) + { + //if (long_multiply(x, y)) + //{ + // return x * y; + //} + //else + // throwOverflowError(); + if (!(long_multiply(x, y))) + { + throwOverflowError(); + } + return x*y; + //else + // throwOverflowError(); + } + + /* shifting */ + /* Talk to Niko about how to get a workaround for PY_* functions in Java */ + + /* floor division */ + public static int floordiv_ovf(int x, int y) + { + if ((y == -1) && (x == INT_MIN)) + { + throwOverflowError(); + } + return x/y; + } + + public static int floordiv_zer_ovf(int x, int y) + { + if (y != 0) + { + return floordiv_ovf(x,y); + } + else + throw new ArithmeticException("Floor Division with integer by 0"); + } + + public static long floordiv_ovf(long x, long y) + { + if ((y == -1) && (x == LONG_MIN)) + { + throwOverflowError(); + } + return x/y; + } + + public static long floordiv_zer_ovf(long x, long y) + { + if (y != 0) + { + return floordiv_ovf(x,y); + } + else + throw new ArithmeticException("Floor Division with integer by 0"); + } + + public static int mod_ovf(int x, int y) + { + if ((y == -1) && (x == INT_MIN)) + { + throwOverflowError(); + } + return x%y; + } + + public static long mod_ovf(long x, long y) + { + if ((y == -1) && (x == LONG_MIN)) + { + throwOverflowError(); + } + return x%y; + } + + // ---------------------------------------------------------------------- // String public static String ll_strconcat(String str1, String str2) { @@ -419,7 +630,7 @@ // Helpers public static byte[] string2bytes(String s) { - return s.getBytes(); + return s.getBytes(); } public static void append(StringBuilder sb, String s) { @@ -438,7 +649,7 @@ // OOString support public static String oostring(int n, int base_) { - // XXX needs special case for unsigned ints + // XXX needs special case for unsigned ints if (base_ == -1) base_ = 10; if (n < 0 && base_ != 10) @@ -598,6 +809,10 @@ interlink.throwIndexError(); } + public static void throwOverflowError() { + interlink.throwOverflowError(); + } + // ---------------------------------------------------------------------- // Self Test Modified: pypy/dist/pypy/translator/jvm/test/test_float.py ============================================================================== --- pypy/dist/pypy/translator/jvm/test/test_float.py (original) +++ pypy/dist/pypy/translator/jvm/test/test_float.py Wed Jul 11 14:36:31 2007 @@ -17,19 +17,19 @@ # expected = fn(i) # res = self.interpret(fn, [i]) # assert res == expected - + + #Works, answer is correct, but not of type r_longlong. def test_longlong_conversion(self): py.test.skip("JVM backend unknown opcode cast_float_to_longlong") - - def test_pow(self): - py.test.skip("JVM backend unknown opcode float_pow") - + def test_float_constant_conversions(self): py.test.skip("JVM backend lacks appropriate percision; 42.000000614400001 == 42.0") - + + #The JVM doesn't even have uints def test_from_r_uint(self): py.test.skip("JVM backend lacks appropriate percision") + #The jvm doesn't even have uints def test_to_r_uint(self): py.test.skip("JVM backend lacks appropriate percision") - \ No newline at end of file + Modified: pypy/dist/pypy/translator/jvm/test/test_int.py ============================================================================== --- pypy/dist/pypy/translator/jvm/test/test_int.py (original) +++ pypy/dist/pypy/translator/jvm/test/test_int.py Wed Jul 11 14:36:31 2007 @@ -18,11 +18,9 @@ def test_rarithmetic(self): pass # does this make more sense in jvm - + + #There is an issue that it doesn't correctly call long code over int code def test_neg_abs_ovf(self): py.test.skip("Unaware how to handle overflow") - - def test_protected_div_mod(self): - py.test.skip("fails because of unusual exception propagation") div_mod_iteration_count = 20 From arigo at codespeak.net Wed Jul 11 14:43:58 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 11 Jul 2007 14:43:58 +0200 (CEST) Subject: [pypy-svn] r44930 - pypy/dist/dotviewer Message-ID: <20070711124358.9671C8141@code0.codespeak.net> Author: arigo Date: Wed Jul 11 14:43:58 2007 New Revision: 44930 Modified: pypy/dist/dotviewer/dotviewer.py pypy/dist/dotviewer/graphclient.py pypy/dist/dotviewer/graphpage.py Log: Small enhancements. Modified: pypy/dist/dotviewer/dotviewer.py ============================================================================== --- pypy/dist/dotviewer/dotviewer.py (original) +++ pypy/dist/dotviewer/dotviewer.py Wed Jul 11 14:43:58 2007 @@ -9,7 +9,8 @@ In the first form, show the graph contained in a .dot file. In the second form, the graph was already compiled to a .plain file. In the third form, listen for connexion on the given port and display -the graphs sent by the remote side. +the graphs sent by the remote side. On the remote site, set the +GRAPHSERVER environment variable to HOST:PORT. """ import sys Modified: pypy/dist/dotviewer/graphclient.py ============================================================================== --- pypy/dist/dotviewer/graphclient.py (original) +++ pypy/dist/dotviewer/graphclient.py Wed Jul 11 14:43:58 2007 @@ -5,16 +5,16 @@ GRAPHSERVER = os.path.join(this_dir, 'graphserver.py') -def display_dot_file(dotfile, wait=True): +def display_dot_file(dotfile, wait=True, save_tmp_file=None): """ Display the given dot file in a subprocess. """ if not os.path.exists(str(dotfile)): raise IOError("No such file: %s" % (dotfile,)) import graphpage page = graphpage.DotFileGraphPage(str(dotfile)) - display_page(page, wait=wait) + display_page(page, wait=wait, save_tmp_file=save_tmp_file) -def display_page(page, wait=True): +def display_page(page, wait=True, save_tmp_file=None): messages = [(msgstruct.CMSG_INIT, msgstruct.MAGIC)] history = [page] pagecache = {} @@ -31,6 +31,10 @@ def reload(graph_id): page = getpage(graph_id) + if save_tmp_file: + f = open(save_tmp_file, 'w') + f.write(page.source) + f.close() messages.extend(page_messages(page, graph_id)) send_graph_messages(io, messages) del messages[:] Modified: pypy/dist/dotviewer/graphpage.py ============================================================================== --- pypy/dist/dotviewer/graphpage.py (original) +++ pypy/dist/dotviewer/graphpage.py Wed Jul 11 14:43:58 2007 @@ -3,6 +3,8 @@ """Base class for the client-side content of one of the 'pages' (one graph) sent over to and displayed by the external process. """ + save_tmp_file = None + def __init__(self, *args): self.args = args @@ -26,7 +28,7 @@ "Display a graph page." import graphclient, msgstruct try: - graphclient.display_page(self) + graphclient.display_page(self, save_tmp_file=self.save_tmp_file) except msgstruct.RemoteError, e: import sys print >> sys.stderr, "Exception in the graph viewer:", str(e) From jlg at codespeak.net Wed Jul 11 15:01:59 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Wed, 11 Jul 2007 15:01:59 +0200 (CEST) Subject: [pypy-svn] r44931 - pypy/dist/pypy/lang/scheme/test Message-ID: <20070711130159.6A1B98120@code0.codespeak.net> Author: jlg Date: Wed Jul 11 15:01:58 2007 New Revision: 44931 Added: pypy/dist/pypy/lang/scheme/test/ex.ss Log: example scheme code for lightning talk Added: pypy/dist/pypy/lang/scheme/test/ex.ss ============================================================================== --- (empty file) +++ pypy/dist/pypy/lang/scheme/test/ex.ss Wed Jul 11 15:01:58 2007 @@ -0,0 +1,24 @@ +(define fac + (lambda (n) + (if (= n 1) + n + (* n (fac (- n 1)))))) +(fac 4) + +(define adder (lambda (x) (lambda (y) (+ x y)))) +((adder 4) 3) + +(letrec ((even? + (lambda (n) + (if (= n 0) + #t + (odd? (- n 1))))) + (odd? + (lambda (n) + (if (= n 0) + #f + (even? (- n 1)))))) + (even? 12)) + +(quit) + From fijal at codespeak.net Wed Jul 11 15:14:18 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 15:14:18 +0200 (CEST) Subject: [pypy-svn] r44932 - pypy/dist/pypy/lang/scheme Message-ID: <20070711131418.662B28150@code0.codespeak.net> Author: fijal Date: Wed Jul 11 15:14:18 2007 New Revision: 44932 Modified: pypy/dist/pypy/lang/scheme/TODO.txt Log: (emil mikulic) another typo Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Wed Jul 11 15:14:18 2007 @@ -28,5 +28,5 @@ - continuations - proper tail-recursion - mutulally recursive procedures (r5rs p.12) -- switch to bytecode generation + eval istead of evaluating AST +- switch to bytecode generation + eval instead of evaluating AST From fijal at codespeak.net Wed Jul 11 16:29:43 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 11 Jul 2007 16:29:43 +0200 (CEST) Subject: [pypy-svn] r44933 - pypy/extradoc/talk/ep2007 Message-ID: <20070711142943.5A5DB8158@code0.codespeak.net> Author: fijal Date: Wed Jul 11 16:29:41 2007 New Revision: 44933 Modified: pypy/extradoc/talk/ep2007/blob.png Log: Bah. Dia now should work. Modified: pypy/extradoc/talk/ep2007/blob.png ============================================================================== Binary files. No diff available. From antocuni at codespeak.net Wed Jul 11 16:41:18 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 11 Jul 2007 16:41:18 +0200 (CEST) Subject: [pypy-svn] r44934 - pypy/extradoc/sprintinfo/post-ep2007 Message-ID: <20070711144118.4C7198160@code0.codespeak.net> Author: antocuni Date: Wed Jul 11 16:41:14 2007 New Revision: 44934 Modified: pypy/extradoc/sprintinfo/post-ep2007/announcement.txt Log: some more possible sprint topics Modified: pypy/extradoc/sprintinfo/post-ep2007/announcement.txt ============================================================================== --- pypy/extradoc/sprintinfo/post-ep2007/announcement.txt (original) +++ pypy/extradoc/sprintinfo/post-ep2007/announcement.txt Wed Jul 11 16:41:14 2007 @@ -36,6 +36,8 @@ * some JIT improvement work * port the stackless transform to ootypesystem +* make the JIT generate fast code for float operations (??) +* experimenting in the perfect_dict direction (?????) * other interesting stuff that you would like to work on ...;-) ------------ From arigo at codespeak.net Thu Jul 12 10:32:36 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 12 Jul 2007 10:32:36 +0200 (CEST) Subject: [pypy-svn] r44947 - pypy/extradoc/talk/dyla2007 Message-ID: <20070712083236.E51888240@code0.codespeak.net> Author: arigo Date: Thu Jul 12 10:32:35 2007 New Revision: 44947 Modified: pypy/extradoc/talk/dyla2007/dyla.tex Log: Start to answer to the reviewer's comments. Modified: pypy/extradoc/talk/dyla2007/dyla.tex ============================================================================== --- pypy/extradoc/talk/dyla2007/dyla.tex (original) +++ pypy/extradoc/talk/dyla2007/dyla.tex Thu Jul 12 10:32:35 2007 @@ -107,12 +107,22 @@ foreseeable one-VM-fits-all standardization attempt. The crux of the argument is that VMs for dynamic languages should not be -written by hand! The PyPy project is the justification, +written by hand! The PyPy project \cite{pypy} is the justification, proving that the approach is feasible in practice. Just as importantly, it also brings new insights and concrete benefits in term of flexibility and performance that go beyond the state of the art. +PyPy contains a Python interpreter implemented in Python, from which +Python VMs can be generated. The reader is referred to +\cite{pypyvmconstruction} for a technical presentation. Let us emphasis +again that the argument we make in the present paper is \emph{not} that +VMs for dynamic languages should be written in their own host language +(as many projects like Squeak \cite{Squeak} and others have done) but +instead that VMs should not be \emph{written} in the first place -- they +should be generated from simple interpreters written in any suitable +high-level language. + In section \ref{sect:approaches} we will explore the way VMs are typically implemented in C and on top of OO VMs and some of the problems of these approaches, using various Python implementations as the main example. In @@ -211,7 +221,7 @@ \emph{Better performance:} So far it seems that performance of highly dynamic languages is not actually significantly improved on OO VMs. Jython is around 5 -times slower than CPython, for IronPython (which +times slower than CPython; for IronPython (which gives up on at least one feature -- frame objects -- to improve performance) the figures vary but it is mostly within the same order of magnitude as CPython. The most important reason for @@ -221,8 +231,11 @@ usage patterns, which is not a simple task. \item -\emph{Better GCs:} While this is obvious in theory, OO VMs tend to have a much -higher memory overhead to start with (XXX ref) +\emph{Better GCs:} While this is obvious in theory, OO VMs tend to have a +higher memory overhead to start with. For example, an instance of Sun's +Java VM which just loaded Jython consumes XXX MB of non-shared memory +(XXX for the JVM and an additional XXX after loading Jython), while a +CPython process fits in XXX MB. \item \emph{Cross-platform portability:} While this is true to some extent, the From arigo at codespeak.net Thu Jul 12 11:00:11 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 12 Jul 2007 11:00:11 +0200 (CEST) Subject: [pypy-svn] r44948 - pypy/extradoc/talk/dyla2007 Message-ID: <20070712090011.EEF3C824E@code0.codespeak.net> Author: arigo Date: Thu Jul 12 11:00:06 2007 New Revision: 44948 Modified: pypy/extradoc/talk/dyla2007/dyla.tex Log: Attempt... Modified: pypy/extradoc/talk/dyla2007/dyla.tex ============================================================================== --- pypy/extradoc/talk/dyla2007/dyla.tex (original) +++ pypy/extradoc/talk/dyla2007/dyla.tex Thu Jul 12 11:00:06 2007 @@ -121,7 +121,10 @@ (as many projects like Squeak \cite{Squeak} and others have done) but instead that VMs should not be \emph{written} in the first place -- they should be generated from simple interpreters written in any suitable -high-level language. +high-level\footnote{``High-level'' is taken by opposition to languages +like PreScheme \cite{kelsey-prescheme} which use the syntax and +metaprogramming facilities of a high-level language but encode +low-level details like memory management.} language. In section \ref{sect:approaches} we will explore the way VMs are typically implemented in C and on top of OO VMs and some of the problems of these From jlg at codespeak.net Thu Jul 12 11:53:38 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 11:53:38 +0200 (CEST) Subject: [pypy-svn] r44949 - pypy/dist/pypy/translator/goal Message-ID: <20070712095338.39BF18282@code0.codespeak.net> Author: jlg Date: Thu Jul 12 11:53:37 2007 New Revision: 44949 Added: pypy/dist/pypy/translator/goal/targetssstandalone.py (contents, props changed) Log: target for scheme interpreter Added: pypy/dist/pypy/translator/goal/targetssstandalone.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetssstandalone.py Thu Jul 12 11:53:37 2007 @@ -0,0 +1,35 @@ +""" +A simple standalone target for the scheme interpreter. +""" + +import autopath +import sys +from pypy.rlib.streamio import open_file_as_stream +from pypy.lang.scheme.ssparser import parse +from pypy.lang.scheme.object import SchemeQuit, ExecutionContext + +# __________ Entry point __________ + +def entry_point(argv): + if len(argv) == 2: + f = open_file_as_stream(argv[1]) + t = parse(f.readall()) + ctx = ExecutionContext() + w_retval = t.eval(ctx) + print w_retval, + return 0 + elif argv[0] == 'foo': + raise SchemeQuit + else: + print "Usage: %s schemesourcefile" % argv[0] + return 1 + +# _____ Define and setup target ___ + +def target(driver, args): + driver.exe_name = 'ss-%(backend)s' + return entry_point, None + +if __name__ == '__main__': + entry_point(sys.argv) + From mwh at codespeak.net Thu Jul 12 14:08:20 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 14:08:20 +0200 (CEST) Subject: [pypy-svn] r44950 - pypy/branch/emptying-the-malloc-zoo-2 Message-ID: <20070712120820.AADAC81CC@code0.codespeak.net> Author: mwh Date: Thu Jul 12 14:08:18 2007 New Revision: 44950 Added: pypy/branch/emptying-the-malloc-zoo-2/ - copied from r44949, pypy/dist/ Log: (pedronis, mwh) new branch for merge forward From jlg at codespeak.net Thu Jul 12 14:10:59 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 14:10:59 +0200 (CEST) Subject: [pypy-svn] r44951 - pypy/dist/pypy/lang/scheme Message-ID: <20070712121059.8DAE281DD@code0.codespeak.net> Author: jlg Date: Thu Jul 12 14:10:57 2007 New Revision: 44951 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py Log: (antocuni, jlg) - __str__ for W_Root is no more Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Thu Jul 12 14:10:57 2007 @@ -7,7 +7,7 @@ Do next ------- -- definitions: let*, letrec +- definitions: let* - symbols vs identifier, which name is better - global dict for symbols _obarray_ Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Thu Jul 12 14:10:57 2007 @@ -31,7 +31,7 @@ if to_exec == "": print raise SchemeQuit - print parse(to_exec).eval(ctx) + print parse(to_exec).eval(ctx).to_string() except SchemeQuit, e: break except BacktrackException, e: Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 12 14:10:57 2007 @@ -8,6 +8,7 @@ return "Unbound variable %s" % self.args[0] class SchemeQuit(SchemeException): + """raised on (quit) evaluation""" pass class W_Root(object): @@ -17,9 +18,6 @@ def to_boolean(self): return True - def __str__(self): - return self.to_string() - def __repr__(self): return "" Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 12 14:10:57 2007 @@ -8,7 +8,7 @@ return s.replace('\\"', '"') class SchemeParser(PackratParser): - r''' + r""" STRING: c = `\"([^\\\"]|\\\"|\\\\)*\"` IGNORE* @@ -78,7 +78,7 @@ cdr = pair return {W_Pair(car, cdr)} | return {W_Nil()}; - ''' + """ def parse(code): p = SchemeParser(code) From mwh at codespeak.net Thu Jul 12 14:15:35 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 14:15:35 +0200 (CEST) Subject: [pypy-svn] r44952 - in pypy/branch/emptying-the-malloc-zoo-2/pypy: rpython/lltypesystem rpython/memory rpython/memory/gctransform translator/c/src Message-ID: <20070712121535.A28FD81FA@code0.codespeak.net> Author: mwh Date: Thu Jul 12 14:15:34 2007 New Revision: 44952 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/llmemory.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gc.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/framework.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h Log: (pedronis, mwh and his Bazaar T-shirt) merge changes from emptying-the-malloc-zoo Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/llmemory.py Thu Jul 12 14:15:34 2007 @@ -22,8 +22,8 @@ return NotImplemented return CompositeOffset(self, other) - def raw_malloc(self, rest, zero): - raise NotImplementedError("raw_malloc(%r, %r)" % (self, rest)) + def _raw_malloc(self, rest, zero): + raise NotImplementedError("_raw_malloc(%r, %r)" % (self, rest)) def raw_memcopy(self, srcadr, dstsrc): raise NotImplementedError("raw_memcopy(%r)" % (self,)) @@ -67,7 +67,7 @@ else: raise TypeError('got %r, expected %r' % (A, self.TYPE)) - def raw_malloc(self, rest, zero): + def _raw_malloc(self, rest, zero): assert not rest if (isinstance(self.TYPE, lltype.ContainerType) and self.TYPE._gckind == 'gc'): @@ -118,12 +118,12 @@ else: return lltype.direct_fieldptr(struct, self.fldname) - def raw_malloc(self, rest, parenttype=None, zero=False): + def _raw_malloc(self, rest, parenttype=None, zero=False): if self.fldname != self.TYPE._arrayfld: # for the error msg - return AddressOffset.raw_malloc(self, rest, zero=zero) + return AddressOffset._raw_malloc(self, rest, zero=zero) assert rest - return rest[0].raw_malloc(rest[1:], parenttype=parenttype or self.TYPE, + return rest[0]._raw_malloc(rest[1:], parenttype=parenttype or self.TYPE, zero=zero) def raw_memcopy(self, srcadr, dstadr): @@ -171,8 +171,8 @@ ptr = item.ref(ptr) return ptr - def raw_malloc(self, rest, zero): - return self.offsets[0].raw_malloc(self.offsets[1:] + rest, zero=zero) + def _raw_malloc(self, rest, zero): + return self.offsets[0]._raw_malloc(self.offsets[1:] + rest, zero=zero) def raw_memcopy(self, srcadr, dstadr): for o in self.offsets[:-1]: @@ -198,7 +198,7 @@ else: return lltype.direct_arrayitems(arrayptr) - def raw_malloc(self, rest, parenttype=None, zero=False): + def _raw_malloc(self, rest, parenttype=None, zero=False): if rest: assert len(rest) == 1 assert isinstance(rest[0], ItemOffset) @@ -245,11 +245,11 @@ gcptr = self.gcheaderbuilder.object_from_header(headerptr) return gcptr - def raw_malloc(self, rest, zero): + def _raw_malloc(self, rest, zero): assert rest if isinstance(rest[0], GCHeaderAntiOffset): - return rest[1].raw_malloc(rest[2:], zero=zero) # just for fun - gcobjadr = rest[0].raw_malloc(rest[1:], zero=zero) + return rest[1]._raw_malloc(rest[2:], zero=zero) # just for fun + gcobjadr = rest[0]._raw_malloc(rest[1:], zero=zero) headerptr = self.gcheaderbuilder.new_header(gcobjadr.ptr) return cast_ptr_to_adr(headerptr) @@ -271,10 +271,10 @@ headerptr = self.gcheaderbuilder.header_of_object(gcptr) return headerptr - def raw_malloc(self, rest, zero): + def _raw_malloc(self, rest, zero): assert len(rest) >= 2 assert isinstance(rest[0], GCHeaderOffset) - return rest[1].raw_malloc(rest[2:], zero=zero) + return rest[1]._raw_malloc(rest[2:], zero=zero) # ____________________________________________________________ @@ -543,7 +543,7 @@ def raw_malloc(size): if not isinstance(size, AddressOffset): raise NotImplementedError(size) - return size.raw_malloc([], zero=False) + return size._raw_malloc([], zero=False) def raw_free(adr): # try to free the whole object if 'adr' is the address of the header @@ -568,7 +568,7 @@ if not isinstance(size, AddressOffset): raise NotImplementedError(size) assert lltype.typeOf(adr) == Address - zeroadr = size.raw_malloc([], zero=True) + zeroadr = size._raw_malloc([], zero=True) size.raw_memcopy(zeroadr, adr) def raw_memcopy(source, dest, size): @@ -615,7 +615,7 @@ self.unitsize = unitsize self.n = n - def raw_malloc(self, rest, zero=False): + def _raw_malloc(self, rest, zero=False): assert not rest arena = _arena(self, zero=zero) return arena.getitemaddr(0) Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py Thu Jul 12 14:15:34 2007 @@ -339,7 +339,7 @@ # __________ address operations __________ - 'raw_malloc': LLOp(canraise=(MemoryError,)), + 'raw_malloc': LLOp(), 'raw_malloc_usage': LLOp(sideeffects=False), 'raw_free': LLOp(), 'raw_memclear': LLOp(), Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gc.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gc.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gc.py Thu Jul 12 14:15:34 2007 @@ -4,7 +4,7 @@ from pypy.rpython.memory.gcheader import GCHeaderBuilder from pypy.rpython.memory import lltypesimulation from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rlib.objectmodel import free_non_gc_object +from pypy.rlib.objectmodel import free_non_gc_object, debug_assert from pypy.rpython.lltypesystem.lloperation import llop from pypy.rlib.rarithmetic import ovfcheck @@ -97,7 +97,10 @@ size = self.fixed_size(typeid) if self.is_varsize(typeid): size += length * self.varsize_item_sizes(typeid) - return raw_malloc(size) + result = raw_malloc(size) + if not result: + raise memoryError + return result def collect(self): self.get_roots() #this is there so that the annotator thinks get_roots is a function @@ -182,6 +185,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) hdr.typeid = typeid << 1 if has_finalizer: @@ -208,6 +213,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError raw_memclear(result, tot_size) hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) hdr.typeid = typeid << 1 @@ -238,6 +245,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError (result + size_gc_header + offset_to_length).signed[0] = length hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) hdr.typeid = typeid << 1 @@ -270,6 +279,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError raw_memclear(result, tot_size) (result + size_gc_header + offset_to_length).signed[0] = length hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) @@ -942,8 +953,10 @@ def setup(self): self.tospace = raw_malloc(self.space_size) + debug_assert(bool(self.tospace), "couldn't allocate tospace") self.top_of_space = self.tospace + self.space_size self.fromspace = raw_malloc(self.space_size) + debug_assert(bool(self.fromspace), "couldn't allocate fromspace") self.free = self.tospace def free_memory(self): @@ -1125,6 +1138,8 @@ size_gc_header = self.size_gc_header() result = raw_malloc(size + size_gc_header) ## print "mallocing %s, size %s at %s" % (typeid, size, result) + if not result: + raise memoryError result.signed[0] = 0 # refcount result.signed[1] = typeid return result + size_gc_header Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/framework.py Thu Jul 12 14:15:34 2007 @@ -6,6 +6,7 @@ from pypy.rpython.memory import gc, lladdress from pypy.rpython.memory.gcheader import GCHeaderBuilder from pypy.rlib.rarithmetic import ovfcheck +from pypy.rlib.objectmodel import debug_assert from pypy.translator.backendopt import graphanalyze from pypy.annotation import model as annmodel from pypy.rpython import annlowlevel @@ -251,6 +252,7 @@ _alloc_flavor_ = 'raw' def setup_root_stack(): stackbase = lladdress.raw_malloc(rootstacksize) + debug_assert(bool(stackbase), "could not allocate root stack") lladdress.raw_memclear(stackbase, rootstacksize) gcdata.root_stack_top = stackbase gcdata.root_stack_base = stackbase Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py Thu Jul 12 14:15:34 2007 @@ -45,7 +45,9 @@ # create incref, etc graph + memoryError = MemoryError() HDRPTR = lltype.Ptr(self.HDR) + def ll_incref(adr): if adr: gcheader = llmemory.cast_adr_to_ptr(adr - gc_header_offset, HDRPTR) @@ -71,6 +73,8 @@ def ll_malloc_fixedsize(size): size = gc_header_offset + size result = lladdress.raw_malloc(size) + if not result: + raise memoryError lladdress.raw_memclear(result, size) result += gc_header_offset return result @@ -80,8 +84,10 @@ varsize = ovfcheck(itemsize * length) tot_size = ovfcheck(fixsize + varsize) except OverflowError: - raise MemoryError + raise memoryError result = lladdress.raw_malloc(tot_size) + if not result: + raise memoryError lladdress.raw_memclear(result, tot_size) result += gc_header_offset return result Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h Thu Jul 12 14:15:34 2007 @@ -8,11 +8,7 @@ #define OP_RAW_MALLOC(size, r, restype) { \ r = (restype) PyObject_Malloc(size); \ - if (r == NULL) { \ - FAIL_EXCEPTION(PyExc_MemoryError, \ - "out of memory"); \ - } \ - else { \ + if (r != NULL) { \ memset((void*)r, 0, size); \ COUNT_MALLOC; \ } \ @@ -22,13 +18,9 @@ #define OP_RAW_MALLOC(size, r, restype) { \ r = (restype) PyObject_Malloc(size); \ - if (r == NULL) { \ - FAIL_EXCEPTION(PyExc_MemoryError, \ - "out of memory"); \ - } \ - else { \ + if (r != NULL) { \ COUNT_MALLOC; \ - } \ + } \ } #endif From antocuni at codespeak.net Thu Jul 12 14:18:36 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 12 Jul 2007 14:18:36 +0200 (CEST) Subject: [pypy-svn] r44953 - pypy/dist/pypy/lang/scheme Message-ID: <20070712121836.7639B81FA@code0.codespeak.net> Author: antocuni Date: Thu Jul 12 14:18:35 2007 New Revision: 44953 Modified: pypy/dist/pypy/lang/scheme/object.py Log: (antocuni, jlg) dict() is not rpython Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 12 14:18:35 2007 @@ -424,7 +424,7 @@ """ def __init__(self, globalscope=None, scope=None, closure=False): if globalscope is None: - self.globalscope = dict(OPERATION_MAP) + self.globalscope = OPERATION_MAP.copy() else: self.globalscope = globalscope From simonb at codespeak.net Thu Jul 12 14:26:35 2007 From: simonb at codespeak.net (simonb at codespeak.net) Date: Thu, 12 Jul 2007 14:26:35 +0200 (CEST) Subject: [pypy-svn] r44954 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070712122635.2FE8B823D@code0.codespeak.net> Author: simonb Date: Thu Jul 12 14:26:30 2007 New Revision: 44954 Added: pypy/dist/pypy/rpython/tool/ pypy/dist/pypy/rpython/tool/mkrffi.py (contents, props changed) pypy/dist/pypy/rpython/tool/test/ Log: start on rffi code generator Added: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Thu Jul 12 14:26:30 2007 @@ -0,0 +1,55 @@ +#!/usr/bin/env python + +import ctypes + +import py +from py.code import Source + + +def proc_func(func): + name = func.__name__ + src = Source(""" + c_%s = rffi.llexternal('%s', [rffi.INT], + lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + """%(name, name)) + return src + +def proc_module(module): + + ns = module.__dict__ + + for key, value in ns.items(): + print "found:", key + if isinstance(value, ctypes._CFuncPtr): + proc_func(value) + + +def test_func(): + + lib = ctypes.CDLL('/usr/lib/libc.dylib') + func = lib.malloc + func.argtypes = [ctypes.c_int] + func.restype = ctypes.c_voidp + + src = proc_func(func) + assert isinstance(src, Source) + _src = Source(""" + c_malloc = rffi.llexternal('malloc', [rffi.INT], + lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + """) + + assert src == _src + +if __name__ == "__main__": + test_1() + + + + + + + + + + + From fijal at codespeak.net Thu Jul 12 14:34:34 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 14:34:34 +0200 (CEST) Subject: [pypy-svn] r44955 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070712123434.928508245@code0.codespeak.net> Author: fijal Date: Thu Jul 12 14:34:33 2007 New Revision: 44955 Removed: pypy/dist/pypy/rpython/lltypesystem/typecache.py Modified: pypy/dist/pypy/rpython/lltypesystem/rfficache.py pypy/dist/pypy/rpython/lltypesystem/test/test_rfficache.py Log: Kill typecache.py Modified: pypy/dist/pypy/rpython/lltypesystem/rfficache.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rfficache.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rfficache.py Thu Jul 12 14:34:33 2007 @@ -1,6 +1,6 @@ -""" This file creates and maintains _cache/rtypes.py, which -keeps information about C type sizes on various platforms +""" This file creates and maintains _cache/stdtypes.py, which +keeps information about C type sizes """ import py @@ -32,13 +32,6 @@ pipe.wait() return int(pipe.stdout.read()) * 8 -def machine_key(): - """ Key unique to machine type, but general enough to share - it between eg different kernels - """ - import platform - return platform.processor(), platform.architecture(), platform.system() - # XXX add float types as well here TYPES = [] @@ -53,31 +46,27 @@ assert isinstance(d, dict) return "{\n%s,\n}" % ",\n".join(["%r:%r" % (k, v) for k, v in d.items()]) -def get_type_sizes(filename, platform_key=machine_key(), types=TYPES, - compiler_exe=None): +def get_type_sizes(filename, compiler_exe=None): try: mod = {} exec py.path.local(filename).read() in mod - platforms = mod['platforms'] + types = mod['types'] except (ImportError, py.error.ENOENT): - platforms = {} + types = {} try: - result = platforms[platform_key] - if py.builtin.sorted(result.keys()) != py.builtin.sorted(TYPES): + if py.builtin.sorted(types.keys()) != py.builtin.sorted(TYPES): # invalidate file - platforms = {} + types = {} raise KeyError - return result + return types except KeyError: - value = dict([(i, sizeof_c_type(i, compiler_exe=compiler_exe)) - for i in types]) - platforms[platform_key] = value - comment = "# this is automatically generated cache files for c types\n" - py.path.local(filename).write(comment + 'platforms = ' + - newline_repr(platforms) + "\n") - return value + types = dict([(i, sizeof_c_type(i, compiler_exe=compiler_exe)) + for i in TYPES]) + py.path.local(filename).write('types = ' + + repr(types) + "\n") + return types from pypy.tool import autopath -CACHE = py.magic.autopath().dirpath().join('typecache.py') +CACHE = py.magic.autopath()/'..'/'..'/'..'/'_cache'/'stdtypes.py' platform = get_type_sizes(CACHE) Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_rfficache.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_rfficache.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_rfficache.py Thu Jul 12 14:34:33 2007 @@ -14,14 +14,6 @@ res = get_type_sizes(tmpfile, compiler_exe='xxx') assert res['char'] == 8 -def test_gettypesizes_platforms(): - tmpfile = udir.join("plat.py") - tmpfile.write(py.code.Source(""" - platforms = {'xxx':{%s}} - """ % ", ".join(['"%s":4' % i for i in TYPES]))) - assert get_type_sizes(tmpfile)['char'] == 8 - assert get_type_sizes(tmpfile, platform_key='xxx', compiler_exe='xxx')['char'] == 4 - def test_types_present(): for name in TYPES: if name.startswith('unsigned'): From fijal at codespeak.net Thu Jul 12 14:44:11 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 14:44:11 +0200 (CEST) Subject: [pypy-svn] r44956 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070712124411.6FB8981D4@code0.codespeak.net> Author: fijal Date: Thu Jul 12 14:44:10 2007 New Revision: 44956 Added: pypy/dist/pypy/rpython/tool/__init__.py (contents, props changed) pypy/dist/pypy/rpython/tool/test/__init__.py (contents, props changed) pypy/dist/pypy/rpython/tool/test/test_mkrffi.py (contents, props changed) Modified: pypy/dist/pypy/rpython/tool/ (props changed) pypy/dist/pypy/rpython/tool/mkrffi.py (contents, props changed) pypy/dist/pypy/rpython/tool/test/ (props changed) Log: Shuffle files around, fixeol Added: pypy/dist/pypy/rpython/tool/__init__.py ============================================================================== Modified: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Thu Jul 12 14:44:10 2007 @@ -23,23 +23,6 @@ if isinstance(value, ctypes._CFuncPtr): proc_func(value) - -def test_func(): - - lib = ctypes.CDLL('/usr/lib/libc.dylib') - func = lib.malloc - func.argtypes = [ctypes.c_int] - func.restype = ctypes.c_voidp - - src = proc_func(func) - assert isinstance(src, Source) - _src = Source(""" - c_malloc = rffi.llexternal('malloc', [rffi.INT], - lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) - """) - - assert src == _src - if __name__ == "__main__": test_1() Added: pypy/dist/pypy/rpython/tool/test/__init__.py ============================================================================== Added: pypy/dist/pypy/rpython/tool/test/test_mkrffi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Thu Jul 12 14:44:10 2007 @@ -0,0 +1,19 @@ + +import ctypes +from pypy.rpython.tool.mkrffi import * + +def test_func(): + + lib = ctypes.CDLL('libc.so.6') + func = lib.malloc + func.argtypes = [ctypes.c_int] + func.restype = ctypes.c_voidp + + src = proc_func(func) + assert isinstance(src, Source) + _src = Source(""" + c_malloc = rffi.llexternal('malloc', [rffi.INT], + lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + """) + + assert src == _src From jlg at codespeak.net Thu Jul 12 14:45:56 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 14:45:56 +0200 (CEST) Subject: [pypy-svn] r44957 - pypy/dist/pypy/lang/scheme Message-ID: <20070712124556.9AE1D81FA@code0.codespeak.net> Author: jlg Date: Thu Jul 12 14:45:51 2007 New Revision: 44957 Modified: pypy/dist/pypy/lang/scheme/object.py Log: (antocuni, jlg) - W_Callable super class for W_Macro, W_Procedure; call instead of eval to call a W_Callable Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 12 14:45:51 2007 @@ -131,20 +131,25 @@ def eval(self, ctx): oper = self.car.eval(ctx) - return oper.eval(ctx, self.cdr) + assert isinstance(oper, W_Callable) + return oper.call(ctx, self.cdr) class W_Nil(W_Root): def to_string(self): return "()" -class W_Procedure(W_Root): +class W_Callable(W_Root): + def call(self, ctx, lst): + raise NotImplementedError + +class W_Procedure(W_Callable): def __init__(self, pname=""): self.pname = pname def to_string(self): return "#" % (self.pname,) - def eval(self, ctx, lst): + def call(self, ctx, lst): #evaluate all arguments into list arg_lst = [] arg = lst @@ -157,14 +162,14 @@ def procedure(self, ctx, lst): raise NotImplementedError -class W_Macro(W_Root): +class W_Macro(W_Callable): def __init__(self, pname=""): self.pname = pname def to_string(self): return "#" % (self.pname,) - def eval(self, ctx, lst=None): + def call(self, ctx, lst=None): raise NotImplementedError class W_Lambda(W_Procedure): @@ -258,7 +263,7 @@ return plst2lst(lst) class Define(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): w_identifier = lst.car assert isinstance(w_identifier, W_Identifier) @@ -267,7 +272,7 @@ return w_val class Sete(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): w_identifier = lst.car assert isinstance(w_identifier, W_Identifier) @@ -276,7 +281,7 @@ return w_val class MacroIf(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): w_condition = lst.car w_then = lst.cdr.car if isinstance(lst.cdr.cdr, W_Nil): @@ -318,13 +323,13 @@ raise SchemeQuit class Lambda(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): w_args = lst.car w_body = lst.cdr #.car return W_Lambda(w_args, w_body, ctx.copy()) class Let(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): local_ctx = ctx.copy() w_formal = lst.car while not isinstance(w_formal, W_Nil): @@ -343,7 +348,7 @@ return body_result class Letrec(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): local_ctx = ctx.copy() #bound variables @@ -373,7 +378,7 @@ return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) class Quote(W_Macro): - def eval(self, ctx, lst): + def call(self, ctx, lst): return lst.car ## @@ -436,7 +441,7 @@ self.closure = closure def copy(self): - return ExecutionContext(self.globalscope, dict(self.scope), True) + return ExecutionContext(self.globalscope, self.scope.copy(), True) def get(self, name): loc = self.scope.get(name, None) From fijal at codespeak.net Thu Jul 12 15:04:38 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 15:04:38 +0200 (CEST) Subject: [pypy-svn] r44958 - pypy/dist/pypy/rpython/tool/test Message-ID: <20070712130438.67D658240@code0.codespeak.net> Author: fijal Date: Thu Jul 12 15:04:36 2007 New Revision: 44958 Added: pypy/dist/pypy/rpython/tool/test/test_c.py (contents, props changed) Log: A bit of a testing utility Added: pypy/dist/pypy/rpython/tool/test/test_c.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/tool/test/test_c.py Thu Jul 12 15:04:36 2007 @@ -0,0 +1,25 @@ + +import os +import sys +from pypy.tool.udir import udir +from distutils import ccompiler +import py +import ctypes + +c_source = """ +void *int_to_void_p(int arg) {} +""" + +class TestBasic: + def setup_class(cls): + compiler = ccompiler.new_compiler() + c_file = udir.join('rffilib.c') + c_file.write(c_source) + compiler.compile([str(c_file)], output_dir='/') + compiler.link_shared_lib([str(udir.join('rffilib.o'))], + 'rffi', output_dir=str(udir)) + cls.lib = ctypes.CDLL(str(udir.join('librffi.so'))) + + def test_basic(self): + assert self.lib + From cfbolz at codespeak.net Thu Jul 12 15:06:14 2007 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jul 2007 15:06:14 +0200 (CEST) Subject: [pypy-svn] r44959 - in pypy/dist/pypy/rlib/parsing: . test Message-ID: <20070712130614.D18568240@code0.codespeak.net> Author: cfbolz Date: Thu Jul 12 15:06:13 2007 New Revision: 44959 Modified: pypy/dist/pypy/rlib/parsing/makepackrat.py pypy/dist/pypy/rlib/parsing/test/test_translate.py Log: make pypackrat grammars that use regular expressions translate too Modified: pypy/dist/pypy/rlib/parsing/makepackrat.py ============================================================================== --- pypy/dist/pypy/rlib/parsing/makepackrat.py (original) +++ pypy/dist/pypy/rlib/parsing/makepackrat.py Thu Jul 12 15:06:13 2007 @@ -414,7 +414,10 @@ self.emit("raise BacktrackException") self.end_block("if") self.emit("_upto = _runner.last_matched_index + 1") - self.emit("_result = self._inputstream[self._pos: _upto]") + self.emit("_pos = self._pos") + self.emit("assert _pos >= 0") + self.emit("assert _upto >= 0") + self.emit("_result = self._inputstream[_pos: _upto]") self.emit("self._pos = _upto") self.emit("return _result") @@ -654,14 +657,19 @@ raise Exception("must import BacktrackException") if 'Status' not in frame.f_globals: raise Exception("must import Status") + result = type.__new__(cls, name_, bases, dct) for key, value in pcls.__dict__.iteritems(): + if isinstance(value, type): + value.__module__ = result.__module__ #XXX help the annotator if isinstance(value, type(lambda: None)): value = new.function(value.func_code, frame.f_globals) - if key not in dct and key not in forbidden: - dct[key] = value - dct['init_parser'] = pcls.__dict__['__init__'] - dct['_code'] = visitor.get_code() - return type.__new__(cls, name_, bases, dct) + if not hasattr(result, key) and key not in forbidden: + setattr(result, key, value) + if result.__init__ is object.__init__: + result.__init__ = pcls.__dict__['__init__'] + result.init_parser = pcls.__dict__['__init__'] + result._code = visitor.get_code() + return result class PackratParser(object): __metaclass__ = MetaPackratParser Modified: pypy/dist/pypy/rlib/parsing/test/test_translate.py ============================================================================== --- pypy/dist/pypy/rlib/parsing/test/test_translate.py (original) +++ pypy/dist/pypy/rlib/parsing/test/test_translate.py Thu Jul 12 15:06:13 2007 @@ -158,3 +158,26 @@ res = func("5-5-5") assert res == '((5 - 5) - 5)' +def test_translate_pypackrat_regex(): + from pypy.rlib.parsing.pypackrat import PackratParser + class parser(PackratParser): + """ + num: + `([1-9][0-9]*)|0`; + """ + print parser._code + def parse(s): + p = parser(s) + return p.num() + res = parse("1234") + assert res == '1234' + t = Translation(parse) + t.annotate([str]) + t.rtype() + t.backendopt() + t.view() + func = t.compile_c() + res = func("12345") + assert res == '12345' + res = func("0") + assert res == '0' From fijal at codespeak.net Thu Jul 12 15:08:14 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 15:08:14 +0200 (CEST) Subject: [pypy-svn] r44960 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070712130814.610B48245@code0.codespeak.net> Author: fijal Date: Thu Jul 12 15:08:13 2007 New Revision: 44960 Modified: pypy/dist/pypy/rpython/tool/mkrffi.py pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Log: Make test a bit more regular (but still quite vague) Modified: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Thu Jul 12 15:08:13 2007 @@ -10,7 +10,7 @@ name = func.__name__ src = Source(""" c_%s = rffi.llexternal('%s', [rffi.INT], - lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) """%(name, name)) return src Modified: pypy/dist/pypy/rpython/tool/test/test_mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/test/test_mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Thu Jul 12 15:08:13 2007 @@ -1,19 +1,19 @@ import ctypes from pypy.rpython.tool.mkrffi import * +from pypy.rpython.tool.test.test_c import TestBasic -def test_func(): +class TestMkrffi(TestBasic): + def test_single_func(self): + func = self.lib.int_to_void_p + func.argtypes = [ctypes.c_int] + func.restype = ctypes.c_voidp - lib = ctypes.CDLL('libc.so.6') - func = lib.malloc - func.argtypes = [ctypes.c_int] - func.restype = ctypes.c_voidp + src = proc_func(func) + assert isinstance(src, Source) + _src = Source(""" + c_int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], + lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + """) - src = proc_func(func) - assert isinstance(src, Source) - _src = Source(""" - c_malloc = rffi.llexternal('malloc', [rffi.INT], - lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) - """) - - assert src == _src + assert src == _src, str(src) + "\n" + str(_src) From fijal at codespeak.net Thu Jul 12 15:14:01 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 15:14:01 +0200 (CEST) Subject: [pypy-svn] r44961 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070712131401.D91B481D2@code0.codespeak.net> Author: fijal Date: Thu Jul 12 15:14:00 2007 New Revision: 44961 Modified: pypy/dist/pypy/rpython/tool/mkrffi.py pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Log: Make this a bit less vague Modified: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Thu Jul 12 15:14:00 2007 @@ -3,36 +3,26 @@ import ctypes import py -from py.code import Source +def primitive_pointer_repr(tp_s): + return 'lltype.Ptr(lltype.FixedSizeArray(%s, 1))' % tp_s + +# XXX any automatic stuff here? +SIMPLE_TYPE_MAPPING = { + ctypes.c_int : 'rffi.INT', + ctypes.c_voidp : primitive_pointer_repr('lltype.Void') +} + +def proc_tp(tp): + try: + return SIMPLE_TYPE_MAPPING[tp] + except KeyError: + raise NotImplementedError("Not implemented mapping for %s" % tp) def proc_func(func): name = func.__name__ - src = Source(""" - c_%s = rffi.llexternal('%s', [rffi.INT], - lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) - """%(name, name)) + src = py.code.Source(""" + c_%s = rffi.llexternal('%s', [%s], %s) + """%(name, name, ",".join([proc_tp(arg) for arg in func.argtypes]), + proc_tp(func.restype))) return src - -def proc_module(module): - - ns = module.__dict__ - - for key, value in ns.items(): - print "found:", key - if isinstance(value, ctypes._CFuncPtr): - proc_func(value) - -if __name__ == "__main__": - test_1() - - - - - - - - - - - Modified: pypy/dist/pypy/rpython/tool/test/test_mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/test/test_mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Thu Jul 12 15:14:00 2007 @@ -2,6 +2,7 @@ import ctypes from pypy.rpython.tool.mkrffi import * from pypy.rpython.tool.test.test_c import TestBasic +import py class TestMkrffi(TestBasic): def test_single_func(self): @@ -10,10 +11,8 @@ func.restype = ctypes.c_voidp src = proc_func(func) - assert isinstance(src, Source) - _src = Source(""" - c_int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], - lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + _src = py.code.Source(""" + c_int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) """) assert src == _src, str(src) + "\n" + str(_src) From mwh at codespeak.net Thu Jul 12 15:14:14 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 15:14:14 +0200 (CEST) Subject: [pypy-svn] r44962 - pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test Message-ID: <20070712131414.8B34B81E2@code0.codespeak.net> Author: mwh Date: Thu Jul 12 15:14:13 2007 New Revision: 44962 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py Log: (mwh, pedronis) cruft-- (in very small units) Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py Thu Jul 12 15:14:13 2007 @@ -5,16 +5,13 @@ from pypy.translator.translator import TranslationContext, graphof from pypy import conftest -class RefcountingGcPolicy2(RefcountingGcPolicy): - transformerclass = RefcountingGCTransformer - def llinterpreter_for_refcounted_graph(f, args_s): from pypy.rpython.llinterp import LLInterpreter from pypy.translator.c.genc import CStandaloneBuilder from pypy.translator.c import gc t = rtype(f, args_s) - cbuild = CStandaloneBuilder(t, f, t.config, gcpolicy=RefcountingGcPolicy2) + cbuild = CStandaloneBuilder(t, f, t.config, gcpolicy=RefcountingGcPolicy) db = cbuild.generate_graphs_for_llinterp() graph = cbuild.getentrypointptr()._obj.graph llinterp = LLInterpreter(t.rtyper) From mwh at codespeak.net Thu Jul 12 15:21:13 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 15:21:13 +0200 (CEST) Subject: [pypy-svn] r44963 - pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test Message-ID: <20070712132113.9434781E2@code0.codespeak.net> Author: mwh Date: Thu Jul 12 15:21:12 2007 New Revision: 44963 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py Log: (pedronis, mwh) these lines were useless and confusing Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py Thu Jul 12 15:21:12 2007 @@ -18,10 +18,6 @@ if conftest.option.view: t.view() return llinterp, graph - res = llinterp.eval_graph(graph, [0]) - assert res == f(0) - res = llinterp.eval_graph(graph, [1]) - assert res == f(1) def test_llinterp_refcounted_graph(): From jlg at codespeak.net Thu Jul 12 15:31:26 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 15:31:26 +0200 (CEST) Subject: [pypy-svn] r44964 - pypy/dist/pypy/lang/scheme Message-ID: <20070712133126.6A3FF8201@code0.codespeak.net> Author: jlg Date: Thu Jul 12 15:31:25 2007 New Revision: 44964 Modified: pypy/dist/pypy/lang/scheme/object.py Log: (antocuni, jlg) - Formal calls for lambda arfuments, ListOper -> RPython Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 12 15:31:25 2007 @@ -172,18 +172,23 @@ def call(self, ctx, lst=None): raise NotImplementedError +class Formal(object): + def __init__(self, name, islist=False): + self.name = name + self.islist = islist + class W_Lambda(W_Procedure): def __init__(self, args, body, closure, pname="#f"): self.args = [] arg = args while not isinstance(arg, W_Nil): if isinstance(arg, W_Identifier): - self.args.append([arg.to_string()]) + self.args.append(Formal(arg.to_string(), True)) break else: assert isinstance(arg.car, W_Identifier) #list of argument names, not evaluated - self.args.append(arg.car.to_string()) + self.args.append(Formal(arg.car.to_string(), False)) arg = arg.cdr self.body = body @@ -200,11 +205,11 @@ #set lambda arguments for idx in range(len(self.args)): - name = self.args[idx] - if isinstance(name, list): - local_ctx.put(name[0], plst2lst(lst[idx:])) + formal = self.args[idx] + if formal.islist: + local_ctx.put(formal.name, plst2lst(lst[idx:])) else: - local_ctx.put(name, lst[idx]) + local_ctx.put(formal.name, lst[idx]) body_expression = self.body body_result = None @@ -231,17 +236,23 @@ acc = None for arg in lst: if acc is None: - acc = arg.eval(ctx).to_number() + acc = arg.eval(ctx) else: - acc = self.oper(acc, arg.eval(ctx).to_number()) + acc = self.oper(acc, arg.eval(ctx)) + + return acc - if isinstance(acc, int): - return W_Fixnum(acc) + def oper(self, x, y): + if isinstance(x, W_Float) or isinstance(y, W_Float): + return W_Float(self.do_oper_float(x.to_number(), y.to_number())) else: - return W_Float(acc) + return W_Fixnum(self.do_oper_int(x.to_number(), y.to_number())) class Add(ListOper): - def oper(self, x, y): + def do_oper_int(self, x, y): + return x + y + + def do_oper_float(self, x, y): return x + y class Sub(ListOper): @@ -251,11 +262,17 @@ else: return ListOper.procedure(self, ctx, lst) - def oper(self, x, y): + def do_oper_int(self, x, y): + return x - y + + def do_oper_float(self, x, y): return x - y class Mul(ListOper): - def oper(self, x, y): + def do_oper_int(self, x, y): + return x * y + + def do_oper_float(self, x, y): return x * y class List(W_Procedure): From fijal at codespeak.net Thu Jul 12 15:36:57 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 15:36:57 +0200 (CEST) Subject: [pypy-svn] r44965 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20070712133657.DF4CA8201@code0.codespeak.net> Author: fijal Date: Thu Jul 12 15:36:55 2007 New Revision: 44965 Modified: pypy/dist/pypy/rpython/lltypesystem/rfficache.py Log: Create _cache in case it's not there Modified: pypy/dist/pypy/rpython/lltypesystem/rfficache.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rfficache.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rfficache.py Thu Jul 12 15:36:55 2007 @@ -66,6 +66,9 @@ repr(types) + "\n") return types +import pypy +import py +py.path.local(pypy.__file__).new(basename='_cache').ensure(dir=1) from pypy.tool import autopath CACHE = py.magic.autopath()/'..'/'..'/'..'/'_cache'/'stdtypes.py' platform = get_type_sizes(CACHE) From mwh at codespeak.net Thu Jul 12 15:40:22 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 15:40:22 +0200 (CEST) Subject: [pypy-svn] r44966 - pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c Message-ID: <20070712134022.748B58201@code0.codespeak.net> Author: mwh Date: Thu Jul 12 15:40:21 2007 New Revision: 44966 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py Log: move BasicGcPolicy fractionally towards being complete Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py Thu Jul 12 15:40:21 2007 @@ -48,6 +48,15 @@ def gc_startup_code(self): return [] + def struct_setup(self, structdefnode, rtti): + return None + + def array_setup(self, arraydefnode): + return None + + def rtti_type(self): + return '' + def OP_GC_PUSH_ALIVE_PYOBJ(self, funcgen, op): expr = funcgen.expr(op.args[0]) if expr == 'NULL': From mwh at codespeak.net Thu Jul 12 15:43:45 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 15:43:45 +0200 (CEST) Subject: [pypy-svn] r44967 - in pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython: . lltypesystem memory/gctransform memory/gctransform/test Message-ID: <20070712134345.9F71A823E@code0.codespeak.net> Author: mwh Date: Thu Jul 12 15:43:43 2007 New Revision: 44967 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/llinterp.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_boehm.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_transform.py Log: (pedronis, mwh) make the boehm gc transformer do most of the things that are now done in the backend. add more tests for the boehm gc transformer, and support for a couple of new operations to the llinterp. Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/llinterp.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/llinterp.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/llinterp.py Thu Jul 12 15:43:43 2007 @@ -758,6 +758,8 @@ assert lltype.typeOf(size) == lltype.Signed return self.heap.raw_malloc(size) + op_boehm_malloc = op_boehm_malloc_atomic = op_raw_malloc + def op_raw_malloc_usage(self, size): assert lltype.typeOf(size) == lltype.Signed return self.heap.raw_malloc_usage(size) Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py Thu Jul 12 15:43:43 2007 @@ -339,6 +339,8 @@ # __________ address operations __________ + 'boehm_malloc': LLOp(), + 'boehm_malloc_atomic': LLOp(), 'raw_malloc': LLOp(), 'raw_malloc_usage': LLOp(sideeffects=False), 'raw_free': LLOp(), Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py Thu Jul 12 15:43:43 2007 @@ -3,11 +3,51 @@ get_rtti, _static_deallocator_body_for_type, LLTransformerOp, ll_call_destructor from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.lltypesystem.lloperation import llop +from pypy.rpython import rmodel +from pypy.rlib.rarithmetic import ovfcheck class BoehmGCTransformer(GCTransformer): def __init__(self, translator, inline=False): super(BoehmGCTransformer, self).__init__(translator, inline=inline) self.finalizer_funcptrs = {} + memoryError = MemoryError() + + def ll_malloc_fixedsize(size): + result = llop.boehm_malloc(llmemory.Address, size) + if not result: + raise memoryError + return result + def ll_malloc_fixedsize_atomic(size): + result = llop.boehm_malloc_atomic(llmemory.Address, size) + if not result: + raise memoryError + return result + # XXX, do we need/want an atomic version of this function? + def ll_malloc_varsize_no_length(length, size, itemsize): + try: + varsize = ovfcheck(itemsize * length) + tot_size = ovfcheck(size + varsize) + except OverflowError: + raise memoryError + result = llop.boehm_malloc(llmemory.Address, tot_size) + if not result: + raise memoryError + return result + def ll_malloc_varsize(length, size, itemsize, lengthoffset): + result = ll_malloc_varsize_no_length(length, size, itemsize) + (result + lengthoffset).signed[0] = length + return result + + if self.translator: + self.malloc_fixedsize_ptr = self.inittime_helper( + ll_malloc_fixedsize, [lltype.Signed], llmemory.Address) + self.malloc_fixedsize_atomic_ptr = self.inittime_helper( + ll_malloc_fixedsize_atomic, [lltype.Signed], llmemory.Address) + self.malloc_varsize_no_length_ptr = self.inittime_helper( + ll_malloc_varsize_no_length, [lltype.Signed]*3, llmemory.Address) + self.malloc_varsize_ptr = self.inittime_helper( + ll_malloc_varsize, [lltype.Signed]*4, llmemory.Address) + self.mixlevelannotator.finish() # for now def push_alive_nopyobj(self, var, llops): pass @@ -23,6 +63,53 @@ """ for boehm it is enough to do nothing""" pass + def gct_malloc(self, hop): + TYPE = hop.spaceop.result.concretetype.TO + assert not TYPE._is_varsize() + c_size = rmodel.inputconst(lltype.Signed, llmemory.sizeof(TYPE)) + if TYPE._is_atomic(): + funcptr = self.malloc_fixedsize_atomic_ptr + else: + funcptr = self.malloc_fixedsize_ptr + v_raw = hop.genop("direct_call", [funcptr, c_size], + resulttype=llmemory.Address) + hop.cast_result(v_raw) + + def gct_malloc_varsize(self, hop): + def intconst(c): return rmodel.inputconst(lltype.Signed, c) + + op = hop.spaceop + TYPE = op.result.concretetype.TO + assert TYPE._is_varsize() + + if isinstance(TYPE, lltype.Struct): + ARRAY = TYPE._flds[TYPE._arrayfld] + else: + ARRAY = TYPE + assert isinstance(ARRAY, lltype.Array) + if ARRAY._hints.get('isrpystring', False): + c_const_size = intconst(llmemory.sizeof(TYPE, 1)) + else: + c_const_size = intconst(llmemory.sizeof(TYPE, 0)) + c_item_size = intconst(llmemory.sizeof(ARRAY.OF)) + + if ARRAY._hints.get("nolength", False): + v_raw = hop.genop("direct_call", + [self.malloc_varsize_no_length_ptr, op.args[-1], + c_const_size, c_item_size], + resulttype=llmemory.Address) + else: + if isinstance(TYPE, lltype.Struct): + offset_to_length = llmemory.FieldOffset(TYPE, TYPE._arrayfld) + \ + llmemory.ArrayLengthOffset(ARRAY) + else: + offset_to_length = llmemory.ArrayLengthOffset(ARRAY) + v_raw = hop.genop("direct_call", + [self.malloc_varsize_ptr, op.args[-1], + c_const_size, c_item_size, intconst(offset_to_length)], + resulttype=llmemory.Address) + hop.cast_result(v_raw) + def finalizer_funcptr_for_type(self, TYPE): if TYPE in self.finalizer_funcptrs: return self.finalizer_funcptrs[TYPE] Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_boehm.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_boehm.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_boehm.py Thu Jul 12 15:43:43 2007 @@ -3,9 +3,14 @@ from pypy.rpython.memory.gctransform.test.test_refcounting import make_deallocator from pypy.rpython.lltypesystem import lltype from pypy.translator.translator import graphof +from pypy.translator.c.gc import BoehmGcPolicy +from pypy.rpython.memory.gctransform.test.test_transform import LLInterpedTranformerTests from pypy import conftest import py +class TestLLInterpedBoehm(LLInterpedTranformerTests): + gcpolicy = BoehmGcPolicy + def make_boehm_finalizer(TYPE): return make_deallocator(TYPE, attr="finalizer_funcptr_for_type", cls=BoehmGCTransformer) Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_refcounting.py Thu Jul 12 15:43:43 2007 @@ -1,117 +1,47 @@ from pypy.rpython.memory.gctransform.test.test_transform import rtype, rtype_and_transform, getops +from pypy.rpython.memory.gctransform.test.test_transform import LLInterpedTranformerTests from pypy.rpython.memory.gctransform.refcounting import RefcountingGCTransformer from pypy.rpython.lltypesystem import lltype -from pypy.translator.c.gc import RefcountingGcPolicy from pypy.translator.translator import TranslationContext, graphof +from pypy.translator.c.gc import RefcountingGcPolicy from pypy import conftest -def llinterpreter_for_refcounted_graph(f, args_s): - from pypy.rpython.llinterp import LLInterpreter - from pypy.translator.c.genc import CStandaloneBuilder - from pypy.translator.c import gc - - t = rtype(f, args_s) - cbuild = CStandaloneBuilder(t, f, t.config, gcpolicy=RefcountingGcPolicy) - db = cbuild.generate_graphs_for_llinterp() - graph = cbuild.getentrypointptr()._obj.graph - llinterp = LLInterpreter(t.rtyper) - if conftest.option.view: - t.view() - return llinterp, graph - - -def test_llinterp_refcounted_graph(): - from pypy.annotation.model import SomeInteger +class TestLLInterpedRefcounting(LLInterpedTranformerTests): + gcpolicy = RefcountingGcPolicy - class C: - pass - c = C() - c.x = 1 - def g(x): - if x: - return c - else: - d = C() - d.x = 2 - return d - def f(x): - return g(x).x - - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeInteger()]) - - res = llinterp.eval_graph(graph, [0]) - assert res == f(0) - res = llinterp.eval_graph(graph, [1]) - assert res == f(1) - -def test_llinterp_refcounted_graph_varsize(): - from pypy.annotation.model import SomeInteger - - def f(x): - r = [] - for i in range(x): - if i % 2: - r.append(x) - return len(r) - - - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeInteger()]) - - res = llinterp.eval_graph(graph, [0]) - assert res == f(0) - res = llinterp.eval_graph(graph, [10]) - assert res == f(10) - -def test_llinterp_refcounted_graph_str(): - from pypy.annotation.model import SomeString - from pypy.rpython.lltypesystem.rstr import string_repr - - def f(x): - return len(x + 'a') - - - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeString()]) - - cc = string_repr.convert_const - - res = llinterp.eval_graph(graph, [cc('a')]) - assert res == f('a') - res = llinterp.eval_graph(graph, [cc('brrrrrr')]) - assert res == f('brrrrrr') - -def test_llinterp_refcounted_graph_with_del(): - from pypy.annotation.model import SomeInteger - - class D: - pass - - delcounter = D() - delcounter.dels = 0 - - class C: - def __del__(self): - delcounter.dels += 1 - c = C() - c.x = 1 - def h(x): - if x: - return c - else: - d = C() - d.x = 2 - return d - def g(x): - return h(x).x - def f(x): - r = g(x) - return r + delcounter.dels + def test_llinterp_refcounted_graph_with_del(self): + from pypy.annotation.model import SomeInteger - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeInteger()]) + class D: + pass - res = llinterp.eval_graph(graph, [1]) - assert res == 1 - res = llinterp.eval_graph(graph, [0]) - assert res == 3 + delcounter = D() + delcounter.dels = 0 + + class C: + def __del__(self): + delcounter.dels += 1 + c = C() + c.x = 1 + def h(x): + if x: + return c + else: + d = C() + d.x = 2 + return d + def g(x): + return h(x).x + def f(x): + r = g(x) + return r + delcounter.dels + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeInteger()]) + + res = llinterp.eval_graph(graph, [1]) + assert res == 1 + res = llinterp.eval_graph(graph, [0]) + assert res == 3 def test_simple_barrier(): S = lltype.GcStruct("S", ('x', lltype.Signed)) Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_transform.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_transform.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/test/test_transform.py Thu Jul 12 15:43:43 2007 @@ -10,6 +10,83 @@ from pypy.rpython.extregistry import ExtRegistryEntry from pypy import conftest +class LLInterpedTranformerTests: + + def llinterpreter_for_transformed_graph(self, f, args_s): + from pypy.rpython.llinterp import LLInterpreter + from pypy.translator.c.genc import CStandaloneBuilder + from pypy.translator.c import gc + + t = rtype(f, args_s) + # XXX we shouldn't need an actual gcpolicy here. + cbuild = CStandaloneBuilder(t, f, t.config, gcpolicy=self.gcpolicy) + db = cbuild.generate_graphs_for_llinterp() + graph = cbuild.getentrypointptr()._obj.graph + llinterp = LLInterpreter(t.rtyper) + if conftest.option.view: + t.view() + return llinterp, graph + + + def test_simple(self): + from pypy.annotation.model import SomeInteger + + class C: + pass + c = C() + c.x = 1 + def g(x): + if x: + return c + else: + d = C() + d.x = 2 + return d + def f(x): + return g(x).x + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeInteger()]) + + res = llinterp.eval_graph(graph, [0]) + assert res == f(0) + res = llinterp.eval_graph(graph, [1]) + assert res == f(1) + + def test_simple_varsize(self): + from pypy.annotation.model import SomeInteger + + def f(x): + r = [] + for i in range(x): + if i % 2: + r.append(x) + return len(r) + + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeInteger()]) + + res = llinterp.eval_graph(graph, [0]) + assert res == f(0) + res = llinterp.eval_graph(graph, [10]) + assert res == f(10) + + def test_str(self): + from pypy.annotation.model import SomeString + from pypy.rpython.lltypesystem.rstr import string_repr + + def f(x): + return len(x + 'a') + + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeString()]) + + cc = string_repr.convert_const + + res = llinterp.eval_graph(graph, [cc('a')]) + assert res == f('a') + res = llinterp.eval_graph(graph, [cc('brrrrrr')]) + assert res == f('brrrrrr') + class _TestGCTransformer(GCTransformer): def push_alive_nopyobj(self, var, llops): From mwh at codespeak.net Thu Jul 12 15:46:26 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 15:46:26 +0200 (CEST) Subject: [pypy-svn] r44968 - pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform Message-ID: <20070712134626.9EFDE80FD@code0.codespeak.net> Author: mwh Date: Thu Jul 12 15:46:24 2007 New Revision: 44968 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/transform.py Log: (mwh, pedronis) don't inline the varsize malloc helpers Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py Thu Jul 12 15:46:24 2007 @@ -44,9 +44,9 @@ self.malloc_fixedsize_atomic_ptr = self.inittime_helper( ll_malloc_fixedsize_atomic, [lltype.Signed], llmemory.Address) self.malloc_varsize_no_length_ptr = self.inittime_helper( - ll_malloc_varsize_no_length, [lltype.Signed]*3, llmemory.Address) + ll_malloc_varsize_no_length, [lltype.Signed]*3, llmemory.Address, inline=False) self.malloc_varsize_ptr = self.inittime_helper( - ll_malloc_varsize, [lltype.Signed]*4, llmemory.Address) + ll_malloc_varsize, [lltype.Signed]*4, llmemory.Address, inline=False) self.mixlevelannotator.finish() # for now def push_alive_nopyobj(self, var, llops): Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/transform.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/transform.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/transform.py Thu Jul 12 15:46:24 2007 @@ -243,8 +243,8 @@ self.graphs_to_inline[graph] = True return self.mixlevelannotator.graph2delayed(graph) - def inittime_helper(self, ll_helper, ll_args, ll_result): - ptr = self.annotate_helper(ll_helper, ll_args, ll_result, inline=True) + def inittime_helper(self, ll_helper, ll_args, ll_result, inline=True): + ptr = self.annotate_helper(ll_helper, ll_args, ll_result, inline=inline) return Constant(ptr, lltype.typeOf(ptr)) def finish_helpers(self): From cfbolz at codespeak.net Thu Jul 12 16:28:21 2007 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jul 2007 16:28:21 +0200 (CEST) Subject: [pypy-svn] r44969 - pypy/extradoc/talk/dyla2007 Message-ID: <20070712142821.35FD08201@code0.codespeak.net> Author: cfbolz Date: Thu Jul 12 16:28:19 2007 New Revision: 44969 Modified: pypy/extradoc/talk/dyla2007/dyla.pdf pypy/extradoc/talk/dyla2007/dyla.tex Log: add slang too Modified: pypy/extradoc/talk/dyla2007/dyla.pdf ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/dyla2007/dyla.tex ============================================================================== --- pypy/extradoc/talk/dyla2007/dyla.tex (original) +++ pypy/extradoc/talk/dyla2007/dyla.tex Thu Jul 12 16:28:19 2007 @@ -122,7 +122,8 @@ instead that VMs should not be \emph{written} in the first place -- they should be generated from simple interpreters written in any suitable high-level\footnote{``High-level'' is taken by opposition to languages -like PreScheme \cite{kelsey-prescheme} which use the syntax and +like PreScheme \cite{kelsey-prescheme} or the subset of Smalltalk that the +Squeak VM is written in \cite{Squeak} which use the syntax and metaprogramming facilities of a high-level language but encode low-level details like memory management.} language. From jlg at codespeak.net Thu Jul 12 16:41:57 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 16:41:57 +0200 (CEST) Subject: [pypy-svn] r44970 - pypy/dist/pypy/lang/scheme Message-ID: <20070712144157.7FDAF823E@code0.codespeak.net> Author: jlg Date: Thu Jul 12 16:41:56 2007 New Revision: 44970 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py Log: (antocuni, jlg) - unquote in ssparse -> RPython; fixnum, float parsing -> RPython Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Thu Jul 12 16:41:56 2007 @@ -84,7 +84,7 @@ class W_Fixnum(W_Root): def __init__(self, val): - self.fixnumval = int(val) + self.fixnumval = val def to_string(self): return str(self.fixnumval) @@ -103,7 +103,7 @@ class W_Float(W_Root): def __init__(self, val): - self.floatval = float(val) + self.floatval = val def to_string(self): return str(self.floatval) Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 12 16:41:56 2007 @@ -5,14 +5,24 @@ W_Nil, W_Boolean, W_Float, Literal def unquote(s): - return s.replace('\\"', '"') + str_lst = [] + last_ch = '' + for c in s[1:]: + if last_ch == '\\' and c == '"': + pass + else: + str_lst.append(last_ch) + + last_ch = c + + return ''.join(str_lst) class SchemeParser(PackratParser): r""" STRING: c = `\"([^\\\"]|\\\"|\\\\)*\"` IGNORE* - return {W_String(unquote(c[1:-1]))}; + return {W_String(unquote(c))}; IDENTIFIER: c = `[\+\-\*\^\?a-zA-Z!<=>_~/$%&:][\+\-\*\^\?a-zA-Z0-9!<=>_~/$%&:]*` @@ -22,12 +32,12 @@ FIXNUM: c = `\-?(0|([1-9][0-9]*))` IGNORE* - return {W_Fixnum(c)}; + return {W_Fixnum(int(c))}; FLOAT: c = `\-?[0-9]*\.[0-9]*` IGNORE* - return {W_Float(c)}; + return {W_Float(float(c))}; BOOLEAN: c = `#(t|f)` From fijal at codespeak.net Thu Jul 12 16:50:29 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 16:50:29 +0200 (CEST) Subject: [pypy-svn] r44971 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20070712145029.2C2958256@code0.codespeak.net> Author: fijal Date: Thu Jul 12 16:50:28 2007 New Revision: 44971 Modified: pypy/dist/pypy/rpython/lltypesystem/rffi.py Log: (simonb, fijal) Some shortcut definitions Modified: pypy/dist/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rffi.py Thu Jul 12 16:50:28 2007 @@ -60,9 +60,15 @@ c_errno = CConstant('errno', lltype.Signed) +# void * +VOIDP = lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1)) + # char * CCHARP = lltype.Ptr(lltype.Array(lltype.Char, hints={'nolength': True})) +# int * +INTP = lltype.Ptr(lltype.Array(lltype.Signed, hints={'nolength': True})) + # various type mapping # str -> char* def str2charp(s): From jlg at codespeak.net Thu Jul 12 17:02:27 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 17:02:27 +0200 (CEST) Subject: [pypy-svn] r44972 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070712150227.326518255@code0.codespeak.net> Author: jlg Date: Thu Jul 12 17:02:22 2007 New Revision: 44972 Modified: pypy/dist/pypy/lang/scheme/interactive.py pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_eval.py pypy/dist/pypy/lang/scheme/test/test_parser.py Log: (antocuni, jlg) - parse() now returns list of sexpressions, not only the first one Modified: pypy/dist/pypy/lang/scheme/interactive.py ============================================================================== --- pypy/dist/pypy/lang/scheme/interactive.py (original) +++ pypy/dist/pypy/lang/scheme/interactive.py Thu Jul 12 17:02:22 2007 @@ -31,7 +31,7 @@ if to_exec == "": print raise SchemeQuit - print parse(to_exec).eval(ctx).to_string() + print parse(to_exec)[0].eval(ctx).to_string() except SchemeQuit, e: break except BacktrackException, e: Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Thu Jul 12 17:02:22 2007 @@ -42,7 +42,7 @@ BOOLEAN: c = `#(t|f)` IGNORE* - return {W_Boolean(c[-1] == 't')}; + return {W_Boolean(c[1] == 't')}; IGNORE: ` |\n|\t|;[^\n]*`; @@ -52,7 +52,7 @@ file: IGNORE* - s = sexpr + s = sexpr* EOF return {s}; Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Thu Jul 12 17:02:22 2007 @@ -8,10 +8,10 @@ assert w_num.eval(None).to_number() == 9 def eval_expr(ctx, expr): - return parse(expr).eval(ctx) + return parse(expr)[0].eval(ctx) def eval_noctx(expr): - return parse(expr).eval(None) + return parse(expr)[0].eval(None) def test_numerical(): w_num = eval_noctx("(+ 4)") Modified: pypy/dist/pypy/lang/scheme/test/test_parser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_parser.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_parser.py Thu Jul 12 17:02:22 2007 @@ -3,6 +3,9 @@ from pypy.lang.scheme.object import W_Boolean, W_Float, W_Fixnum, W_String from pypy.lang.scheme.object import W_Pair, W_Nil, W_Symbol, W_Identifier +def parse_sexpr(expr): + return parse(expr)[0] + def unwrap(w_obj): """for testing purposes: unwrap a scheme object into a python object""" if isinstance(w_obj, W_Float): @@ -26,43 +29,43 @@ raise NotImplementedError("don't know what to do with: %s" % (w_obj, )) def test_simple(): - w_fixnum = parse('1') + w_fixnum = parse_sexpr('1') assert isinstance(w_fixnum, W_Fixnum) assert unwrap(w_fixnum) == 1 - w_fixnum = parse('0') + w_fixnum = parse_sexpr('0') assert unwrap(w_fixnum) == 0 assert isinstance(w_fixnum, W_Fixnum) - w_fixnum = parse('1123') + w_fixnum = parse_sexpr('1123') assert unwrap(w_fixnum) == 1123 assert isinstance(w_fixnum, W_Fixnum) - w_fixnum = parse('abfa__') + w_fixnum = parse_sexpr('abfa__') assert isinstance(w_fixnum, W_Identifier) - t = parse(r'''"don't believe \"them\""''') + t = parse_sexpr(r'''"don't believe \"them\""''') assert isinstance(t, W_String) assert unwrap(t) == 'don\'t believe "them"' def test_objects(): - w_fixnum = parse('-12345') + w_fixnum = parse_sexpr('-12345') assert isinstance(w_fixnum, W_Fixnum) assert unwrap(w_fixnum) == -12345 - w_float = parse('123456.1234') + w_float = parse_sexpr('123456.1234') assert isinstance(w_float, W_Float) assert unwrap(w_float) == 123456.1234 - w_float = parse('-123456.1234') + w_float = parse_sexpr('-123456.1234') assert isinstance(w_float, W_Float) assert unwrap(w_float) == -123456.1234 def test_sexpr(): - w_list = parse('( 1 )') + w_list = parse_sexpr('( 1 )') assert isinstance(w_list, W_Pair) assert isinstance(w_list.car, W_Fixnum) assert isinstance(w_list.cdr, W_Nil) - #w_list = parse('()') + #w_list = parse_sexpr('()') #assert isinstance(w_list, W_Nil) - w_list = parse('(+ 1 2)') + w_list = parse_sexpr('(+ 1 2)') assert isinstance(w_list, W_Pair) assert isinstance(w_list.car, W_Identifier) assert isinstance(w_list.cdr, W_Pair) @@ -72,7 +75,7 @@ def test_complex_sexpr(): #parse more complex sexpr - t = parse(r''' + t = parse_sexpr(r''' (define (fac n) ; comment (if (< n 2) n (* (fac (- n 1)) n))) @@ -88,38 +91,38 @@ yield check_ident_ch, char def check_ident_ch(char): - t = parse("(" + char + ")") + t = parse_sexpr("(" + char + ")") assert isinstance(t, W_Pair) assert isinstance(t.car, W_Identifier) assert unwrap(t.car) == char assert isinstance(t.cdr, W_Nil) def test_truth_values(): - t = parse("#f") + t = parse_sexpr("#f") assert unwrap(t) == False - t = parse("#t") + t = parse_sexpr("#t") assert unwrap(t) == True def test_list_dotted(): - t = parse("(1 . 2)") + t = parse_sexpr("(1 . 2)") assert isinstance(t, W_Pair) assert unwrap(t.car) == 1 assert unwrap(t.cdr) == 2 - t = parse("(1 . (2 . 3))") + t = parse_sexpr("(1 . (2 . 3))") assert unwrap(t.car) == 1 assert unwrap(t.cdr.car) == 2 assert unwrap(t.cdr.cdr) == 3 - t = parse("(1 . (2 . (3 . ())))") + t = parse_sexpr("(1 . (2 . (3 . ())))") assert unwrap(t) == [1, 2, 3] def test_list_mixed(): - t = parse("(1 2 . 3)") + t = parse_sexpr("(1 2 . 3)") assert unwrap(t.car) == 1 assert unwrap(t.cdr.car) == 2 assert unwrap(t.cdr.cdr) == 3 - t = parse("(1 2 . (3 4))") + t = parse_sexpr("(1 2 . (3 4))") assert unwrap(t) == [1, 2, 3, 4] From cfbolz at codespeak.net Thu Jul 12 17:11:29 2007 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jul 2007 17:11:29 +0200 (CEST) Subject: [pypy-svn] r44973 - pypy/dist/pypy/translator/goal Message-ID: <20070712151129.2B8FD80FD@code0.codespeak.net> Author: cfbolz Date: Thu Jul 12 17:11:26 2007 New Revision: 44973 Added: pypy/dist/pypy/translator/goal/targetscheme.py - copied unchanged from r44972, pypy/dist/pypy/translator/goal/targetssstandalone.py Removed: pypy/dist/pypy/translator/goal/targetssstandalone.py Log: make the name more explicit: nowadays there are nearly no targets that are not standalone anyway. From jlg at codespeak.net Thu Jul 12 17:16:07 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Thu, 12 Jul 2007 17:16:07 +0200 (CEST) Subject: [pypy-svn] r44974 - pypy/dist/pypy/translator/goal Message-ID: <20070712151607.5369F818C@code0.codespeak.net> Author: jlg Date: Thu Jul 12 17:16:05 2007 New Revision: 44974 Modified: pypy/dist/pypy/translator/goal/targetscheme.py Log: (antocuni, jlg) - forgot to update it earlier Modified: pypy/dist/pypy/translator/goal/targetscheme.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetscheme.py (original) +++ pypy/dist/pypy/translator/goal/targetscheme.py Thu Jul 12 17:16:05 2007 @@ -10,16 +10,20 @@ # __________ Entry point __________ + def entry_point(argv): if len(argv) == 2: f = open_file_as_stream(argv[1]) t = parse(f.readall()) ctx = ExecutionContext() - w_retval = t.eval(ctx) - print w_retval, + try: + for sexpr in t: + w_retval = sexpr.eval(ctx) + print w_retval.to_string() + except SchemeQuit, e: + return 0 + return 0 - elif argv[0] == 'foo': - raise SchemeQuit else: print "Usage: %s schemesourcefile" % argv[0] return 1 From fijal at codespeak.net Thu Jul 12 17:21:51 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 12 Jul 2007 17:21:51 +0200 (CEST) Subject: [pypy-svn] r44976 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070712152151.30DE7818C@code0.codespeak.net> Author: fijal Date: Thu Jul 12 17:21:50 2007 New Revision: 44976 Modified: pypy/dist/pypy/rpython/tool/mkrffi.py pypy/dist/pypy/rpython/tool/test/test_c.py pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Log: (simonb, fijal) Make this work for structures as well Modified: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Thu Jul 12 17:21:50 2007 @@ -9,20 +9,68 @@ # XXX any automatic stuff here? SIMPLE_TYPE_MAPPING = { - ctypes.c_int : 'rffi.INT', - ctypes.c_voidp : primitive_pointer_repr('lltype.Void') + ctypes.c_int : 'rffi.INT', + ctypes.c_uint : 'rffi.UINT', + ctypes.c_voidp : 'rffi.VOIDP', } -def proc_tp(tp): - try: - return SIMPLE_TYPE_MAPPING[tp] - except KeyError: +class RffiSource(object): + def __init__(self, structures=None, code=None): + # ctypes structure -> code mapping + if structures is None: + self.structures = {} + else: + self.structures = structures + # rest of the code + if code is None: + self.code = py.code.Source() + else: + self.code = code + + def __add__(self, other): + structs = self.structures.copy() + structs.update(other.structures) + code = py.code.Source(self.code, other.code) + return RffiSource(structs, code) + + def __iadd__(self, other): + self.structures.update(other.structures) + self.code = py.code.Source(self.code, other.code) + return self + + def proc_struct(self, tp): + name = tp.__name__ + real_name = 'c_' + name + real_name = real_name.rstrip('_structure') + real_name += '_structure' + if tp not in self.structures: + fields = ["('%s', %s)" % (name_, self.proc_tp(field_tp)) + for name_, field_tp in tp._fields_] + fields_repr = ", ".join(fields) + self.structures[tp] = py.code.Source( + "%s = rffi.CStruct('%s', %s)"%(real_name, name, + fields_repr)) + return real_name + + def proc_tp(self, tp): + try: + return SIMPLE_TYPE_MAPPING[tp] + except KeyError: + pass + if issubclass(tp, ctypes._Pointer): + if issubclass(tp._type_, ctypes._SimpleCData): + return "lltype.Ptr(lltype.Array(%s, hints={'nolength': True}))"\ + % self.proc_tp(tp._type_) + return "lltype.Ptr(%s)" % self.proc_tp(tp._type_) + elif issubclass(tp, ctypes.Structure): + return self.proc_struct(tp) raise NotImplementedError("Not implemented mapping for %s" % tp) -def proc_func(func): - name = func.__name__ - src = py.code.Source(""" - c_%s = rffi.llexternal('%s', [%s], %s) - """%(name, name, ",".join([proc_tp(arg) for arg in func.argtypes]), - proc_tp(func.restype))) - return src + def proc_func(self, func): + name = func.__name__ + src = py.code.Source(""" + c_%s = rffi.llexternal('%s', [%s], %s) + """%(name, name, ", ".join([self.proc_tp(arg) for + arg in func.argtypes]), + self.proc_tp(func.restype))) + self.code = py.code.Source(self.code, src) Modified: pypy/dist/pypy/rpython/tool/test/test_c.py ============================================================================== --- pypy/dist/pypy/rpython/tool/test/test_c.py (original) +++ pypy/dist/pypy/rpython/tool/test/test_c.py Thu Jul 12 17:21:50 2007 @@ -8,6 +8,13 @@ c_source = """ void *int_to_void_p(int arg) {} + +struct random_strucutre { + int one; + int *two; +}; + +struct random_structure* int_int_to_struct_p(int one, int two) {} """ class TestBasic: Modified: pypy/dist/pypy/rpython/tool/test/test_mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/test/test_mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Thu Jul 12 17:21:50 2007 @@ -4,15 +4,56 @@ from pypy.rpython.tool.test.test_c import TestBasic import py +class random_structure(ctypes.Structure): + _fields_ = [('one', ctypes.c_int), + ('two', ctypes.POINTER(ctypes.c_int))] + +def test_rffisource(): + res = RffiSource({1:2, 3:4}, "ab") + RffiSource(None, "c") + assert res.structures == {1:2, 3:4} + assert str(res.code) == "ab\nc" + res += RffiSource({5:6}) + assert 5 in res.structures.keys() + +def test_proc_tp_simple(): + rffi_source = RffiSource() + assert rffi_source.proc_tp(ctypes.c_int) == 'rffi.INT' + assert rffi_source.proc_tp(ctypes.c_void_p) == 'rffi.VOIDP' + +def test_proc_tp_complicated(): + rffi_source = RffiSource() + assert rffi_source.proc_tp(ctypes.POINTER(ctypes.c_uint)) == \ + "lltype.Ptr(lltype.Array(rffi.UINT, hints={'nolength': True}))" + rffi_source.proc_tp(random_structure) + _src = py.code.Source(""" + c_random_structure = rffi.CStruct('random_structure', ('one', rffi.INT), ('two', lltype.Ptr(lltype.Array(rffi.INT, hints={'nolength': True})))) + """) + src = rffi_source.structures[random_structure] + assert src.strip() == _src.strip(), str(src) + "\n" + str(_src) + class TestMkrffi(TestBasic): def test_single_func(self): func = self.lib.int_to_void_p func.argtypes = [ctypes.c_int] func.restype = ctypes.c_voidp - src = proc_func(func) + src = RffiSource() + src.proc_func(func) _src = py.code.Source(""" - c_int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], lltype.Ptr(lltype.FixedSizeArray(lltype.Void, 1))) + c_int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], rffi.VOIDP) """) - assert src == _src, str(src) + "\n" + str(_src) + assert src.code == _src, str(src) + "\n" + str(_src) + + def test_struct_return(self): + func = self.lib.int_int_to_struct_p + func.argtypes = [ctypes.c_int, ctypes.c_int] + func.restype = ctypes.POINTER(random_structure) + rffi_source = RffiSource() + rffi_source.proc_func(func) + assert random_structure in rffi_source.structures + _src = py.code.Source(""" + c_int_int_to_struct_p = rffi.llexternal('int_int_to_struct_p', [rffi.INT, rffi.INT], lltype.Ptr(c_random_structure)) + """) + src = rffi_source.code + assert src.strip() == _src.strip(), str(src) + "\n" + str(_src) From mwh at codespeak.net Thu Jul 12 17:33:47 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 17:33:47 +0200 (CEST) Subject: [pypy-svn] r44977 - in pypy/branch/emptying-the-malloc-zoo-2/pypy: rpython/lltypesystem rpython/memory/gctransform translator/c translator/c/src translator/c/test Message-ID: <20070712153347.5A99D824D@code0.codespeak.net> Author: mwh Date: Thu Jul 12 17:33:45 2007 New Revision: 44977 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/funcgen.py pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/support.py pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/test/test_boehm.py Log: (mwh, pedronis) make the boehm gc transformer do way more and the boehm gcpolicy in genc do way less Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/lltypesystem/lloperation.py Thu Jul 12 17:33:45 2007 @@ -341,6 +341,7 @@ 'boehm_malloc': LLOp(), 'boehm_malloc_atomic': LLOp(), + 'boehm_register_finalizer': LLOp(), 'raw_malloc': LLOp(), 'raw_malloc_usage': LLOp(sideeffects=False), 'raw_free': LLOp(), Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/boehm.py Thu Jul 12 17:33:45 2007 @@ -5,22 +5,29 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython import rmodel from pypy.rlib.rarithmetic import ovfcheck +from pypy.objspace.flow.model import Constant class BoehmGCTransformer(GCTransformer): + FINALIZER_PTR = lltype.Ptr(lltype.FuncType([llmemory.Address], lltype.Void)) + def __init__(self, translator, inline=False): super(BoehmGCTransformer, self).__init__(translator, inline=inline) self.finalizer_funcptrs = {} memoryError = MemoryError() - def ll_malloc_fixedsize(size): + def ll_malloc_fixedsize(size, finalizer): result = llop.boehm_malloc(llmemory.Address, size) if not result: raise memoryError + if finalizer: # XXX runtime check here is bad? + llop.boehm_register_finalizer(lltype.Void, result, finalizer) return result - def ll_malloc_fixedsize_atomic(size): + def ll_malloc_fixedsize_atomic(size, finalizer): result = llop.boehm_malloc_atomic(llmemory.Address, size) if not result: raise memoryError + if finalizer: # XXX runtime check here is bad? + llop.boehm_register_finalizer(lltype.Void, result, finalizer) return result # XXX, do we need/want an atomic version of this function? def ll_malloc_varsize_no_length(length, size, itemsize): @@ -40,9 +47,9 @@ if self.translator: self.malloc_fixedsize_ptr = self.inittime_helper( - ll_malloc_fixedsize, [lltype.Signed], llmemory.Address) + ll_malloc_fixedsize, [lltype.Signed, self.FINALIZER_PTR], llmemory.Address) self.malloc_fixedsize_atomic_ptr = self.inittime_helper( - ll_malloc_fixedsize_atomic, [lltype.Signed], llmemory.Address) + ll_malloc_fixedsize_atomic, [lltype.Signed, self.FINALIZER_PTR], llmemory.Address) self.malloc_varsize_no_length_ptr = self.inittime_helper( ll_malloc_varsize_no_length, [lltype.Signed]*3, llmemory.Address, inline=False) self.malloc_varsize_ptr = self.inittime_helper( @@ -71,10 +78,15 @@ funcptr = self.malloc_fixedsize_atomic_ptr else: funcptr = self.malloc_fixedsize_ptr - v_raw = hop.genop("direct_call", [funcptr, c_size], + c_finalizer_ptr = Constant(self.finalizer_funcptr_for_type(TYPE), self.FINALIZER_PTR) + v_raw = hop.genop("direct_call", + [funcptr, c_size, c_finalizer_ptr], resulttype=llmemory.Address) hop.cast_result(v_raw) + # XXX In theory this is wrong: + gct_zero_malloc = gct_malloc + def gct_malloc_varsize(self, hop): def intconst(c): return rmodel.inputconst(lltype.Signed, c) @@ -82,6 +94,8 @@ TYPE = op.result.concretetype.TO assert TYPE._is_varsize() + assert not self.finalizer_funcptr_for_type(TYPE) + if isinstance(TYPE, lltype.Struct): ARRAY = TYPE._flds[TYPE._arrayfld] else: @@ -110,6 +124,8 @@ resulttype=llmemory.Address) hop.cast_result(v_raw) + gct_zero_malloc_varsize = gct_malloc_varsize + def finalizer_funcptr_for_type(self, TYPE): if TYPE in self.finalizer_funcptrs: return self.finalizer_funcptrs[TYPE] @@ -144,7 +160,7 @@ llop.gc_restore_exception(lltype.Void, exc_instance) fptr = self.annotate_helper(ll_finalizer, [llmemory.Address], lltype.Void) else: - fptr = None + fptr = lltype.nullptr(self.FINALIZER_PTR.TO) self.finalizer_funcptrs[TYPE] = fptr return fptr Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/rpython/memory/gctransform/refcounting.py Thu Jul 12 17:33:45 2007 @@ -155,6 +155,8 @@ resulttype=llmemory.Address) hop.cast_result(v_raw) + gct_zero_malloc = gct_malloc + def gct_malloc_varsize(self, hop): def intconst(c): return rmodel.inputconst(lltype.Signed, c) @@ -190,6 +192,8 @@ resulttype=llmemory.Address) hop.cast_result(v_raw) + gct_zero_malloc_varsize = gct_malloc_varsize + def gct_gc_deallocate(self, hop): TYPE = hop.spaceop.args[0].value v_addr = hop.spaceop.args[1] Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/funcgen.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/funcgen.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/funcgen.py Thu Jul 12 17:33:45 2007 @@ -510,62 +510,17 @@ self.expr(op.args[0]), self.expr(op.args[1])) - def OP_ZERO_MALLOC(self, op): - TYPE = self.lltypemap(op.result).TO - typename = self.db.gettype(TYPE) - eresult = self.expr(op.result) - esize = 'sizeof(%s)' % cdecl(typename, '') - - return self.gcpolicy.zero_malloc(TYPE, esize, eresult) - - def OP_MALLOC(self, op): - TYPE = self.lltypemap(op.result).TO - typename = self.db.gettype(TYPE) - eresult = self.expr(op.result) - esize = 'sizeof(%s)' % cdecl(typename, '') - - return self.gcpolicy.malloc(TYPE, esize, eresult) - - OP_ZERO_MALLOC = OP_MALLOC - - def OP_MALLOC_VARSIZE(self, op): - TYPE = self.lltypemap(op.result).TO - typename = self.db.gettype(TYPE) - lenfld = 'length' - nodedef = self.db.gettypedefnode(TYPE) - if isinstance(TYPE, Struct): - arfld = TYPE._arrayfld - lenfld = "%s.length" % nodedef.c_struct_field_name(arfld) - VARPART = TYPE._flds[TYPE._arrayfld] - else: - VARPART = TYPE - assert isinstance(VARPART, Array) - itemtypename = self.db.gettype(VARPART.OF) - elength = self.expr(op.args[1]) - eresult = self.expr(op.result) - if VARPART.OF is Void: # strange - esize = 'sizeof(%s)' % (cdecl(typename, ''),) - result = '{\n' - else: - itemtype = cdecl(itemtypename, '') - result = 'IF_VARSIZE_OVERFLOW(%s, %s, %s)\nelse {\n' % ( - elength, - itemtype, - eresult) - esize = 'sizeof(%s)-sizeof(%s)+%s*sizeof(%s)' % ( - cdecl(typename, ''), - itemtype, - elength, - itemtype) - result += self.gcpolicy.zero_malloc(TYPE, esize, eresult) - - # ctypes Arrays have no length field - if not VARPART._hints.get('nolength', False): - result += '\nif(%s) %s->%s = %s;' % (eresult, eresult, lenfld, elength) - result += '\n}' - return result - - OP_ZERO_MALLOC_VARSIZE = OP_MALLOC_VARSIZE + def OP_BOEHM_MALLOC(self, op): + return 'OP_BOEHM_ZERO_MALLOC(%s, %s, void*, 0, 0);' % (self.expr(op.args[0]), + self.expr(op.result)) + + def OP_BOEHM_MALLOC_ATOMIC(self, op): + return 'OP_BOEHM_ZERO_MALLOC(%s, %s, void*, 1, 0);' % (self.expr(op.args[0]), + self.expr(op.result)) + + def OP_BOEHM_REGISTER_FINALIZER(self, op): + return 'GC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' \ + % (self.expr(op.args[0]), self.expr(op.args[1])) def OP_RAW_MALLOC(self, op): eresult = self.expr(op.result) @@ -573,6 +528,7 @@ return "OP_RAW_MALLOC(%s, %s, void *);" % (esize, eresult) def OP_FLAVORED_MALLOC(self, op): + # XXX this function should DIE! TYPE = self.lltypemap(op.result).TO typename = self.db.gettype(TYPE) eresult = self.expr(op.result) @@ -590,6 +546,7 @@ raise NotImplementedError def OP_FLAVORED_MALLOC_VARSIZE(self, op): + # XXX this function should DIE!, at least twice over # XXX I know this working in just one case, probably makes # sense to assert it here, rest is just copied flavor = op.args[0].value Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/gc.py Thu Jul 12 17:33:45 2007 @@ -117,14 +117,6 @@ # zero malloc impl - def zero_malloc(self, TYPE, esize, eresult): - assert TYPE._gckind == 'gc' # we don't really support this - typename = self.db.gettype(TYPE) - erestype = cdecl(typename, '*') - return 'OP_ZERO_MALLOC(%s, %s, %s);' % (esize, eresult, erestype) - - malloc = zero_malloc - def OP_GC_CALL_RTTI_DESTRUCTOR(self, funcgen, op): args = [funcgen.expr(v) for v in op.args] line = '%s(%s);' % (args[0], ', '.join(args[1:])) @@ -180,18 +172,11 @@ class BoehmGcPolicy(BasicGcPolicy): transformerclass = boehm.BoehmGCTransformer - def setup_gcinfo(self, defnode): - transformer = defnode.db.gctransformer - fptr = transformer.finalizer_funcptr_for_type(defnode.LLTYPE) - if fptr: - defnode.gcinfo = BoehmInfo() - defnode.gcinfo.finalizer = defnode.db.get(fptr) - def array_setup(self, arraydefnode): - self.setup_gcinfo(arraydefnode) + pass def struct_setup(self, structdefnode, rtti): - self.setup_gcinfo(structdefnode) + pass def rtti_type(self): return BoehmGcRuntimeTypeInfo_OpaqueNode.typename @@ -199,25 +184,6 @@ def rtti_node_factory(self): return BoehmGcRuntimeTypeInfo_OpaqueNode - def zero_malloc(self, TYPE, esize, eresult): - gcinfo = self.db.gettypedefnode(TYPE).gcinfo - assert TYPE._gckind == 'gc' # _is_atomic() depends on this! - is_atomic = TYPE._is_atomic() - is_varsize = TYPE._is_varsize() - typename = self.db.gettype(TYPE) - erestype = cdecl(typename, '*') - result = 'OP_BOEHM_ZERO_MALLOC(%s, %s, %s, %d, %d);' % (esize, - eresult, - erestype, - is_atomic, - is_varsize) - if gcinfo and gcinfo.finalizer: - result += ('\nGC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' - % (eresult, gcinfo.finalizer)) - return result - - malloc = zero_malloc - def gc_libraries(self): if sys.platform == 'win32': return ['gc_pypy'] @@ -301,7 +267,6 @@ yield "#include " def struct_setup(self, structdefnode, rtti): - self.setup_gcinfo(structdefnode) T = structdefnode.STRUCT if T._is_atomic(): malloc_exact = False @@ -336,37 +301,11 @@ self.get_descr_name(defnode), T) yield "}" - def zero_malloc(self, TYPE, esize, eresult): - defnode = self.db.gettypedefnode(TYPE) - gcinfo = defnode.gcinfo - if gcinfo: - if not gcinfo.malloc_exact: - assert TYPE._gckind == 'gc' # _is_atomic() depends on this! - is_atomic = TYPE._is_atomic() - is_varsize = TYPE._is_varsize() - typename = self.db.gettype(TYPE) - erestype = cdecl(typename, '*') - result = 'OP_BOEHM_ZERO_MALLOC(%s, %s, %s, %d, %d);' % ( - esize, eresult, erestype, is_atomic, is_varsize) - else: - result = '%s = GC_MALLOC_EXPLICITLY_TYPED(%s, %s);' % ( - eresult, esize, self.get_descr_name(defnode)) - if gcinfo.finalizer: - result += ('\nGC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' - % (eresult, gcinfo.finalizer)) - else: - return super(MoreExactBoehmGcPolicy, self).zero_malloc( - TYPE, esize, eresult) - return result - - malloc = zero_malloc # to get an idea how it looks like with no refcount/gc at all class NoneGcPolicy(BoehmGcPolicy): - zero_malloc = RefcountingGcPolicy.zero_malloc.im_func - malloc = RefcountingGcPolicy.malloc.im_func gc_libraries = RefcountingGcPolicy.gc_libraries.im_func gc_startup_code = RefcountingGcPolicy.gc_startup_code.im_func @@ -421,11 +360,6 @@ o = top_container(defnode.obj) return defnode.db.gctransformer.gc_field_values_for(o) - def zero_malloc(self, TYPE, esize, eresult): - assert False, "a malloc operation in a framework build??" - - malloc = zero_malloc - class StacklessFrameworkGcPolicy(FrameworkGcPolicy): transformerclass = stacklessframework.StacklessFrameworkGCTransformer requires_stackless = True Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h Thu Jul 12 17:33:45 2007 @@ -116,7 +116,6 @@ #define OP_BOEHM_ZERO_MALLOC(size, r, restype, is_atomic, is_varsize) { \ r = (restype) BOEHM_MALLOC_ ## is_atomic ## _ ## is_varsize (size); \ - if (r == NULL) {FAIL_EXCEPTION(PyExc_MemoryError, "out of memory");} \ else { \ if (is_atomic) /* the non-atomic versions return cleared memory */ \ memset((void*) r, 0, size); \ Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/support.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/support.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/support.py Thu Jul 12 17:33:45 2007 @@ -52,6 +52,10 @@ else: return s[:2].lower() +def is_pointer_to_forward_ref(T): + if not isinstance(T, lltype.Ptr): + return False + return isinstance(T.TO, lltype.ForwardReference) def llvalue_from_constant(c): try: @@ -64,7 +68,12 @@ if T == lltype.Void: return None else: - assert lltype.typeOf(c.value) == T + ACTUAL_TYPE = lltype.typeOf(c.value) + # If the type is still uncomputed, we can't make this + # check. Something else will blow up instead, probably + # very confusingly. + if not is_pointer_to_forward_ref(ACTUAL_TYPE): + assert ACTUAL_TYPE == T return c.value Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/test/test_boehm.py ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/test/test_boehm.py (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/test/test_boehm.py Thu Jul 12 17:33:45 2007 @@ -1,6 +1,6 @@ import py from pypy.translator.translator import TranslationContext -from pypy.rpython.lltypesystem.lltype import Void +from pypy.rpython.lltypesystem import lltype from pypy.translator.tool.cbuild import check_boehm_presence from pypy.translator.c.genc import CExtModuleBuilder from pypy import conftest @@ -62,7 +62,6 @@ def test__del__(self): from pypy.rpython.lltypesystem.lloperation import llop - from pypy.rpython.lltypesystem import lltype class State: pass s = State() @@ -98,7 +97,6 @@ def test_weakgcaddress_is_weak(self): from pypy.rpython.lltypesystem.lloperation import llop - from pypy.rpython.lltypesystem import lltype from pypy.rlib.objectmodel import cast_object_to_weakgcaddress class State: pass @@ -130,7 +128,6 @@ def test_del_raises(self): from pypy.rpython.lltypesystem.lloperation import llop - from pypy.rpython.lltypesystem import lltype import os class A(object): def __del__(self): @@ -160,7 +157,6 @@ assert res > 0 def test_memory_error_varsize(self): - from pypy.rpython.lltypesystem import lltype N = int(2**31-1) A = lltype.GcArray(lltype.Char) def alloc(n): @@ -183,7 +179,7 @@ def prob_with_pyobj(b): return 3, b def collect(): - llop.gc__collect(Void) + llop.gc__collect(lltype.Void) f = self.getcompiled(prob_with_pyobj, [object]) c = self.getcompiled(collect, []) from sys import getrefcount as g @@ -202,6 +198,23 @@ after = g(obj) assert abs(before - after) < 5 + def test_zero_malloc(self): + T = lltype.GcStruct("C", ('x', lltype.Signed)) + def fixed_size(): + t = lltype.malloc(T, zero=True) + return t.x + c_fixed_size = self.getcompiled(fixed_size, []) + res = c_fixed_size() + assert res == 0 + A = lltype.GcArray(lltype.Signed) + def var_size(): + a = lltype.malloc(A, 1, zero=True) + return a[0] + c_var_size = self.getcompiled(var_size, []) + res = c_var_size() + assert res == 0 + + class TestUsingExactBoehm(TestUsingBoehm): gcpolicy = "exact_boehm" From arigo at codespeak.net Thu Jul 12 17:34:44 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 12 Jul 2007 17:34:44 +0200 (CEST) Subject: [pypy-svn] r44978 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070712153444.601CF824D@code0.codespeak.net> Author: arigo Date: Thu Jul 12 17:34:42 2007 New Revision: 44978 Added: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py (contents, props changed) pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py - copied, changed from r44949, pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: (lac, arigo) Intermediate check-in: a very broken attempt. Will be removed anyway. Added: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/compactlltype.py Thu Jul 12 17:34:42 2007 @@ -0,0 +1,205 @@ +import ctypes, sys, weakref +from pypy.rpython.lltypesystem.lltype import Signed, Struct, GcStruct, Ptr +from pypy.rpython.lltypesystem.lltype import ContainerType, Array, GcArray +from pypy.rpython.lltypesystem.lltype import typeOf, castable + +_Ctypes_PointerType = type(ctypes.POINTER(ctypes.c_int)) + + +def cast_pointer(PTRTYPE, ptr): + CURTYPE = typeOf(ptr) + if not isinstance(CURTYPE, Ptr) or not isinstance(PTRTYPE, Ptr): + raise TypeError, "can only cast pointers to other pointers" + return ptr._cast_to(PTRTYPE) + + +class _parentable(ctypes.Structure): # won't work with Union + __slots__ = () + +# ____________________________________________________________ + +_ctypes_cache = { + Signed: ctypes.c_long, + } + +def _build_ctypes_struct(S, max_n=None): + fields = [] + for fieldname in S._names: + FIELDTYPE = S._flds[fieldname] + if max_n is not None and fieldname == S._arrayfld: + cls = _build_ctypes_array(FIELDTYPE, max_n) + else: + cls = _get_ctypes_type(FIELDTYPE) + fields.append((fieldname, cls)) + + class CStruct(_parentable): + _fields_ = fields + _TYPE = S + + def malloc(cls, n=None): + S = cls._TYPE + if S._arrayfld is None: + if n is not None: + raise TypeError("%r is not variable-sized" % (S,)) + return ctypes.pointer(cls()) + else: + if n is None: + raise TypeError("%r is variable-sized" % (S,)) + smallercls = _build_ctypes_struct(S, n) + smallstruct = smallercls() + getattr(smallstruct, S._arrayfld).length = n + structptr = ctypes.cast(ctypes.pointer(smallstruct), + ctypes.POINTER(cls)) + return structptr + malloc = classmethod(malloc) + + CStruct.__name__ = 'ctypes_%s' % (S,) + return CStruct + +def _build_ctypes_array(A, max_n=None): + ITEM = A.OF + ctypes_item = _get_ctypes_type(ITEM) + if max_n is None: + max_n = sys.maxint // ctypes.sizeof(ctypes_item) + max_n //= 2 # XXX better safe than sorry about ctypes bugs + + class CArray(_parentable): + _fields_ = [('length', ctypes.c_int), + ('items', max_n * ctypes_item)] + _TYPE = A + + def malloc(cls, n=None): + if not isinstance(n, int): + raise TypeError, "array length must be an int" + smallercls = _build_ctypes_array(cls._TYPE, n) + smallarray = smallercls() + smallarray.length = n + arrayptr = ctypes.cast(ctypes.pointer(smallarray), + ctypes.POINTER(cls)) + return arrayptr + malloc = classmethod(malloc) + + CArray.__name__ = 'ctypes_%s' % (A,) + return CArray + +def _get_ctypes_type(T): + try: + return _ctypes_cache[T] + except KeyError: + if isinstance(T, Ptr): + cls = ctypes.POINTER(_get_ctypes_type(T.TO)) + elif isinstance(T, Struct): + cls = _build_ctypes_struct(T) + elif isinstance(T, Array): + cls = _build_ctypes_array(T) + else: + raise NotImplementedError(T) + _ctypes_cache[T] = cls + return cls + +def _expose(val): + if isinstance(type(val), _Ctypes_PointerType): + val = val.contents + T = typeOf(val) + if isinstance(T, ContainerType): + val = _ptr(ctypes.pointer(val)) + return val + +def _lltype2ctypes(val): + T = typeOf(val) + if isinstance(T, Ptr): + return val._storageptr + return val + +class _ptr(object): + __slots__ = ['_storageptr', '_TYPE'] + + def __init__(self, storageptr): + assert isinstance(type(storageptr), _Ctypes_PointerType) + _ptr._storageptr.__set__(self, storageptr) + _ptr._TYPE.__set__(self, Ptr(type(storageptr)._type_._TYPE)) + + def __getattr__(self, field_name): + if isinstance(self._TYPE.TO, Struct): + if field_name in self._TYPE.TO._flds: + return _expose(getattr(self._storageptr.contents, field_name)) + raise AttributeError("%r instance has no field %r" % (self._TYPE.TO, + field_name)) + + def __setattr__(self, field_name, value): + if isinstance(self._TYPE.TO, Struct): + if field_name in self._TYPE.TO._flds: + setattr(self._storageptr.contents, field_name, + _lltype2ctypes(value)) + return + raise AttributeError("%r instance has no field %r" % (self._TYPE.TO, + field_name)) + + def __nonzero__(self): + return bool(self._storageptr) + + def __len__(self): + T = self._TYPE.TO + if isinstance(T, Array):# ,FixedSizeArray)): + #if self._T._hints.get('nolength', False): + # raise TypeError("%r instance has no length attribute" % + # (self._T,)) + return self._storageptr.contents.length + raise TypeError("%r instance is not an array" % (T,)) + + def __getitem__(self, i): + T = self._TYPE.TO + if isinstance(T, Array): + start, stop = 0, self._storageptr.contents.length + if not (start <= i < stop): + if isinstance(i, slice): + raise TypeError("array slicing not supported") + raise IndexError("array index out of bounds") + return _expose(self._storageptr.contents.items[i]) + raise TypeError("%r instance is not an array" % (T,)) + + def _cast_to(self, PTRTYPE): + CURTYPE = self._TYPE + down_or_up = castable(PTRTYPE, CURTYPE) + if down_or_up == 0: + return self + if not self: # null pointer cast + return PTRTYPE._defl() + WAAA + if isinstance(self._obj, int): + return _ptr(PTRTYPE, self._obj, solid=True) + if down_or_up > 0: + p = self + while down_or_up: + p = getattr(p, typeOf(p).TO._names[0]) + down_or_up -= 1 + return _ptr(PTRTYPE, p._obj, solid=self._solid) + u = -down_or_up + struc = self._obj + while u: + parent = struc._parentstructure() + if parent is None: + raise RuntimeError("widening to trash: %r" % self) + PARENTTYPE = struc._parent_type + if getattr(parent, PARENTTYPE._names[0]) is not struc: + raise InvalidCast(CURTYPE, PTRTYPE) # xxx different exception perhaps? + struc = parent + u -= 1 + if PARENTTYPE != PTRTYPE.TO: + raise RuntimeError("widening %r inside %r instead of %r" % (CURTYPE, PARENTTYPE, PTRTYPE.TO)) + return _ptr(PTRTYPE, struc, solid=self._solid) + +# ____________________________________________________________ + +def malloc(T, n=None, immortal=False): + if T._gckind != 'gc' and not immortal:# and flavor.startswith('gc'): + raise TypeError, "gc flavor malloc of a non-GC non-immortal structure" + if isinstance(T, (Struct, Array)): + cls = _get_ctypes_type(T) + return _ptr(cls.malloc(n)) + else: + raise TypeError, "malloc for Structs and Arrays only" + +def nullptr(T): + cls = _get_ctypes_type(T) + return _ptr(ctypes.POINTER(cls)()) From bea at codespeak.net Thu Jul 12 18:05:28 2007 From: bea at codespeak.net (bea at codespeak.net) Date: Thu, 12 Jul 2007 18:05:28 +0200 (CEST) Subject: [pypy-svn] r44979 - pypy/extradoc/talk/ep2007 Message-ID: <20070712160528.7B3A88274@code0.codespeak.net> Author: bea Date: Thu Jul 12 18:05:26 2007 New Revision: 44979 Added: pypy/extradoc/talk/ep2007/openspace_discussions.txt Log: A summary of the post-its and discussions at the open space discussion at ep2007 - please review and adjust We should post a news item in pypy page referring to this summar and the talks done as well as post open space summary n europython wiki Added: pypy/extradoc/talk/ep2007/openspace_discussions.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ep2007/openspace_discussions.txt Thu Jul 12 18:05:26 2007 @@ -0,0 +1,108 @@ + +What do you want PyPy to do? +Open Space session at EP2007 2007-07-11 +================================================ +Purpose: +- collect information about needs from PyPy and + would-be PyPy users +- answers from PyPy developers when/where applicable + +How was it done: +- everyone present wrote their primary question + and/or interest on a post-it +- these were grouped to identify overlapping areas + interest, largest areas were adressed first +- a summary of the questions/interests grouped per + area can be found below and the "answers" discussed + +Areas of questions/interests: + +RPython: stand alone and/or interfacing external libraries +-------------------------------------------------------------- +Questions/interests: +- using RPython in CPython (unless PyPy can run all my Python + 2 code next month) +- how many are interested in RPython (in CPython, support + available, stability) +- RPython for Python extensions +- external library interfacing (extension modules) +- 3D graphics: Fast floating points? Fast math? + +Answers/comments: +- CPython API support in PyPy is slow +- RPython not complete yet (ex. ref counting) +- The state of RCtypes (support it, refactor and change + direction, future at all) is unclear - needs discussions + (how - to use ctypes at (PyPy) application level?) +- we need strategic discussions regarding how to interface + with external clibraries, to be discussed during sprint +- general need for refactoring - timespan will be roughly + a year to have a PyPy that usable (while still not covering + all extension modules) +- for stand alone RPython applications there is need for usability + fixes (shared interest) and speed/IO extensions + (app-level solution) +- regarding the question on 3D graphics (fast floating points, fast + math) the answer is that it is fairly easy to add this and there is + a matching interest in the PyPy dev-team in this area. + The need to display/visualize is then a matter of general C extension + interfacing + +Suggestions: +- RCtypes: let?s experiment here and discuss approaches during the sprint + +Javascript backend/related areas +------------------------------------- +Questions: +- I want to write a parser in Python and port it to javascript +- usable javascript backend +- emulate pygame on flash player + +Answers/comments: +- a parser is doable but you will encounter bugs, report them + and they will be fixed - there is a matching interest here + in the PyPy dev-team +- there is also a document on http://codespeak.net/pypyXXX + regarding parser creation +- regarding emulating PyGame on a flahs player - this is doable + but work needed, some rough estimates discussed were between + 2-4 months because it?s connected to tweaking a backend or + creating a new backend (which is an area we know a lot about;-) + So doable, requires work and we are not too sure about the results ;-) + +Refactoring/usability +-------------------------------- +Questions: +- how pypy goes about maintaining parts? +- SLES/Solaris package? Light production usage? +- be less hackish to use? +- pypy 1.1.0 planning, which cleanups are we to tackle + and who? +- fixing/improving stackless, pickling and OO-support + +Answers/comments: +- this was not discussed in particular due to time constraints +- was also not prioritized because it contained several PyPy + developers interests and we prioritized to try to answer + "users/would-be "users" interests +- it was apparent though during the entire discussion that + we where discussing answers in the form of "doable" and + "refactoring needed" which is what this area is all about + +Suggestions: +- general refactoring/technical direction discussion in + connection to the sprint + + +Other areas of interests/questions +------------------------------------- +Questions: +- reducing the memory footprint of the Python interpreter + (I want to run preferable several, securely separated interpreters + on a resource constrained system (a phone specifically) +- different language interpreters/front-ends + +Answers/comments: +- these were not discussed due to time constraints +- they will be taken into account when discussing general + refactoring/technical direction From tismer at codespeak.net Thu Jul 12 18:08:27 2007 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 12 Jul 2007 18:08:27 +0200 (CEST) Subject: [pypy-svn] r44980 - pypy/extradoc/talk/ep2007 Message-ID: <20070712160827.6E726825C@code0.codespeak.net> Author: tismer Date: Thu Jul 12 18:08:26 2007 New Revision: 44980 Added: pypy/extradoc/talk/ep2007/The Essentials Of Stackless Python V2.pdf (contents, props changed) Log: pdv version of my talk Added: pypy/extradoc/talk/ep2007/The Essentials Of Stackless Python V2.pdf ============================================================================== Binary file. No diff available. From mwh at codespeak.net Thu Jul 12 18:30:25 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 12 Jul 2007 18:30:25 +0200 (CEST) Subject: [pypy-svn] r44981 - pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src Message-ID: <20070712163025.6665581E2@code0.codespeak.net> Author: mwh Date: Thu Jul 12 18:30:21 2007 New Revision: 44981 Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h Log: *coughcoughcough* Modified: pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h ============================================================================== --- pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h (original) +++ pypy/branch/emptying-the-malloc-zoo-2/pypy/translator/c/src/mem.h Thu Jul 12 18:30:21 2007 @@ -116,10 +116,8 @@ #define OP_BOEHM_ZERO_MALLOC(size, r, restype, is_atomic, is_varsize) { \ r = (restype) BOEHM_MALLOC_ ## is_atomic ## _ ## is_varsize (size); \ - else { \ - if (is_atomic) /* the non-atomic versions return cleared memory */ \ - memset((void*) r, 0, size); \ - } \ + if (r && is_atomic) /* the non-atomic versions return cleared memory */ \ + memset((void*) r, 0, size); \ } /* as we said in rbuiltin.py: From pdg at codespeak.net Thu Jul 12 20:14:38 2007 From: pdg at codespeak.net (pdg at codespeak.net) Date: Thu, 12 Jul 2007 20:14:38 +0200 (CEST) Subject: [pypy-svn] r44982 - in pypy/dist/pypy/translator/jvm: . test Message-ID: <20070712181438.58E5E81CE@code0.codespeak.net> Author: pdg Date: Thu Jul 12 20:14:34 2007 New Revision: 44982 Modified: pypy/dist/pypy/translator/jvm/generator.py pypy/dist/pypy/translator/jvm/test/test_int.py pypy/dist/pypy/translator/jvm/test/test_overflow.py Log: \translator\jvm - fixed a typo so that all int tests pass (pdg) Modified: pypy/dist/pypy/translator/jvm/generator.py ============================================================================== --- pypy/dist/pypy/translator/jvm/generator.py (original) +++ pypy/dist/pypy/translator/jvm/generator.py Thu Jul 12 20:14:34 2007 @@ -369,7 +369,7 @@ LABSOVF = Method.s(jPyPy, 'abs_ovf', (jLong,), jLong) MATHDABS = Method.s(jMath, 'abs', (jDouble,), jDouble) INEGOVF = Method.s(jPyPy, 'negate_ovf', (jInt,), jInt) -LNEGOVF = Method.s(jPyPy, 'negate_ovf', (jInt,), jInt) +LNEGOVF = Method.s(jPyPy, 'negate_ovf', (jLong,), jLong) IADDOVF = Method.s(jPyPy, 'add_ovf', (jInt, jInt), jInt) LADDOVF = Method.s(jPyPy, 'add_ovf', (jLong, jLong), jLong) ISUBOVF = Method.s(jPyPy, 'sub_ovf', (jInt, jInt), jInt) Modified: pypy/dist/pypy/translator/jvm/test/test_int.py ============================================================================== --- pypy/dist/pypy/translator/jvm/test/test_int.py (original) +++ pypy/dist/pypy/translator/jvm/test/test_int.py Thu Jul 12 20:14:34 2007 @@ -19,8 +19,4 @@ def test_rarithmetic(self): pass # does this make more sense in jvm - #There is an issue that it doesn't correctly call long code over int code - def test_neg_abs_ovf(self): - py.test.skip("Unaware how to handle overflow") - div_mod_iteration_count = 20 Modified: pypy/dist/pypy/translator/jvm/test/test_overflow.py ============================================================================== --- pypy/dist/pypy/translator/jvm/test/test_overflow.py (original) +++ pypy/dist/pypy/translator/jvm/test/test_overflow.py Thu Jul 12 20:14:34 2007 @@ -1,3 +1,4 @@ +#Why can't I import check? #from pypy.translator.jvm.test.runtest import check import py from pypy.translator.jvm.test.runtest import JvmTest @@ -14,7 +15,7 @@ # return ovfcheck(x+y) # except OverflowError: # return 42 -# + #def op_sub(x, y): # try: # return ovfcheck(x-y) @@ -45,3 +46,4 @@ # yield check, op_mul, [int, int], (sys.maxint/2 + 1, 2) # yield check, op_lshift, [int, int], (2, 30) # yield check, op_neg, [int], (-sys.maxint-1,) + From mwh at codespeak.net Fri Jul 13 09:21:04 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 13 Jul 2007 09:21:04 +0200 (CEST) Subject: [pypy-svn] r44994 - in pypy/dist: lib-python/2.4.1/test pypy/annotation pypy/lib pypy/module/pypyjit/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/memory pypy/rpython/memory/gctransform pypy/rpython/memory/gctransform/test pypy/translator/c pypy/translator/c/src pypy/translator/c/test Message-ID: <20070713072104.9D53D81DF@code0.codespeak.net> Author: mwh Date: Fri Jul 13 09:21:00 2007 New Revision: 44994 Modified: pypy/dist/lib-python/2.4.1/test/test_threading.py pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/policy.py pypy/dist/pypy/lib/_structseq.py pypy/dist/pypy/module/pypyjit/test/test_pypy_c.py pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/memory/gc.py pypy/dist/pypy/rpython/memory/gctransform/boehm.py pypy/dist/pypy/rpython/memory/gctransform/framework.py pypy/dist/pypy/rpython/memory/gctransform/refcounting.py pypy/dist/pypy/rpython/memory/gctransform/test/test_boehm.py pypy/dist/pypy/rpython/memory/gctransform/test/test_refcounting.py pypy/dist/pypy/rpython/memory/gctransform/test/test_transform.py pypy/dist/pypy/rpython/memory/gctransform/transform.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/gc.py pypy/dist/pypy/translator/c/src/mem.h pypy/dist/pypy/translator/c/support.py pypy/dist/pypy/translator/c/test/test_boehm.py Log: (pedronis, mwh) Merge our emptying-the-malloc-zoo-2 branch. We simplified and reduced the memory operations a backend needs to implement, especially when using Boehm. ------------------------------------------------------------------------ r44950 | mwh | 2007-07-12 15:08:18 +0300 (Thu, 12 Jul 2007) | 3 lines (pedronis, mwh) new branch for merge forward ------------------------------------------------------------------------ r44952 | mwh | 2007-07-12 15:15:34 +0300 (Thu, 12 Jul 2007) | 3 lines (pedronis, mwh and his Bazaar T-shirt) merge changes from emptying-the-malloc-zoo ------------------------------------------------------------------------ r44962 | mwh | 2007-07-12 16:14:13 +0300 (Thu, 12 Jul 2007) | 3 lines (mwh, pedronis) cruft-- (in very small units) ------------------------------------------------------------------------ r44963 | mwh | 2007-07-12 16:21:12 +0300 (Thu, 12 Jul 2007) | 1 line (pedronis, mwh) these lines were useless and confusing ------------------------------------------------------------------------ r44966 | mwh | 2007-07-12 16:40:21 +0300 (Thu, 12 Jul 2007) | 2 lines move BasicGcPolicy fractionally towards being complete ------------------------------------------------------------------------ r44967 | mwh | 2007-07-12 16:43:43 +0300 (Thu, 12 Jul 2007) | 6 lines (pedronis, mwh) make the boehm gc transformer do most of the things that are now done in the backend. add more tests for the boehm gc transformer, and support for a couple of new operations to the llinterp. ------------------------------------------------------------------------ r44968 | mwh | 2007-07-12 16:46:24 +0300 (Thu, 12 Jul 2007) | 3 lines (mwh, pedronis) don't inline the varsize malloc helpers ------------------------------------------------------------------------ r44977 | mwh | 2007-07-12 18:33:45 +0300 (Thu, 12 Jul 2007) | 4 lines (mwh, pedronis) make the boehm gc transformer do way more and the boehm gcpolicy in genc do way less ------------------------------------------------------------------------ r44981 | mwh | 2007-07-12 19:30:21 +0300 (Thu, 12 Jul 2007) | 2 lines *coughcoughcough* ------------------------------------------------------------------------ Modified: pypy/dist/lib-python/2.4.1/test/test_threading.py ============================================================================== --- pypy/dist/lib-python/2.4.1/test/test_threading.py (original) +++ pypy/dist/lib-python/2.4.1/test/test_threading.py Fri Jul 13 09:21:00 2007 @@ -2,54 +2,61 @@ # Create a bunch of threads, let each do some work, wait until all are done +print 'a' from test.test_support import verbose +print 'a' import random +print 'a' import threading +print 'a' import time +print 'a' -# This takes about n/3 seconds to run (about n/3 clumps of tasks, times -# about 1 second per clump). -numtasks = 10 - -# no more than 3 of the 10 can run at once -sema = threading.BoundedSemaphore(value=3) -mutex = threading.RLock() -running = 0 - -class TestThread(threading.Thread): - def run(self): - global running - delay = random.random() * 2 - if verbose: - print 'task', self.getName(), 'will run for', delay, 'sec' - sema.acquire() - mutex.acquire() - running = running + 1 - if verbose: - print running, 'tasks are running' - mutex.release() - time.sleep(delay) - if verbose: - print 'task', self.getName(), 'done' - mutex.acquire() - running = running - 1 - if verbose: - print self.getName(), 'is finished.', running, 'tasks are running' - mutex.release() - sema.release() - -threads = [] -def starttasks(): - for i in range(numtasks): - t = TestThread(name=""%i) - threads.append(t) - t.start() - -starttasks() - -if verbose: - print 'waiting for all tasks to complete' -for t in threads: - t.join() -if verbose: - print 'all tasks done' +print 'a' +def test_main(): + # This takes about n/3 seconds to run (about n/3 clumps of tasks, times + # about 1 second per clump). + numtasks = 10 + + # no more than 3 of the 10 can run at once + sema = threading.BoundedSemaphore(value=3) + mutex = threading.RLock() + running = 0 + + class TestThread(threading.Thread): + def run(self): + global running + delay = random.random() * 2 + if verbose: + print 'task', self.getName(), 'will run for', delay, 'sec' + sema.acquire() + mutex.acquire() + running = running + 1 + if verbose: + print running, 'tasks are running' + mutex.release() + time.sleep(delay) + if verbose: + print 'task', self.getName(), 'done' + mutex.acquire() + running = running - 1 + if verbose: + print self.getName(), 'is finished.', running, 'tasks are running' + mutex.release() + sema.release() + + threads = [] + def starttasks(): + for i in range(numtasks): + t = TestThread(name=""%i) + threads.append(t) + t.start() + + starttasks() + + if verbose: + print 'waiting for all tasks to complete' + for t in threads: + t.join() + if verbose: + print 'all tasks done' Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Fri Jul 13 09:21:00 2007 @@ -229,6 +229,8 @@ if clsdef.issubclass(clsdef2) and clsdef is not clsdef2: del self.needs_hash_support[clsdef] break + + self.annotator.policy.compute_at_fixpoint(self.annotator) finally: self.leave() @@ -531,6 +533,9 @@ return result def see_mutable(self, x): + if x.__class__.__name__ == 'ExecutionContext': + import pdb + pdb.set_trace() if x in self.seen_mutable: return clsdef = self.getuniqueclassdef(x.__class__) Modified: pypy/dist/pypy/annotation/policy.py ============================================================================== --- pypy/dist/pypy/annotation/policy.py (original) +++ pypy/dist/pypy/annotation/policy.py Fri Jul 13 09:21:00 2007 @@ -13,6 +13,9 @@ class BasicAnnotatorPolicy(object): allow_someobjects = True + def compute_at_fixpoint(self, annotator): + pass + def event(pol, bookkeeper, what, *args): pass Modified: pypy/dist/pypy/lib/_structseq.py ============================================================================== --- pypy/dist/pypy/lib/_structseq.py (original) +++ pypy/dist/pypy/lib/_structseq.py Fri Jul 13 09:21:00 2007 @@ -23,7 +23,7 @@ if obj is None: return self if self.index is None: - return obj.__dict__[self.__name__] + return getattr(obj, 'data_' + self.__name__) else: return obj[self.index] @@ -36,11 +36,13 @@ def __new__(metacls, classname, bases, dict): assert not bases fields_by_index = {} + slots = [] for name, field in dict.items(): if isinstance(field, structseqfield): assert field._index not in fields_by_index fields_by_index[field._index] = field field.__name__ = name + slots.append('data_' + name) dict['n_fields'] = len(fields_by_index) extra_fields = fields_by_index.items() @@ -60,6 +62,7 @@ dict['_extra_fields'] = tuple(extra_fields) dict['__new__'] = structseq_new dict['__reduce__'] = structseq_reduce + dict['__slots__'] = tuple(slots) return type.__new__(metacls, classname, (tuple,), dict) @@ -91,11 +94,12 @@ dict[name] = value sequence = sequence[:N] result = tuple.__new__(cls, sequence) - result.__dict__ = dict for field in cls._extra_fields: name = field.__name__ if name not in dict: dict[name] = field._default(result) + for k, v in dict.iteritems(): + setattr(result, k, v) return result def structseq_reduce(self): Modified: pypy/dist/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/dist/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/dist/pypy/module/pypyjit/test/test_pypy_c.py Fri Jul 13 09:21:00 2007 @@ -38,7 +38,7 @@ child_stdout.close() assert result assert result.splitlines()[-1].strip() == 'OK :-)' - assert logfilepath.check() + #assert logfilepath.check() def test_f(self): Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Jul 13 09:21:00 2007 @@ -758,6 +758,8 @@ assert lltype.typeOf(size) == lltype.Signed return self.heap.raw_malloc(size) + op_boehm_malloc = op_boehm_malloc_atomic = op_raw_malloc + def op_raw_malloc_usage(self, size): assert lltype.typeOf(size) == lltype.Signed return self.heap.raw_malloc_usage(size) Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Fri Jul 13 09:21:00 2007 @@ -22,8 +22,8 @@ return NotImplemented return CompositeOffset(self, other) - def raw_malloc(self, rest, zero): - raise NotImplementedError("raw_malloc(%r, %r)" % (self, rest)) + def _raw_malloc(self, rest, zero): + raise NotImplementedError("_raw_malloc(%r, %r)" % (self, rest)) def raw_memcopy(self, srcadr, dstsrc): raise NotImplementedError("raw_memcopy(%r)" % (self,)) @@ -67,7 +67,7 @@ else: raise TypeError('got %r, expected %r' % (A, self.TYPE)) - def raw_malloc(self, rest, zero): + def _raw_malloc(self, rest, zero): assert not rest if (isinstance(self.TYPE, lltype.ContainerType) and self.TYPE._gckind == 'gc'): @@ -118,12 +118,12 @@ else: return lltype.direct_fieldptr(struct, self.fldname) - def raw_malloc(self, rest, parenttype=None, zero=False): + def _raw_malloc(self, rest, parenttype=None, zero=False): if self.fldname != self.TYPE._arrayfld: # for the error msg - return AddressOffset.raw_malloc(self, rest, zero=zero) + return AddressOffset._raw_malloc(self, rest, zero=zero) assert rest - return rest[0].raw_malloc(rest[1:], parenttype=parenttype or self.TYPE, + return rest[0]._raw_malloc(rest[1:], parenttype=parenttype or self.TYPE, zero=zero) def raw_memcopy(self, srcadr, dstadr): @@ -171,8 +171,8 @@ ptr = item.ref(ptr) return ptr - def raw_malloc(self, rest, zero): - return self.offsets[0].raw_malloc(self.offsets[1:] + rest, zero=zero) + def _raw_malloc(self, rest, zero): + return self.offsets[0]._raw_malloc(self.offsets[1:] + rest, zero=zero) def raw_memcopy(self, srcadr, dstadr): for o in self.offsets[:-1]: @@ -198,7 +198,7 @@ else: return lltype.direct_arrayitems(arrayptr) - def raw_malloc(self, rest, parenttype=None, zero=False): + def _raw_malloc(self, rest, parenttype=None, zero=False): if rest: assert len(rest) == 1 assert isinstance(rest[0], ItemOffset) @@ -245,11 +245,11 @@ gcptr = self.gcheaderbuilder.object_from_header(headerptr) return gcptr - def raw_malloc(self, rest, zero): + def _raw_malloc(self, rest, zero): assert rest if isinstance(rest[0], GCHeaderAntiOffset): - return rest[1].raw_malloc(rest[2:], zero=zero) # just for fun - gcobjadr = rest[0].raw_malloc(rest[1:], zero=zero) + return rest[1]._raw_malloc(rest[2:], zero=zero) # just for fun + gcobjadr = rest[0]._raw_malloc(rest[1:], zero=zero) headerptr = self.gcheaderbuilder.new_header(gcobjadr.ptr) return cast_ptr_to_adr(headerptr) @@ -271,10 +271,10 @@ headerptr = self.gcheaderbuilder.header_of_object(gcptr) return headerptr - def raw_malloc(self, rest, zero): + def _raw_malloc(self, rest, zero): assert len(rest) >= 2 assert isinstance(rest[0], GCHeaderOffset) - return rest[1].raw_malloc(rest[2:], zero=zero) + return rest[1]._raw_malloc(rest[2:], zero=zero) # ____________________________________________________________ @@ -543,7 +543,7 @@ def raw_malloc(size): if not isinstance(size, AddressOffset): raise NotImplementedError(size) - return size.raw_malloc([], zero=False) + return size._raw_malloc([], zero=False) def raw_free(adr): # try to free the whole object if 'adr' is the address of the header @@ -568,7 +568,7 @@ if not isinstance(size, AddressOffset): raise NotImplementedError(size) assert lltype.typeOf(adr) == Address - zeroadr = size.raw_malloc([], zero=True) + zeroadr = size._raw_malloc([], zero=True) size.raw_memcopy(zeroadr, adr) def raw_memcopy(source, dest, size): @@ -615,7 +615,7 @@ self.unitsize = unitsize self.n = n - def raw_malloc(self, rest, zero=False): + def _raw_malloc(self, rest, zero=False): assert not rest arena = _arena(self, zero=zero) return arena.getitemaddr(0) Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Fri Jul 13 09:21:00 2007 @@ -339,7 +339,10 @@ # __________ address operations __________ - 'raw_malloc': LLOp(canraise=(MemoryError,)), + 'boehm_malloc': LLOp(), + 'boehm_malloc_atomic': LLOp(), + 'boehm_register_finalizer': LLOp(), + 'raw_malloc': LLOp(), 'raw_malloc_usage': LLOp(sideeffects=False), 'raw_free': LLOp(), 'raw_memclear': LLOp(), Modified: pypy/dist/pypy/rpython/memory/gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gc.py (original) +++ pypy/dist/pypy/rpython/memory/gc.py Fri Jul 13 09:21:00 2007 @@ -4,7 +4,7 @@ from pypy.rpython.memory.gcheader import GCHeaderBuilder from pypy.rpython.memory import lltypesimulation from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rlib.objectmodel import free_non_gc_object +from pypy.rlib.objectmodel import free_non_gc_object, debug_assert from pypy.rpython.lltypesystem.lloperation import llop from pypy.rlib.rarithmetic import ovfcheck @@ -97,7 +97,10 @@ size = self.fixed_size(typeid) if self.is_varsize(typeid): size += length * self.varsize_item_sizes(typeid) - return raw_malloc(size) + result = raw_malloc(size) + if not result: + raise memoryError + return result def collect(self): self.get_roots() #this is there so that the annotator thinks get_roots is a function @@ -182,6 +185,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) hdr.typeid = typeid << 1 if has_finalizer: @@ -208,6 +213,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError raw_memclear(result, tot_size) hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) hdr.typeid = typeid << 1 @@ -238,6 +245,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError (result + size_gc_header + offset_to_length).signed[0] = length hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) hdr.typeid = typeid << 1 @@ -270,6 +279,8 @@ except OverflowError: raise memoryError result = raw_malloc(tot_size) + if not result: + raise memoryError raw_memclear(result, tot_size) (result + size_gc_header + offset_to_length).signed[0] = length hdr = llmemory.cast_adr_to_ptr(result, self.HDRPTR) @@ -942,8 +953,10 @@ def setup(self): self.tospace = raw_malloc(self.space_size) + debug_assert(bool(self.tospace), "couldn't allocate tospace") self.top_of_space = self.tospace + self.space_size self.fromspace = raw_malloc(self.space_size) + debug_assert(bool(self.fromspace), "couldn't allocate fromspace") self.free = self.tospace def free_memory(self): @@ -1125,6 +1138,8 @@ size_gc_header = self.size_gc_header() result = raw_malloc(size + size_gc_header) ## print "mallocing %s, size %s at %s" % (typeid, size, result) + if not result: + raise memoryError result.signed[0] = 0 # refcount result.signed[1] = typeid return result + size_gc_header Modified: pypy/dist/pypy/rpython/memory/gctransform/boehm.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/boehm.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/boehm.py Fri Jul 13 09:21:00 2007 @@ -3,11 +3,58 @@ get_rtti, _static_deallocator_body_for_type, LLTransformerOp, ll_call_destructor from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.lltypesystem.lloperation import llop +from pypy.rpython import rmodel +from pypy.rlib.rarithmetic import ovfcheck +from pypy.objspace.flow.model import Constant class BoehmGCTransformer(GCTransformer): + FINALIZER_PTR = lltype.Ptr(lltype.FuncType([llmemory.Address], lltype.Void)) + def __init__(self, translator, inline=False): super(BoehmGCTransformer, self).__init__(translator, inline=inline) self.finalizer_funcptrs = {} + memoryError = MemoryError() + + def ll_malloc_fixedsize(size, finalizer): + result = llop.boehm_malloc(llmemory.Address, size) + if not result: + raise memoryError + if finalizer: # XXX runtime check here is bad? + llop.boehm_register_finalizer(lltype.Void, result, finalizer) + return result + def ll_malloc_fixedsize_atomic(size, finalizer): + result = llop.boehm_malloc_atomic(llmemory.Address, size) + if not result: + raise memoryError + if finalizer: # XXX runtime check here is bad? + llop.boehm_register_finalizer(lltype.Void, result, finalizer) + return result + # XXX, do we need/want an atomic version of this function? + def ll_malloc_varsize_no_length(length, size, itemsize): + try: + varsize = ovfcheck(itemsize * length) + tot_size = ovfcheck(size + varsize) + except OverflowError: + raise memoryError + result = llop.boehm_malloc(llmemory.Address, tot_size) + if not result: + raise memoryError + return result + def ll_malloc_varsize(length, size, itemsize, lengthoffset): + result = ll_malloc_varsize_no_length(length, size, itemsize) + (result + lengthoffset).signed[0] = length + return result + + if self.translator: + self.malloc_fixedsize_ptr = self.inittime_helper( + ll_malloc_fixedsize, [lltype.Signed, self.FINALIZER_PTR], llmemory.Address) + self.malloc_fixedsize_atomic_ptr = self.inittime_helper( + ll_malloc_fixedsize_atomic, [lltype.Signed, self.FINALIZER_PTR], llmemory.Address) + self.malloc_varsize_no_length_ptr = self.inittime_helper( + ll_malloc_varsize_no_length, [lltype.Signed]*3, llmemory.Address, inline=False) + self.malloc_varsize_ptr = self.inittime_helper( + ll_malloc_varsize, [lltype.Signed]*4, llmemory.Address, inline=False) + self.mixlevelannotator.finish() # for now def push_alive_nopyobj(self, var, llops): pass @@ -23,6 +70,62 @@ """ for boehm it is enough to do nothing""" pass + def gct_malloc(self, hop): + TYPE = hop.spaceop.result.concretetype.TO + assert not TYPE._is_varsize() + c_size = rmodel.inputconst(lltype.Signed, llmemory.sizeof(TYPE)) + if TYPE._is_atomic(): + funcptr = self.malloc_fixedsize_atomic_ptr + else: + funcptr = self.malloc_fixedsize_ptr + c_finalizer_ptr = Constant(self.finalizer_funcptr_for_type(TYPE), self.FINALIZER_PTR) + v_raw = hop.genop("direct_call", + [funcptr, c_size, c_finalizer_ptr], + resulttype=llmemory.Address) + hop.cast_result(v_raw) + + # XXX In theory this is wrong: + gct_zero_malloc = gct_malloc + + def gct_malloc_varsize(self, hop): + def intconst(c): return rmodel.inputconst(lltype.Signed, c) + + op = hop.spaceop + TYPE = op.result.concretetype.TO + assert TYPE._is_varsize() + + assert not self.finalizer_funcptr_for_type(TYPE) + + if isinstance(TYPE, lltype.Struct): + ARRAY = TYPE._flds[TYPE._arrayfld] + else: + ARRAY = TYPE + assert isinstance(ARRAY, lltype.Array) + if ARRAY._hints.get('isrpystring', False): + c_const_size = intconst(llmemory.sizeof(TYPE, 1)) + else: + c_const_size = intconst(llmemory.sizeof(TYPE, 0)) + c_item_size = intconst(llmemory.sizeof(ARRAY.OF)) + + if ARRAY._hints.get("nolength", False): + v_raw = hop.genop("direct_call", + [self.malloc_varsize_no_length_ptr, op.args[-1], + c_const_size, c_item_size], + resulttype=llmemory.Address) + else: + if isinstance(TYPE, lltype.Struct): + offset_to_length = llmemory.FieldOffset(TYPE, TYPE._arrayfld) + \ + llmemory.ArrayLengthOffset(ARRAY) + else: + offset_to_length = llmemory.ArrayLengthOffset(ARRAY) + v_raw = hop.genop("direct_call", + [self.malloc_varsize_ptr, op.args[-1], + c_const_size, c_item_size, intconst(offset_to_length)], + resulttype=llmemory.Address) + hop.cast_result(v_raw) + + gct_zero_malloc_varsize = gct_malloc_varsize + def finalizer_funcptr_for_type(self, TYPE): if TYPE in self.finalizer_funcptrs: return self.finalizer_funcptrs[TYPE] @@ -57,7 +160,7 @@ llop.gc_restore_exception(lltype.Void, exc_instance) fptr = self.annotate_helper(ll_finalizer, [llmemory.Address], lltype.Void) else: - fptr = None + fptr = lltype.nullptr(self.FINALIZER_PTR.TO) self.finalizer_funcptrs[TYPE] = fptr return fptr Modified: pypy/dist/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/framework.py Fri Jul 13 09:21:00 2007 @@ -6,6 +6,7 @@ from pypy.rpython.memory import gc, lladdress from pypy.rpython.memory.gcheader import GCHeaderBuilder from pypy.rlib.rarithmetic import ovfcheck +from pypy.rlib.objectmodel import debug_assert from pypy.translator.backendopt import graphanalyze from pypy.annotation import model as annmodel from pypy.rpython import annlowlevel @@ -251,6 +252,7 @@ _alloc_flavor_ = 'raw' def setup_root_stack(): stackbase = lladdress.raw_malloc(rootstacksize) + debug_assert(bool(stackbase), "could not allocate root stack") lladdress.raw_memclear(stackbase, rootstacksize) gcdata.root_stack_top = stackbase gcdata.root_stack_base = stackbase Modified: pypy/dist/pypy/rpython/memory/gctransform/refcounting.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/refcounting.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/refcounting.py Fri Jul 13 09:21:00 2007 @@ -45,7 +45,9 @@ # create incref, etc graph + memoryError = MemoryError() HDRPTR = lltype.Ptr(self.HDR) + def ll_incref(adr): if adr: gcheader = llmemory.cast_adr_to_ptr(adr - gc_header_offset, HDRPTR) @@ -71,6 +73,8 @@ def ll_malloc_fixedsize(size): size = gc_header_offset + size result = lladdress.raw_malloc(size) + if not result: + raise memoryError lladdress.raw_memclear(result, size) result += gc_header_offset return result @@ -80,8 +84,10 @@ varsize = ovfcheck(itemsize * length) tot_size = ovfcheck(fixsize + varsize) except OverflowError: - raise MemoryError + raise memoryError result = lladdress.raw_malloc(tot_size) + if not result: + raise memoryError lladdress.raw_memclear(result, tot_size) result += gc_header_offset return result @@ -149,6 +155,8 @@ resulttype=llmemory.Address) hop.cast_result(v_raw) + gct_zero_malloc = gct_malloc + def gct_malloc_varsize(self, hop): def intconst(c): return rmodel.inputconst(lltype.Signed, c) @@ -184,6 +192,8 @@ resulttype=llmemory.Address) hop.cast_result(v_raw) + gct_zero_malloc_varsize = gct_malloc_varsize + def gct_gc_deallocate(self, hop): TYPE = hop.spaceop.args[0].value v_addr = hop.spaceop.args[1] Modified: pypy/dist/pypy/rpython/memory/gctransform/test/test_boehm.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/test/test_boehm.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/test/test_boehm.py Fri Jul 13 09:21:00 2007 @@ -3,9 +3,14 @@ from pypy.rpython.memory.gctransform.test.test_refcounting import make_deallocator from pypy.rpython.lltypesystem import lltype from pypy.translator.translator import graphof +from pypy.translator.c.gc import BoehmGcPolicy +from pypy.rpython.memory.gctransform.test.test_transform import LLInterpedTranformerTests from pypy import conftest import py +class TestLLInterpedBoehm(LLInterpedTranformerTests): + gcpolicy = BoehmGcPolicy + def make_boehm_finalizer(TYPE): return make_deallocator(TYPE, attr="finalizer_funcptr_for_type", cls=BoehmGCTransformer) Modified: pypy/dist/pypy/rpython/memory/gctransform/test/test_refcounting.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/test/test_refcounting.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/test/test_refcounting.py Fri Jul 13 09:21:00 2007 @@ -1,124 +1,47 @@ from pypy.rpython.memory.gctransform.test.test_transform import rtype, rtype_and_transform, getops +from pypy.rpython.memory.gctransform.test.test_transform import LLInterpedTranformerTests from pypy.rpython.memory.gctransform.refcounting import RefcountingGCTransformer from pypy.rpython.lltypesystem import lltype -from pypy.translator.c.gc import RefcountingGcPolicy from pypy.translator.translator import TranslationContext, graphof +from pypy.translator.c.gc import RefcountingGcPolicy from pypy import conftest -class RefcountingGcPolicy2(RefcountingGcPolicy): - transformerclass = RefcountingGCTransformer - -def llinterpreter_for_refcounted_graph(f, args_s): - from pypy.rpython.llinterp import LLInterpreter - from pypy.translator.c.genc import CStandaloneBuilder - from pypy.translator.c import gc - - t = rtype(f, args_s) - cbuild = CStandaloneBuilder(t, f, t.config, gcpolicy=RefcountingGcPolicy2) - db = cbuild.generate_graphs_for_llinterp() - graph = cbuild.getentrypointptr()._obj.graph - llinterp = LLInterpreter(t.rtyper) - if conftest.option.view: - t.view() - return llinterp, graph - res = llinterp.eval_graph(graph, [0]) - assert res == f(0) - res = llinterp.eval_graph(graph, [1]) - assert res == f(1) - - -def test_llinterp_refcounted_graph(): - from pypy.annotation.model import SomeInteger - - class C: - pass - c = C() - c.x = 1 - def g(x): - if x: - return c - else: - d = C() - d.x = 2 - return d - def f(x): - return g(x).x - - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeInteger()]) - - res = llinterp.eval_graph(graph, [0]) - assert res == f(0) - res = llinterp.eval_graph(graph, [1]) - assert res == f(1) - -def test_llinterp_refcounted_graph_varsize(): - from pypy.annotation.model import SomeInteger +class TestLLInterpedRefcounting(LLInterpedTranformerTests): + gcpolicy = RefcountingGcPolicy - def f(x): - r = [] - for i in range(x): - if i % 2: - r.append(x) - return len(r) - - - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeInteger()]) - - res = llinterp.eval_graph(graph, [0]) - assert res == f(0) - res = llinterp.eval_graph(graph, [10]) - assert res == f(10) - -def test_llinterp_refcounted_graph_str(): - from pypy.annotation.model import SomeString - from pypy.rpython.lltypesystem.rstr import string_repr - - def f(x): - return len(x + 'a') - - - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeString()]) - - cc = string_repr.convert_const - - res = llinterp.eval_graph(graph, [cc('a')]) - assert res == f('a') - res = llinterp.eval_graph(graph, [cc('brrrrrr')]) - assert res == f('brrrrrr') - -def test_llinterp_refcounted_graph_with_del(): - from pypy.annotation.model import SomeInteger - - class D: - pass - - delcounter = D() - delcounter.dels = 0 - - class C: - def __del__(self): - delcounter.dels += 1 - c = C() - c.x = 1 - def h(x): - if x: - return c - else: - d = C() - d.x = 2 - return d - def g(x): - return h(x).x - def f(x): - r = g(x) - return r + delcounter.dels + def test_llinterp_refcounted_graph_with_del(self): + from pypy.annotation.model import SomeInteger - llinterp, graph = llinterpreter_for_refcounted_graph(f, [SomeInteger()]) + class D: + pass - res = llinterp.eval_graph(graph, [1]) - assert res == 1 - res = llinterp.eval_graph(graph, [0]) - assert res == 3 + delcounter = D() + delcounter.dels = 0 + + class C: + def __del__(self): + delcounter.dels += 1 + c = C() + c.x = 1 + def h(x): + if x: + return c + else: + d = C() + d.x = 2 + return d + def g(x): + return h(x).x + def f(x): + r = g(x) + return r + delcounter.dels + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeInteger()]) + + res = llinterp.eval_graph(graph, [1]) + assert res == 1 + res = llinterp.eval_graph(graph, [0]) + assert res == 3 def test_simple_barrier(): S = lltype.GcStruct("S", ('x', lltype.Signed)) Modified: pypy/dist/pypy/rpython/memory/gctransform/test/test_transform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/test/test_transform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/test/test_transform.py Fri Jul 13 09:21:00 2007 @@ -10,6 +10,83 @@ from pypy.rpython.extregistry import ExtRegistryEntry from pypy import conftest +class LLInterpedTranformerTests: + + def llinterpreter_for_transformed_graph(self, f, args_s): + from pypy.rpython.llinterp import LLInterpreter + from pypy.translator.c.genc import CStandaloneBuilder + from pypy.translator.c import gc + + t = rtype(f, args_s) + # XXX we shouldn't need an actual gcpolicy here. + cbuild = CStandaloneBuilder(t, f, t.config, gcpolicy=self.gcpolicy) + db = cbuild.generate_graphs_for_llinterp() + graph = cbuild.getentrypointptr()._obj.graph + llinterp = LLInterpreter(t.rtyper) + if conftest.option.view: + t.view() + return llinterp, graph + + + def test_simple(self): + from pypy.annotation.model import SomeInteger + + class C: + pass + c = C() + c.x = 1 + def g(x): + if x: + return c + else: + d = C() + d.x = 2 + return d + def f(x): + return g(x).x + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeInteger()]) + + res = llinterp.eval_graph(graph, [0]) + assert res == f(0) + res = llinterp.eval_graph(graph, [1]) + assert res == f(1) + + def test_simple_varsize(self): + from pypy.annotation.model import SomeInteger + + def f(x): + r = [] + for i in range(x): + if i % 2: + r.append(x) + return len(r) + + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeInteger()]) + + res = llinterp.eval_graph(graph, [0]) + assert res == f(0) + res = llinterp.eval_graph(graph, [10]) + assert res == f(10) + + def test_str(self): + from pypy.annotation.model import SomeString + from pypy.rpython.lltypesystem.rstr import string_repr + + def f(x): + return len(x + 'a') + + + llinterp, graph = self.llinterpreter_for_transformed_graph(f, [SomeString()]) + + cc = string_repr.convert_const + + res = llinterp.eval_graph(graph, [cc('a')]) + assert res == f('a') + res = llinterp.eval_graph(graph, [cc('brrrrrr')]) + assert res == f('brrrrrr') + class _TestGCTransformer(GCTransformer): def push_alive_nopyobj(self, var, llops): Modified: pypy/dist/pypy/rpython/memory/gctransform/transform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/transform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/transform.py Fri Jul 13 09:21:00 2007 @@ -243,8 +243,8 @@ self.graphs_to_inline[graph] = True return self.mixlevelannotator.graph2delayed(graph) - def inittime_helper(self, ll_helper, ll_args, ll_result): - ptr = self.annotate_helper(ll_helper, ll_args, ll_result, inline=True) + def inittime_helper(self, ll_helper, ll_args, ll_result, inline=True): + ptr = self.annotate_helper(ll_helper, ll_args, ll_result, inline=inline) return Constant(ptr, lltype.typeOf(ptr)) def finish_helpers(self): Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Jul 13 09:21:00 2007 @@ -510,62 +510,17 @@ self.expr(op.args[0]), self.expr(op.args[1])) - def OP_ZERO_MALLOC(self, op): - TYPE = self.lltypemap(op.result).TO - typename = self.db.gettype(TYPE) - eresult = self.expr(op.result) - esize = 'sizeof(%s)' % cdecl(typename, '') - - return self.gcpolicy.zero_malloc(TYPE, esize, eresult) - - def OP_MALLOC(self, op): - TYPE = self.lltypemap(op.result).TO - typename = self.db.gettype(TYPE) - eresult = self.expr(op.result) - esize = 'sizeof(%s)' % cdecl(typename, '') - - return self.gcpolicy.malloc(TYPE, esize, eresult) - - OP_ZERO_MALLOC = OP_MALLOC - - def OP_MALLOC_VARSIZE(self, op): - TYPE = self.lltypemap(op.result).TO - typename = self.db.gettype(TYPE) - lenfld = 'length' - nodedef = self.db.gettypedefnode(TYPE) - if isinstance(TYPE, Struct): - arfld = TYPE._arrayfld - lenfld = "%s.length" % nodedef.c_struct_field_name(arfld) - VARPART = TYPE._flds[TYPE._arrayfld] - else: - VARPART = TYPE - assert isinstance(VARPART, Array) - itemtypename = self.db.gettype(VARPART.OF) - elength = self.expr(op.args[1]) - eresult = self.expr(op.result) - if VARPART.OF is Void: # strange - esize = 'sizeof(%s)' % (cdecl(typename, ''),) - result = '{\n' - else: - itemtype = cdecl(itemtypename, '') - result = 'IF_VARSIZE_OVERFLOW(%s, %s, %s)\nelse {\n' % ( - elength, - itemtype, - eresult) - esize = 'sizeof(%s)-sizeof(%s)+%s*sizeof(%s)' % ( - cdecl(typename, ''), - itemtype, - elength, - itemtype) - result += self.gcpolicy.zero_malloc(TYPE, esize, eresult) - - # ctypes Arrays have no length field - if not VARPART._hints.get('nolength', False): - result += '\nif(%s) %s->%s = %s;' % (eresult, eresult, lenfld, elength) - result += '\n}' - return result - - OP_ZERO_MALLOC_VARSIZE = OP_MALLOC_VARSIZE + def OP_BOEHM_MALLOC(self, op): + return 'OP_BOEHM_ZERO_MALLOC(%s, %s, void*, 0, 0);' % (self.expr(op.args[0]), + self.expr(op.result)) + + def OP_BOEHM_MALLOC_ATOMIC(self, op): + return 'OP_BOEHM_ZERO_MALLOC(%s, %s, void*, 1, 0);' % (self.expr(op.args[0]), + self.expr(op.result)) + + def OP_BOEHM_REGISTER_FINALIZER(self, op): + return 'GC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' \ + % (self.expr(op.args[0]), self.expr(op.args[1])) def OP_RAW_MALLOC(self, op): eresult = self.expr(op.result) @@ -573,6 +528,7 @@ return "OP_RAW_MALLOC(%s, %s, void *);" % (esize, eresult) def OP_FLAVORED_MALLOC(self, op): + # XXX this function should DIE! TYPE = self.lltypemap(op.result).TO typename = self.db.gettype(TYPE) eresult = self.expr(op.result) @@ -590,6 +546,7 @@ raise NotImplementedError def OP_FLAVORED_MALLOC_VARSIZE(self, op): + # XXX this function should DIE!, at least twice over # XXX I know this working in just one case, probably makes # sense to assert it here, rest is just copied flavor = op.args[0].value Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Fri Jul 13 09:21:00 2007 @@ -48,6 +48,15 @@ def gc_startup_code(self): return [] + def struct_setup(self, structdefnode, rtti): + return None + + def array_setup(self, arraydefnode): + return None + + def rtti_type(self): + return '' + def OP_GC_PUSH_ALIVE_PYOBJ(self, funcgen, op): expr = funcgen.expr(op.args[0]) if expr == 'NULL': @@ -108,14 +117,6 @@ # zero malloc impl - def zero_malloc(self, TYPE, esize, eresult): - assert TYPE._gckind == 'gc' # we don't really support this - typename = self.db.gettype(TYPE) - erestype = cdecl(typename, '*') - return 'OP_ZERO_MALLOC(%s, %s, %s);' % (esize, eresult, erestype) - - malloc = zero_malloc - def OP_GC_CALL_RTTI_DESTRUCTOR(self, funcgen, op): args = [funcgen.expr(v) for v in op.args] line = '%s(%s);' % (args[0], ', '.join(args[1:])) @@ -171,18 +172,11 @@ class BoehmGcPolicy(BasicGcPolicy): transformerclass = boehm.BoehmGCTransformer - def setup_gcinfo(self, defnode): - transformer = defnode.db.gctransformer - fptr = transformer.finalizer_funcptr_for_type(defnode.LLTYPE) - if fptr: - defnode.gcinfo = BoehmInfo() - defnode.gcinfo.finalizer = defnode.db.get(fptr) - def array_setup(self, arraydefnode): - self.setup_gcinfo(arraydefnode) + pass def struct_setup(self, structdefnode, rtti): - self.setup_gcinfo(structdefnode) + pass def rtti_type(self): return BoehmGcRuntimeTypeInfo_OpaqueNode.typename @@ -190,25 +184,6 @@ def rtti_node_factory(self): return BoehmGcRuntimeTypeInfo_OpaqueNode - def zero_malloc(self, TYPE, esize, eresult): - gcinfo = self.db.gettypedefnode(TYPE).gcinfo - assert TYPE._gckind == 'gc' # _is_atomic() depends on this! - is_atomic = TYPE._is_atomic() - is_varsize = TYPE._is_varsize() - typename = self.db.gettype(TYPE) - erestype = cdecl(typename, '*') - result = 'OP_BOEHM_ZERO_MALLOC(%s, %s, %s, %d, %d);' % (esize, - eresult, - erestype, - is_atomic, - is_varsize) - if gcinfo and gcinfo.finalizer: - result += ('\nGC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' - % (eresult, gcinfo.finalizer)) - return result - - malloc = zero_malloc - def gc_libraries(self): if sys.platform == 'win32': return ['gc_pypy'] @@ -292,7 +267,6 @@ yield "#include " def struct_setup(self, structdefnode, rtti): - self.setup_gcinfo(structdefnode) T = structdefnode.STRUCT if T._is_atomic(): malloc_exact = False @@ -327,37 +301,11 @@ self.get_descr_name(defnode), T) yield "}" - def zero_malloc(self, TYPE, esize, eresult): - defnode = self.db.gettypedefnode(TYPE) - gcinfo = defnode.gcinfo - if gcinfo: - if not gcinfo.malloc_exact: - assert TYPE._gckind == 'gc' # _is_atomic() depends on this! - is_atomic = TYPE._is_atomic() - is_varsize = TYPE._is_varsize() - typename = self.db.gettype(TYPE) - erestype = cdecl(typename, '*') - result = 'OP_BOEHM_ZERO_MALLOC(%s, %s, %s, %d, %d);' % ( - esize, eresult, erestype, is_atomic, is_varsize) - else: - result = '%s = GC_MALLOC_EXPLICITLY_TYPED(%s, %s);' % ( - eresult, esize, self.get_descr_name(defnode)) - if gcinfo.finalizer: - result += ('\nGC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' - % (eresult, gcinfo.finalizer)) - else: - return super(MoreExactBoehmGcPolicy, self).zero_malloc( - TYPE, esize, eresult) - return result - - malloc = zero_malloc # to get an idea how it looks like with no refcount/gc at all class NoneGcPolicy(BoehmGcPolicy): - zero_malloc = RefcountingGcPolicy.zero_malloc.im_func - malloc = RefcountingGcPolicy.malloc.im_func gc_libraries = RefcountingGcPolicy.gc_libraries.im_func gc_startup_code = RefcountingGcPolicy.gc_startup_code.im_func @@ -412,11 +360,6 @@ o = top_container(defnode.obj) return defnode.db.gctransformer.gc_field_values_for(o) - def zero_malloc(self, TYPE, esize, eresult): - assert False, "a malloc operation in a framework build??" - - malloc = zero_malloc - class StacklessFrameworkGcPolicy(FrameworkGcPolicy): transformerclass = stacklessframework.StacklessFrameworkGCTransformer requires_stackless = True Modified: pypy/dist/pypy/translator/c/src/mem.h ============================================================================== --- pypy/dist/pypy/translator/c/src/mem.h (original) +++ pypy/dist/pypy/translator/c/src/mem.h Fri Jul 13 09:21:00 2007 @@ -8,11 +8,7 @@ #define OP_RAW_MALLOC(size, r, restype) { \ r = (restype) PyObject_Malloc(size); \ - if (r == NULL) { \ - FAIL_EXCEPTION(PyExc_MemoryError, \ - "out of memory"); \ - } \ - else { \ + if (r != NULL) { \ memset((void*)r, 0, size); \ COUNT_MALLOC; \ } \ @@ -22,13 +18,9 @@ #define OP_RAW_MALLOC(size, r, restype) { \ r = (restype) PyObject_Malloc(size); \ - if (r == NULL) { \ - FAIL_EXCEPTION(PyExc_MemoryError, \ - "out of memory"); \ - } \ - else { \ + if (r != NULL) { \ COUNT_MALLOC; \ - } \ + } \ } #endif @@ -124,11 +116,8 @@ #define OP_BOEHM_ZERO_MALLOC(size, r, restype, is_atomic, is_varsize) { \ r = (restype) BOEHM_MALLOC_ ## is_atomic ## _ ## is_varsize (size); \ - if (r == NULL) {FAIL_EXCEPTION(PyExc_MemoryError, "out of memory");} \ - else { \ - if (is_atomic) /* the non-atomic versions return cleared memory */ \ - memset((void*) r, 0, size); \ - } \ + if (r && is_atomic) /* the non-atomic versions return cleared memory */ \ + memset((void*) r, 0, size); \ } /* as we said in rbuiltin.py: Modified: pypy/dist/pypy/translator/c/support.py ============================================================================== --- pypy/dist/pypy/translator/c/support.py (original) +++ pypy/dist/pypy/translator/c/support.py Fri Jul 13 09:21:00 2007 @@ -52,6 +52,10 @@ else: return s[:2].lower() +def is_pointer_to_forward_ref(T): + if not isinstance(T, lltype.Ptr): + return False + return isinstance(T.TO, lltype.ForwardReference) def llvalue_from_constant(c): try: @@ -64,7 +68,12 @@ if T == lltype.Void: return None else: - assert lltype.typeOf(c.value) == T + ACTUAL_TYPE = lltype.typeOf(c.value) + # If the type is still uncomputed, we can't make this + # check. Something else will blow up instead, probably + # very confusingly. + if not is_pointer_to_forward_ref(ACTUAL_TYPE): + assert ACTUAL_TYPE == T return c.value Modified: pypy/dist/pypy/translator/c/test/test_boehm.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_boehm.py (original) +++ pypy/dist/pypy/translator/c/test/test_boehm.py Fri Jul 13 09:21:00 2007 @@ -1,6 +1,6 @@ import py from pypy.translator.translator import TranslationContext -from pypy.rpython.lltypesystem.lltype import Void +from pypy.rpython.lltypesystem import lltype from pypy.translator.tool.cbuild import check_boehm_presence from pypy.translator.c.genc import CExtModuleBuilder from pypy import conftest @@ -62,7 +62,6 @@ def test__del__(self): from pypy.rpython.lltypesystem.lloperation import llop - from pypy.rpython.lltypesystem import lltype class State: pass s = State() @@ -98,7 +97,6 @@ def test_weakgcaddress_is_weak(self): from pypy.rpython.lltypesystem.lloperation import llop - from pypy.rpython.lltypesystem import lltype from pypy.rlib.objectmodel import cast_object_to_weakgcaddress class State: pass @@ -130,7 +128,6 @@ def test_del_raises(self): from pypy.rpython.lltypesystem.lloperation import llop - from pypy.rpython.lltypesystem import lltype import os class A(object): def __del__(self): @@ -160,7 +157,6 @@ assert res > 0 def test_memory_error_varsize(self): - from pypy.rpython.lltypesystem import lltype N = int(2**31-1) A = lltype.GcArray(lltype.Char) def alloc(n): @@ -183,7 +179,7 @@ def prob_with_pyobj(b): return 3, b def collect(): - llop.gc__collect(Void) + llop.gc__collect(lltype.Void) f = self.getcompiled(prob_with_pyobj, [object]) c = self.getcompiled(collect, []) from sys import getrefcount as g @@ -202,6 +198,23 @@ after = g(obj) assert abs(before - after) < 5 + def test_zero_malloc(self): + T = lltype.GcStruct("C", ('x', lltype.Signed)) + def fixed_size(): + t = lltype.malloc(T, zero=True) + return t.x + c_fixed_size = self.getcompiled(fixed_size, []) + res = c_fixed_size() + assert res == 0 + A = lltype.GcArray(lltype.Signed) + def var_size(): + a = lltype.malloc(A, 1, zero=True) + return a[0] + c_var_size = self.getcompiled(var_size, []) + res = c_var_size() + assert res == 0 + + class TestUsingExactBoehm(TestUsingBoehm): gcpolicy = "exact_boehm" From antocuni at codespeak.net Fri Jul 13 09:32:00 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 13 Jul 2007 09:32:00 +0200 (CEST) Subject: [pypy-svn] r44995 - in pypy/extradoc/talk/ep2007: . demo howtoeu interpreter pypy_10_and_jit pytest rpython Message-ID: <20070713073200.2D83981B6@code0.codespeak.net> Author: antocuni Date: Fri Jul 13 09:31:55 2007 New Revision: 44995 Modified: pypy/extradoc/talk/ep2007/demo/ (props changed) pypy/extradoc/talk/ep2007/demo/pickledtasklet.py (props changed) pypy/extradoc/talk/ep2007/howtoeu/ (props changed) pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (contents, props changed) pypy/extradoc/talk/ep2007/interpreter/ (props changed) pypy/extradoc/talk/ep2007/interpreter/interpreter.txt (props changed) pypy/extradoc/talk/ep2007/openspace_discussions.txt (contents, props changed) pypy/extradoc/talk/ep2007/pypy_10_and_jit/ (props changed) pypy/extradoc/talk/ep2007/pypy_10_and_jit/talk.txt (props changed) pypy/extradoc/talk/ep2007/pytest/ (props changed) pypy/extradoc/talk/ep2007/pytest/pytest.txt (props changed) pypy/extradoc/talk/ep2007/rpython/ (props changed) pypy/extradoc/talk/ep2007/stackless-essentials.txt (contents, props changed) Log: fixeol Modified: pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt ============================================================================== --- pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt (original) +++ pypy/extradoc/talk/ep2007/howtoeu/howtoeu.txt Fri Jul 13 09:31:55 2007 @@ -1,219 +1,219 @@ - -.. include:: - - -================================================== -How to do an open source EU research project -================================================== - - -:authors: Lene Wagner, Holger Krekel (merlinux GmbH) -:event: 10.7.2007, EuroPython 2007, Vilnius - - -Intro -=================== - -- we have participated in the PyPy project -- both on technical and management levels -- founded merlinux GmbH in 2004, doing various projects -- we discuss *open source* research projects -- skip "Intellectucal Property" issues - -EU Framework Research Programme -==================================== - -- IST = Information Society and Technology -- PyPy was a STREP -- FP7 spends some 20.000.000.000 Euro in 7 years -- regularly publishes Work Programmes and Topics -- puts out CALLS for Proposals -- we mainly discuss FP7 (only few changes from FP6) - -Getting started -===================== - -- visions / ideas? -- reading Work programmes / CALL topics -- finding people to collaborate -- finding a set of organisations -- drafting a proposal ... - -(Research) Objectives? -============================== - -- formulate core research objective(s) - (state of the art & how the project aims beyond) -- formulate core technical objectives? -- core other objectives? (methodological/social) - -can you summarize it on one page? - -The Proposal -=============== - -- High level Objectives -- Impact on industry and society -- management structures -- work plan in long form -- detailed description of work packages -- Project duration, budget? - -Work Packages -=================== - -- each has a specific objective -- Person Months allocated from Partners -- Lead Partner -- Deliverables (Reports, Releases, Papers) - -Consortium -============================ - -- looking for interested parties -- consortium consists of organisations -- is collective entity towards EU -- requires co-ordinator/contact person for EU -- receives and distributes money - - -Applying to the EU -========================= - -- answer to CALLS within stated deadline -- co-ordinator registers project and proposal -- proposals gets reviewed by *independent* experts -- proposals receive points (PyPy had 26 out of 30) -- Invitation to "negotiation" - -Negotiation -============== - -- discuss criticism of reviewers, change proposal -- Prepare paperwork (CPF = Contract Preparation Forms) -- sign CONTRACT with the European Union - -Funding Schemes -===================== - -- Universities receive (basically) 75% funding -- Small/Medium Companies (SME) receive 75% funding of costs -- actual or 20% flat administrative overheads accountable - -Signing the contract? -======================== - -signature of the (negotiated) contract: - -- is legally binding for all contractors and the EU -- afterwards the EU does pre-payments (40-80%) -- the consortium can ask for amendments later - -Infrastructure -================== -- ensure close collaboration of non co-located developers -- ensure openness for external contributors -- ensure consistency of codebase/project -- PyPy: IRC, svn, mailing lists - - -Management structures -============================ - -- Administrative team (preparing Consortium decisions) -- Technical Board (discussing problems, deciding roadmap) -- developer body (everybody can participate) -- Consortium and Co-ordinator - - -Meetings and Sprints -============================ - -- sprints provide the rhythm of the project -- plan for core for newcomer-friendly sprints -- integrate a model for funding new contributors! - - -Contractual and actual developments -===================================== -- Contract implies a 'plan' for the development -- Project may evolve into other directions -- do checkpoints for assessing the work package status - -Planning for changes -=========================== - -- be able to shift work among partners -- if necessary: amend work plan, change the contract with the EU -- but: don't touch the objectives! - - -Reporting / Deliverables -============================ - -- interim reviews each 12-18 months -- Deliverables are: Reports, Releases etc. -- Deadlines -- official submission of Reports/Documentation to EU - -Recommendations -==================== - -- sprint-driven model of development; -- share some funding with upcoming contributors; -- balance contractual with community interests -- consider changes to the contractual work plan if - it fits the objectives better; - -More Recommendations -======================== -- transparent and flat communication and decision culture -- be ready to adjust roles and not stick to formal issues too much. -- dual-use Deliverables for Community/Industry and EU contract purposes - - -Roles and Responsbilities -============================ - -- assign responsibilities to people - - - per aspect (administration, technical, research) - - per work package - -- plan for changing responsibilities - -Changes from FP6 to FP7 -============================= - -- apparently FP7 improves some regulations that PyPy found burdonsome -- less collective financial responsibility -- 75% funding for (small/medium) companies -- subcontracting relaxed (important for international companies) - - -What Remains? -================================ - -- build the project on a shared vision and shared interest -- don't build it on money (too much) -- get into contact with national support offices! -- use the national and EU help infrastructure! - - -Further information -========================== - -- official EU FP7 website: - http://cordis.europa.eu/fp7/home_en.html - -- information leaflet on FP7: - http://ec.europa.eu/research/fp7/pdf/fp7-inbrief_en.pdf - -- Information and Communication Technology department: - http://cordis.europa.eu/fp7/ict/ - -- PyPy project http://pypy.org - - - - + +.. include:: + + +================================================== +How to do an open source EU research project +================================================== + + +:authors: Lene Wagner, Holger Krekel (merlinux GmbH) +:event: 10.7.2007, EuroPython 2007, Vilnius + + +Intro +=================== + +- we have participated in the PyPy project +- both on technical and management levels +- founded merlinux GmbH in 2004, doing various projects +- we discuss *open source* research projects +- skip "Intellectucal Property" issues + +EU Framework Research Programme +==================================== + +- IST = Information Society and Technology +- PyPy was a STREP +- FP7 spends some 20.000.000.000 Euro in 7 years +- regularly publishes Work Programmes and Topics +- puts out CALLS for Proposals +- we mainly discuss FP7 (only few changes from FP6) + +Getting started +===================== + +- visions / ideas? +- reading Work programmes / CALL topics +- finding people to collaborate +- finding a set of organisations +- drafting a proposal ... + +(Research) Objectives? +============================== + +- formulate core research objective(s) + (state of the art & how the project aims beyond) +- formulate core technical objectives? +- core other objectives? (methodological/social) + +can you summarize it on one page? + +The Proposal +=============== + +- High level Objectives +- Impact on industry and society +- management structures +- work plan in long form +- detailed description of work packages +- Project duration, budget? + +Work Packages +=================== + +- each has a specific objective +- Person Months allocated from Partners +- Lead Partner +- Deliverables (Reports, Releases, Papers) + +Consortium +============================ + +- looking for interested parties +- consortium consists of organisations +- is collective entity towards EU +- requires co-ordinator/contact person for EU +- receives and distributes money + + +Applying to the EU +========================= + +- answer to CALLS within stated deadline +- co-ordinator registers project and proposal +- proposals gets reviewed by *independent* experts +- proposals receive points (PyPy had 26 out of 30) +- Invitation to "negotiation" + +Negotiation +============== + +- discuss criticism of reviewers, change proposal +- Prepare paperwork (CPF = Contract Preparation Forms) +- sign CONTRACT with the European Union + +Funding Schemes +===================== + +- Universities receive (basically) 75% funding +- Small/Medium Companies (SME) receive 75% funding of costs +- actual or 20% flat administrative overheads accountable + +Signing the contract? +======================== + +signature of the (negotiated) contract: + +- is legally binding for all contractors and the EU +- afterwards the EU does pre-payments (40-80%) +- the consortium can ask for amendments later + +Infrastructure +================== +- ensure close collaboration of non co-located developers +- ensure openness for external contributors +- ensure consistency of codebase/project +- PyPy: IRC, svn, mailing lists + + +Management structures +============================ + +- Administrative team (preparing Consortium decisions) +- Technical Board (discussing problems, deciding roadmap) +- developer body (everybody can participate) +- Consortium and Co-ordinator + + +Meetings and Sprints +============================ + +- sprints provide the rhythm of the project +- plan for core for newcomer-friendly sprints +- integrate a model for funding new contributors! + + +Contractual and actual developments +===================================== +- Contract implies a 'plan' for the development +- Project may evolve into other directions +- do checkpoints for assessing the work package status + +Planning for changes +=========================== + +- be able to shift work among partners +- if necessary: amend work plan, change the contract with the EU +- but: don't touch the objectives! + + +Reporting / Deliverables +============================ + +- interim reviews each 12-18 months +- Deliverables are: Reports, Releases etc. +- Deadlines +- official submission of Reports/Documentation to EU + +Recommendations +==================== + +- sprint-driven model of development; +- share some funding with upcoming contributors; +- balance contractual with community interests +- consider changes to the contractual work plan if + it fits the objectives better; + +More Recommendations +======================== +- transparent and flat communication and decision culture +- be ready to adjust roles and not stick to formal issues too much. +- dual-use Deliverables for Community/Industry and EU contract purposes + + +Roles and Responsbilities +============================ + +- assign responsibilities to people + + - per aspect (administration, technical, research) + - per work package + +- plan for changing responsibilities + +Changes from FP6 to FP7 +============================= + +- apparently FP7 improves some regulations that PyPy found burdonsome +- less collective financial responsibility +- 75% funding for (small/medium) companies +- subcontracting relaxed (important for international companies) + + +What Remains? +================================ + +- build the project on a shared vision and shared interest +- don't build it on money (too much) +- get into contact with national support offices! +- use the national and EU help infrastructure! + + +Further information +========================== + +- official EU FP7 website: + http://cordis.europa.eu/fp7/home_en.html + +- information leaflet on FP7: + http://ec.europa.eu/research/fp7/pdf/fp7-inbrief_en.pdf + +- Information and Communication Technology department: + http://cordis.europa.eu/fp7/ict/ + +- PyPy project http://pypy.org + + + + Modified: pypy/extradoc/talk/ep2007/openspace_discussions.txt ============================================================================== --- pypy/extradoc/talk/ep2007/openspace_discussions.txt (original) +++ pypy/extradoc/talk/ep2007/openspace_discussions.txt Fri Jul 13 09:31:55 2007 @@ -1,108 +1,108 @@ - -What do you want PyPy to do? -Open Space session at EP2007 2007-07-11 -================================================ -Purpose: -- collect information about needs from PyPy and - would-be PyPy users -- answers from PyPy developers when/where applicable - -How was it done: -- everyone present wrote their primary question - and/or interest on a post-it -- these were grouped to identify overlapping areas - interest, largest areas were adressed first -- a summary of the questions/interests grouped per - area can be found below and the "answers" discussed - -Areas of questions/interests: - -RPython: stand alone and/or interfacing external libraries --------------------------------------------------------------- -Questions/interests: -- using RPython in CPython (unless PyPy can run all my Python - 2 code next month) -- how many are interested in RPython (in CPython, support - available, stability) -- RPython for Python extensions -- external library interfacing (extension modules) -- 3D graphics: Fast floating points? Fast math? - -Answers/comments: -- CPython API support in PyPy is slow -- RPython not complete yet (ex. ref counting) -- The state of RCtypes (support it, refactor and change - direction, future at all) is unclear - needs discussions - (how - to use ctypes at (PyPy) application level?) -- we need strategic discussions regarding how to interface - with external clibraries, to be discussed during sprint -- general need for refactoring - timespan will be roughly - a year to have a PyPy that usable (while still not covering - all extension modules) -- for stand alone RPython applications there is need for usability - fixes (shared interest) and speed/IO extensions - (app-level solution) -- regarding the question on 3D graphics (fast floating points, fast - math) the answer is that it is fairly easy to add this and there is - a matching interest in the PyPy dev-team in this area. - The need to display/visualize is then a matter of general C extension - interfacing - -Suggestions: -- RCtypes: let?s experiment here and discuss approaches during the sprint - -Javascript backend/related areas -------------------------------------- -Questions: -- I want to write a parser in Python and port it to javascript -- usable javascript backend -- emulate pygame on flash player - -Answers/comments: -- a parser is doable but you will encounter bugs, report them - and they will be fixed - there is a matching interest here - in the PyPy dev-team -- there is also a document on http://codespeak.net/pypyXXX - regarding parser creation -- regarding emulating PyGame on a flahs player - this is doable - but work needed, some rough estimates discussed were between - 2-4 months because it?s connected to tweaking a backend or - creating a new backend (which is an area we know a lot about;-) - So doable, requires work and we are not too sure about the results ;-) - -Refactoring/usability --------------------------------- -Questions: -- how pypy goes about maintaining parts? -- SLES/Solaris package? Light production usage? -- be less hackish to use? -- pypy 1.1.0 planning, which cleanups are we to tackle - and who? -- fixing/improving stackless, pickling and OO-support - -Answers/comments: -- this was not discussed in particular due to time constraints -- was also not prioritized because it contained several PyPy - developers interests and we prioritized to try to answer - "users/would-be "users" interests -- it was apparent though during the entire discussion that - we where discussing answers in the form of "doable" and - "refactoring needed" which is what this area is all about - -Suggestions: -- general refactoring/technical direction discussion in - connection to the sprint - - -Other areas of interests/questions -------------------------------------- -Questions: -- reducing the memory footprint of the Python interpreter - (I want to run preferable several, securely separated interpreters - on a resource constrained system (a phone specifically) -- different language interpreters/front-ends - -Answers/comments: -- these were not discussed due to time constraints -- they will be taken into account when discussing general - refactoring/technical direction + +What do you want PyPy to do? +Open Space session at EP2007 2007-07-11 +================================================ +Purpose: +- collect information about needs from PyPy and + would-be PyPy users +- answers from PyPy developers when/where applicable + +How was it done: +- everyone present wrote their primary question + and/or interest on a post-it +- these were grouped to identify overlapping areas + interest, largest areas were adressed first +- a summary of the questions/interests grouped per + area can be found below and the "answers" discussed + +Areas of questions/interests: + +RPython: stand alone and/or interfacing external libraries +-------------------------------------------------------------- +Questions/interests: +- using RPython in CPython (unless PyPy can run all my Python + 2 code next month) +- how many are interested in RPython (in CPython, support + available, stability) +- RPython for Python extensions +- external library interfacing (extension modules) +- 3D graphics: Fast floating points? Fast math? + +Answers/comments: +- CPython API support in PyPy is slow +- RPython not complete yet (ex. ref counting) +- The state of RCtypes (support it, refactor and change + direction, future at all) is unclear - needs discussions + (how - to use ctypes at (PyPy) application level?) +- we need strategic discussions regarding how to interface + with external clibraries, to be discussed during sprint +- general need for refactoring - timespan will be roughly + a year to have a PyPy that usable (while still not covering + all extension modules) +- for stand alone RPython applications there is need for usability + fixes (shared interest) and speed/IO extensions + (app-level solution) +- regarding the question on 3D graphics (fast floating points, fast + math) the answer is that it is fairly easy to add this and there is + a matching interest in the PyPy dev-team in this area. + The need to display/visualize is then a matter of general C extension + interfacing + +Suggestions: +- RCtypes: let?s experiment here and discuss approaches during the sprint + +Javascript backend/related areas +------------------------------------- +Questions: +- I want to write a parser in Python and port it to javascript +- usable javascript backend +- emulate pygame on flash player + +Answers/comments: +- a parser is doable but you will encounter bugs, report them + and they will be fixed - there is a matching interest here + in the PyPy dev-team +- there is also a document on http://codespeak.net/pypyXXX + regarding parser creation +- regarding emulating PyGame on a flahs player - this is doable + but work needed, some rough estimates discussed were between + 2-4 months because it?s connected to tweaking a backend or + creating a new backend (which is an area we know a lot about;-) + So doable, requires work and we are not too sure about the results ;-) + +Refactoring/usability +-------------------------------- +Questions: +- how pypy goes about maintaining parts? +- SLES/Solaris package? Light production usage? +- be less hackish to use? +- pypy 1.1.0 planning, which cleanups are we to tackle + and who? +- fixing/improving stackless, pickling and OO-support + +Answers/comments: +- this was not discussed in particular due to time constraints +- was also not prioritized because it contained several PyPy + developers interests and we prioritized to try to answer + "users/would-be "users" interests +- it was apparent though during the entire discussion that + we where discussing answers in the form of "doable" and + "refactoring needed" which is what this area is all about + +Suggestions: +- general refactoring/technical direction discussion in + connection to the sprint + + +Other areas of interests/questions +------------------------------------- +Questions: +- reducing the memory footprint of the Python interpreter + (I want to run preferable several, securely separated interpreters + on a resource constrained system (a phone specifically) +- different language interpreters/front-ends + +Answers/comments: +- these were not discussed due to time constraints +- they will be taken into account when discussing general + refactoring/technical direction Modified: pypy/extradoc/talk/ep2007/stackless-essentials.txt ============================================================================== --- pypy/extradoc/talk/ep2007/stackless-essentials.txt (original) +++ pypy/extradoc/talk/ep2007/stackless-essentials.txt Fri Jul 13 09:31:55 2007 @@ -1,78 +1,78 @@ -Abstract ID: 62 -Abstract content: - -This is a re-worked, actualized and improved version of -my talk at PyCon 2007. Repeating the abstract: - -As a surprise for people who think they know Stackless, -we present the new Stackless implementation For PyPy, -which has led to a significant amount of new insight -about parallel programming and its possible implementations. -We will isolate the known Stackless as a special case of -a general concept. - -This is a Stackless, not a PyPy talk. But the insights presented -here would not exist without PyPy's existance. - -Summary: - -Stackless has been around for a long time now. After several -versions with different goals in mind, the basic concepts of -channels and tasklets turned out to be useful abstractions, -and since many versions, Stackless is only ported from version -to version, without fundamental changes to the principles. - -As some spin-off, Armin Rigo invented Greenlets at a Stackless -sprint. They are some kind of coroutines and a bit of special -semantics. The major benefit is that Greenlets can run on -unmodified CPython. - -In parallel to that, the PyPy project is in its fourth year -now, and one of its goals was Stackless integration as an option. -And of course, Stackless has been integrated into PyPy in a very -nice and elegant way, much nicer than expected. During the design -of the Stackless extension to PyPy, it turned out, that tasklets, -greenlets and coroutines are not that different in principle, -and it was possible to base all known parallel paradigms on -one simple coroutine layout, which is as minimalistic as possible. - -It is a side effect of PyPy's simplicity, that even led to a -pretty new concept for Stackless, that allows all the different -switching paradigms to co-exist without interference. Users -could go further and implement their own concurrency model, -and it would neither interfere with others nor cost performance. - -Today's Stackless can be seen as a special case of the more general -implementation given for PyPy. This implementation can also be taken -as a reference about how Stackless is meant to be implemented. -This reference implementation is completely written in Python. - -The talk will try to isolate the crucial design decisions in -Stackless from implementation details. The reduced concepts -are together the essentials of Stackless. - -Special emphasis is given to interactive examples, simple use-cases, -and an animation that visually explains the new concept of -composability. - -Primary authors: TISMER, Christian (tismerysoft GmbH) -Co-Authors: -- none -- -Presenters: TISMER, Christian -Track classification: Python Language and Libraries -Submitted by: TISMER, Christian -Submitted on: 25 May 2007 23:56 -Last modified on: 25 May 2007 23:56 -Status: SUBMITTED -Comments : - - -As this is a re-worked version of my PyCon 2007 talk, I would like -to improve it by giving more examples, including some of -Andrew Dahlke's PyCon talk. -The original was quite short in explaining concepts, and moving -over to the new concept of composability, quickly. -I would lke to give more space to introducing the basic -concepts this time, having more visualisation, iteractive sessions, -and short repetitions and summaries. -This talk intends to give some insight both to the novice and also +Abstract ID: 62 +Abstract content: + +This is a re-worked, actualized and improved version of +my talk at PyCon 2007. Repeating the abstract: + +As a surprise for people who think they know Stackless, +we present the new Stackless implementation For PyPy, +which has led to a significant amount of new insight +about parallel programming and its possible implementations. +We will isolate the known Stackless as a special case of +a general concept. + +This is a Stackless, not a PyPy talk. But the insights presented +here would not exist without PyPy's existance. + +Summary: + +Stackless has been around for a long time now. After several +versions with different goals in mind, the basic concepts of +channels and tasklets turned out to be useful abstractions, +and since many versions, Stackless is only ported from version +to version, without fundamental changes to the principles. + +As some spin-off, Armin Rigo invented Greenlets at a Stackless +sprint. They are some kind of coroutines and a bit of special +semantics. The major benefit is that Greenlets can run on +unmodified CPython. + +In parallel to that, the PyPy project is in its fourth year +now, and one of its goals was Stackless integration as an option. +And of course, Stackless has been integrated into PyPy in a very +nice and elegant way, much nicer than expected. During the design +of the Stackless extension to PyPy, it turned out, that tasklets, +greenlets and coroutines are not that different in principle, +and it was possible to base all known parallel paradigms on +one simple coroutine layout, which is as minimalistic as possible. + +It is a side effect of PyPy's simplicity, that even led to a +pretty new concept for Stackless, that allows all the different +switching paradigms to co-exist without interference. Users +could go further and implement their own concurrency model, +and it would neither interfere with others nor cost performance. + +Today's Stackless can be seen as a special case of the more general +implementation given for PyPy. This implementation can also be taken +as a reference about how Stackless is meant to be implemented. +This reference implementation is completely written in Python. + +The talk will try to isolate the crucial design decisions in +Stackless from implementation details. The reduced concepts +are together the essentials of Stackless. + +Special emphasis is given to interactive examples, simple use-cases, +and an animation that visually explains the new concept of +composability. + +Primary authors: TISMER, Christian (tismerysoft GmbH) +Co-Authors: -- none -- +Presenters: TISMER, Christian +Track classification: Python Language and Libraries +Submitted by: TISMER, Christian +Submitted on: 25 May 2007 23:56 +Last modified on: 25 May 2007 23:56 +Status: SUBMITTED +Comments : + + +As this is a re-worked version of my PyCon 2007 talk, I would like +to improve it by giving more examples, including some of +Andrew Dahlke's PyCon talk. +The original was quite short in explaining concepts, and moving +over to the new concept of composability, quickly. +I would lke to give more space to introducing the basic +concepts this time, having more visualisation, iteractive sessions, +and short repetitions and summaries. +This talk intends to give some insight both to the novice and also to experienced stackless users. \ No newline at end of file From antocuni at codespeak.net Fri Jul 13 09:39:59 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 13 Jul 2007 09:39:59 +0200 (CEST) Subject: [pypy-svn] r44996 - pypy/extradoc/talk/ep2007 Message-ID: <20070713073959.A6E2E81DE@code0.codespeak.net> Author: antocuni Date: Fri Jul 13 09:39:58 2007 New Revision: 44996 Modified: pypy/extradoc/talk/ep2007/openspace_discussions.txt Log: fix the link Modified: pypy/extradoc/talk/ep2007/openspace_discussions.txt ============================================================================== --- pypy/extradoc/talk/ep2007/openspace_discussions.txt (original) +++ pypy/extradoc/talk/ep2007/openspace_discussions.txt Fri Jul 13 09:39:58 2007 @@ -59,10 +59,10 @@ - emulate pygame on flash player Answers/comments: -- a parser is doable but you will encounter bugs, report them +- a parser is doable but you could encounter bugs, report them and they will be fixed - there is a matching interest here in the PyPy dev-team -- there is also a document on http://codespeak.net/pypyXXX +- there is also a document on http://codespeak.net/pypy/dist/pypy/doc/rlib.html#parsing regarding parser creation - regarding emulating PyGame on a flahs player - this is doable but work needed, some rough estimates discussed were between From jlg at codespeak.net Fri Jul 13 09:40:09 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Fri, 13 Jul 2007 09:40:09 +0200 (CEST) Subject: [pypy-svn] r44997 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070713074009.D9F9C8217@code0.codespeak.net> Author: jlg Date: Fri Jul 13 09:40:08 2007 New Revision: 44997 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/ex.ss Log: arithmetic operations uses to_fixnum, to_float instead of generic to_number Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Fri Jul 13 09:40:08 2007 @@ -18,6 +18,8 @@ - comparison: < > eq? eqv? +- rational, complex number types + Do in some future ----------------- Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Fri Jul 13 09:40:08 2007 @@ -244,9 +244,9 @@ def oper(self, x, y): if isinstance(x, W_Float) or isinstance(y, W_Float): - return W_Float(self.do_oper_float(x.to_number(), y.to_number())) + return W_Float(self.do_oper_float(x.to_float(), y.to_float())) else: - return W_Fixnum(self.do_oper_int(x.to_number(), y.to_number())) + return W_Fixnum(self.do_oper_int(x.to_fixnum(), y.to_fixnum())) class Add(ListOper): def do_oper_int(self, x, y): Modified: pypy/dist/pypy/lang/scheme/test/ex.ss ============================================================================== --- pypy/dist/pypy/lang/scheme/test/ex.ss (original) +++ pypy/dist/pypy/lang/scheme/test/ex.ss Fri Jul 13 09:40:08 2007 @@ -20,5 +20,7 @@ (even? (- n 1)))))) (even? 12)) +(+ 1 2 3) +(+ 1 2.9 2) (quit) From mwh at codespeak.net Fri Jul 13 09:43:50 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 13 Jul 2007 09:43:50 +0200 (CEST) Subject: [pypy-svn] r44998 - pypy/dist/pypy/translator/llvm/test Message-ID: <20070713074350.5A35581E1@code0.codespeak.net> Author: mwh Date: Fri Jul 13 09:43:49 2007 New Revision: 44998 Modified: pypy/dist/pypy/translator/llvm/test/test_symbolic.py Log: skipped failing test i don't remember writing Modified: pypy/dist/pypy/translator/llvm/test/test_symbolic.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_symbolic.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_symbolic.py Fri Jul 13 09:43:49 2007 @@ -166,7 +166,17 @@ fn = compile_function(vararray, [int]) assert fn(21) == 42 - +def test_itemoffset_void(): + py.test.skip("not supported") + A = lltype.GcArray(lltype.Void) + s = llmemory.sizeof(A, 1) + s += llmemory.sizeof(lltype.Signed) + print s + def f(): + return s + fn = compile_function(f, []) + res = fn() + assert res > 0 def test_is_early_constant(): from pypy.rlib import jit From jacob at codespeak.net Fri Jul 13 09:50:20 2007 From: jacob at codespeak.net (jacob at codespeak.net) Date: Fri, 13 Jul 2007 09:50:20 +0200 (CEST) Subject: [pypy-svn] r44999 - in pypy/dist/pypy/interpreter: pyparser/data pyparser/test test Message-ID: <20070713075020.E431481DF@code0.codespeak.net> Author: jacob Date: Fri Jul 13 09:50:19 2007 New Revision: 44999 Modified: pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a pypy/dist/pypy/interpreter/pyparser/test/test_parser.py pypy/dist/pypy/interpreter/test/test_syntax.py Log: Modified grammar towards supporting future imports. Modified: pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a (original) +++ pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a Fri Jul 13 09:50:19 2007 @@ -24,10 +24,15 @@ # file_input is a module or sequence of commands read from an input file; # eval_input is the input for the eval() and input() functions. # NB: compound_stmt in single_input is followed by extra NEWLINE! -single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE -file_input: (NEWLINE | stmt)* ENDMARKER +single_input: NEWLINE | future_import_list [';'] | future_import_list ';' simple_stmt | compound_stmt NEWLINE +file_input: (future_import_list [';'])* (NEWLINE | stmt)* ENDMARKER eval_input: testlist NEWLINE* ENDMARKER +future_import_list: import_from_future (';' import_from_future)* +import_from_future: 'from' '__future__' 'import' ('(' future_import_as_names [','] ')' | future_import_as_names) +future_import_feature: NAME [('as'|NAME) NAME] +future_import_as_names: future_import_feature (',' future_import_feature)* + decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE decorators: decorator+ funcdef: [decorators] 'def' NAME parameters ':' suite @@ -63,7 +68,8 @@ exec_stmt: 'exec' expr ['in' test [',' test]] assert_stmt: 'assert' test [',' test] -compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef +# with_stmt is added dynamically to compund_stmt if imported from __future__ +compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] while_stmt: 'while' test ':' suite ['else' ':' suite] for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite] Modified: pypy/dist/pypy/interpreter/pyparser/test/test_parser.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_parser.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Fri Jul 13 09:50:19 2007 @@ -2,7 +2,6 @@ from pypy.interpreter.pyparser.grammar import Parser - def test_symbols(): p = Parser() x1 = p.add_symbol('sym') @@ -43,3 +42,90 @@ assert v == 6 v = p.add_symbol( 'sym3' ) assert v == 9 + +class FakeSpace: + w_None = None + w_str = str + w_basestring = basestring + w_int = int + + def wrap(self,obj): + return obj + + def isinstance(self, obj, wtype ): + return isinstance(obj,wtype) + + def is_true(self, obj): + return obj + + def eq_w(self, obj1, obj2): + return obj1 == obj2 + + def is_w(self, obj1, obj2): + return obj1 is obj2 + + def type(self, obj): + return type(obj) + + def newlist(self, lst): + return list(lst) + + def newtuple(self, lst): + return tuple(lst) + + def call_method(self, obj, meth, *args): + return getattr(obj, meth)(*args) + + def call_function(self, func, *args): + return func(*args) + + builtin = dict(int=int, long=long, float=float, complex=complex) + +from pypy.interpreter.pyparser.asthelper import get_atoms +class RuleTracer(dict): + + def __init__(self, *args, **kw): + self.trace = [] + + def __getitem__(self, attr): + if attr in ['file_input', 'future_import_list', 'import_from_future', + 'future_import_as_names']: + return lambda *args: None + + def record_trace(builder, number): + result = [t.value for t in get_atoms(builder, number)] + self.trace.append((attr, result)) + return record_trace + + def get(self, attr, default): + return self.__getitem__(attr) + +from pypy.interpreter.pyparser.astbuilder import AstBuilder +class MockBuilder(AstBuilder): + + def __init__(self, *args, **kw): + AstBuilder.__init__(self, *args, **kw) + self.build_rules = RuleTracer() + +class TestFuture(object): + + def setup_class(self): + from pypy.interpreter.pyparser.pythonparse import make_pyparser + self.parser = make_pyparser('2.5a') + + def setup_method(self, method): + self.builder = MockBuilder(self.parser, space=FakeSpace()) + def check_parse(self, tst, expected): + self.parser.parse_source(tst, 'exec' , self.builder) + assert self.builder.build_rules.trace == expected + + def test_single_future_import(self): + tst = 'from __future__ import na\n' + expected = [('future_import_feature', ['na'])] + self.check_parse(tst, expected) + + def test_double_future_import(self): + tst = 'from __future__ import na, xx\n' + expected = [('future_import_feature', ['na']), + ('future_import_feature', ['xx'])] + self.check_parse(tst, expected) Modified: pypy/dist/pypy/interpreter/test/test_syntax.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_syntax.py (original) +++ pypy/dist/pypy/interpreter/test/test_syntax.py Fri Jul 13 09:50:19 2007 @@ -491,6 +491,30 @@ else: assert False, 'Assignment to with did not raise SyntaxError' + def test_with_as_keyword_and_docstring(self): + try: + exec "'Docstring'\nfrom __future__ import with_statement\nwith = 9" + except SyntaxError: + pass + else: + assert False, 'Assignment to with did not raise SyntaxError' + + def test_with_as_keyword_compound(self): + try: + exec "from __future__ import generators, with_statement\nwith = 9" + except SyntaxError: + pass + else: + assert False, 'Assignment to with did not raise SyntaxError' + + def test_with_as_keyword_multiple(self): + try: + exec "from __future__ import generators\nfrom __future__ import with_statement\nwith = 9" + except SyntaxError: + pass + else: + assert False, 'Assignment to with did not raise SyntaxError' + def test_as_as_identifier(self): exec "as = 9" exec "import sys as foo" From mwh at codespeak.net Fri Jul 13 09:51:05 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 13 Jul 2007 09:51:05 +0200 (CEST) Subject: [pypy-svn] r45000 - in pypy/dist/pypy: rpython/memory/gctransform translator/llvm Message-ID: <20070713075105.EBA3981DF@code0.codespeak.net> Author: mwh Date: Fri Jul 13 09:51:03 2007 New Revision: 45000 Modified: pypy/dist/pypy/rpython/memory/gctransform/boehm.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/opwriter.py Log: usual putting-llvm-back-together-with-sellotape, but pleasingly this mostly involves deleting code. Modified: pypy/dist/pypy/rpython/memory/gctransform/boehm.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform/boehm.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform/boehm.py Fri Jul 13 09:51:03 2007 @@ -55,6 +55,7 @@ self.malloc_varsize_ptr = self.inittime_helper( ll_malloc_varsize, [lltype.Signed]*4, llmemory.Address, inline=False) self.mixlevelannotator.finish() # for now + self.mixlevelannotator.backend_optimize() def push_alive_nopyobj(self, var, llops): pass Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Fri Jul 13 09:51:03 2007 @@ -36,67 +36,6 @@ exc_flag=False): raise NotImplementedError, 'GcPolicy should not be used directly' - def zeromalloc(self, codewriter, targetvar, type_, size=1, atomic=False, - exc_flag=False): - uword = self.db.get_machine_uword() - malloc_ptr = '%malloc_ptr' + self.get_count(True) - malloc_size = '%malloc_size' + self.get_count() - malloc_sizeu = '%malloc_sizeu' + self.get_count() - - codewriter.getelementptr(malloc_size, type_, 'null', - [(uword, size)], getptr=False) - codewriter.cast(malloc_sizeu, type_, malloc_size, uword) - self._zeromalloc(codewriter, malloc_ptr, malloc_sizeu, atomic, exc_flag) - codewriter.cast(targetvar, 'sbyte*', malloc_ptr, type_) - - def var_zeromalloc(self, codewriter, targetvar, - type_, node, len, atomic=False): - - word = lentype = self.db.get_machine_word() - uword = self.db.get_machine_uword() - malloc_ptr = '%malloc_ptr' + self.get_count(True) - malloc_size = '%malloc_size' + self.get_count() - malloc_sizeu = '%malloc_sizeu' + self.get_count() - actuallen = '%actuallen' + self.get_count() - arraylength = '%arraylength' + self.get_count() - - ARRAY, indices_to_array = node.var_malloc_info() - - #varsized arrays and structs look like this: - #Array: {int length , elemtype*} - #Struct: {...., Array} - - # the following indices access the last element in the array - elemtype = self.db.repr_type(ARRAY.OF) - word = lentype = self.db.get_machine_word() - uword = self.db.get_machine_uword() - - # need room for NUL terminator - if ARRAY is STR.chars: - codewriter.binaryop('add', actuallen, lentype, len, 1) - else: - codewriter.cast(actuallen, lentype, len, lentype) - - elemindices = list(indices_to_array) - elemindices += [('uint', 1), (lentype, actuallen)] - codewriter.getelementptr(malloc_size, type_, 'null', elemindices) - codewriter.cast(malloc_sizeu, elemtype + '*', malloc_size, uword) - - self._zeromalloc(codewriter, malloc_ptr, malloc_sizeu, atomic=atomic) - - indices_to_arraylength = tuple(indices_to_array) + (('uint', 0),) - - codewriter.cast(targetvar, 'sbyte*', malloc_ptr, type_) - - #XXX ctypes Arrays have no length field - #XXXif not VARPART._hints.get('nolength', False): - - # the following accesses the length field of the array - codewriter.getelementptr(arraylength, type_, - targetvar, indices_to_arraylength) - codewriter.store(lentype, len, arraylength) - - def op_call_rtti_destructor(self, codewriter, opr): raise Exception, 'GcPolicy should not be used directly' @@ -201,14 +140,14 @@ ## atomic = False # malloc_size is unsigned right now - sizei = '%malloc_sizei' + self.get_count() - codewriter.cast(sizei, uword, size, word) - codewriter.call(targetvar, 'sbyte*', fnname, [word], [sizei]) + sizeu = '%malloc_sizeu' + self.get_count() + codewriter.cast(sizeu, word, size, uword) + codewriter.call(targetvar, 'sbyte*', fnname, [word], [size]) if atomic: codewriter.call(None, 'void', '%llvm.memset' + postfix(), ['sbyte*', 'ubyte', uword, uword], - [targetvar, 0, size, boundary_size], + [targetvar, 0, sizeu, boundary_size], cconv='ccc') Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Fri Jul 13 09:51:03 2007 @@ -279,36 +279,15 @@ # is of type Void, which is removed by direct_call indirect_call = direct_call - def malloc(self, opr): - arg_type = opr.op.args[0].value + def boehm_malloc(self, opr): + self.db.gcpolicy._zeromalloc(self.codewriter, opr.retref, opr.argrefs[0], atomic=False) - # XXX hack better to look at the actual structure - name = str(opr.op.args[0]) - exc = False - if 'Exception' in name or 'Error' in name: - exc = True - - self.db.gcpolicy.zeromalloc(self.codewriter, opr.retref, opr.rettype, - atomic=arg_type._is_atomic(), exc_flag=exc) - - zero_malloc = malloc - - def malloc_varsize(self, opr): - - # XXX transformation perhaps? - arg_type = opr.op.args[0].value - if isinstance(arg_type, lltype.Array) and arg_type.OF is lltype.Void: - # This is a backend decision to NOT represent a void array with - # anything and save space - therefore not varsized anymore - self.malloc(opr) - return - - node = self.db.obj2node[arg_type] - self.db.gcpolicy.var_zeromalloc(self.codewriter, opr.retref, - opr.rettype, node, opr.argrefs[1], - atomic=arg_type._is_atomic()) + def boehm_malloc_atomic(self, opr): + self.db.gcpolicy._zeromalloc(self.codewriter, opr.retref, opr.argrefs[0], atomic=True) - zero_malloc_varsize = malloc_varsize + def boehm_register_finalizer(self, opr): + # ha, ha + pass def flavored_malloc(self, opr): flavor = opr.op.args[0].value From mwh at codespeak.net Fri Jul 13 10:34:54 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 13 Jul 2007 10:34:54 +0200 (CEST) Subject: [pypy-svn] r45001 - pypy/extradoc/sprintinfo/post-ep2007 Message-ID: <20070713083454.65851824D@code0.codespeak.net> Author: mwh Date: Fri Jul 13 10:34:51 2007 New Revision: 45001 Added: pypy/extradoc/sprintinfo/post-ep2007/planning.txt (contents, props changed) Log: some sort of planning.txt Added: pypy/extradoc/sprintinfo/post-ep2007/planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/post-ep2007/planning.txt Fri Jul 13 10:34:51 2007 @@ -0,0 +1,36 @@ +------------------------------ +Goals and topics of the sprint +------------------------------ + +There are many possible and interesting sprint topics to work on - here +we list some possible task areas: + +* completing the missing python 2.5 features and support +* write or port more extension modules (e.g. zlib is missing) +* identify slow areas of PyPy through benchmarking and work on improvements, + possibly moving app-level parts of the Python interpreter to interp-level + if useful. +* there are some parts of PyPy in need of refactoring, we may spend some time + on those, for example: + + - rctypes and the extension compiler need some rethinking + - support for LLVM 2.0 for the llvm backend + - ... + +* some JIT improvement work +* port the stackless transform to ootypesystem +* make the JIT generate fast code for float operations (??) +* experimenting in the perfect_dict direction (?????) +* other interesting stuff that you would like to work on ...;-) + + +IDEAS +----- + + * jgustak/antocuni -- pypy.lang.scheme + * rene/alecu/lucio -- js/actionscript + * maciej -- pypy-c self hosting + * christion -- tasklet pickling bugs + * simon -- rctypes/rffi + * michael/pedronis -- emptying-the-malloc-zoo (a bit) + * arlo/jacob -- python 2.5 features/extension modules From antocuni at codespeak.net Fri Jul 13 10:35:24 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 13 Jul 2007 10:35:24 +0200 (CEST) Subject: [pypy-svn] r45002 - pypy/dist/pypy/translator/cli/test Message-ID: <20070713083524.917E48250@code0.codespeak.net> Author: antocuni Date: Fri Jul 13 10:35:23 2007 New Revision: 45002 Added: pypy/dist/pypy/translator/cli/test/mylib.py (contents, props changed) Log: forgot to add this file Added: pypy/dist/pypy/translator/cli/test/mylib.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/test/mylib.py Fri Jul 13 10:35:23 2007 @@ -0,0 +1,5 @@ +from pypy.translator.cli.carbonpython import export + + at export(int, int) +def sum(a, b): + return a+b From antocuni at codespeak.net Fri Jul 13 10:38:05 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 13 Jul 2007 10:38:05 +0200 (CEST) Subject: [pypy-svn] r45003 - pypy/dist/pypy/doc Message-ID: <20070713083805.07813824D@code0.codespeak.net> Author: antocuni Date: Fri Jul 13 10:38:05 2007 New Revision: 45003 Modified: pypy/dist/pypy/doc/faq.txt Log: add an anchor Modified: pypy/dist/pypy/doc/faq.txt ============================================================================== --- pypy/dist/pypy/doc/faq.txt (original) +++ pypy/dist/pypy/doc/faq.txt Fri Jul 13 10:38:05 2007 @@ -368,6 +368,8 @@ See the `getting-started`_ guide. +.. _`how do I compile my own interpreters`: + -------------------------------------- How do I compile my own interpreters? -------------------------------------- From fijal at codespeak.net Fri Jul 13 11:09:57 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 13 Jul 2007 11:09:57 +0200 (CEST) Subject: [pypy-svn] r45004 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20070713090957.37EE6825D@code0.codespeak.net> Author: fijal Date: Fri Jul 13 11:09:52 2007 New Revision: 45004 Modified: pypy/dist/pypy/rpython/lltypesystem/rfficache.py Log: Move import one level below to allow tests to run on translated pypy-c Modified: pypy/dist/pypy/rpython/lltypesystem/rfficache.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rfficache.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rfficache.py Fri Jul 13 11:09:52 2007 @@ -6,10 +6,10 @@ import py import os from pypy.translator.tool.cbuild import build_executable -from py.compat.subprocess import PIPE, Popen from pypy.tool.udir import udir def sizeof_c_type(c_typename, includes={}, compiler_exe=None): + from py.compat.subprocess import PIPE, Popen includes['stdio.h'] = True includes['sys' + os.path.sep + 'types.h'] = True include_string = "\n".join(["#include <%s>" % i for i in includes.keys()]) From pedronis at codespeak.net Fri Jul 13 11:23:44 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 13 Jul 2007 11:23:44 +0200 (CEST) Subject: [pypy-svn] r45005 - pypy/branch/emptying-the-malloc-zoo Message-ID: <20070713092344.AD25481F9@code0.codespeak.net> Author: pedronis Date: Fri Jul 13 11:23:42 2007 New Revision: 45005 Removed: pypy/branch/emptying-the-malloc-zoo/ Log: remove merged branch From pedronis at codespeak.net Fri Jul 13 11:24:07 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 13 Jul 2007 11:24:07 +0200 (CEST) Subject: [pypy-svn] r45006 - pypy/branch/emptying-the-malloc-zoo-2 Message-ID: <20070713092407.E56658205@code0.codespeak.net> Author: pedronis Date: Fri Jul 13 11:24:06 2007 New Revision: 45006 Removed: pypy/branch/emptying-the-malloc-zoo-2/ Log: remove merged branch From mwh at codespeak.net Fri Jul 13 11:29:03 2007 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 13 Jul 2007 11:29:03 +0200 (CEST) Subject: [pypy-svn] r45007 - pypy/dist/pypy/doc/discussion Message-ID: <20070713092903.BA08880D9@code0.codespeak.net> Author: mwh Date: Fri Jul 13 11:29:02 2007 New Revision: 45007 Added: pypy/dist/pypy/doc/discussion/emptying-the-malloc-zoo.txt (contents, props changed) Log: a half finished discussion document Added: pypy/dist/pypy/doc/discussion/emptying-the-malloc-zoo.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/discussion/emptying-the-malloc-zoo.txt Fri Jul 13 11:29:02 2007 @@ -0,0 +1,38 @@ +.. coding: utf-8 + +Emptying the malloc zoo +======================= + +Around the end-of-the-EU-project time there were two major areas of +obscurity in the memory management area: + + 1. The confusing set of operations that the low-level backend are + expected to implement. + + 2. The related, but slightly different, confusion of the various + "flavours" of malloc: what's the difference between + lltype.malloc(T, flavour='raw') and llmemory.raw_malloc(sizeof(T))? + +At the post-ep2007 sprint, Samuele and Michael attacked the first +problem a bit: making the Boehm GC transformer only require three +simple operations of the backend. This could be extending still +further by having the gc transformer use rffi to insert calls to the +relevant Boehm functions^Wmacros, and then the backend wouldn't need +to know anything about Boehm at all (but... LLVM). + +A potential next step is to work out what we want the "llpython" +interface to memory management to be. + +There are various use cases: + +**lltype.malloc(T) ? T is a fixed-size GC container** + + This is the default case. Non-pointers inside the allocated memory + will not be zeroed. The object will be managed by the GC, no + deallocation required. + +**lltype.malloc(T, zero=True) ? T is a GC container** + + As above, but all fields will be cleared. + + - lltype.malloc(U, raw=True) ? From jlg at codespeak.net Fri Jul 13 13:05:39 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Fri, 13 Jul 2007 13:05:39 +0200 (CEST) Subject: [pypy-svn] r45009 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070713110539.6B1DC825C@code0.codespeak.net> Author: jlg Date: Fri Jul 13 13:05:38 2007 New Revision: 45009 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/ex.ss pypy/dist/pypy/lang/scheme/test/test_eval.py Log: ListOper subclasses generated dynamically; Div added; porcedure/macro with refactored Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Fri Jul 13 13:05:38 2007 @@ -19,6 +19,7 @@ - comparison: < > eq? eqv? - rational, complex number types + - exact/inexact Do in some future ----------------- Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Fri Jul 13 13:05:38 2007 @@ -1,4 +1,4 @@ -import autopath +import py class SchemeException(Exception): pass @@ -142,6 +142,15 @@ def call(self, ctx, lst): raise NotImplementedError + def eval_body(self, ctx, body): + body_expression = body + body_result = None + while not isinstance(body_expression, W_Nil): + body_result = body_expression.car.eval(ctx) + body_expression = body_expression.cdr + + return body_result + class W_Procedure(W_Callable): def __init__(self, pname=""): self.pname = pname @@ -211,13 +220,7 @@ else: local_ctx.put(formal.name, lst[idx]) - body_expression = self.body - body_result = None - while not isinstance(body_expression, W_Nil): - body_result = body_expression.car.eval(local_ctx) - body_expression = body_expression.cdr - - return body_result # self.body.eval(local_ctx) + return self.eval_body(local_ctx, self.body) def plst2lst(plst): """coverts python list() of W_Root into W_Pair scheme list""" @@ -233,6 +236,9 @@ ## class ListOper(W_Procedure): def procedure(self, ctx, lst): + if len(lst) == 1: + return self.unary_oper(lst[0].eval(ctx)) + acc = None for arg in lst: if acc is None: @@ -242,38 +248,50 @@ return acc + def unary_oper(self, x): + if isinstance(x, W_Float): + return W_Float(self.do_unary_oper_float(x.to_float())) + else: + return W_Fixnum(self.do_unary_oper_int(x.to_fixnum())) + def oper(self, x, y): if isinstance(x, W_Float) or isinstance(y, W_Float): return W_Float(self.do_oper_float(x.to_float(), y.to_float())) else: return W_Fixnum(self.do_oper_int(x.to_fixnum(), y.to_fixnum())) -class Add(ListOper): - def do_oper_int(self, x, y): - return x + y - - def do_oper_float(self, x, y): - return x + y - -class Sub(ListOper): - def procedure(self, ctx, lst): - if len(lst) == 1: - return ListOper.procedure(self, ctx, [W_Fixnum(0), lst[0]]) - else: - return ListOper.procedure(self, ctx, lst) - - def do_oper_int(self, x, y): - return x - y - - def do_oper_float(self, x, y): - return x - y - -class Mul(ListOper): - def do_oper_int(self, x, y): - return x * y - - def do_oper_float(self, x, y): - return x * y +def create_op_class(oper, unary_oper): + class Op(ListOper): + pass + + local_locals = {} + for name in ["int", "float"]: + + attr_name = "do_oper_%s" % (name, ) + + code = py.code.Source(""" + def %s(self, x, y): + return x %s y + """ % (attr_name, oper)) + + exec code.compile() in local_locals + setattr(Op, attr_name, local_locals[attr_name]) + + attr_name = "do_unary_oper_%s" % (name, ) + code = py.code.Source(""" + def %s(self, x): + return %s x + """ % (attr_name, unary_oper)) + + exec code.compile() in local_locals + setattr(Op, attr_name, local_locals[attr_name]) + + return Op + +Add = create_op_class('+', '') +Sub = create_op_class('-', '-') +Mul = create_op_class('*', '') +Div = create_op_class('/', '1 /') class List(W_Procedure): def procedure(self, ctx, lst): @@ -356,13 +374,7 @@ local_ctx.put(name, val) w_formal = w_formal.cdr - body_expression = lst.cdr - body_result = None - while not isinstance(body_expression, W_Nil): - body_result = body_expression.car.eval(local_ctx) - body_expression = body_expression.cdr - - return body_result + return self.eval_body(local_ctx, lst.cdr) class Letrec(W_Macro): def call(self, ctx, lst): @@ -383,13 +395,7 @@ local_ctx.set(name, val) w_formal = w_formal.cdr - body_expression = lst.cdr - body_result = None - while not isinstance(body_expression, W_Nil): - body_result = body_expression.car.eval(local_ctx) - body_expression = body_expression.cdr - - return body_result + return self.eval_body(local_ctx, lst.cdr) def Literal(sexpr): return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) @@ -417,6 +423,7 @@ '+': Add, '-': Sub, '*': Mul, + '/': Div, #list operations 'cons': Cons, 'car': Car, Modified: pypy/dist/pypy/lang/scheme/test/ex.ss ============================================================================== --- pypy/dist/pypy/lang/scheme/test/ex.ss (original) +++ pypy/dist/pypy/lang/scheme/test/ex.ss Fri Jul 13 13:05:38 2007 @@ -21,6 +21,13 @@ (even? 12)) (+ 1 2 3) -(+ 1 2.9 2) -(quit) +(- 2) +(- 1 2.9 2) +(* 1 2 2) +(/ 1 2.9 2) +(/ 1) +(/ 2 1) +(/ 2 1 7) +(/ 2) +;(quit) Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Fri Jul 13 13:05:38 2007 @@ -28,6 +28,13 @@ w_num = eval_noctx("(* 4 -5 6.1)") assert w_num.to_number() == (4 * -5 * 6.1) + w_num = eval_noctx("(/ 4)") + assert w_num.to_number() == 1 / 4 + w_num = eval_noctx("(/ 4 -5)") + assert w_num.to_number() == 4 / -5 + w_num = eval_noctx("(/ 4 -5 6.1)") + assert w_num.to_number() == (4 / -5 / 6.1) + w_num = eval_noctx("(- 4)") assert w_num.to_number() == -4 w_num = eval_noctx("(- 4 5)") From jacob at codespeak.net Fri Jul 13 13:30:43 2007 From: jacob at codespeak.net (jacob at codespeak.net) Date: Fri, 13 Jul 2007 13:30:43 +0200 (CEST) Subject: [pypy-svn] r45011 - in pypy/dist/pypy/interpreter/pyparser: data test Message-ID: <20070713113043.39A0B8274@code0.codespeak.net> Author: jacob Date: Fri Jul 13 13:30:42 2007 New Revision: 45011 Modified: pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Log: Fixed problems in grammar changes.Addedfurther tests. Modified: pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a (original) +++ pypy/dist/pypy/interpreter/pyparser/data/Grammar2.5a Fri Jul 13 13:30:42 2007 @@ -24,8 +24,9 @@ # file_input is a module or sequence of commands read from an input file; # eval_input is the input for the eval() and input() functions. # NB: compound_stmt in single_input is followed by extra NEWLINE! -single_input: NEWLINE | future_import_list [';'] | future_import_list ';' simple_stmt | compound_stmt NEWLINE -file_input: (future_import_list [';'])* (NEWLINE | stmt)* ENDMARKER + +single_input: NEWLINE | future_import_list [';'] | [future_import_list ';'] simple_stmt | compound_stmt NEWLINE +file_input: (future_import_list [';'] NEWLINE)* (NEWLINE | stmt)* ENDMARKER eval_input: testlist NEWLINE* ENDMARKER future_import_list: import_from_future (';' import_from_future)* @@ -68,7 +69,8 @@ exec_stmt: 'exec' expr ['in' test [',' test]] assert_stmt: 'assert' test [',' test] -# with_stmt is added dynamically to compund_stmt if imported from __future__ +# TODO: with_stmt should be added dynamically to compund_stmt if +# imported from __future__ compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] while_stmt: 'while' test ':' suite ['else' ':' suite] Modified: pypy/dist/pypy/interpreter/pyparser/test/test_parser.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_parser.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Fri Jul 13 13:30:42 2007 @@ -1,6 +1,5 @@ - from pypy.interpreter.pyparser.grammar import Parser - +from pypy.interpreter.pyparser import error def test_symbols(): p = Parser() @@ -88,9 +87,11 @@ self.trace = [] def __getitem__(self, attr): - if attr in ['file_input', 'future_import_list', 'import_from_future', - 'future_import_as_names']: - return lambda *args: None + if attr in ['dotted_name', 'dotted_as_name', 'dotted_as_names', + 'import_stmt', 'small_stmt', 'simple_stmt', 'stmt', + 'single_input', 'file_input', 'future_import_list', + 'import_from_future', 'future_import_as_names']: + return None def record_trace(builder, number): result = [t.value for t in get_atoms(builder, number)] @@ -107,18 +108,26 @@ AstBuilder.__init__(self, *args, **kw) self.build_rules = RuleTracer() -class TestFuture(object): +class TestFuture(object): + def setup_class(self): from pypy.interpreter.pyparser.pythonparse import make_pyparser self.parser = make_pyparser('2.5a') def setup_method(self, method): self.builder = MockBuilder(self.parser, space=FakeSpace()) - def check_parse(self, tst, expected): - self.parser.parse_source(tst, 'exec' , self.builder) + + def check_parse_mode(self, tst, expected, mode): + self.parser.parse_source(tst, mode, self.builder) assert self.builder.build_rules.trace == expected + def check_parse(self, tst, expected): + self.check_parse_mode(tst, expected, 'exec') + self.builder.build_rules.trace = [] + self.check_parse_mode(tst, expected, 'single') + + def test_single_future_import(self): tst = 'from __future__ import na\n' expected = [('future_import_feature', ['na'])] @@ -129,3 +138,40 @@ expected = [('future_import_feature', ['na']), ('future_import_feature', ['xx'])] self.check_parse(tst, expected) + + def test_two_future_imports(self): + tst = 'from __future__ import na;from __future__ import xx\n' + expected = [('future_import_feature', ['na']), + ('future_import_feature', ['xx'])] + self.check_parse(tst, expected) + + def test_future_imports_nl(self): + tst = ''' +from __future__ import na +from __future__ import xx; +from __future__ import yy +''' + expected = [('future_import_feature', ['na']), + ('future_import_feature', ['xx']), + ('future_import_feature', ['yy'])] + self.check_parse_mode(tst, expected,'exec') + + def test_single_future_as(self): + tst = 'from __future__ import na as x\n' + expected = [('future_import_feature', ['na', 'as', 'x'])] + self.check_parse(tst, expected) + + def test_single_future_as(self): + tst = 'import sys;from __future__ import na as x\n' + expected = [] + try: + self.check_parse_mode(tst, expected,'exec') + assert False == 'An import before a future import should throw an error.' + except error.SyntaxError: + pass + + def test_regular_import(self): + tst = 'import sys' + expected = [('import_name', ['import', 'sys'])] + self.check_parse(tst, expected) + From arigo at codespeak.net Fri Jul 13 13:42:27 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 13 Jul 2007 13:42:27 +0200 (CEST) Subject: [pypy-svn] r45012 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070713114227.D0BDF806B@code0.codespeak.net> Author: arigo Date: Fri Jul 13 13:42:26 2007 New Revision: 45012 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: (lac, arigo) Set the _dead attribute correctly when lltype.free() is called. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Fri Jul 13 13:42:26 2007 @@ -1664,6 +1664,7 @@ T = typeOf(p) if not isinstance(T, Ptr) or p._togckind() != 'raw': raise TypeError, "free(): only for pointers to non-gc containers" + p._obj._free() def functionptr(TYPE, name, **attrs): if not isinstance(TYPE, FuncType): Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Fri Jul 13 13:42:26 2007 @@ -427,7 +427,9 @@ p = malloc(S, flavor="raw") assert typeOf(p).TO == S assert not isweak(p, S) + p.x = 2 free(p, flavor="raw") + py.test.raises(RuntimeError, "p.x") T = GcStruct('T', ('y', Signed)) p = malloc(T, flavor="gc") assert typeOf(p).TO == T From jlg at codespeak.net Fri Jul 13 14:01:34 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Fri, 13 Jul 2007 14:01:34 +0200 (CEST) Subject: [pypy-svn] r45013 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070713120134.1A7E68234@code0.codespeak.net> Author: jlg Date: Fri Jul 13 14:01:33 2007 New Revision: 45013 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/ex.ss Log: (fijal, jlg) generated ListOper subclasses uses _annspecialcase_ for methods Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Fri Jul 13 14:01:33 2007 @@ -126,8 +126,9 @@ self.cdr = cdr def to_string(self): - return "(" + self.car.to_string() + " . " \ - + self.cdr.to_string() + ")" + car = self.car.to_string() + cdr = self.cdr.to_string() + return "(" + car + " . " + cdr + ")" def eval(self, ctx): oper = self.car.eval(ctx) @@ -250,48 +251,49 @@ def unary_oper(self, x): if isinstance(x, W_Float): - return W_Float(self.do_unary_oper_float(x.to_float())) + return W_Float(self.do_unary_oper(x.to_float())) else: - return W_Fixnum(self.do_unary_oper_int(x.to_fixnum())) + return W_Fixnum(self.do_unary_oper(x.to_fixnum())) def oper(self, x, y): if isinstance(x, W_Float) or isinstance(y, W_Float): - return W_Float(self.do_oper_float(x.to_float(), y.to_float())) + return W_Float(self.do_oper(x.to_float(), y.to_float())) else: - return W_Fixnum(self.do_oper_int(x.to_fixnum(), y.to_fixnum())) + return W_Fixnum(self.do_oper(x.to_fixnum(), y.to_fixnum())) -def create_op_class(oper, unary_oper): +def create_op_class(oper, unary_oper, title): class Op(ListOper): pass local_locals = {} - for name in ["int", "float"]: + attr_name = "do_oper" - attr_name = "do_oper_%s" % (name, ) - - code = py.code.Source(""" - def %s(self, x, y): - return x %s y - """ % (attr_name, oper)) - - exec code.compile() in local_locals - setattr(Op, attr_name, local_locals[attr_name]) - - attr_name = "do_unary_oper_%s" % (name, ) - code = py.code.Source(""" - def %s(self, x): - return %s x - """ % (attr_name, unary_oper)) - - exec code.compile() in local_locals - setattr(Op, attr_name, local_locals[attr_name]) + code = py.code.Source(""" + def %s(self, x, y): + return x %s y + """ % (attr_name, oper)) + + exec code.compile() in local_locals + local_locals[attr_name]._annspecialcase_ = 'specialize:argtype(1,2)' + setattr(Op, attr_name, local_locals[attr_name]) + + attr_name = "do_unary_oper" + code = py.code.Source(""" + def %s(self, x): + return %s x + """ % (attr_name, unary_oper)) + + exec code.compile() in local_locals + local_locals[attr_name]._annspecialcase_ = 'specialize:argtype(1)' + setattr(Op, attr_name, local_locals[attr_name]) + Op.__name__ = "Op" + title return Op -Add = create_op_class('+', '') -Sub = create_op_class('-', '-') -Mul = create_op_class('*', '') -Div = create_op_class('/', '1 /') +Add = create_op_class('+', '', "Add") +Sub = create_op_class('-', '-', "Sub") +Mul = create_op_class('*', '', "Mul") +Div = create_op_class('/', '1 /', "Div") class List(W_Procedure): def procedure(self, ctx, lst): Modified: pypy/dist/pypy/lang/scheme/test/ex.ss ============================================================================== --- pypy/dist/pypy/lang/scheme/test/ex.ss (original) +++ pypy/dist/pypy/lang/scheme/test/ex.ss Fri Jul 13 14:01:33 2007 @@ -22,12 +22,13 @@ (+ 1 2 3) (- 2) +(- 2.1) (- 1 2.9 2) (* 1 2 2) (/ 1 2.9 2) (/ 1) (/ 2 1) (/ 2 1 7) -(/ 2) +(/ 2.1) ;(quit) From fijal at codespeak.net Fri Jul 13 14:24:06 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 13 Jul 2007 14:24:06 +0200 (CEST) Subject: [pypy-svn] r45014 - in pypy/dist/pypy/objspace/flow: . test Message-ID: <20070713122406.C21A78243@code0.codespeak.net> Author: fijal Date: Fri Jul 13 14:24:05 2007 New Revision: 45014 Modified: pypy/dist/pypy/objspace/flow/flowcontext.py pypy/dist/pypy/objspace/flow/test/test_objspace.py Log: (anto, fijal, pedronis) - Patch flow objspace to take care about introduced opcodes. Modified: pypy/dist/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/dist/pypy/objspace/flow/flowcontext.py (original) +++ pypy/dist/pypy/objspace/flow/flowcontext.py Fri Jul 13 14:24:05 2007 @@ -19,6 +19,23 @@ self.block = block self.currentstate = currentstate +class PyFrame(pyframe.PyFrame): + def LOOKUP_METHOD(f, nameindex, *ignored): + space = f.space + w_obj = f.popvalue() + w_name = f.getname_w(nameindex) + w_value = space.getattr(w_obj, w_name) + f.pushvalue(w_value) + #f.pushvalue(None) + + def CALL_METHOD(f, nargs, *ignored): + # 'nargs' is the argument count excluding the implicit 'self' + w_callable = f.peekvalue(nargs) + try: + w_result = f.space.call_valuestack(w_callable, nargs, f) + finally: + f.dropvalues(nargs + 1) + f.pushvalue(w_result) class SpamBlock(Block): # make slots optional, for debugging @@ -219,7 +236,7 @@ # create an empty frame suitable for the code object # while ignoring any operation like the creation of the locals dict self.recorder = [] - frame = pyframe.PyFrame(self.space, self.code, + frame = PyFrame(self.space, self.code, self.w_globals, self.closure) frame.last_instr = 0 return frame Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_objspace.py Fri Jul 13 14:24:05 2007 @@ -795,6 +795,34 @@ simplify_graph(graph) assert self.all_operations(graph) == {'getitem': 1} + def test_callmethod_opcode(self): + """ Tests code generated by pypy-c compiled with CALL_METHOD + bytecode + """ + class X: + def m(self): + return 3 + + def f(): + x = X() + return x.m() + + # monkey patch code + import new + c = f.func_code + # this code is generated by pypy-c when compiling above f + pypy_code = 't\x00\x00\x83\x00\x00}\x00\x00|\x00\x00\x91\x02\x00\x92\x00\x00Sd\x00\x00S' + new_c = new.code(c.co_argcount, c.co_nlocals, 3, + 3, pypy_code, c.co_consts, ('X', 'x', 'm'), + ('x',), c.co_filename, + 'f', c.co_firstlineno, c.co_lnotab) + f2 = new.function(new_c, locals(), 'f') + + graph = self.codetest(f2) + all_ops = self.all_operations(graph) + assert all_ops['simple_call'] == 2 + assert all_ops['getattr'] == 1 + class TestFlowObjSpaceDelay(Base): def setup_class(cls): cls.space = FlowObjSpace() From fijal at codespeak.net Fri Jul 13 14:31:14 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 13 Jul 2007 14:31:14 +0200 (CEST) Subject: [pypy-svn] r45015 - in pypy/dist/dotviewer: . test Message-ID: <20070713123114.A3613825F@code0.codespeak.net> Author: fijal Date: Fri Jul 13 14:31:14 2007 New Revision: 45015 Modified: pypy/dist/dotviewer/graphclient.py pypy/dist/dotviewer/graphpage.py pypy/dist/dotviewer/graphparse.py pypy/dist/dotviewer/test/test_interactive.py Log: Make it run on top of pypy-c (well, not really...) Modified: pypy/dist/dotviewer/graphclient.py ============================================================================== --- pypy/dist/dotviewer/graphclient.py (original) +++ pypy/dist/dotviewer/graphclient.py Fri Jul 13 14:31:14 2007 @@ -125,7 +125,11 @@ return msgstruct.SocketIO(s) def spawn_local_handler(): - cmdline = '"%s" -u "%s" --stdio' % (sys.executable, GRAPHSERVER) + if hasattr(sys, 'pypy_objspaceclass'): + python = 'python' + else: + python = sys.executable + cmdline = '"%s" -u "%s" --stdio' % (python, GRAPHSERVER) child_in, child_out = os.popen2(cmdline, 'tb', 0) io = msgstruct.FileIO(child_out, child_in) return io Modified: pypy/dist/dotviewer/graphpage.py ============================================================================== --- pypy/dist/dotviewer/graphpage.py (original) +++ pypy/dist/dotviewer/graphpage.py Fri Jul 13 14:31:14 2007 @@ -35,9 +35,11 @@ def display_background(self): "Display a graph page in a background thread." - import thread - thread.start_new_thread(self.display, ()) - + try: + import thread + thread.start_new_thread(self.display, ()) + except ImportError: + self.display() class DotFileGraphPage(GraphPage): def compute(self, dotfile): Modified: pypy/dist/dotviewer/graphparse.py ============================================================================== --- pypy/dist/dotviewer/graphparse.py (original) +++ pypy/dist/dotviewer/graphparse.py Fri Jul 13 14:31:14 2007 @@ -2,7 +2,7 @@ Graph file parsing. """ -import os, sys, re, thread +import os, sys, re import msgstruct re_nonword = re.compile(r'([^0-9a-zA-Z_.]+)') @@ -46,7 +46,12 @@ cmdline = 'neato -Tplain' #print >> sys.stderr, '* running:', cmdline child_in, child_out = os.popen2(cmdline, 'r') - thread.start_new_thread(bkgndwrite, (child_in, content)) + try: + import thread + except ImportError: + bkgndwrite(child_in, content) + else: + thread.start_new_thread(bkgndwrite, (child_in, content)) plaincontent = child_out.read() child_out.close() if not plaincontent: # 'dot' is likely not installed Modified: pypy/dist/dotviewer/test/test_interactive.py ============================================================================== --- pypy/dist/dotviewer/test/test_interactive.py (original) +++ pypy/dist/dotviewer/test/test_interactive.py Fri Jul 13 14:31:14 2007 @@ -122,7 +122,12 @@ host, port = s.getsockname() # pick a random free port s.close() - cmdargs = [sys.executable, str(pkgdir.join('graphserver.py')), + if hasattr(sys, 'pypy_objspaceclass'): + python = 'python' + else: + python = sys.executable + + cmdargs = [python, str(pkgdir.join('graphserver.py')), str(port)] print '* starting:', ' '.join(cmdargs) pid = os.spawnv(os.P_NOWAIT, cmdargs[0], cmdargs) From pedronis at codespeak.net Fri Jul 13 14:41:05 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 13 Jul 2007 14:41:05 +0200 (CEST) Subject: [pypy-svn] r45016 - pypy/extradoc/talk/ep2007 Message-ID: <20070713124105.49A39825E@code0.codespeak.net> Author: pedronis Date: Fri Jul 13 14:41:02 2007 New Revision: 45016 Modified: pypy/extradoc/talk/ep2007/openspace_discussions.txt Log: tweaks Modified: pypy/extradoc/talk/ep2007/openspace_discussions.txt ============================================================================== --- pypy/extradoc/talk/ep2007/openspace_discussions.txt (original) +++ pypy/extradoc/talk/ep2007/openspace_discussions.txt Fri Jul 13 14:41:02 2007 @@ -30,12 +30,12 @@ Answers/comments: - CPython API support in PyPy is slow -- RPython not complete yet (ex. ref counting) +- Not much attention went into improving refcounting. - The state of RCtypes (support it, refactor and change direction, future at all) is unclear - needs discussions (how - to use ctypes at (PyPy) application level?) - we need strategic discussions regarding how to interface - with external clibraries, to be discussed during sprint + with external c libraries, to be discussed during sprint - general need for refactoring - timespan will be roughly a year to have a PyPy that usable (while still not covering all extension modules) From antocuni at codespeak.net Fri Jul 13 14:45:58 2007 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 13 Jul 2007 14:45:58 +0200 (CEST) Subject: [pypy-svn] r45017 - pypy/dist/pypy/objspace/flow/test Message-ID: <20070713124558.0A18F825E@code0.codespeak.net> Author: antocuni Date: Fri Jul 13 14:45:57 2007 New Revision: 45017 Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py Log: factor out the monkey-patching Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_objspace.py Fri Jul 13 14:45:57 2007 @@ -1,3 +1,4 @@ +import new import py from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse from pypy.objspace.flow.model import flatten @@ -795,6 +796,13 @@ simplify_graph(graph) assert self.all_operations(graph) == {'getitem': 1} + def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames): + c = code + return new.code(c.co_argcount, c.co_nlocals, stacksize, flags, + codestring, c.co_consts, names, varnames, + c.co_filename, c.co_name, c.co_firstlineno, + c.co_lnotab) + def test_callmethod_opcode(self): """ Tests code generated by pypy-c compiled with CALL_METHOD bytecode @@ -807,15 +815,9 @@ x = X() return x.m() - # monkey patch code - import new - c = f.func_code # this code is generated by pypy-c when compiling above f pypy_code = 't\x00\x00\x83\x00\x00}\x00\x00|\x00\x00\x91\x02\x00\x92\x00\x00Sd\x00\x00S' - new_c = new.code(c.co_argcount, c.co_nlocals, 3, - 3, pypy_code, c.co_consts, ('X', 'x', 'm'), - ('x',), c.co_filename, - 'f', c.co_firstlineno, c.co_lnotab) + new_c = self.monkey_patch_code(f.func_code, 3, 3, pypy_code, ('X', 'x', 'm'), ('x',)) f2 = new.function(new_c, locals(), 'f') graph = self.codetest(f2) @@ -823,6 +825,7 @@ assert all_ops['simple_call'] == 2 assert all_ops['getattr'] == 1 + class TestFlowObjSpaceDelay(Base): def setup_class(cls): cls.space = FlowObjSpace() From fijal at codespeak.net Fri Jul 13 14:47:23 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 13 Jul 2007 14:47:23 +0200 (CEST) Subject: [pypy-svn] r45018 - pypy/dist/pypy/objspace/std Message-ID: <20070713124723.E03A2825E@code0.codespeak.net> Author: fijal Date: Fri Jul 13 14:47:23 2007 New Revision: 45018 Modified: pypy/dist/pypy/objspace/std/objspace.py Log: Don't patch call_likely_builtin, unless we want to Modified: pypy/dist/pypy/objspace/std/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/std/objspace.py (original) +++ pypy/dist/pypy/objspace/std/objspace.py Fri Jul 13 14:47:23 2007 @@ -16,6 +16,7 @@ from pypy.rlib.rarithmetic import base_int from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.jit import hint, we_are_jitted +from pypy.objspace.flow.flowcontext import PyFrame as FlowPyFrame import sys import os import __builtin__ @@ -58,7 +59,7 @@ # Import all the object types and implementations self.model = StdTypeModel(self.config) - class StdObjSpaceFrame(pyframe.PyFrame): + class StdObjSpaceFrame(FlowPyFrame): if self.config.objspace.std.optimized_int_add: if self.config.objspace.std.withsmallint: def BINARY_ADD(f, oparg, *ignored): @@ -103,19 +104,20 @@ w_result = f.space.getitem(w_1, w_2) f.pushvalue(w_result) - def CALL_LIKELY_BUILTIN(f, oparg, *ignored): - from pypy.module.__builtin__ import OPTIMIZED_BUILTINS, Module - from pypy.objspace.std.dictmultiobject import W_DictMultiObject - w_globals = f.w_globals - num = oparg >> 8 - assert isinstance(w_globals, W_DictMultiObject) - w_value = w_globals.implementation.get_builtin_indexed(num) - if w_value is None: - builtins = f.get_builtin() - assert isinstance(builtins, Module) - w_builtin_dict = builtins.w_dict - assert isinstance(w_builtin_dict, W_DictMultiObject) - w_value = w_builtin_dict.implementation.get_builtin_indexed(num) + if self.config.objspace.opcodes.CALL_LIKELY_BUILTIN: + def CALL_LIKELY_BUILTIN(f, oparg, *ignored): + from pypy.module.__builtin__ import OPTIMIZED_BUILTINS, Module + from pypy.objspace.std.dictmultiobject import W_DictMultiObject + w_globals = f.w_globals + num = oparg >> 8 + assert isinstance(w_globals, W_DictMultiObject) + w_value = w_globals.implementation.get_builtin_indexed(num) + if w_value is None: + builtins = f.get_builtin() + assert isinstance(builtins, Module) + w_builtin_dict = builtins.w_dict + assert isinstance(w_builtin_dict, W_DictMultiObject) + w_value = w_builtin_dict.implementation.get_builtin_indexed(num) ## if w_value is not None: ## print "CALL_LIKELY_BUILTIN fast" if w_value is None: From fijal at codespeak.net Fri Jul 13 14:52:44 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 13 Jul 2007 14:52:44 +0200 (CEST) Subject: [pypy-svn] r45019 - pypy/dist/pypy/objspace/std Message-ID: <20070713125244.EA024825E@code0.codespeak.net> Author: fijal Date: Fri Jul 13 14:52:44 2007 New Revision: 45019 Modified: pypy/dist/pypy/objspace/std/objspace.py Log: Fix indentation Modified: pypy/dist/pypy/objspace/std/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/std/objspace.py (original) +++ pypy/dist/pypy/objspace/std/objspace.py Fri Jul 13 14:52:44 2007 @@ -120,20 +120,20 @@ w_value = w_builtin_dict.implementation.get_builtin_indexed(num) ## if w_value is not None: ## print "CALL_LIKELY_BUILTIN fast" - if w_value is None: - varname = OPTIMIZED_BUILTINS[num] - message = "global name '%s' is not defined" % varname - raise OperationError(f.space.w_NameError, - f.space.wrap(message)) - nargs = oparg & 0xff - w_function = w_value - try: - w_result = f.space.call_valuestack(w_function, nargs, f) - # XXX XXX fix the problem of resume points! - #rstack.resume_point("CALL_FUNCTION", f, nargs, returns=w_result) - finally: - f.dropvalues(nargs) - f.pushvalue(w_result) + if w_value is None: + varname = OPTIMIZED_BUILTINS[num] + message = "global name '%s' is not defined" % varname + raise OperationError(f.space.w_NameError, + f.space.wrap(message)) + nargs = oparg & 0xff + w_function = w_value + try: + w_result = f.space.call_valuestack(w_function, nargs, f) + # XXX XXX fix the problem of resume points! + #rstack.resume_point("CALL_FUNCTION", f, nargs, returns=w_result) + finally: + f.dropvalues(nargs) + f.pushvalue(w_result) if self.config.objspace.std.logspaceoptypes: _space_op_types = [] From lac at codespeak.net Fri Jul 13 14:53:46 2007 From: lac at codespeak.net (lac at codespeak.net) Date: Fri, 13 Jul 2007 14:53:46 +0200 (CEST) Subject: [pypy-svn] r45020 - in pypy/dist/pypy/interpreter/pyparser: . test Message-ID: <20070713125346.7BE64825E@code0.codespeak.net> Author: lac Date: Fri Jul 13 14:53:45 2007 New Revision: 45020 Added: pypy/dist/pypy/interpreter/pyparser/test/fakes.py (contents, props changed) pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py (contents, props changed) Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py pypy/dist/pypy/interpreter/pyparser/grammar.py pypy/dist/pypy/interpreter/pyparser/test/__init__.py pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder.py pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Log: - Refactored FakeSpace helper class to a separate file - Added a dummy builder for future_import_feature - Ate some of the trailing whitespace Justas Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/dist/pypy/interpreter/pyparser/astbuilder.py Fri Jul 13 14:53:45 2007 @@ -1038,6 +1038,7 @@ 'while_stmt' : build_while_stmt, 'import_name' : build_import_name, 'import_from' : build_import_from, + 'future_import_feature': None, 'yield_stmt' : build_yield_stmt, 'continue_stmt' : build_continue_stmt, 'del_stmt' : build_del_stmt, Modified: pypy/dist/pypy/interpreter/pyparser/grammar.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/grammar.py (original) +++ pypy/dist/pypy/interpreter/pyparser/grammar.py Fri Jul 13 14:53:45 2007 @@ -112,7 +112,8 @@ # This attribute is here for convenience self.debug = debug # the parser that represent the grammar used - assert isinstance( parser, Parser ) + # Commented the assert: this eases the testing + #assert isinstance( parser, Parser ) self.parser = parser def context(self): @@ -122,13 +123,13 @@ def restore(self, ctx): """Accept an opaque context object""" pass - + def alternative(self, rule, source): return False - + def sequence(self, rule, source, elts_number): return False - + def token(self, name, value, source): return False @@ -191,7 +192,7 @@ self.stack.append(SyntaxNode(rule.codename, [], source.current_lineno())) else: self.stack.append(TempSyntaxNode(rule.codename, [], source.current_lineno())) - + if self.debug: self.stack[-1].dumpstr() return True @@ -674,11 +675,11 @@ else: # error unknown or negative integer """ - # XXX (adim): this is trunk's keyword management + # XXX (adim): this is trunk's keyword management # if (self.value is not None and builder.keywords is not None # and self.value not in builder.keywords): # return 0 - + ctx = source.context() tk = source.next() if tk.codename == self.codename: @@ -749,7 +750,7 @@ self.symbols = {} # mapping symbol name -> symbol code self.tokens = { 'NULLTOKEN' : -1 } self.EmptyToken = Token( self, -1, None ) - self.tok_name = {} + self.tok_name = {} self.tok_values = {} self.tok_rvalues = {} self._ann_sym_count = -10 @@ -795,7 +796,7 @@ self.tok_values[value] = val # XXX : this reverse mapping seemed only to be used # because of pycodegen visitAugAssign - self.tok_rvalues[val] = value + self.tok_rvalues[val] = value return val return self.tokens[ tok ] @@ -837,10 +838,10 @@ def build_alternative( self, name_id, args ): # assert isinstance( name_id, int ) assert isinstance(args, list) - alt = Alternative( self, name_id, args ) + alt = Alternative( self, name_id, args ) self.all_rules.append( alt ) return alt - + def Alternative_n(self, name, args ): # assert isinstance(name, str) name_id = self.add_symbol( name ) @@ -851,7 +852,7 @@ alt = Sequence( self, name_id, args ) self.all_rules.append( alt ) return alt - + def Sequence_n(self, name, args ): # assert isinstance(name, str) name_id = self.add_symbol( name ) @@ -862,7 +863,7 @@ alt = KleeneStar( self, name_id, _min, _max, rule ) self.all_rules.append( alt ) return alt - + def KleeneStar_n(self, name, _min = 0, _max = -1, rule = None ): # assert isinstance(name, str) name_id = self.add_symbol( name ) Modified: pypy/dist/pypy/interpreter/pyparser/test/__init__.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/__init__.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/__init__.py Fri Jul 13 14:53:45 2007 @@ -0,0 +1 @@ + Added: pypy/dist/pypy/interpreter/pyparser/test/fakes.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/interpreter/pyparser/test/fakes.py Fri Jul 13 14:53:45 2007 @@ -0,0 +1,38 @@ + +class FakeSpace: + w_None = None + w_str = str + w_basestring = basestring + w_int = int + + def wrap(self,obj): + return obj + + def isinstance(self, obj, wtype ): + return isinstance(obj,wtype) + + def is_true(self, obj): + return obj + + def eq_w(self, obj1, obj2): + return obj1 == obj2 + + def is_w(self, obj1, obj2): + return obj1 is obj2 + + def type(self, obj): + return type(obj) + + def newlist(self, lst): + return list(lst) + + def newtuple(self, lst): + return tuple(lst) + + def call_method(self, obj, meth, *args): + return getattr(obj, meth)(*args) + + def call_function(self, func, *args): + return func(*args) + + builtin = dict(int=int, long=long, float=float, complex=complex) Modified: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder.py Fri Jul 13 14:53:45 2007 @@ -13,7 +13,7 @@ from pypy.interpreter.astcompiler import ast - +from fakes import FakeSpace from expressions import TESTS, SINGLE_INPUTS, EXEC_INPUTS def arglist_equal(left,right): @@ -49,7 +49,7 @@ left_nodes = list(left.getChildren()) right_nodes = list(right.getChildren()) if len(left_nodes) != len(right_nodes): - print "Number of children mismatch:", left, right + print "Number of children mismatch:", left, right return False for left_node, right_node in zip(left_nodes, right_nodes): if not nodes_equal(left_node, right_node, check_lineno): @@ -60,7 +60,7 @@ return left==right if left.__class__.__name__ != right.__class__.__name__: print "Node type mismatch:", left, right - return False + return False if isinstance(left,test_ast.Function) and isinstance(right,ast_ast.Function): left_nodes = list(left.getChildren()) right_nodes = [] # generated ast differ here because argnames is a list of nodes in @@ -118,7 +118,7 @@ if right.value is None: right_nodes = (ast_ast.Const(None),) else: - right_nodes = right.getChildren() + right_nodes = right.getChildren() elif isinstance(left,test_ast.Subscript): # test_ast.Subscript is not expressive enough to tell the difference # between a[x] and a[x,] :-( @@ -131,14 +131,14 @@ left_nodes = left.getChildren() right_nodes = right.getChildren() if len(left_nodes)!=len(right_nodes): - print "Number of children mismatch:", left, right + print "Number of children mismatch:", left, right return False for i,j in zip(left_nodes,right_nodes): if not nodes_equal(i,j, check_lineno): return False if check_lineno: # left is a stablecompiler.ast node which means and stable compiler - # doesn't set a lineno on each Node. + # doesn't set a lineno on each Node. # (stablecompiler.ast.Expression doesn't have a lineno attribute) if hasattr(left, 'lineno') and left.lineno is not None and left.lineno != right.lineno: print "(1) (%s) left: %s, right: %s" % (left, left.lineno, right.lineno) @@ -175,44 +175,6 @@ } -class FakeSpace: - w_None = None - w_str = str - w_basestring = basestring - w_int = int - - def wrap(self,obj): - return obj - - def isinstance(self, obj, wtype ): - return isinstance(obj,wtype) - - def is_true(self, obj): - return obj - - def eq_w(self, obj1, obj2): - return obj1 == obj2 - - def is_w(self, obj1, obj2): - return obj1 is obj2 - - def type(self, obj): - return type(obj) - - def newlist(self, lst): - return list(lst) - - def newtuple(self, lst): - return tuple(lst) - - def call_method(self, obj, meth, *args): - return getattr(obj, meth)(*args) - - def call_function(self, func, *args): - return func(*args) - - builtin = dict(int=int, long=long, float=float, complex=complex) - # Create parser from Grammar_stable, not current grammar. stable_parser = pythonparse.make_pyparser('stable') python_parser = pythonparse.make_pyparser() # 'native') # 2.5a') @@ -241,7 +203,7 @@ check_lineno = False print "-" * 30 print "ORIG :", python_ast - print + print print "BUILT:", pypy_ast print "-" * 30 assert nodes_equal(python_ast, pypy_ast, check_lineno), 'failed on %r' % (expr) @@ -258,12 +220,12 @@ yield check_expression, expr, 'exec' -NEW_GRAMMAR_SNIPPETS = [ +NEW_GRAMMAR_SNIPPETS = [ 'snippet_with_1.py', 'snippet_with_2.py', ] -SNIPPETS = [ +SNIPPETS = [ 'snippet_1.py', 'snippet_several_statements.py', 'snippet_simple_function.py', Added: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Fri Jul 13 14:53:45 2007 @@ -0,0 +1,37 @@ +from pypy.interpreter.pyparser.astbuilder import AstBuilder +from pypy.interpreter.pyparser.grammar import Parser +from pypy.interpreter.pyparser.pytoken import setup_tokens +from fakes import FakeSpace + + +class ParserStub(): + def __init__(self): + self.tokens = {} + self._sym_count = 0 + self.tok_values = {} + self.tok_rvalues = {} + + def add_token( self, tok, value = None ): + # assert isinstance( tok, str ) + if not tok in self.tokens: + val = self._sym_count + self._sym_count += 1 + self.tokens[tok] = val + #self.tok_name[val] = tok + if value is not None: + self.tok_values[value] = val + self.tok_rvalues[val] = value + return val + return self.tokens[ tok ] + + +class TestBuilderFuture: + def setup_class(self): + self.parser = ParserStub() + setup_tokens(self.parser) + + def setup_method(self, method): + self.builder = AstBuilder(self.parser, space=FakeSpace()) + + def test_future_rules(self): + assert 'future_import_feature' in self.builder.build_rules Modified: pypy/dist/pypy/interpreter/pyparser/test/test_parser.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_parser.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Fri Jul 13 14:53:45 2007 @@ -1,5 +1,7 @@ from pypy.interpreter.pyparser.grammar import Parser from pypy.interpreter.pyparser import error +from fakes import FakeSpace + def test_symbols(): p = Parser() @@ -42,47 +44,10 @@ v = p.add_symbol( 'sym3' ) assert v == 9 -class FakeSpace: - w_None = None - w_str = str - w_basestring = basestring - w_int = int - - def wrap(self,obj): - return obj - - def isinstance(self, obj, wtype ): - return isinstance(obj,wtype) - - def is_true(self, obj): - return obj - - def eq_w(self, obj1, obj2): - return obj1 == obj2 - - def is_w(self, obj1, obj2): - return obj1 is obj2 - - def type(self, obj): - return type(obj) - - def newlist(self, lst): - return list(lst) - - def newtuple(self, lst): - return tuple(lst) - - def call_method(self, obj, meth, *args): - return getattr(obj, meth)(*args) - - def call_function(self, func, *args): - return func(*args) - - builtin = dict(int=int, long=long, float=float, complex=complex) from pypy.interpreter.pyparser.asthelper import get_atoms class RuleTracer(dict): - + def __init__(self, *args, **kw): self.trace = [] @@ -92,7 +57,7 @@ 'single_input', 'file_input', 'future_import_list', 'import_from_future', 'future_import_as_names']: return None - + def record_trace(builder, number): result = [t.value for t in get_atoms(builder, number)] self.trace.append((attr, result)) @@ -100,7 +65,7 @@ def get(self, attr, default): return self.__getitem__(attr) - + from pypy.interpreter.pyparser.astbuilder import AstBuilder class MockBuilder(AstBuilder): @@ -110,7 +75,7 @@ class TestFuture(object): - + def setup_class(self): from pypy.interpreter.pyparser.pythonparse import make_pyparser self.parser = make_pyparser('2.5a') @@ -121,18 +86,18 @@ def check_parse_mode(self, tst, expected, mode): self.parser.parse_source(tst, mode, self.builder) assert self.builder.build_rules.trace == expected - + def check_parse(self, tst, expected): self.check_parse_mode(tst, expected, 'exec') self.builder.build_rules.trace = [] self.check_parse_mode(tst, expected, 'single') - + def test_single_future_import(self): tst = 'from __future__ import na\n' expected = [('future_import_feature', ['na'])] self.check_parse(tst, expected) - + def test_double_future_import(self): tst = 'from __future__ import na, xx\n' expected = [('future_import_feature', ['na']), @@ -160,7 +125,7 @@ tst = 'from __future__ import na as x\n' expected = [('future_import_feature', ['na', 'as', 'x'])] self.check_parse(tst, expected) - + def test_single_future_as(self): tst = 'import sys;from __future__ import na as x\n' expected = [] @@ -174,4 +139,4 @@ tst = 'import sys' expected = [('import_name', ['import', 'sys'])] self.check_parse(tst, expected) - + From arigo at codespeak.net Fri Jul 13 15:27:34 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 13 Jul 2007 15:27:34 +0200 (CEST) Subject: [pypy-svn] r45021 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070713132734.AAEB8827B@code0.codespeak.net> Author: arigo Date: Fri Jul 13 15:27:33 2007 New Revision: 45021 Modified: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py Log: (lac, arigo) First test passes: allocating and freeing ctypes-based structs and arrays. Modified: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/compactlltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/compactlltype.py Fri Jul 13 15:27:33 2007 @@ -1,51 +1,46 @@ -import ctypes, sys, weakref -from pypy.rpython.lltypesystem.lltype import Signed, Struct, GcStruct, Ptr -from pypy.rpython.lltypesystem.lltype import ContainerType, Array, GcArray -from pypy.rpython.lltypesystem.lltype import typeOf, castable +import sys +import ctypes +from pypy.rpython.lltypesystem import lltype +from pypy.tool.uid import fixid _Ctypes_PointerType = type(ctypes.POINTER(ctypes.c_int)) - -def cast_pointer(PTRTYPE, ptr): - CURTYPE = typeOf(ptr) - if not isinstance(CURTYPE, Ptr) or not isinstance(PTRTYPE, Ptr): - raise TypeError, "can only cast pointers to other pointers" - return ptr._cast_to(PTRTYPE) - - -class _parentable(ctypes.Structure): # won't work with Union - __slots__ = () +def uaddressof(obj): + return fixid(ctypes.addressof(obj)) # ____________________________________________________________ +_allocated = [] + _ctypes_cache = { - Signed: ctypes.c_long, + lltype.Signed: ctypes.c_long, } -def _build_ctypes_struct(S, max_n=None): +def build_ctypes_struct(S, max_n=None): fields = [] for fieldname in S._names: FIELDTYPE = S._flds[fieldname] if max_n is not None and fieldname == S._arrayfld: - cls = _build_ctypes_array(FIELDTYPE, max_n) + cls = build_ctypes_array(FIELDTYPE, max_n) else: - cls = _get_ctypes_type(FIELDTYPE) + cls = get_ctypes_type(FIELDTYPE) fields.append((fieldname, cls)) - class CStruct(_parentable): + class CStruct(ctypes.Structure): _fields_ = fields - _TYPE = S def malloc(cls, n=None): - S = cls._TYPE if S._arrayfld is None: if n is not None: raise TypeError("%r is not variable-sized" % (S,)) - return ctypes.pointer(cls()) + storage = cls() + _allocated.append(storage) + return _ctypes_struct(S, storage) else: + XXX if n is None: raise TypeError("%r is variable-sized" % (S,)) - smallercls = _build_ctypes_struct(S, n) + smallercls = build_ctypes_struct(S, n) smallstruct = smallercls() getattr(smallstruct, S._arrayfld).length = n structptr = ctypes.cast(ctypes.pointer(smallstruct), @@ -56,150 +51,152 @@ CStruct.__name__ = 'ctypes_%s' % (S,) return CStruct -def _build_ctypes_array(A, max_n=None): +def build_ctypes_array(A, max_n=0): + assert max_n >= 0 ITEM = A.OF - ctypes_item = _get_ctypes_type(ITEM) - if max_n is None: - max_n = sys.maxint // ctypes.sizeof(ctypes_item) - max_n //= 2 # XXX better safe than sorry about ctypes bugs + ctypes_item = get_ctypes_type(ITEM) - class CArray(_parentable): + class CArray(ctypes.Structure): _fields_ = [('length', ctypes.c_int), ('items', max_n * ctypes_item)] - _TYPE = A def malloc(cls, n=None): if not isinstance(n, int): raise TypeError, "array length must be an int" - smallercls = _build_ctypes_array(cls._TYPE, n) - smallarray = smallercls() - smallarray.length = n - arrayptr = ctypes.cast(ctypes.pointer(smallarray), - ctypes.POINTER(cls)) - return arrayptr + biggercls = build_ctypes_array(A, n) + bigarray = biggercls() + _allocated.append(bigarray) + bigarray.length = n + return _ctypes_array(A, bigarray) malloc = classmethod(malloc) - CArray.__name__ = 'ctypes_%s' % (A,) + CArray.__name__ = 'ctypes_%s*%d' % (A, max_n) return CArray -def _get_ctypes_type(T): +def get_ctypes_type(T): try: return _ctypes_cache[T] except KeyError: - if isinstance(T, Ptr): - cls = ctypes.POINTER(_get_ctypes_type(T.TO)) - elif isinstance(T, Struct): - cls = _build_ctypes_struct(T) - elif isinstance(T, Array): - cls = _build_ctypes_array(T) + if isinstance(T, lltype.Ptr): + cls = ctypes.POINTER(get_ctypes_type(T.TO)) + elif isinstance(T, lltype.Struct): + cls = build_ctypes_struct(T) + elif isinstance(T, lltype.Array): + cls = build_ctypes_array(T) else: raise NotImplementedError(T) _ctypes_cache[T] = cls return cls -def _expose(val): - if isinstance(type(val), _Ctypes_PointerType): - val = val.contents - T = typeOf(val) - if isinstance(T, ContainerType): - val = _ptr(ctypes.pointer(val)) +def ctypes2lltype(val, RESTYPE): + if isinstance(RESTYPE, lltype.Struct): + return _ctypes_struct(RESTYPE, val) + if isinstance(RESTYPE, lltype.Array): + return _ctypes_array(RESTYPE, val) + if isinstance(RESTYPE, lltype.Ptr): + if isinstance(RESTYPE.TO, lltype.Array): + ArrayType = build_ctypes_array(RESTYPE.TO, + max_n=val.contents.length) + val = ctypes.cast(val, ctypes.POINTER(ArrayType)) + obj = ctypes2lltype(val.contents, RESTYPE.TO) + return lltype._ptr(RESTYPE, obj, solid=True) return val -def _lltype2ctypes(val): - T = typeOf(val) - if isinstance(T, Ptr): - return val._storageptr +def lltype2ctypes(val): + T = lltype.typeOf(val) + if isinstance(T, lltype.Ptr): + return val._obj._convert_to_ctypes_pointer() return val -class _ptr(object): - __slots__ = ['_storageptr', '_TYPE'] - def __init__(self, storageptr): - assert isinstance(type(storageptr), _Ctypes_PointerType) - _ptr._storageptr.__set__(self, storageptr) - _ptr._TYPE.__set__(self, Ptr(type(storageptr)._type_._TYPE)) +class _ctypes_parentable(lltype._parentable): - def __getattr__(self, field_name): - if isinstance(self._TYPE.TO, Struct): - if field_name in self._TYPE.TO._flds: - return _expose(getattr(self._storageptr.contents, field_name)) - raise AttributeError("%r instance has no field %r" % (self._TYPE.TO, - field_name)) + __slots__ = ('_ctypes_storage',) + + def __init__(self, TYPE, ctypes_storage): + lltype._parentable.__init__(self, TYPE) + self._ctypes_storage = ctypes_storage + + def _free(self): + # XXX REALLY SLOW! + myaddress = ctypes.addressof(self._ctypes_storage) + for i, obj in enumerate(_allocated): + if ctypes.addressof(obj) == myaddress: + # found it + del _allocated[i] + lltype._parentable._free(self) + break + else: + raise RuntimeError("lltype.free() on a pointer that was not " + "obtained by lltype.malloc()") + + +class _ctypes_struct(_ctypes_parentable): + _kind = "structure" + __slots__ = () + + def __repr__(self): + return '' % ( + self._TYPE._name, + uaddressof(self._ctypes_storage)) def __setattr__(self, field_name, value): - if isinstance(self._TYPE.TO, Struct): - if field_name in self._TYPE.TO._flds: - setattr(self._storageptr.contents, field_name, - _lltype2ctypes(value)) - return - raise AttributeError("%r instance has no field %r" % (self._TYPE.TO, - field_name)) - - def __nonzero__(self): - return bool(self._storageptr) - - def __len__(self): - T = self._TYPE.TO - if isinstance(T, Array):# ,FixedSizeArray)): - #if self._T._hints.get('nolength', False): - # raise TypeError("%r instance has no length attribute" % - # (self._T,)) - return self._storageptr.contents.length - raise TypeError("%r instance is not an array" % (T,)) - - def __getitem__(self, i): - T = self._TYPE.TO - if isinstance(T, Array): - start, stop = 0, self._storageptr.contents.length - if not (start <= i < stop): - if isinstance(i, slice): - raise TypeError("array slicing not supported") - raise IndexError("array index out of bounds") - return _expose(self._storageptr.contents.items[i]) - raise TypeError("%r instance is not an array" % (T,)) - - def _cast_to(self, PTRTYPE): - CURTYPE = self._TYPE - down_or_up = castable(PTRTYPE, CURTYPE) - if down_or_up == 0: - return self - if not self: # null pointer cast - return PTRTYPE._defl() - WAAA - if isinstance(self._obj, int): - return _ptr(PTRTYPE, self._obj, solid=True) - if down_or_up > 0: - p = self - while down_or_up: - p = getattr(p, typeOf(p).TO._names[0]) - down_or_up -= 1 - return _ptr(PTRTYPE, p._obj, solid=self._solid) - u = -down_or_up - struc = self._obj - while u: - parent = struc._parentstructure() - if parent is None: - raise RuntimeError("widening to trash: %r" % self) - PARENTTYPE = struc._parent_type - if getattr(parent, PARENTTYPE._names[0]) is not struc: - raise InvalidCast(CURTYPE, PTRTYPE) # xxx different exception perhaps? - struc = parent - u -= 1 - if PARENTTYPE != PTRTYPE.TO: - raise RuntimeError("widening %r inside %r instead of %r" % (CURTYPE, PARENTTYPE, PTRTYPE.TO)) - return _ptr(PTRTYPE, struc, solid=self._solid) + if field_name.startswith('_'): + lltype._parentable.__setattr__(self, field_name, value) + else: + setattr(self._ctypes_storage, field_name, lltype2ctypes(value)) + + def _getattr(self, field_name, uninitialized_ok=False): + return getattr(self, field_name) + + def __getattr__(self, field_name): + if field_name.startswith('_'): + return lltype._parentable.__getattr__(self, field_name) + else: + return ctypes2lltype(getattr(self._ctypes_storage, field_name), + getattr(self._TYPE, field_name)) + + def _convert_to_ctypes_pointer(self): + return ctypes.pointer(self._ctypes_storage) + +class _ctypes_array(_ctypes_parentable): + _kind = "array" + __slots__ = () + + def __init__(self, TYPE, ctypes_storage): + _ctypes_parentable.__init__(self, TYPE, ctypes_storage) + assert ctypes_storage.length == len(ctypes_storage.items) + + def __repr__(self): + return '' % ( + uaddressof(self._ctypes_storage),) + + def getlength(self): + length = self._ctypes_storage.length + assert length == len(self._ctypes_storage.items) + return length + + def getbounds(self): + return 0, self.getlength() + + def getitem(self, index): + return ctypes2lltype(self._ctypes_storage.items[index], + self._TYPE.OF) + + def _convert_to_ctypes_pointer(self): + PtrType = ctypes.POINTER(get_ctypes_type(self._TYPE)) + return ctypes.cast(ctypes.pointer(self._ctypes_storage), PtrType) # ____________________________________________________________ -def malloc(T, n=None, immortal=False): - if T._gckind != 'gc' and not immortal:# and flavor.startswith('gc'): - raise TypeError, "gc flavor malloc of a non-GC non-immortal structure" - if isinstance(T, (Struct, Array)): - cls = _get_ctypes_type(T) - return _ptr(cls.malloc(n)) - else: - raise TypeError, "malloc for Structs and Arrays only" - -def nullptr(T): - cls = _get_ctypes_type(T) - return _ptr(ctypes.POINTER(cls)()) +def malloc(T, n=None, flavor='gc', immortal=False): + # XXX for now, let's only worry about raw malloc + assert flavor == 'raw' + assert T._gckind == 'raw' + assert isinstance(T, (lltype.Struct, lltype.Array)) + cls = get_ctypes_type(T) + container = cls.malloc(n) + return lltype._ptr(lltype.Ptr(T), container, solid=True) + +def getobjcount(): + return len(_allocated) Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Fri Jul 13 15:27:33 2007 @@ -1199,6 +1199,7 @@ self._dead = False def _free(self): + self._check() # no double-frees self._dead = True def _setparentstructure(self, parent, parentindex): @@ -1664,7 +1665,7 @@ T = typeOf(p) if not isinstance(T, Ptr) or p._togckind() != 'raw': raise TypeError, "free(): only for pointers to non-gc containers" - p._obj._free() + p._obj0._free() def functionptr(TYPE, name, **attrs): if not isinstance(TYPE, FuncType): Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py Fri Jul 13 15:27:33 2007 @@ -1,64 +1,69 @@ -import py -from pypy.rpython.lltypesystem.compactlltype import * +from pypy.rpython.lltypesystem.lltype import * +from pypy.rpython.lltypesystem.compactlltype import malloc, getobjcount -def isweak(p, T): - return True# p._weak and typeOf(p).TO == T +py.test.skip("in-progress") -def test_nullptr(): - S = Struct('s') - p0 = nullptr(S) - assert not p0 - assert typeOf(p0) == Ptr(S) +def isctypes(p): + return hasattr(p._obj, '_ctypes_storage') def test_basics(): - S0 = GcStruct("s0", ('a', Signed), ('b', Signed)) + objcount = getobjcount() + S0 = Struct("s0", ('a', Signed), ('b', Signed)) assert S0.a == Signed assert S0.b == Signed - s0 = malloc(S0) + s0 = malloc(S0, flavor='raw') print s0 + assert isctypes(s0) assert typeOf(s0) == Ptr(S0) - #py.test.raises(UninitializedMemoryAccess, "s0.a") XXX! + #py.test.raises(UninitializedMemoryAccess, "s0.a") s0.a = 1 s0.b = s0.a assert s0.a == 1 assert s0.b == 1 assert typeOf(s0.a) == Signed + free(s0, flavor='raw') + py.test.raises(RuntimeError, "s0.a") # accessing freed struct s0 + py.test.raises(RuntimeError, "free(s0, flavor='raw')") # double free # simple array - Ar = GcArray(('v', Signed)) - x = malloc(Ar,0) + Ar = Array(('v', Signed)) + x = malloc(Ar, 0, flavor='raw') print x + assert isctypes(x) assert len(x) == 0 - x = malloc(Ar,3) + free(x, flavor='raw') + x = malloc(Ar, 3, flavor='raw') print x + assert isctypes(x) assert typeOf(x) == Ptr(Ar) assert len(x) == 3 - assert isweak(x[0], Ar.OF) assert typeOf(x[0]) == Ptr(Ar.OF) x[0].v = 1 x[1].v = 2 x[2].v = 3 assert typeOf(x[0].v) == Signed assert [x[z].v for z in range(3)] == [1, 2, 3] + free(x, flavor='raw') # def define_list(T): - List_typ = GcStruct("list", - ("items", Ptr(GcArray(('item',T))))) + List_typ = Struct("list", + ("items", Ptr(Array(('item',T))))) def newlist(): - l = malloc(List_typ) - items = malloc(List_typ.items.TO, 0) + l = malloc(List_typ, flavor='raw') + items = malloc(List_typ.items.TO, 0, flavor='raw') l.items = items return l def append(l, newitem): length = len(l.items) print 'append! length', length, 'newitem', newitem - newitems = malloc(List_typ.items.TO, length+1) + newitems = malloc(List_typ.items.TO, length+1, flavor='raw') i = 0 while i Author: pedronis Date: Fri Jul 13 15:33:20 2007 New Revision: 45023 Modified: pypy/dist/pypy/rpython/llinterp.py Log: dummy implementation Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Jul 13 15:33:20 2007 @@ -760,6 +760,9 @@ op_boehm_malloc = op_boehm_malloc_atomic = op_raw_malloc + def op_boehm_register_finalizer(self, p, finalizer): + pass + def op_raw_malloc_usage(self, size): assert lltype.typeOf(size) == lltype.Signed return self.heap.raw_malloc_usage(size) From justas at codespeak.net Fri Jul 13 15:36:02 2007 From: justas at codespeak.net (justas at codespeak.net) Date: Fri, 13 Jul 2007 15:36:02 +0200 (CEST) Subject: [pypy-svn] r45024 - in pypy/dist/pypy/interpreter/pyparser: . test Message-ID: <20070713133602.29A27827F@code0.codespeak.net> Author: justas Date: Fri Jul 13 15:36:01 2007 New Revision: 45024 Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Log: Dummy import from __future__ builder functions + tests Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/dist/pypy/interpreter/pyparser/astbuilder.py Fri Jul 13 15:36:01 2007 @@ -879,6 +879,14 @@ builder.push(ast.From(from_name, names, atoms[0].lineno)) +def build_future_import_feature(builder, nb): + """ + future_import_feature: NAME [('as'|NAME) NAME] + + Enables python language future imports. Called once per feature imported, + no matter how you got to this one particular feature. + """ + def build_yield_stmt(builder, nb): atoms = get_atoms(builder, nb) builder.push(ast.Yield(atoms[1], atoms[0].lineno)) @@ -1038,7 +1046,8 @@ 'while_stmt' : build_while_stmt, 'import_name' : build_import_name, 'import_from' : build_import_from, - 'future_import_feature': None, + 'future_import_feature': build_future_import_feature, + 'import_from_future': build_import_from, 'yield_stmt' : build_yield_stmt, 'continue_stmt' : build_continue_stmt, 'del_stmt' : build_del_stmt, Modified: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Fri Jul 13 15:36:01 2007 @@ -1,6 +1,7 @@ -from pypy.interpreter.pyparser.astbuilder import AstBuilder from pypy.interpreter.pyparser.grammar import Parser from pypy.interpreter.pyparser.pytoken import setup_tokens +from pypy.interpreter.pyparser import astbuilder + from fakes import FakeSpace @@ -25,13 +26,27 @@ return self.tokens[ tok ] +class RuleStub: + def __init__(self, name, root=False): + self.codename = name + self.root = root + is_root = lambda self: self.root + + class TestBuilderFuture: def setup_class(self): self.parser = ParserStub() setup_tokens(self.parser) def setup_method(self, method): - self.builder = AstBuilder(self.parser, space=FakeSpace()) + self.builder = astbuilder.AstBuilder(self.parser, space=FakeSpace()) def test_future_rules(self): - assert 'future_import_feature' in self.builder.build_rules + assert (self.builder.build_rules['future_import_feature'] is + astbuilder.build_future_import_feature) + assert (self.builder.build_rules['import_from_future'] is + astbuilder.build_import_from) + + def test_future_import(self): + #self.builder.push(RuleStub('future_import_feature', root=True)) + pass From jlg at codespeak.net Fri Jul 13 16:21:18 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Fri, 13 Jul 2007 16:21:18 +0200 (CEST) Subject: [pypy-svn] r45026 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070713142118.88FDE828E@code0.codespeak.net> Author: jlg Date: Fri Jul 13 16:21:17 2007 New Revision: 45026 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: Equal accept any number of arguments Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Fri Jul 13 16:21:17 2007 @@ -2,7 +2,6 @@ ------ - lambda called with wrong number of args issue -- RPythonize (during EP sprint) Do next ------- @@ -18,8 +17,7 @@ - comparison: < > eq? eqv? -- rational, complex number types - - exact/inexact + - exact/inexact numbers Do in some future ----------------- Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Fri Jul 13 16:21:17 2007 @@ -7,6 +7,10 @@ def __str__(self): return "Unbound variable %s" % self.args[0] +class WrongArgsNumber(SchemeException): + def __str__(self): + return "Wrong number of args" + class SchemeQuit(SchemeException): """raised on (quit) evaluation""" pass @@ -237,6 +241,12 @@ ## class ListOper(W_Procedure): def procedure(self, ctx, lst): + if len(lst) == 0: + if self.default_result is None: + raise WrongArgsNumber() + + return self.default_result + if len(lst) == 1: return self.unary_oper(lst[0].eval(ctx)) @@ -261,7 +271,7 @@ else: return W_Fixnum(self.do_oper(x.to_fixnum(), y.to_fixnum())) -def create_op_class(oper, unary_oper, title): +def create_op_class(oper, unary_oper, title, default_result=None): class Op(ListOper): pass @@ -287,14 +297,32 @@ local_locals[attr_name]._annspecialcase_ = 'specialize:argtype(1)' setattr(Op, attr_name, local_locals[attr_name]) + if default_result is None: + Op.default_result = None + else: + Op.default_result = W_Fixnum(default_result) + Op.__name__ = "Op" + title return Op -Add = create_op_class('+', '', "Add") +Add = create_op_class('+', '', "Add", 0) Sub = create_op_class('-', '-', "Sub") -Mul = create_op_class('*', '', "Mul") +Mul = create_op_class('*', '', "Mul", 1) Div = create_op_class('/', '1 /', "Div") +class Equal(W_Procedure): + def procedure(self, ctx, lst): + if len(lst) < 2: + return W_Boolean(True) + + prev = lst[0] + for arg in lst[1:]: + if prev.to_number() != arg.to_number(): + return W_Boolean(False) + prev = arg + + return W_Boolean(True) + class List(W_Procedure): def procedure(self, ctx, lst): return plst2lst(lst) @@ -349,12 +377,6 @@ w_pair = lst[0] return w_pair.cdr -class Equal(W_Procedure): - def procedure(self, ctx, lst): - w_first = lst[0] - w_second = lst[1] - return W_Boolean(w_first.equal(w_second)) - class Quit(W_Procedure): def procedure(self, ctx, lst): raise SchemeQuit Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Fri Jul 13 16:21:17 2007 @@ -14,6 +14,8 @@ return parse(expr)[0].eval(None) def test_numerical(): + w_num = eval_noctx("(+)") + assert w_num.to_number() == 0 w_num = eval_noctx("(+ 4)") assert w_num.to_number() == 4 w_num = eval_noctx("(+ 4 -5)") @@ -21,6 +23,8 @@ w_num = eval_noctx("(+ 4 -5 6.1)") assert w_num.to_number() == 5.1 + w_num = eval_noctx("(*)") + assert w_num.to_number() == 1 w_num = eval_noctx("(* 4)") assert w_num.to_number() == 4 w_num = eval_noctx("(* 4 -5)") @@ -28,6 +32,7 @@ w_num = eval_noctx("(* 4 -5 6.1)") assert w_num.to_number() == (4 * -5 * 6.1) + py.test.raises(WrongArgsNumber, eval_noctx, "(/)") w_num = eval_noctx("(/ 4)") assert w_num.to_number() == 1 / 4 w_num = eval_noctx("(/ 4 -5)") @@ -35,6 +40,7 @@ w_num = eval_noctx("(/ 4 -5 6.1)") assert w_num.to_number() == (4 / -5 / 6.1) + py.test.raises(WrongArgsNumber, eval_noctx, "(-)") w_num = eval_noctx("(- 4)") assert w_num.to_number() == -4 w_num = eval_noctx("(- 4 5)") @@ -148,23 +154,41 @@ assert w_cddr.to_number() == 3 def test_comparison_homonums(): + w_bool = eval_noctx("(=)") + assert w_bool.to_boolean() is True + + w_bool = eval_noctx("(= 1)") + assert w_bool.to_boolean() is True + w_bool = eval_noctx("(= 1 2)") assert w_bool.to_boolean() is False w_bool = eval_noctx("(= 2 2)") assert w_bool.to_boolean() is True + w_bool = eval_noctx("(= 2 2 2 2)") + assert w_bool.to_boolean() is True + + w_bool = eval_noctx("(= 2 2 3 2)") + assert w_bool.to_boolean() is False + w_bool = eval_noctx("(= 2.1 1.2)") assert w_bool.to_boolean() is False w_bool = eval_noctx("(= 2.1 2.1)") assert w_bool.to_boolean() is True + w_bool = eval_noctx("(= 2.1 2.1 2.1 2.1)") + assert w_bool.to_boolean() is True + + w_bool = eval_noctx("(= 2.1 2.1 2.1 2)") + assert w_bool.to_boolean() is False + def test_comparison_heteronums(): - w_bool = eval_noctx("(= 1 2.2)") + w_bool = eval_noctx("(= 1 1.0 1.1)") assert w_bool.to_boolean() is False - w_bool = eval_noctx("(= 2.0 2)") + w_bool = eval_noctx("(= 2.0 2 2.0)") assert w_bool.to_boolean() is True def test_lambda_noargs(): From fijal at codespeak.net Fri Jul 13 16:33:43 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 13 Jul 2007 16:33:43 +0200 (CEST) Subject: [pypy-svn] r45027 - in pypy/dist/pypy/module/operator: . test Message-ID: <20070713143343.7D826814E@code0.codespeak.net> Author: fijal Date: Fri Jul 13 16:33:43 2007 New Revision: 45027 Added: pypy/dist/pypy/module/operator/test/ (props changed) pypy/dist/pypy/module/operator/test/test_operator.py (contents, props changed) Modified: pypy/dist/pypy/module/operator/__init__.py pypy/dist/pypy/module/operator/interp_operator.py Log: (arigo, fijal, anto, pedronis) Whack a bit around operator.py to support operator.eq == operator.__eq__ Modified: pypy/dist/pypy/module/operator/__init__.py ============================================================================== --- pypy/dist/pypy/module/operator/__init__.py (original) +++ pypy/dist/pypy/module/operator/__init__.py Fri Jul 13 16:33:43 2007 @@ -4,25 +4,29 @@ class Module(MixedModule): """Operator Builtin Module. """ + # HACK! override loaders to be able to access different operations + # under same name. I.e., operator.eq == operator.__eq__ + + def __init__(self, space, w_name): + def create_lambda(name, alsoname): + return lambda space : self.getdictvalue(space, space.wrap(alsoname)) + + MixedModule.__init__(self, space, w_name) + for name, alsoname in self.mapping.iteritems(): + self.loaders[name] = create_lambda(name, alsoname) + appleveldefs = {} app_names = ['__delslice__', '__getslice__', '__repeat__', '__setslice__', - 'attrgetter', 'countOf', 'delslice', 'getslice', 'indexOf', - 'isMappingType', 'isNumberType', 'isSequenceType', - 'itemgetter','repeat', 'setslice', + 'attrgetter', 'countOf', 'delslice', 'getslice', 'indexOf', + 'isMappingType', 'isNumberType', 'isSequenceType', + 'itemgetter','repeat', 'setslice', ] for name in app_names: appleveldefs[name] = 'app_operator.%s' % name - interp_names = ['index', '__abs__', '__add__', '__and__', - '__concat__', '__contains__', '__delitem__','__div__', - '__eq__', '__floordiv__', '__ge__', '__getitem__', - '__gt__', '__inv__', '__invert__', '__le__', - '__lshift__', '__lt__', '__mod__', '__mul__', - '__ne__', '__neg__', '__not__', '__or__', - '__pos__', '__pow__', '__rshift__', '__setitem__', - '__sub__', '__truediv__', '__xor__', 'abs', 'add', + interp_names = ['index', 'abs', 'add', 'and_', 'concat', 'contains', 'delitem', 'div', 'eq', 'floordiv', 'ge', 'getitem', 'gt', 'inv', 'invert', 'is_', 'is_not', 'isCallable', 'le', @@ -34,3 +38,37 @@ for name in interp_names: interpleveldefs[name] = 'interp_operator.%s' % name + + mapping = { + '__abs__' : 'abs', + '__add__' : 'add', + '__and__' : 'and_', + '__concat__' : 'concat', + '__contains__' : 'contains', + 'sequenceIncludes' : 'contains', + '__delitem__' : 'delitem', + '__div__' : 'div', + '__eq__' : 'eq', + '__floordiv__' : 'floordiv', + '__ge__' : 'ge', + '__getitem__' : 'getitem', + '__gt__' : 'gt', + '__inv__' : 'inv', + '__invert__' : 'invert', + '__le__' : 'le', + '__lshift__' : 'lshift', + '__lt__' : 'lt', + '__mod__' : 'mod', + '__mul__' : 'mul', + '__ne__' : 'ne', + '__neg__' : 'neg', + '__not__' : 'not_', + '__or__' : 'or_', + '__pos__' : 'pos', + '__pow__' : 'pow', + '__rshift__' : 'rshift', + '__setitem__' : 'setitem', + '__sub__' : 'sub', + '__truediv__' : 'truediv', + '__xor__' : 'xor', + } Modified: pypy/dist/pypy/module/operator/interp_operator.py ============================================================================== --- pypy/dist/pypy/module/operator/interp_operator.py (original) +++ pypy/dist/pypy/module/operator/interp_operator.py Fri Jul 13 16:33:43 2007 @@ -5,17 +5,13 @@ 'abs(a) -- Same as abs(a).' return space.abs(w_obj) -__abs__ = abs - def add(space, w_obj1, w_obj2): 'add(a, b) -- Same as a a + b' return space.add(w_obj1, w_obj2) -__add__ = add def and_(space, w_obj1, w_obj2): 'and_(a, b) -- Same as a a & b' return space.and_(w_obj1, w_obj2) -__and__ = and_ # attrgetter @@ -23,13 +19,10 @@ 'concat(a, b) -- Same as a a + b, for a and b sequences.' return space.add(w_obj1, w_obj2) # XXX cPython only works on types with sequence api # we support any with __add__ -__concat__ = concat def contains(space, w_obj1, w_obj2): 'contains(a, b) -- Same as b in a (note reversed operands).' return space.contains(w_obj1, w_obj2) -__contains__ = contains -sequenceIncludes = contains # countOf @@ -37,53 +30,43 @@ 'delitem(a,b) -- Same as del a[b]' space.delitem(w_obj, w_key) -__delitem__ = delitem - # delslice def div(space, w_a, w_b): 'div(a, b) -- Same as a / b when __future__.division is no in effect' return space.div(w_a, w_b) -__div__ = div def eq(space, w_a, w_b): 'eq(a, b) -- Same as a==b' return space.eq(w_a, w_b) -__eq__ = eq def floordiv(space, w_a, w_b): 'floordiv(a, b) -- Same as a // b.' return space.floordiv(w_a, w_b) -__floordiv__ = floordiv def ge(space, w_a, w_b): 'ge(a, b) -- Same as a>=b.' return space.ge(w_a, w_b) -__ge__ = ge def getitem(space, w_a, w_b): 'getitem(a, b) -- Same as a[b].' return space.getitem(w_a, w_b) -__getitem__ = getitem # getslice def gt(space, w_a, w_b): 'gt(a, b) -- Same as a>b.' return space.gt(w_a, w_b) -__gt__ = gt # indexOf def inv(space, w_obj,): 'inv(a) -- Same as ~a.' return space.invert(w_obj) -__inv__ = inv def invert(space, w_obj,): 'invert(a) -- Same as ~a.' return space.invert(w_obj) -__invert__ = invert def isCallable(space, w_obj): 'isCallable(a) -- Same as callable(a).' @@ -108,83 +91,68 @@ def le(space, w_a, w_b): 'le(a, b) -- Same as a<=b.' return space.le(w_a, w_b) -__le__ = le def lshift(space, w_a, w_b): 'lshift(a, b) -- Same as a << b.' return space.lshift(w_a, w_b) -__lshift__ = lshift def lt(space, w_a, w_b): 'lt(a, b) -- Same as a> b.' return space.rshift(w_a, w_b) -__rshift__ = rshift # sequenceIncludes def setitem(space, w_obj, w_key, w_value): 'setitem(a, b, c) -- Same as a[b] = c.' space.setitem(w_obj, w_key, w_value) -__setitem__ = setitem # setslice def sub(space, w_a, w_b): 'sub(a, b) -- Same as a - b.' return space.sub(w_a, w_b) -__sub__ = sub def truediv(space, w_a, w_b): 'truediv(a, b) -- Same as a / b when __future__.division is in effect.' return space.truediv(w_a, w_b) -__truediv__ = truediv def truth(space, w_a,): 'truth(a) -- Return True if a is true, False otherwise.' @@ -193,4 +161,3 @@ def xor(space, w_a, w_b): 'xor(a, b) -- Same as a ^ b.' return space.xor(w_a, w_b) -__xor__ = xor Added: pypy/dist/pypy/module/operator/test/test_operator.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/operator/test/test_operator.py Fri Jul 13 16:33:43 2007 @@ -0,0 +1,5 @@ + +class AppTestOperator: + def test_equality(self): + import operator + assert operator.eq == operator.__eq__ From arigo at codespeak.net Fri Jul 13 16:48:22 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 13 Jul 2007 16:48:22 +0200 (CEST) Subject: [pypy-svn] r45028 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070713144822.E90C881E6@code0.codespeak.net> Author: arigo Date: Fri Jul 13 16:48:22 2007 New Revision: 45028 Modified: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py Log: (lac, arigo) Some progress on compactlltype, which is again going to be changed anyway :-/ Modified: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/compactlltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/compactlltype.py Fri Jul 13 16:48:22 2007 @@ -37,15 +37,13 @@ _allocated.append(storage) return _ctypes_struct(S, storage) else: - XXX if n is None: raise TypeError("%r is variable-sized" % (S,)) - smallercls = build_ctypes_struct(S, n) - smallstruct = smallercls() - getattr(smallstruct, S._arrayfld).length = n - structptr = ctypes.cast(ctypes.pointer(smallstruct), - ctypes.POINTER(cls)) - return structptr + biggercls = build_ctypes_struct(S, n) + bigstruct = biggercls() + _allocated.append(bigstruct) + getattr(bigstruct, S._arrayfld).length = n + return _ctypes_struct(S, bigstruct) malloc = classmethod(malloc) CStruct.__name__ = 'ctypes_%s' % (S,) @@ -130,11 +128,24 @@ raise RuntimeError("lltype.free() on a pointer that was not " "obtained by lltype.malloc()") + def _convert_to_ctypes_pointer(self): + if self._TYPE._is_varsize(): + PtrType = ctypes.POINTER(get_ctypes_type(self._TYPE)) + return ctypes.cast(ctypes.pointer(self._ctypes_storage), PtrType) + else: + return ctypes.pointer(self._ctypes_storage) + class _ctypes_struct(_ctypes_parentable): _kind = "structure" __slots__ = () + def __init__(self, TYPE, ctypes_storage): + _ctypes_parentable.__init__(self, TYPE, ctypes_storage) + if TYPE._arrayfld is not None: + array = getattr(ctypes_storage, TYPE._arrayfld) + assert array.length == len(array.items) + def __repr__(self): return '' % ( self._TYPE._name, @@ -156,9 +167,6 @@ return ctypes2lltype(getattr(self._ctypes_storage, field_name), getattr(self._TYPE, field_name)) - def _convert_to_ctypes_pointer(self): - return ctypes.pointer(self._ctypes_storage) - class _ctypes_array(_ctypes_parentable): _kind = "array" __slots__ = () @@ -183,10 +191,6 @@ return ctypes2lltype(self._ctypes_storage.items[index], self._TYPE.OF) - def _convert_to_ctypes_pointer(self): - PtrType = ctypes.POINTER(get_ctypes_type(self._TYPE)) - return ctypes.cast(ctypes.pointer(self._ctypes_storage), PtrType) - # ____________________________________________________________ def malloc(T, n=None, flavor='gc', immortal=False): Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py Fri Jul 13 16:48:22 2007 @@ -120,14 +120,12 @@ iwfreelist(l) def test_varsizestruct(): - S1 = GcStruct("s1", ('a', Signed), ('rest', Array(('v', Signed)))) - py.test.raises(TypeError, "malloc(S1)") - s1 = malloc(S1, 4) + S1 = Struct("s1", ('a', Signed), ('rest', Array(('v', Signed)))) + py.test.raises(TypeError, "malloc(S1, flavor='raw')") + s1 = malloc(S1, 4, flavor='raw') s1.a = 0 assert s1.a == 0 - assert isweak(s1.rest, S1.rest) assert len(s1.rest) == 4 - assert isweak(s1.rest[0], S1.rest.OF) s1.rest[0].v = 0 assert typeOf(s1.rest[0].v) == Signed assert s1.rest[0].v == 0 @@ -138,35 +136,31 @@ s1.rest[3].v = 5 assert s1.a == 17 assert s1.rest[3].v == 5 - - py.test.raises(TypeError, "Struct('invalid', ('rest', Array(('v', Signed))), ('a', Signed))") - py.test.raises(TypeError, "Struct('invalid', ('rest', GcArray(('v', Signed))), ('a', Signed))") - py.test.raises(TypeError, "Struct('invalid', ('x', Struct('s1', ('a', Signed), ('rest', Array(('v', Signed))))))") - py.test.raises(TypeError, "Struct('invalid', ('x', S1))") + free(s1, flavor='raw') def test_substructure_ptr(): + objcount = getobjcount() S3 = Struct("s3", ('a', Signed)) S2 = Struct("s2", ('s3', S3)) - S1 = GcStruct("s1", ('sub1', S2), ('sub2', S2)) - p1 = malloc(S1) - assert isweak(p1.sub1, S2) - assert isweak(p1.sub2, S2) - assert isweak(p1.sub1.s3, S3) - p2 = p1.sub1 - assert isweak(p2.s3, S3) - -def test_gc_substructure_ptr(): - S1 = GcStruct("s2", ('a', Signed)) - S2 = Struct("s3", ('a', Signed)) - S0 = GcStruct("s1", ('sub1', S1), ('sub2', S2)) - p1 = malloc(S0) - assert typeOf(p1.sub1) == Ptr(S1) - assert isweak(p1.sub2, S2) + S1 = Struct("s1", ('sub1', S2), ('sub2', S2)) + p1 = malloc(S1, flavor='raw') + p1.sub1.s3.a = 123 + p1.sub2.s3.a = 456 + assert p1.sub1.s3.a == 123 + assert p1.sub2.s3.a == 456 + p2 = p1.sub2 + py.test.raises(RuntimeError, "free(p2, flavor='raw')") + free(p1, flavor='raw') + py.test.raises(RuntimeError, "free(p1, flavor='raw')") + + p2 = malloc(S1, flavor='raw') + free(p2.sub1, flavor='raw') + assert objcount == getobjcount() # check for leaks def test_cast_simple_widening(): S2 = Struct("s2", ('a', Signed)) S1 = Struct("s1", ('sub1', S2), ('sub2', S2)) - p1 = malloc(S1, immortal=True) + p1 = malloc(S1, flavor='raw') p2 = p1.sub1 p3 = p2 assert typeOf(p3) == Ptr(S2) From arigo at codespeak.net Fri Jul 13 17:09:26 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 13 Jul 2007 17:09:26 +0200 (CEST) Subject: [pypy-svn] r45029 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070713150926.994C5821E@code0.codespeak.net> Author: arigo Date: Fri Jul 13 17:09:25 2007 New Revision: 45029 Added: pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py (contents, props changed) pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py (contents, props changed) Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py Log: (lac, arigo, pedronis around) Another approach: can convert _struct in-place to internally use a ctypes.Structure. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Fri Jul 13 17:09:25 2007 @@ -1191,16 +1191,19 @@ '_parent_type', '_parent_index', '_keepparent', '_wrparent', '__weakref__', - '_dead') + '_dead', + '_ctypes_storage') def __init__(self, TYPE): self._wrparent = None self._TYPE = TYPE self._dead = False + self._ctypes_storage = None def _free(self): self._check() # no double-frees self._dead = True + self._ctypes_storage = None def _setparentstructure(self, parent, parentindex): self._wrparent = weakref.ref(parent) @@ -1317,7 +1320,18 @@ if isinstance(r, _uninitialized) and not uninitialized_ok: raise UninitializedMemoryAccess("%r.%s"%(self, field_name)) return r - + + def __getattr__(self, field_name): + if self._ctypes_storage is not None: + return self._ctypes_storage._getattr(field_name) + raise AttributeError(field_name) + + def __setattr__(self, field_name, value): + if field_name.startswith('_') or self._ctypes_storage is None: + _parentable.__setattr__(self, field_name, value) + else: + self._ctypes_storage._setattr(field_name, value) + # for FixedSizeArray kind of structs: def getlength(self): Added: pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py Fri Jul 13 17:09:25 2007 @@ -0,0 +1,108 @@ +import ctypes +from pypy.rpython.lltypesystem import lltype + + +_ctypes_cache = { + lltype.Signed: ctypes.c_long, + } + +def build_ctypes_struct(S, max_n=None): + fields = [] + for fieldname in S._names: + FIELDTYPE = S._flds[fieldname] + if max_n is not None and fieldname == S._arrayfld: + cls = build_ctypes_array(FIELDTYPE, max_n) + else: + cls = get_ctypes_type(FIELDTYPE) + fields.append((fieldname, cls)) + + class CStruct(ctypes.Structure): + _fields_ = fields + + def _malloc(cls, n=None): + if S._arrayfld is None: + if n is not None: + raise TypeError("%r is not variable-sized" % (S,)) + storage = cls() + return storage + else: + if n is None: + raise TypeError("%r is variable-sized" % (S,)) + biggercls = build_ctypes_struct(S, n) + bigstruct = biggercls() + getattr(bigstruct, S._arrayfld).length = n + return bigstruct + _malloc = classmethod(_malloc) + + def _getattr(self, field_name): + cobj = getattr(self, field_name) + return ctypes2lltype(cobj) + + def _setattr(self, field_name, value): + cobj = lltype2ctypes(value) + setattr(self, field_name, cobj) + + CStruct.__name__ = 'ctypes_%s' % (S,) + return CStruct + +def build_ctypes_array(A, max_n=0): + assert max_n >= 0 + ITEM = A.OF + ctypes_item = get_ctypes_type(ITEM) + + class CArray(ctypes.Structure): + _fields_ = [('length', ctypes.c_int), + ('items', max_n * ctypes_item)] + + def _malloc(cls, n=None): + if not isinstance(n, int): + raise TypeError, "array length must be an int" + biggercls = build_ctypes_array(A, n) + bigarray = biggercls() + bigarray.length = n + return bigarray + _malloc = classmethod(_malloc) + + CArray.__name__ = 'ctypes_%s*%d' % (A, max_n) + return CArray + +def get_ctypes_type(T): + try: + return _ctypes_cache[T] + except KeyError: + if isinstance(T, lltype.Ptr): + cls = ctypes.POINTER(get_ctypes_type(T.TO)) + elif isinstance(T, lltype.Struct): + cls = build_ctypes_struct(T) + elif isinstance(T, lltype.Array): + cls = build_ctypes_array(T) + else: + raise NotImplementedError(T) + _ctypes_cache[T] = cls + return cls + + +def convert_struct(container): + STRUCT = container._TYPE + cls = get_ctypes_type(STRUCT) + cstruct = cls._malloc() + container._ctypes_storage = cstruct + for field_name in STRUCT._names: + field_value = getattr(container, field_name) + delattr(container, field_name) + setattr(cstruct, field_name, lltype2ctypes(field_value)) + +def lltype2ctypes(llobj): + T = lltype.typeOf(llobj) + if isinstance(T, lltype.Ptr): + container = llobj._obj + if container._ctypes_storage is None: + if isinstance(T.TO, lltype.Struct): + convert_struct(container) + else: + raise NotImplementedError(T) + return ctypes.pointer(container._ctypes_storage) + return llobj + +def ctypes2lltype(cobj): + return cobj Added: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py Fri Jul 13 17:09:25 2007 @@ -0,0 +1,21 @@ +import ctypes +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem.lltype2ctypes import lltype2ctypes + + +def test_primitive(): + assert lltype2ctypes(5) == 5 + assert lltype2ctypes('?') == '?' + +def test_simple_struct(): + S = lltype.Struct('S', ('x', lltype.Signed)) + s = lltype.malloc(S, flavor='raw') + s.x = 123 + sc = lltype2ctypes(s) + assert isinstance(sc.contents, ctypes.Structure) + assert sc.contents.x == 123 + sc.contents.x = 456 + assert s.x == 456 + s.x = 789 + assert sc.contents.x == 789 + lltype.free(s, flavor='raw') From pedronis at codespeak.net Fri Jul 13 17:16:19 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 13 Jul 2007 17:16:19 +0200 (CEST) Subject: [pypy-svn] r45030 - pypy/dist/lib-python/2.4.1/test Message-ID: <20070713151619.B1839821E@code0.codespeak.net> Author: pedronis Date: Fri Jul 13 17:16:19 2007 New Revision: 45030 Added: pypy/dist/lib-python/2.4.1/test/test_threading.py - copied unchanged from r39456, pypy/dist/lib-python/2.4.1/test/test_threading.py Log: revert From arigo at codespeak.net Fri Jul 13 17:20:59 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 13 Jul 2007 17:20:59 +0200 (CEST) Subject: [pypy-svn] r45031 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070713152059.1F24C81FD@code0.codespeak.net> Author: arigo Date: Fri Jul 13 17:20:58 2007 New Revision: 45031 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py Log: (lac, arigo) Support for arrays. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Fri Jul 13 17:20:58 2007 @@ -1411,6 +1411,8 @@ return 0, stop def getitem(self, index, uninitialized_ok=False): + if self._ctypes_storage is not None: + return self._ctypes_storage._getitem(index) try: v = self.items[index] if isinstance(v, _uninitialized) and not uninitialized_ok: @@ -1425,6 +1427,9 @@ raise def setitem(self, index, value): + if self._ctypes_storage is not None: + self._ctypes_storage._setitem(index, value) + return try: self.items[index] = value except IndexError: Modified: pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py Fri Jul 13 17:20:58 2007 @@ -63,6 +63,14 @@ return bigarray _malloc = classmethod(_malloc) + def _getitem(self, index): + cobj = self.items[index] + return ctypes2lltype(cobj) + + def _setitem(self, index, value): + cobj = lltype2ctypes(value) + self.items[index] = cobj + CArray.__name__ = 'ctypes_%s*%d' % (A, max_n) return CArray @@ -90,7 +98,19 @@ for field_name in STRUCT._names: field_value = getattr(container, field_name) delattr(container, field_name) - setattr(cstruct, field_name, lltype2ctypes(field_value)) + if not isinstance(field_value, lltype._uninitialized): + setattr(cstruct, field_name, lltype2ctypes(field_value)) + +def convert_array(container): + ARRAY = container._TYPE + cls = get_ctypes_type(ARRAY) + carray = cls._malloc(container.getlength()) + container._ctypes_storage = carray + for i in range(container.getlength()): + item_value = container.items[i] # fish fish + container.items[i] = None + if not isinstance(item_value, lltype._uninitialized): + carray.items[i] = lltype2ctypes(item_value) def lltype2ctypes(llobj): T = lltype.typeOf(llobj) @@ -99,6 +119,8 @@ if container._ctypes_storage is None: if isinstance(T.TO, lltype.Struct): convert_struct(container) + elif isinstance(T.TO, lltype.Array): + convert_array(container) else: raise NotImplementedError(T) return ctypes.pointer(container._ctypes_storage) Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py Fri Jul 13 17:20:58 2007 @@ -8,7 +8,7 @@ assert lltype2ctypes('?') == '?' def test_simple_struct(): - S = lltype.Struct('S', ('x', lltype.Signed)) + S = lltype.Struct('S', ('x', lltype.Signed), ('y', lltype.Signed)) s = lltype.malloc(S, flavor='raw') s.x = 123 sc = lltype2ctypes(s) @@ -18,4 +18,22 @@ assert s.x == 456 s.x = 789 assert sc.contents.x == 789 + s.y = 52 + assert sc.contents.y == 52 lltype.free(s, flavor='raw') + +def test_simple_array(): + A = lltype.Array(lltype.Signed) + a = lltype.malloc(A, 10, flavor='raw') + a[0] = 100 + a[1] = 101 + a[2] = 102 + ac = lltype2ctypes(a) + assert isinstance(ac.contents, ctypes.Structure) + assert ac.contents.length == 10 + assert ac.contents.items[1] == 101 + ac.contents.items[2] = 456 + assert a[2] == 456 + a[3] = 789 + assert ac.contents.items[3] == 789 + lltype.free(a, flavor='raw') From justas at codespeak.net Fri Jul 13 17:24:19 2007 From: justas at codespeak.net (justas at codespeak.net) Date: Fri, 13 Jul 2007 17:24:19 +0200 (CEST) Subject: [pypy-svn] r45032 - pypy/dist/pypy/interpreter/pyparser/test Message-ID: <20070713152419.0AF7D821E@code0.codespeak.net> Author: justas Date: Fri Jul 13 17:24:19 2007 New Revision: 45032 Modified: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Log: Fixed syntax incompatible with python2.4 Modified: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Fri Jul 13 17:24:19 2007 @@ -5,7 +5,7 @@ from fakes import FakeSpace -class ParserStub(): +class ParserStub: def __init__(self): self.tokens = {} self._sym_count = 0 From tismer at codespeak.net Fri Jul 13 17:41:34 2007 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 13 Jul 2007 17:41:34 +0200 (CEST) Subject: [pypy-svn] r45033 - pypy/dist/pypy/lib/test2 Message-ID: <20070713154134.2DA0A825D@code0.codespeak.net> Author: tismer Date: Fri Jul 13 17:41:33 2007 New Revision: 45033 Added: pypy/dist/pypy/lib/test2/test_stackless.py Log: stackless pickling is broken. This test breaks because we are pickling main via the parent. lib/stackless does not yet use usercostate. Also, pickling parent is generally a bad idea. I guess the parent must go away Added: pypy/dist/pypy/lib/test2/test_stackless.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/test2/test_stackless.py Fri Jul 13 17:41:33 2007 @@ -0,0 +1,31 @@ +from pypy.conftest import gettestobjspace + +class AppTest_Stackless: + + def setup_class(cls): + space = gettestobjspace(usemodules=('_stackless',)) + cls.space = space + + def test_pickle(self): + import pickle, sys + import stackless + + ch = stackless.channel() + + def recurs(depth, level=1): + print 'enter level %s%d' % (level*' ', level) + if level >= depth: + ch.send('hi') + if level < depth: + recurs(depth, level+1) + print 'leave level %s%d' % (level*' ', level) + + def demo(depth): + t = stackless.tasklet(recurs)(depth) + print ch.receive() + blob = pickle.dumps(t) + + t = stackless.tasklet(demo)(14) + stackless.run() + +# remark: think of fixing cells etc. on the sprint From justas at codespeak.net Fri Jul 13 17:59:35 2007 From: justas at codespeak.net (justas at codespeak.net) Date: Fri, 13 Jul 2007 17:59:35 +0200 (CEST) Subject: [pypy-svn] r45034 - in pypy/dist/pypy/interpreter/pyparser: . test Message-ID: <20070713155935.4D8C381DE@code0.codespeak.net> Author: justas Date: Fri Jul 13 17:59:33 2007 New Revision: 45034 Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py pypy/dist/pypy/interpreter/pyparser/asthelper.py pypy/dist/pypy/interpreter/pyparser/pythonparse.py pypy/dist/pypy/interpreter/pyparser/test/fakes.py pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Log: - Expanded test_parser.py import tests to validate the right atoms - Implemented most of the build_future_import_feature - Ate more trailing whitespace Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/dist/pypy/interpreter/pyparser/astbuilder.py Fri Jul 13 17:59:33 2007 @@ -886,6 +886,20 @@ Enables python language future imports. Called once per feature imported, no matter how you got to this one particular feature. """ + atoms = peek_atoms(builder, nb) + feature_name = atoms[0].value + space = builder.space + w_feature_code = space.appexec([space.wrap(feature_name)], + """(feature): + import __future__ as f + feature = getattr(f, feature, None) + return feature and feature.compiler_flag or 0 + """) + + # We will call a method on the parser (the method exists only in unit + # tests). + builder.parser.add_production(space.unwrap(w_feature_code)) + def build_yield_stmt(builder, nb): atoms = get_atoms(builder, nb) @@ -1062,7 +1076,7 @@ 'eval_input' : build_eval_input, 'with_stmt' : build_with_stmt, } - + class AstBuilderContext(AbstractContext): """specific context management for AstBuidler""" @@ -1088,7 +1102,7 @@ self.with_enabled = True # XXX # self.keywords.update({'with':None, 'as': None}) - + def context(self): return AstBuilderContext(self.rule_stack) Modified: pypy/dist/pypy/interpreter/pyparser/asthelper.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/asthelper.py (original) +++ pypy/dist/pypy/interpreter/pyparser/asthelper.py Fri Jul 13 17:59:33 2007 @@ -417,6 +417,26 @@ atoms.reverse() return atoms + +def peek_atoms(builder, nb): + atoms = [] + + i = nb + current = len(builder.rule_stack) - 1 + while i > 0: + assert current >= 0 + obj = builder.rule_stack[current] + if isinstance(obj, BaseRuleObject): + i += obj.count + else: + atoms.append( obj ) + i -= 1 + current -= 1 + + atoms.reverse() + return atoms + + #def eval_string(value): # """temporary implementation # Modified: pypy/dist/pypy/interpreter/pyparser/pythonparse.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/pythonparse.py (original) +++ pypy/dist/pypy/interpreter/pyparser/pythonparse.py Fri Jul 13 17:59:33 2007 @@ -90,7 +90,7 @@ 'single' : "single_input", 'exec' : "file_input", } - + def __init__(self): # , predefined_symbols=None): grammar.Parser.__init__(self) pytoken.setup_tokens(self) @@ -102,7 +102,7 @@ def is_base_token(self, tokvalue): return tokvalue < 0 or tokvalue >= self._basetokens_count - + def parse_source(self, textsrc, mode, builder, flags=0): """Parse a python source according to goal""" goal = self.targets[mode] @@ -149,7 +149,7 @@ if real_rule is not rule.args[i]: rule.args[i] = real_rule - + def insert_rule(self, ruledef): """parses and inserts corresponding rules in the parser""" # parse the ruledef(s) Modified: pypy/dist/pypy/interpreter/pyparser/test/fakes.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/fakes.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/fakes.py Fri Jul 13 17:59:33 2007 @@ -5,7 +5,10 @@ w_basestring = basestring w_int = int - def wrap(self,obj): + def wrap(self, obj): + return obj + + def unwrap(self, obj): return obj def isinstance(self, obj, wtype ): Modified: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Fri Jul 13 17:59:33 2007 @@ -1,6 +1,8 @@ +from pypy.interpreter.astcompiler import consts from pypy.interpreter.pyparser.grammar import Parser from pypy.interpreter.pyparser.pytoken import setup_tokens from pypy.interpreter.pyparser import astbuilder +from pypy.interpreter.pyparser.asthelper import TokenObject from fakes import FakeSpace @@ -11,6 +13,7 @@ self._sym_count = 0 self.tok_values = {} self.tok_rvalues = {} + self.trace = [] def add_token( self, tok, value = None ): # assert isinstance( tok, str ) @@ -25,6 +28,8 @@ return val return self.tokens[ tok ] + def add_production(self, rule): + self.trace.append(rule) class RuleStub: def __init__(self, name, root=False): @@ -33,13 +38,26 @@ is_root = lambda self: self.root +class TokenForTest(TokenObject): + def __init__(self, value, parser): + TokenObject.__init__(self, 'dummy', value, -1, parser) + + +class FakeSpaceForFeatureLookup(FakeSpace): + feature_code_lookup = {'with_statement': consts.CO_FUTURE_WITH_STATEMENT} + def appexec(self, posargs_w, code): + feature_name = posargs_w[0] + return self.feature_code_lookup.get(feature_name, 0) + + class TestBuilderFuture: def setup_class(self): self.parser = ParserStub() setup_tokens(self.parser) def setup_method(self, method): - self.builder = astbuilder.AstBuilder(self.parser, space=FakeSpace()) + self.builder = astbuilder.AstBuilder( + self.parser, space=FakeSpaceForFeatureLookup()) def test_future_rules(self): assert (self.builder.build_rules['future_import_feature'] is @@ -48,5 +66,12 @@ astbuilder.build_import_from) def test_future_import(self): - #self.builder.push(RuleStub('future_import_feature', root=True)) - pass + token_values = ['with_statement', 'as', 'stuff'] + for val in token_values: + self.builder.push(TokenForTest(val, self.parser)) + + astbuilder.build_future_import_feature(self.builder, len(token_values)) + assert ([rule.value for rule in self.builder.rule_stack] == + token_values) + assert self.parser.trace == [32768] + Modified: pypy/dist/pypy/interpreter/pyparser/test/test_parser.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_parser.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_parser.py Fri Jul 13 17:59:33 2007 @@ -1,3 +1,4 @@ +from pypy.interpreter.pyparser.asthelper import get_atoms from pypy.interpreter.pyparser.grammar import Parser from pypy.interpreter.pyparser import error from fakes import FakeSpace @@ -45,17 +46,18 @@ assert v == 9 -from pypy.interpreter.pyparser.asthelper import get_atoms class RuleTracer(dict): def __init__(self, *args, **kw): self.trace = [] + self.exclude_rules = [ + 'dotted_name', 'dotted_as_name', 'dotted_as_names', + 'import_stmt', 'small_stmt', 'simple_stmt', 'stmt', + 'single_input', 'file_input', 'future_import_list', + 'import_from_future', 'future_import_as_names'] def __getitem__(self, attr): - if attr in ['dotted_name', 'dotted_as_name', 'dotted_as_names', - 'import_stmt', 'small_stmt', 'simple_stmt', 'stmt', - 'single_input', 'file_input', 'future_import_list', - 'import_from_future', 'future_import_as_names']: + if attr in self.exclude_rules: return None def record_trace(builder, number): @@ -140,3 +142,19 @@ expected = [('import_name', ['import', 'sys'])] self.check_parse(tst, expected) + def test_future_import_atoms(self): + self.builder.build_rules.exclude_rules.remove('import_from_future') + self.builder.build_rules.exclude_rules.append('future_import_feature') + tst = 'from __future__ import na as x\n' + expected = [('import_from_future', + ['from', '__future__', 'import', 'na', 'as', 'x'])] + self.check_parse(tst, expected) + + def test_regular_from(self): + self.builder.build_rules.exclude_rules.extend([ + 'import_as_name', 'import_as_names']) + + tst = 'from stuff import na as x\n' + expected = [('import_from', + ['from', 'stuff', 'import', 'na', 'as', 'x'])] + self.check_parse(tst, expected) From jlg at codespeak.net Fri Jul 13 18:12:00 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Fri, 13 Jul 2007 18:12:00 +0200 (CEST) Subject: [pypy-svn] r45036 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070713161200.A2FAF826E@code0.codespeak.net> Author: jlg Date: Fri Jul 13 18:12:00 2007 New Revision: 45036 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: WrongArgType exception; number, float, fixnum refactoring -> r5rs; number predicates Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Fri Jul 13 18:12:00 2007 @@ -11,6 +11,10 @@ def __str__(self): return "Wrong number of args" +class WrongArgType(SchemeException): + def __str__(self): + return "Wrong argument type" + class SchemeQuit(SchemeException): """raised on (quit) evaluation""" pass @@ -86,43 +90,56 @@ def __repr__(self): return "" -class W_Fixnum(W_Root): +class W_Number(W_Root): + pass + +class W_Float(W_Number): def __init__(self, val): - self.fixnumval = val + self.exact = False + self.floatval = val def to_string(self): - return str(self.fixnumval) + return str(self.floatval) def to_number(self): - return self.to_fixnum() + return self.to_float() def to_fixnum(self): - return self.fixnumval + return int(self.floatval) def to_float(self): - return float(self.fixnumval) + return self.floatval + + def round(self): + int_part = int(self.floatval) + if self.floatval > 0: + if self.floatval >= (int_part + 0.5): + return int_part + 1 + + return int_part - def equal(self, w_obj): - return self.fixnumval == w_obj.to_number() + else: + if self.floatval <= (int_part - 0.5): + return int_part - 1 + + return int_part -class W_Float(W_Root): +class W_Fixnum(W_Float): def __init__(self, val): - self.floatval = val + self.fixnumval = val + self.exact = True def to_string(self): - return str(self.floatval) + return str(self.fixnumval) def to_number(self): - return self.to_float() + return self.to_fixnum() def to_fixnum(self): - return int(self.floatval) + return self.fixnumval def to_float(self): - return self.floatval - - def equal(self, w_obj): - return self.floatval == w_obj.to_number() + return float(self.fixnumval) class W_Pair(W_Root): def __init__(self, car, cdr): @@ -260,16 +277,16 @@ return acc def unary_oper(self, x): - if isinstance(x, W_Float): - return W_Float(self.do_unary_oper(x.to_float())) - else: + if isinstance(x, W_Fixnum): return W_Fixnum(self.do_unary_oper(x.to_fixnum())) + else: + return W_Float(self.do_unary_oper(x.to_float())) def oper(self, x, y): - if isinstance(x, W_Float) or isinstance(y, W_Float): - return W_Float(self.do_oper(x.to_float(), y.to_float())) - else: + if isinstance(x, W_Fixnum) and isinstance(y, W_Fixnum): return W_Fixnum(self.do_oper(x.to_fixnum(), y.to_fixnum())) + else: + return W_Float(self.do_oper(x.to_float(), y.to_float())) def create_op_class(oper, unary_oper, title, default_result=None): class Op(ListOper): @@ -317,6 +334,7 @@ prev = lst[0] for arg in lst[1:]: + assert isinstance(arg, W_Number) if prev.to_number() != arg.to_number(): return W_Boolean(False) prev = arg @@ -327,6 +345,45 @@ def procedure(self, ctx, lst): return plst2lst(lst) +## +# Predicate +## +class PredicateNumber(W_Procedure): + def procedure(self, ctx, lst): + if len(lst) != 1: + raise WrongArgsNumber + + if not isinstance(lst[0], W_Number): + raise WrongArgType + + return W_Boolean(self.predicate(lst[0])) + +class IntegerP(PredicateNumber): + def predicate(self, w_obj): + if not w_obj.exact: + return w_obj.to_float() == w_obj.round() + + return True + +class RealP(PredicateNumber): + def predicate(self, w_obj): + return isinstance(w_obj, W_Float) + +class NumberP(PredicateNumber): + def predicate(self, w_obj): + return isinstance(w_obj, W_Number) + +class ExactP(PredicateNumber): + def predicate(self, w_obj): + return w_obj.exact + +class InexactP(PredicateNumber): + def predicate(self, w_obj): + return not w_obj.exact + +## +# Macro +## class Define(W_Macro): def call(self, ctx, lst): w_identifier = lst.car @@ -456,6 +513,14 @@ 'quit': Quit, #comparisons '=': Equal, + #predicates + 'integer?': IntegerP, + 'rational?': RealP, + 'real?': RealP, + 'complex?': NumberP, + 'number?': NumberP, + 'exact?': ExactP, + 'inexact?': InexactP, #macros 'define': Define, 'set!': Sete, Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Fri Jul 13 18:12:00 2007 @@ -377,3 +377,29 @@ def test_quit(): py.test.raises(SchemeQuit, eval_noctx, "(quit)") +def test_numbers(): + assert eval_noctx("(integer? 42)").to_boolean() + assert eval_noctx("(integer? 42.0)").to_boolean() + assert not eval_noctx("(integer? 42.1)").to_boolean() + + assert eval_noctx("(rational? 42)").to_boolean() + assert eval_noctx("(rational? 42.1)").to_boolean() + + assert eval_noctx("(real? 42)").to_boolean() + assert eval_noctx("(real? 42.1)").to_boolean() + + assert eval_noctx("(complex? 42)").to_boolean() + assert eval_noctx("(complex? 42.1)").to_boolean() + + assert eval_noctx("(number? 42)").to_boolean() + assert eval_noctx("(number? 42.1)").to_boolean() + + py.test.raises(WrongArgType, eval_noctx, "(number? 'a)" ) + +def test_exactness(): + assert eval_noctx("(exact? 42)").to_boolean() + assert not eval_noctx("(exact? 42.0)").to_boolean() + + assert not eval_noctx("(inexact? 42)").to_boolean() + assert eval_noctx("(inexact? 42.0)").to_boolean() + From jlg at codespeak.net Fri Jul 13 18:19:49 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Fri, 13 Jul 2007 18:19:49 +0200 (CEST) Subject: [pypy-svn] r45037 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070713161949.99A588278@code0.codespeak.net> Author: jlg Date: Fri Jul 13 18:19:49 2007 New Revision: 45037 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_eval.py pypy/dist/pypy/lang/scheme/test/test_object.py pypy/dist/pypy/lang/scheme/test/test_parser.py Log: s/W_Float/W_Real/; s/W_Fixnum/W_Integer -> r5rs Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Fri Jul 13 18:19:49 2007 @@ -93,7 +93,7 @@ class W_Number(W_Root): pass -class W_Float(W_Number): +class W_Real(W_Number): def __init__(self, val): self.exact = False self.floatval = val @@ -124,7 +124,7 @@ return int_part -class W_Fixnum(W_Float): +class W_Integer(W_Real): def __init__(self, val): self.fixnumval = val self.exact = True @@ -277,16 +277,16 @@ return acc def unary_oper(self, x): - if isinstance(x, W_Fixnum): - return W_Fixnum(self.do_unary_oper(x.to_fixnum())) + if isinstance(x, W_Integer): + return W_Integer(self.do_unary_oper(x.to_fixnum())) else: - return W_Float(self.do_unary_oper(x.to_float())) + return W_Real(self.do_unary_oper(x.to_float())) def oper(self, x, y): - if isinstance(x, W_Fixnum) and isinstance(y, W_Fixnum): - return W_Fixnum(self.do_oper(x.to_fixnum(), y.to_fixnum())) + if isinstance(x, W_Integer) and isinstance(y, W_Integer): + return W_Integer(self.do_oper(x.to_fixnum(), y.to_fixnum())) else: - return W_Float(self.do_oper(x.to_float(), y.to_float())) + return W_Real(self.do_oper(x.to_float(), y.to_float())) def create_op_class(oper, unary_oper, title, default_result=None): class Op(ListOper): @@ -317,7 +317,7 @@ if default_result is None: Op.default_result = None else: - Op.default_result = W_Fixnum(default_result) + Op.default_result = W_Integer(default_result) Op.__name__ = "Op" + title return Op @@ -367,7 +367,7 @@ class RealP(PredicateNumber): def predicate(self, w_obj): - return isinstance(w_obj, W_Float) + return isinstance(w_obj, W_Real) class NumberP(PredicateNumber): def predicate(self, w_obj): Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Fri Jul 13 18:19:49 2007 @@ -1,8 +1,8 @@ import autopath from pypy.rlib.parsing.pypackrat import PackratParser from pypy.rlib.parsing.makepackrat import BacktrackException, Status -from pypy.lang.scheme.object import W_Pair, W_Fixnum, W_String, W_Identifier, \ - W_Nil, W_Boolean, W_Float, Literal +from pypy.lang.scheme.object import W_Pair, W_Integer, W_String, W_Identifier, \ + W_Nil, W_Boolean, W_Real, Literal def unquote(s): str_lst = [] @@ -32,12 +32,12 @@ FIXNUM: c = `\-?(0|([1-9][0-9]*))` IGNORE* - return {W_Fixnum(int(c))}; + return {W_Integer(int(c))}; FLOAT: c = `\-?[0-9]*\.[0-9]*` IGNORE* - return {W_Float(float(c))}; + return {W_Real(float(c))}; BOOLEAN: c = `#(t|f)` Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Fri Jul 13 18:19:49 2007 @@ -4,7 +4,7 @@ def test_eval_obj(): w_num = W_Pair(W_Identifier("+"), - W_Pair(W_Fixnum(4), W_Pair(W_Fixnum(5), W_Nil()))) + W_Pair(W_Integer(4), W_Pair(W_Integer(5), W_Nil()))) assert w_num.eval(None).to_number() == 9 def eval_expr(ctx, expr): @@ -54,13 +54,13 @@ def test_ctx_simple(): ctx = ExecutionContext() - ctx.put("v1", W_Fixnum(4)) - ctx.put("v2", W_Fixnum(5)) + ctx.put("v1", W_Integer(4)) + ctx.put("v2", W_Integer(5)) w_num = eval_expr(ctx, "(+ 1 v1 v2)") assert w_num.to_number() == 10 - ctx.put("v2", W_Float(3.2)) + ctx.put("v2", W_Real(3.2)) w_num = eval_expr(ctx, "(+ 1 v1 v2)") assert w_num.to_number() == 8.2 @@ -199,7 +199,7 @@ ctx.put("f1", w_lambda) w_result = eval_expr(ctx, "(f1)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 12 def test_lambda_args(): @@ -207,11 +207,11 @@ eval_expr(ctx, "(define f1 (lambda (n) n))") w_result = eval_expr(ctx, "(f1 42)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 42 w_result = eval_expr(ctx, "((lambda (n m) (+ n m)) 42 -42)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 0 def test_lambda_top_ctx(): @@ -219,12 +219,12 @@ eval_expr(ctx, "(define n 42)") eval_expr(ctx, "(define f1 (lambda (m) (+ n m)))") w_result = eval_expr(ctx, "(f1 -42)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 0 eval_expr(ctx, "(define n 84)") w_result = eval_expr(ctx, "(f1 -42)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 42 def test_lambda_fac(): @@ -250,11 +250,11 @@ eval_expr(ctx, """(define add6 (adder 6))""") w_result = eval_expr(ctx, "(add6 5)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 11 w_result = eval_expr(ctx, "((adder 6) 5)") - assert isinstance(w_result, W_Fixnum) + assert isinstance(w_result, W_Integer) assert w_result.to_number() == 11 def test_lambda_long_body(): @@ -285,7 +285,7 @@ def test_quote(): w_fnum = eval_noctx("(quote 42)") - assert isinstance(w_fnum, W_Fixnum) + assert isinstance(w_fnum, W_Integer) assert w_fnum.to_number() == 42 w_sym = eval_noctx("(quote symbol)") @@ -313,7 +313,7 @@ def test_quote_parse(): w_fnum = eval_noctx("'42") - assert isinstance(w_fnum, W_Fixnum) + assert isinstance(w_fnum, W_Integer) assert w_fnum.to_number() == 42 w_sym = eval_noctx("'symbol") @@ -341,7 +341,7 @@ def test_list(): ctx = ExecutionContext() - ctx.put("var", W_Fixnum(42)) + ctx.put("var", W_Integer(42)) w_lst = eval_expr(ctx, "(list 1 var (+ 2 1) 'a)") assert isinstance(w_lst, W_Pair) assert w_lst.car.to_number() == 1 @@ -352,7 +352,7 @@ def test_let(): ctx = ExecutionContext() - w_global = W_Fixnum(0) + w_global = W_Integer(0) ctx.put("var", w_global) w_result = eval_expr(ctx, "(let ((var 42) (x (+ 2 var))) (+ var x))") assert w_result.to_number() == 44 Modified: pypy/dist/pypy/lang/scheme/test/test_object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_object.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_object.py Fri Jul 13 18:19:49 2007 @@ -17,22 +17,22 @@ def test_fixnum(): num = 12345 - w_num = W_Fixnum(num) + w_num = W_Integer(num) assert num == w_num.to_fixnum() assert float(num) == w_num.to_float() assert w_num.to_boolean() is True def test_float(): num = 12345.567 - w_num = W_Float(num) + w_num = W_Real(num) assert num == w_num.to_float() assert int(num) == w_num.to_fixnum() assert w_num.to_boolean() is True def test_pair(): - c1 = W_Fixnum(1) + c1 = W_Integer(1) c2 = W_String("c2") - c3 = W_Float(0.3) + c3 = W_Real(0.3) c4 = W_Nil() p2 = W_Pair(c3, c4) p1 = W_Pair(c2, p2) @@ -55,7 +55,7 @@ assert w_id.to_boolean() is True def test_ctx(): - w_fnum = W_Fixnum(12) + w_fnum = W_Integer(12) w_symb = W_Symbol("symb") ctx = ExecutionContext({}) @@ -67,15 +67,15 @@ assert ctx.get("no_such_key") is None def test_location(): - w_fnum = W_Fixnum(42) + w_fnum = W_Integer(42) loc = Location(w_fnum) assert isinstance(loc, Location) assert loc.obj is w_fnum def test_ctx_sets(): - w_fnum = W_Fixnum(42) - w_fnum2 = W_Fixnum(43) - w_fnum3 = W_Fixnum(44) + w_fnum = W_Integer(42) + w_fnum2 = W_Integer(43) + w_fnum3 = W_Integer(44) ctx = ExecutionContext({}) ctx.put("v1", w_fnum) Modified: pypy/dist/pypy/lang/scheme/test/test_parser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_parser.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_parser.py Fri Jul 13 18:19:49 2007 @@ -1,6 +1,6 @@ import py from pypy.lang.scheme.ssparser import parse -from pypy.lang.scheme.object import W_Boolean, W_Float, W_Fixnum, W_String +from pypy.lang.scheme.object import W_Boolean, W_Real, W_Integer, W_String from pypy.lang.scheme.object import W_Pair, W_Nil, W_Symbol, W_Identifier def parse_sexpr(expr): @@ -8,9 +8,9 @@ def unwrap(w_obj): """for testing purposes: unwrap a scheme object into a python object""" - if isinstance(w_obj, W_Float): + if isinstance(w_obj, W_Real): return w_obj.to_number() - elif isinstance(w_obj, W_Fixnum): + elif isinstance(w_obj, W_Integer): return w_obj.to_number() elif isinstance(w_obj, W_String): return w_obj.strval @@ -30,14 +30,14 @@ def test_simple(): w_fixnum = parse_sexpr('1') - assert isinstance(w_fixnum, W_Fixnum) + assert isinstance(w_fixnum, W_Integer) assert unwrap(w_fixnum) == 1 w_fixnum = parse_sexpr('0') assert unwrap(w_fixnum) == 0 - assert isinstance(w_fixnum, W_Fixnum) + assert isinstance(w_fixnum, W_Integer) w_fixnum = parse_sexpr('1123') assert unwrap(w_fixnum) == 1123 - assert isinstance(w_fixnum, W_Fixnum) + assert isinstance(w_fixnum, W_Integer) w_fixnum = parse_sexpr('abfa__') assert isinstance(w_fixnum, W_Identifier) t = parse_sexpr(r'''"don't believe \"them\""''') @@ -46,20 +46,20 @@ def test_objects(): w_fixnum = parse_sexpr('-12345') - assert isinstance(w_fixnum, W_Fixnum) + assert isinstance(w_fixnum, W_Integer) assert unwrap(w_fixnum) == -12345 w_float = parse_sexpr('123456.1234') - assert isinstance(w_float, W_Float) + assert isinstance(w_float, W_Real) assert unwrap(w_float) == 123456.1234 w_float = parse_sexpr('-123456.1234') - assert isinstance(w_float, W_Float) + assert isinstance(w_float, W_Real) assert unwrap(w_float) == -123456.1234 def test_sexpr(): w_list = parse_sexpr('( 1 )') assert isinstance(w_list, W_Pair) - assert isinstance(w_list.car, W_Fixnum) + assert isinstance(w_list.car, W_Integer) assert isinstance(w_list.cdr, W_Nil) #w_list = parse_sexpr('()') @@ -69,8 +69,8 @@ assert isinstance(w_list, W_Pair) assert isinstance(w_list.car, W_Identifier) assert isinstance(w_list.cdr, W_Pair) - assert isinstance(w_list.cdr.car, W_Fixnum) - assert isinstance(w_list.cdr.cdr.car, W_Fixnum) + assert isinstance(w_list.cdr.car, W_Integer) + assert isinstance(w_list.cdr.cdr.car, W_Integer) assert isinstance(w_list.cdr.cdr.cdr, W_Nil) def test_complex_sexpr(): From simonb at codespeak.net Fri Jul 13 19:26:41 2007 From: simonb at codespeak.net (simonb at codespeak.net) Date: Fri, 13 Jul 2007 19:26:41 +0200 (CEST) Subject: [pypy-svn] r45038 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070713172641.52FBE823E@code0.codespeak.net> Author: simonb Date: Fri Jul 13 19:26:40 2007 New Revision: 45038 Modified: pypy/dist/pypy/rpython/tool/mkrffi.py pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Log: put structs in with rest of the code (they dont need to be all at the top for any reason), rename some stuff, add more types (manual maping for now) Modified: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Fri Jul 13 19:26:40 2007 @@ -9,48 +9,68 @@ # XXX any automatic stuff here? SIMPLE_TYPE_MAPPING = { - ctypes.c_int : 'rffi.INT', - ctypes.c_uint : 'rffi.UINT', - ctypes.c_voidp : 'rffi.VOIDP', + ctypes.c_ubyte : 'rffi.UCHAR', + ctypes.c_byte : 'rffi.CHAR', + ctypes.c_char : 'rffi.CHAR', + ctypes.c_int8 : 'rffi.CHAR', + ctypes.c_ushort : 'rffi.USHORT', + ctypes.c_short : 'rffi.SHORT', + ctypes.c_uint16 : 'rffi.USHORT', + ctypes.c_int16 : 'rffi.SHORT', + ctypes.c_int : 'rffi.INT', + ctypes.c_uint : 'rffi.UINT', + ctypes.c_int32 : 'rffi.INT', + ctypes.c_uint32 : 'rffi.UINT', + #ctypes.c_long : 'rffi.LONG', # same as c_int.. + #ctypes.c_ulong : 'rffi.ULONG', + ctypes.c_longlong : 'rffi.LONGLONG', + ctypes.c_ulonglong : 'rffi.ULONGLONG', + ctypes.c_int64 : 'rffi.LONGLONG', + ctypes.c_uint64 : 'rffi.ULONGLONG', + ctypes.c_voidp : 'rffi.VOIDP', + None : 'rffi.lltype.Void', # XXX make a type in rffi + ctypes.c_char_p : 'rffi.CCHARP', + ctypes.c_double : 'rffi.lltype.Float', # XXX make a type in rffi } class RffiSource(object): - def __init__(self, structures=None, code=None): - # ctypes structure -> code mapping - if structures is None: - self.structures = {} + def __init__(self, structs=None, source=None): + # set of ctypes structs + if structs is None: + self.structs = set() else: - self.structures = structures - # rest of the code - if code is None: - self.code = py.code.Source() + self.structs = structs + if source is None: + self.source = py.code.Source() else: - self.code = code + self.source = source + + def __str__(self): + return str(self.source) def __add__(self, other): - structs = self.structures.copy() - structs.update(other.structures) - code = py.code.Source(self.code, other.code) - return RffiSource(structs, code) + structs = self.structs.copy() + structs.update(other.structs) + source = py.code.Source(self.source, other.source) + return RffiSource(structs, source) def __iadd__(self, other): - self.structures.update(other.structures) - self.code = py.code.Source(self.code, other.code) + self.structs.update(other.structs) + self.source = py.code.Source(self.source, other.source) return self def proc_struct(self, tp): name = tp.__name__ - real_name = 'c_' + name - real_name = real_name.rstrip('_structure') - real_name += '_structure' - if tp not in self.structures: - fields = ["('%s', %s)" % (name_, self.proc_tp(field_tp)) + if tp not in self.structs: + fields = ["('%s', %s), " % (name_, self.proc_tp(field_tp)) for name_, field_tp in tp._fields_] - fields_repr = ", ".join(fields) - self.structures[tp] = py.code.Source( - "%s = rffi.CStruct('%s', %s)"%(real_name, name, - fields_repr)) - return real_name + fields_repr = ''.join(fields) + self.structs.add(tp) + src = py.code.Source( + "%s = lltype.Struct('%s', %s hints={'external':'C'})"%( + name, name, fields_repr)) + self.source = py.code.Source(self.source, src) + return name def proc_tp(self, tp): try: @@ -67,10 +87,29 @@ raise NotImplementedError("Not implemented mapping for %s" % tp) def proc_func(self, func): + print "proc_func", func name = func.__name__ src = py.code.Source(""" - c_%s = rffi.llexternal('%s', [%s], %s) + %s = rffi.llexternal('%s', [%s], %s) """%(name, name, ", ".join([self.proc_tp(arg) for arg in func.argtypes]), self.proc_tp(func.restype))) - self.code = py.code.Source(self.code, src) + self.source = py.code.Source(self.source, src) + + def proc_namespace(self, ns): + exempt = set(id(value) for value in ctypes.__dict__.values()) + for key, value in ns.items(): + if id(value) in exempt: + continue + if isinstance(value, ctypes._CFuncPtr): + print "func", key, value + try: + self.proc_func(value) + except NotImplementedError: + print "skipped:", value + #print value, value.__class__.__name__ + + + + + Modified: pypy/dist/pypy/rpython/tool/test/test_mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/test/test_mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Fri Jul 13 19:26:40 2007 @@ -10,10 +10,10 @@ def test_rffisource(): res = RffiSource({1:2, 3:4}, "ab") + RffiSource(None, "c") - assert res.structures == {1:2, 3:4} - assert str(res.code) == "ab\nc" + assert res.structs == {1:2, 3:4} + assert str(res.source) == "ab\nc" res += RffiSource({5:6}) - assert 5 in res.structures.keys() + assert 5 in res.structs.keys() def test_proc_tp_simple(): rffi_source = RffiSource() @@ -26,9 +26,9 @@ "lltype.Ptr(lltype.Array(rffi.UINT, hints={'nolength': True}))" rffi_source.proc_tp(random_structure) _src = py.code.Source(""" - c_random_structure = rffi.CStruct('random_structure', ('one', rffi.INT), ('two', lltype.Ptr(lltype.Array(rffi.INT, hints={'nolength': True})))) + random_structure = lltype.Struct('random_structure', ('one', rffi.INT), ('two', lltype.Ptr(lltype.Array(rffi.INT, hints={'nolength': True}))), hints={'external':'C'}) """) - src = rffi_source.structures[random_structure] + src = rffi_source.source assert src.strip() == _src.strip(), str(src) + "\n" + str(_src) class TestMkrffi(TestBasic): @@ -40,10 +40,10 @@ src = RffiSource() src.proc_func(func) _src = py.code.Source(""" - c_int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], rffi.VOIDP) + int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], rffi.VOIDP) """) - assert src.code == _src, str(src) + "\n" + str(_src) + assert src.source == _src, str(src) + "\n" + str(_src) def test_struct_return(self): func = self.lib.int_int_to_struct_p @@ -51,9 +51,11 @@ func.restype = ctypes.POINTER(random_structure) rffi_source = RffiSource() rffi_source.proc_func(func) - assert random_structure in rffi_source.structures + assert random_structure in rffi_source.structs _src = py.code.Source(""" - c_int_int_to_struct_p = rffi.llexternal('int_int_to_struct_p', [rffi.INT, rffi.INT], lltype.Ptr(c_random_structure)) + random_structure = lltype.Struct('random_structure', ('one', rffi.INT), ('two', lltype.Ptr(lltype.Array(rffi.INT, hints={'nolength': True}))), hints={'external':'C'}) + + int_int_to_struct_p = rffi.llexternal('int_int_to_struct_p', [rffi.INT, rffi.INT], lltype.Ptr(random_structure)) """) - src = rffi_source.code + src = rffi_source.source assert src.strip() == _src.strip(), str(src) + "\n" + str(_src) From arigo at codespeak.net Sat Jul 14 08:50:17 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jul 2007 08:50:17 +0200 (CEST) Subject: [pypy-svn] r45041 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070714065017.6D8E781C8@code0.codespeak.net> Author: arigo Date: Sat Jul 14 08:50:16 2007 New Revision: 45041 Removed: pypy/dist/pypy/rpython/lltypesystem/compactlltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_compactlltype.py Log: Kill old code (from yesterday :-). From arigo at codespeak.net Sat Jul 14 08:52:27 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jul 2007 08:52:27 +0200 (CEST) Subject: [pypy-svn] r45042 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070714065227.6FEB480DC@code0.codespeak.net> Author: arigo Date: Sat Jul 14 08:52:26 2007 New Revision: 45042 Added: pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py - copied unchanged from r45031, pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py - copied, changed from r45031, pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py Removed: pypy/dist/pypy/rpython/lltypesystem/lltype2ctypes.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype2ctypes.py Log: Rename the module to avoid tab-completion conflict with 'lltype.py'. From jlg at codespeak.net Sat Jul 14 09:59:20 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Sat, 14 Jul 2007 09:59:20 +0200 (CEST) Subject: [pypy-svn] r45043 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070714075920.F34A4819B@code0.codespeak.net> Author: jlg Date: Sat Jul 14 09:59:18 2007 New Revision: 45043 Modified: pypy/dist/pypy/lang/scheme/TODO.txt pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/ssparser.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: more work on numbers Modified: pypy/dist/pypy/lang/scheme/TODO.txt ============================================================================== --- pypy/dist/pypy/lang/scheme/TODO.txt (original) +++ pypy/dist/pypy/lang/scheme/TODO.txt Sat Jul 14 09:59:18 2007 @@ -11,23 +11,22 @@ - symbols vs identifier, which name is better - global dict for symbols _obarray_ -- implement key funcions +- implement key functions (apply, reduce, mapcar and so on) # reduce will not make into python 3.0 ;) - comparison: < > eq? eqv? - - exact/inexact numbers - Do in some future ----------------- Here starts the real fun! +- quasi-quotations - macros - delayed evaluation - continuations - proper tail-recursion - - mutulally recursive procedures (r5rs p.12) -- switch to bytecode generation + eval instead of evaluating AST + - mutually recursive procedures (r5rs p.12) +- switch to byte-code generation + eval instead of evaluating AST Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Sat Jul 14 09:59:18 2007 @@ -13,7 +13,7 @@ class WrongArgType(SchemeException): def __str__(self): - return "Wrong argument type" + return "Wrong argument type: {%s}!=%s" % (self.args[0], self.args[1]) class SchemeQuit(SchemeException): """raised on (quit) evaluation""" @@ -96,50 +96,54 @@ class W_Real(W_Number): def __init__(self, val): self.exact = False - self.floatval = val + self.realval = val def to_string(self): - return str(self.floatval) + return str(self.realval) def to_number(self): return self.to_float() def to_fixnum(self): - return int(self.floatval) + return int(self.realval) def to_float(self): - return self.floatval + return self.realval def round(self): - int_part = int(self.floatval) - if self.floatval > 0: - if self.floatval >= (int_part + 0.5): + int_part = int(self.realval) + if self.realval > 0: + if self.realval >= (int_part + 0.5): return int_part + 1 return int_part else: - if self.floatval <= (int_part - 0.5): + if self.realval <= (int_part - 0.5): return int_part - 1 return int_part + def is_integer(self): + return self.realval == self.round() + class W_Integer(W_Real): def __init__(self, val): - self.fixnumval = val + self.intval = val + self.realval = val self.exact = True def to_string(self): - return str(self.fixnumval) + return str(self.intval) def to_number(self): return self.to_fixnum() def to_fixnum(self): - return self.fixnumval + return self.intval def to_float(self): - return float(self.fixnumval) + return float(self.intval) class W_Pair(W_Root): def __init__(self, car, cdr): @@ -185,6 +189,7 @@ arg_lst = [] arg = lst while not isinstance(arg, W_Nil): + assert isinstance(arg, W_Pair) arg_lst.append(arg.car.eval(ctx)) arg = arg.cdr @@ -217,6 +222,7 @@ self.args.append(Formal(arg.to_string(), True)) break else: + assert isinstance(arg, W_Pair) assert isinstance(arg.car, W_Identifier) #list of argument names, not evaluated self.args.append(Formal(arg.car.to_string(), False)) @@ -301,7 +307,7 @@ """ % (attr_name, oper)) exec code.compile() in local_locals - local_locals[attr_name]._annspecialcase_ = 'specialize:argtype(1,2)' + local_locals[attr_name]._annspecialcase_ = 'specialize:argtype(1)' setattr(Op, attr_name, local_locals[attr_name]) attr_name = "do_unary_oper" @@ -354,14 +360,14 @@ raise WrongArgsNumber if not isinstance(lst[0], W_Number): - raise WrongArgType + raise WrongArgType(1, 'Number') return W_Boolean(self.predicate(lst[0])) class IntegerP(PredicateNumber): def predicate(self, w_obj): if not w_obj.exact: - return w_obj.to_float() == w_obj.round() + return w_obj.is_integer() return True @@ -381,11 +387,30 @@ def predicate(self, w_obj): return not w_obj.exact +class ZeroP(PredicateNumber): + def predicate(self, w_obj): + return w_obj.to_number() == 0.0 + +class OddP(PredicateNumber): + def predicate(self, w_obj): + if not w_obj.is_integer(): + raise WrongArgType(1, "Integer") + + return w_obj.round() % 2 != 0 + +class EvenP(PredicateNumber): + def predicate(self, w_obj): + if not w_obj.is_integer(): + raise WrongArgType(1, "Integer") + + return w_obj.round() % 2 == 0 + ## # Macro ## class Define(W_Macro): def call(self, ctx, lst): + assert isinstance(lst, W_Pair) w_identifier = lst.car assert isinstance(w_identifier, W_Identifier) @@ -395,6 +420,7 @@ class Sete(W_Macro): def call(self, ctx, lst): + assert isinstance(lst, W_Pair) w_identifier = lst.car assert isinstance(w_identifier, W_Identifier) @@ -404,6 +430,7 @@ class MacroIf(W_Macro): def call(self, ctx, lst): + assert isinstance(lst, W_Pair) w_condition = lst.car w_then = lst.cdr.car if isinstance(lst.cdr.cdr, W_Nil): @@ -427,11 +454,13 @@ class Car(W_Procedure): def procedure(self, ctx, lst): w_pair = lst[0] + assert isinstance(w_pair, W_Pair) return w_pair.car class Cdr(W_Procedure): def procedure(self, ctx, lst): w_pair = lst[0] + assert isinstance(w_pair, W_Pair) return w_pair.cdr class Quit(W_Procedure): @@ -441,11 +470,12 @@ class Lambda(W_Macro): def call(self, ctx, lst): w_args = lst.car - w_body = lst.cdr #.car + w_body = lst.cdr return W_Lambda(w_args, w_body, ctx.copy()) class Let(W_Macro): def call(self, ctx, lst): + assert isinstance(lst, W_Pair) local_ctx = ctx.copy() w_formal = lst.car while not isinstance(w_formal, W_Nil): @@ -459,6 +489,7 @@ class Letrec(W_Macro): def call(self, ctx, lst): + assert isinstance(lst, W_Pair) local_ctx = ctx.copy() #bound variables @@ -478,11 +509,12 @@ return self.eval_body(local_ctx, lst.cdr) -def Literal(sexpr): +def literal(sexpr): return W_Pair(W_Identifier('quote'), W_Pair(sexpr, W_Nil())) class Quote(W_Macro): def call(self, ctx, lst): + assert isinstance(lst, W_Pair) return lst.car ## @@ -521,6 +553,9 @@ 'number?': NumberP, 'exact?': ExactP, 'inexact?': InexactP, + 'zero?': ZeroP, + 'odd?': OddP, + 'even?': EvenP, #macros 'define': Define, 'set!': Sete, Modified: pypy/dist/pypy/lang/scheme/ssparser.py ============================================================================== --- pypy/dist/pypy/lang/scheme/ssparser.py (original) +++ pypy/dist/pypy/lang/scheme/ssparser.py Sat Jul 14 09:59:18 2007 @@ -2,7 +2,7 @@ from pypy.rlib.parsing.pypackrat import PackratParser from pypy.rlib.parsing.makepackrat import BacktrackException, Status from pypy.lang.scheme.object import W_Pair, W_Integer, W_String, W_Identifier, \ - W_Nil, W_Boolean, W_Real, Literal + W_Nil, W_Boolean, W_Real, literal def unquote(s): str_lst = [] @@ -59,7 +59,7 @@ literal: `'` s = sexpr - return {Literal(s)}; + return {literal(s)}; sexpr: list Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Sat Jul 14 09:59:18 2007 @@ -399,7 +399,23 @@ def test_exactness(): assert eval_noctx("(exact? 42)").to_boolean() assert not eval_noctx("(exact? 42.0)").to_boolean() + py.test.raises(WrongArgType, eval_noctx, "(exact? 'a)" ) assert not eval_noctx("(inexact? 42)").to_boolean() assert eval_noctx("(inexact? 42.0)").to_boolean() + py.test.raises(WrongArgType, eval_noctx, "(inexact? 'a)" ) + +def test_number_predicates(): + assert eval_noctx("(zero? 0)").to_boolean() + assert eval_noctx("(zero? 0.0)").to_boolean() + assert not eval_noctx("(zero? 1.0)").to_boolean() + py.test.raises(WrongArgType, eval_noctx, "(zero? 'a)" ) + + assert not eval_noctx("(odd? 0)").to_boolean() + assert eval_noctx("(odd? 1)").to_boolean() + py.test.raises(WrongArgType, eval_noctx, "(odd? 1.1)" ) + + assert eval_noctx("(even? 0)").to_boolean() + assert not eval_noctx("(even? 1)").to_boolean() + py.test.raises(WrongArgType, eval_noctx, "(even? 1.1)" ) From arigo at codespeak.net Sat Jul 14 10:17:46 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jul 2007 10:17:46 +0200 (CEST) Subject: [pypy-svn] r45044 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070714081746.A70DD81CF@code0.codespeak.net> Author: arigo Date: Sat Jul 14 10:17:46 2007 New Revision: 45044 Modified: pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py pypy/dist/pypy/rpython/lltypesystem/rffi.py pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Log: (lac, arigo) First tests of calling an external rffi function via ctypes. Modified: pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py Sat Jul 14 10:17:46 2007 @@ -1,9 +1,12 @@ +import sys import ctypes +import ctypes.util from pypy.rpython.lltypesystem import lltype _ctypes_cache = { lltype.Signed: ctypes.c_long, + lltype.Char: ctypes.c_ubyte, } def build_ctypes_struct(S, max_n=None): @@ -30,19 +33,24 @@ raise TypeError("%r is variable-sized" % (S,)) biggercls = build_ctypes_struct(S, n) bigstruct = biggercls() - getattr(bigstruct, S._arrayfld).length = n + array = getattr(bigstruct, S._arrayfld) + if hasattr(array, 'length'): + array.length = n return bigstruct _malloc = classmethod(_malloc) def _getattr(self, field_name): + T = getattr(S, field_name) cobj = getattr(self, field_name) - return ctypes2lltype(cobj) + return ctypes2lltype(T, cobj) def _setattr(self, field_name, value): cobj = lltype2ctypes(value) setattr(self, field_name, cobj) CStruct.__name__ = 'ctypes_%s' % (S,) + if max_n is not None: + CStruct._normalized_ctype = get_ctypes_type(S) return CStruct def build_ctypes_array(A, max_n=0): @@ -51,27 +59,33 @@ ctypes_item = get_ctypes_type(ITEM) class CArray(ctypes.Structure): - _fields_ = [('length', ctypes.c_int), - ('items', max_n * ctypes_item)] + if not A._hints.get('nolength'): + _fields_ = [('length', ctypes.c_int), + ('items', max_n * ctypes_item)] + else: + _fields_ = [('items', max_n * ctypes_item)] def _malloc(cls, n=None): if not isinstance(n, int): raise TypeError, "array length must be an int" biggercls = build_ctypes_array(A, n) bigarray = biggercls() - bigarray.length = n + if hasattr(bigarray, 'length'): + bigarray.length = n return bigarray _malloc = classmethod(_malloc) def _getitem(self, index): cobj = self.items[index] - return ctypes2lltype(cobj) + return ctypes2lltype(ITEM, cobj) def _setitem(self, index, value): cobj = lltype2ctypes(value) self.items[index] = cobj CArray.__name__ = 'ctypes_%s*%d' % (A, max_n) + if max_n > 0: + CArray._normalized_ctype = get_ctypes_type(A) return CArray def get_ctypes_type(T): @@ -112,7 +126,13 @@ if not isinstance(item_value, lltype._uninitialized): carray.items[i] = lltype2ctypes(item_value) -def lltype2ctypes(llobj): +# ____________________________________________________________ + +def lltype2ctypes(llobj, normalize=True): + """Convert the lltype object 'llobj' to its ctypes equivalent. + 'normalize' should only be False in tests, where we want to + inspect the resulting ctypes object manually. + """ T = lltype.typeOf(llobj) if isinstance(T, lltype.Ptr): container = llobj._obj @@ -123,8 +143,84 @@ convert_array(container) else: raise NotImplementedError(T) - return ctypes.pointer(container._ctypes_storage) + storage = container._ctypes_storage + p = ctypes.pointer(storage) + if normalize and hasattr(storage, '_normalized_ctype'): + p = ctypes.cast(p, ctypes.POINTER(storage._normalized_ctype)) + return p + + if T is lltype.Char: + return ord(llobj) + return llobj -def ctypes2lltype(cobj): - return cobj +def ctypes2lltype(T, cobj): + """Convert the ctypes object 'cobj' to its lltype equivalent. + 'T' is the expected lltype type. + """ + if T is lltype.Char: + llobj = chr(cobj) + else: + llobj = cobj + + assert lltype.typeOf(llobj) == T + return llobj + +# __________ the standard C library __________ + +if sys.platform == 'win32': + standard_c_lib = ctypes.cdll.LoadLibrary('msvcrt.dll') +else: + standard_c_lib = ctypes.cdll.LoadLibrary(ctypes.util.find_library('c')) + +# ____________________________________________ + +def get_ctypes_callable(funcptr): + if getattr(funcptr._obj, 'source', None) is not None: + # give up - for tests with an inlined bit of C code + raise NotImplementedError("cannot call a C function defined in " + "a custom C source snippet") + FUNCTYPE = lltype.typeOf(funcptr).TO + funcname = funcptr._obj._name + libraries = getattr(funcptr._obj, 'libraries', None) + if not libraries: + cfunc = getattr(standard_c_lib, funcname, None) + else: + cfunc = None + for libname in libraries: + libpath = ctypes.util.find_library(libname) + if libpath: + clib = ctypes.cdll.LoadLibrary(libpath) + cfunc = getattr(clib, funcname, None) + if cfunc is not None: + break + + if cfunc is None: + # function name not found in any of the libraries + if not libraries: + place = 'the standard C library' + elif len(libraries) == 1: + place = 'library %r' % (libraries[0],) + else: + place = 'any of the libraries %r' % (libraries,) + raise NotImplementedError("function %r not found in %s" % ( + funcname, place)) + + # get_ctypes_type() can raise NotImplementedError too + cfunc.argtypes = [get_ctypes_type(T) for T in FUNCTYPE.ARGS] + cfunc.restype = get_ctypes_type(FUNCTYPE.RESULT) + return cfunc + +def make_callable_via_ctypes(funcptr): + try: + cfunc = get_ctypes_callable(funcptr) + except NotImplementedError, e: + def invoke_via_ctypes(*argvalues): + raise NotImplementedError, e + else: + RESULT = lltype.typeOf(funcptr).TO.RESULT + def invoke_via_ctypes(*argvalues): + cargs = [lltype2ctypes(value) for value in argvalues] + cres = cfunc(*cargs) + return ctypes2lltype(RESULT, cres) + funcptr._obj._callable = invoke_via_ctypes Modified: pypy/dist/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rffi.py Sat Jul 14 10:17:46 2007 @@ -20,11 +20,15 @@ def llexternal(name, args, result, _callable=None, sources=[], includes=[], libraries=[]): ext_type = lltype.FuncType(args, result) - return lltype.functionptr(ext_type, name, external='C', - sources=tuple(sources), - includes=tuple(includes), - libraries=tuple(libraries), - _callable=_callable) + funcptr = lltype.functionptr(ext_type, name, external='C', + sources=tuple(sources), + includes=tuple(includes), + libraries=tuple(libraries), + _callable=_callable) + if _callable is None: + from pypy.rpython.lltypesystem import ll2ctypes + ll2ctypes.make_callable_via_ctypes(funcptr) + return funcptr def setup(): """ creates necessary c-level types Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Sat Jul 14 10:17:46 2007 @@ -1,11 +1,16 @@ +import py import ctypes -from pypy.rpython.lltypesystem import lltype -from pypy.rpython.lltypesystem.ll2ctypes import lltype2ctypes +from pypy.rpython.lltypesystem import lltype, rffi +from pypy.rpython.lltypesystem.ll2ctypes import lltype2ctypes, ctypes2lltype def test_primitive(): assert lltype2ctypes(5) == 5 - assert lltype2ctypes('?') == '?' + assert lltype2ctypes('?') == ord('?') + assert lltype2ctypes('\xE0') == 0xE0 + assert ctypes2lltype(lltype.Signed, 5) == 5 + assert ctypes2lltype(lltype.Char, ord('a')) == 'a' + assert ctypes2lltype(lltype.Char, 0xFF) == '\xFF' def test_simple_struct(): S = lltype.Struct('S', ('x', lltype.Signed), ('y', lltype.Signed)) @@ -28,7 +33,7 @@ a[0] = 100 a[1] = 101 a[2] = 102 - ac = lltype2ctypes(a) + ac = lltype2ctypes(a, normalize=False) assert isinstance(ac.contents, ctypes.Structure) assert ac.contents.length == 10 assert ac.contents.items[1] == 101 @@ -37,3 +42,58 @@ a[3] = 789 assert ac.contents.items[3] == 789 lltype.free(a, flavor='raw') + +def test_array_nolength(): + A = lltype.Array(lltype.Signed, hints={'nolength': True}) + a = lltype.malloc(A, 10, flavor='raw') + a[0] = 100 + a[1] = 101 + a[2] = 102 + ac = lltype2ctypes(a, normalize=False) + assert isinstance(ac.contents, ctypes.Structure) + assert ac.contents.items[1] == 101 + ac.contents.items[2] = 456 + assert a[2] == 456 + a[3] = 789 + assert ac.contents.items[3] == 789 + assert ctypes.sizeof(ac.contents) == 10 * ctypes.sizeof(ctypes.c_long) + lltype.free(a, flavor='raw') + +def test_charp(): + s = rffi.str2charp("hello") + sc = lltype2ctypes(s, normalize=False) + assert sc.contents.items[0] == ord('h') + assert sc.contents.items[1] == ord('e') + assert sc.contents.items[2] == ord('l') + assert sc.contents.items[3] == ord('l') + assert sc.contents.items[4] == ord('o') + assert sc.contents.items[5] == 0 + assert not hasattr(sc.contents, 'length') + sc.contents.items[1] = ord('E') + assert s[1] == 'E' + s[0] = 'H' + assert sc.contents.items[0] == ord('H') + +def test_strlen(): + strlen = rffi.llexternal('strlen', [rffi.CCHARP], lltype.Signed, + includes=['string.h']) + s = rffi.str2charp("xxx") + res = strlen(s) + rffi.free_charp(s) + assert res == 3 + +def test_func_not_in_clib(): + foobar = rffi.llexternal('I_really_dont_exist', [], lltype.Signed) + py.test.raises(NotImplementedError, foobar) + + foobar = rffi.llexternal('I_really_dont_exist', [], lltype.Signed, + libraries=['m']) # math library + py.test.raises(NotImplementedError, foobar) + + foobar = rffi.llexternal('I_really_dont_exist', [], lltype.Signed, + libraries=['m', 'z']) # math and zlib libraries + py.test.raises(NotImplementedError, foobar) + + foobar = rffi.llexternal('I_really_dont_exist', [], lltype.Signed, + libraries=['I_really_dont_exist_either']) + py.test.raises(NotImplementedError, foobar) From fijal at codespeak.net Sat Jul 14 10:21:32 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 14 Jul 2007 10:21:32 +0200 (CEST) Subject: [pypy-svn] r45045 - pypy/dist/pypy/interpreter/pyparser Message-ID: <20070714082132.2011A81CF@code0.codespeak.net> Author: fijal Date: Sat Jul 14 10:21:31 2007 New Revision: 45045 Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py Log: s/value/get_value()/ Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/dist/pypy/interpreter/pyparser/astbuilder.py Sat Jul 14 10:21:31 2007 @@ -887,7 +887,7 @@ no matter how you got to this one particular feature. """ atoms = peek_atoms(builder, nb) - feature_name = atoms[0].value + feature_name = atoms[0].get_value() space = builder.space w_feature_code = space.appexec([space.wrap(feature_name)], """(feature): From justas at codespeak.net Sat Jul 14 10:30:29 2007 From: justas at codespeak.net (justas at codespeak.net) Date: Sat, 14 Jul 2007 10:30:29 +0200 (CEST) Subject: [pypy-svn] r45046 - in pypy/dist/pypy/interpreter/pyparser: . test Message-ID: <20070714083029.7E78181CF@code0.codespeak.net> Author: justas Date: Sat Jul 14 10:30:29 2007 New Revision: 45046 Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py pypy/dist/pypy/interpreter/pyparser/asthelper.py pypy/dist/pypy/interpreter/pyparser/pythonparse.py pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Log: A flag to enable a chosen grammar and its build rules Modified: pypy/dist/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/dist/pypy/interpreter/pyparser/astbuilder.py Sat Jul 14 10:30:29 2007 @@ -11,6 +11,7 @@ #import pypy.interpreter.pyparser.pytoken as tok from pypy.interpreter.pyparser.error import SyntaxError from pypy.interpreter.pyparser.parsestring import parsestr +from pypy.interpreter.pyparser.pythonparse import ENABLE_GRAMMAR_VERSION from pypy.interpreter.gateway import interp2app from asthelper import * @@ -888,6 +889,7 @@ """ atoms = peek_atoms(builder, nb) feature_name = atoms[0].get_value() + assert type(feature_name) is str space = builder.space w_feature_code = space.appexec([space.wrap(feature_name)], """(feature): @@ -1060,8 +1062,6 @@ 'while_stmt' : build_while_stmt, 'import_name' : build_import_name, 'import_from' : build_import_from, - 'future_import_feature': build_future_import_feature, - 'import_from_future': build_import_from, 'yield_stmt' : build_yield_stmt, 'continue_stmt' : build_continue_stmt, 'del_stmt' : build_del_stmt, @@ -1087,7 +1087,8 @@ class AstBuilder(BaseGrammarBuilder): """A builder that directly produce the AST""" - def __init__(self, parser, debug=0, space=None): + def __init__(self, parser, debug=0, space=None, + grammar_version=ENABLE_GRAMMAR_VERSION): BaseGrammarBuilder.__init__(self, parser, debug) self.rule_stack = [] self.space = space @@ -1095,6 +1096,11 @@ self.with_enabled = False self.build_rules = ASTRULES_Template self.user_build_rules = {} + if grammar_version >= "2.5": + self.build_rules.update({ + 'future_import_feature': build_future_import_feature, + 'import_from_future': build_import_from, + }) def enable_with(self): if self.with_enabled: Modified: pypy/dist/pypy/interpreter/pyparser/asthelper.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/asthelper.py (original) +++ pypy/dist/pypy/interpreter/pyparser/asthelper.py Sat Jul 14 10:30:29 2007 @@ -562,8 +562,10 @@ class TokenObject(ast.Node): """A simple object used to wrap a rule or token""" + def __init__(self, name, value, lineno, parser): self.name = name + assert value is None or type(value) is str self.value = value self.count = 0 # self.line = 0 # src.getline() Modified: pypy/dist/pypy/interpreter/pyparser/pythonparse.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/pythonparse.py (original) +++ pypy/dist/pypy/interpreter/pyparser/pythonparse.py Sat Jul 14 10:30:29 2007 @@ -28,6 +28,8 @@ from codeop import PyCF_DONT_IMPLY_DEDENT +ENABLE_GRAMMAR_VERSION = "2.4" + ## files encoding management ############################################ _recode_to_utf8 = gateway.applevel(r''' @@ -165,7 +167,7 @@ # recompute first sets self.build_first_sets() -def make_pyparser(version="2.4"): +def make_pyparser(version=ENABLE_GRAMMAR_VERSION): parser = PythonParser() return build_parser_for_version(version, parser=parser) Modified: pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/test_astbuilder_future.py Sat Jul 14 10:30:29 2007 @@ -57,7 +57,8 @@ def setup_method(self, method): self.builder = astbuilder.AstBuilder( - self.parser, space=FakeSpaceForFeatureLookup()) + self.parser, space=FakeSpaceForFeatureLookup(), + grammar_version="2.5a") def test_future_rules(self): assert (self.builder.build_rules['future_import_feature'] is From pedronis at codespeak.net Sat Jul 14 10:49:58 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 14 Jul 2007 10:49:58 +0200 (CEST) Subject: [pypy-svn] r45047 - pypy/dist/pypy/annotation Message-ID: <20070714084958.6C8F081B4@code0.codespeak.net> Author: pedronis Date: Sat Jul 14 10:49:58 2007 New Revision: 45047 Added: pypy/dist/pypy/annotation/bookkeeper.py - copied unchanged from r44993, pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/policy.py - copied unchanged from r44993, pypy/dist/pypy/annotation/policy.py Log: revert From fijal at codespeak.net Sat Jul 14 10:50:02 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 14 Jul 2007 10:50:02 +0200 (CEST) Subject: [pypy-svn] r45048 - pypy/dist/pypy/interpreter/pyparser Message-ID: <20070714085002.ABE6F81CF@code0.codespeak.net> Author: fijal Date: Sat Jul 14 10:50:02 2007 New Revision: 45048 Modified: pypy/dist/pypy/interpreter/pyparser/asthelper.py Log: Remove non-rpython assertion Modified: pypy/dist/pypy/interpreter/pyparser/asthelper.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/asthelper.py (original) +++ pypy/dist/pypy/interpreter/pyparser/asthelper.py Sat Jul 14 10:50:02 2007 @@ -565,7 +565,6 @@ def __init__(self, name, value, lineno, parser): self.name = name - assert value is None or type(value) is str self.value = value self.count = 0 # self.line = 0 # src.getline() From pedronis at codespeak.net Sat Jul 14 10:51:42 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 14 Jul 2007 10:51:42 +0200 (CEST) Subject: [pypy-svn] r45049 - pypy/dist/pypy/lib Message-ID: <20070714085142.3135881B4@code0.codespeak.net> Author: pedronis Date: Sat Jul 14 10:51:41 2007 New Revision: 45049 Added: pypy/dist/pypy/lib/_structseq.py - copied unchanged from r44993, pypy/dist/pypy/lib/_structseq.py Log: revert From pedronis at codespeak.net Sat Jul 14 10:53:56 2007 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 14 Jul 2007 10:53:56 +0200 (CEST) Subject: [pypy-svn] r45050 - pypy/dist/pypy/module/pypyjit/test Message-ID: <20070714085356.929DE81B4@code0.codespeak.net> Author: pedronis Date: Sat Jul 14 10:53:56 2007 New Revision: 45050 Added: pypy/dist/pypy/module/pypyjit/test/test_pypy_c.py - copied unchanged from r44993, pypy/dist/pypy/module/pypyjit/test/test_pypy_c.py Log: revert From jlg at codespeak.net Sat Jul 14 11:09:45 2007 From: jlg at codespeak.net (jlg at codespeak.net) Date: Sat, 14 Jul 2007 11:09:45 +0200 (CEST) Subject: [pypy-svn] r45051 - in pypy/dist/pypy/lang/scheme: . test Message-ID: <20070714090945.378CF8195@code0.codespeak.net> Author: jlg Date: Sat Jul 14 11:09:44 2007 New Revision: 45051 Modified: pypy/dist/pypy/lang/scheme/object.py pypy/dist/pypy/lang/scheme/test/test_eval.py Log: procedures, macros raises SchemeSyntaxError or WrongArgType Modified: pypy/dist/pypy/lang/scheme/object.py ============================================================================== --- pypy/dist/pypy/lang/scheme/object.py (original) +++ pypy/dist/pypy/lang/scheme/object.py Sat Jul 14 11:09:44 2007 @@ -5,7 +5,11 @@ class UnboundVariable(SchemeException): def __str__(self): - return "Unbound variable %s" % self.args[0] + return "Unbound variable %s" % (self.args[0], ) + +class NotCallable(SchemeException): + def __str__(self): + return "%s is not a callable" % (self.args[0].to_string(), ) class WrongArgsNumber(SchemeException): def __str__(self): @@ -13,7 +17,12 @@ class WrongArgType(SchemeException): def __str__(self): - return "Wrong argument type: {%s}!=%s" % (self.args[0], self.args[1]) + return "Wrong argument type: %s is not %s" % \ + (self.args[0].to_string(), self.args[1]) + +class SchemeSyntaxError(SchemeException): + def __str__(self): + return "Syntax error" class SchemeQuit(SchemeException): """raised on (quit) evaluation""" @@ -157,7 +166,8 @@ def eval(self, ctx): oper = self.car.eval(ctx) - assert isinstance(oper, W_Callable) + if not isinstance(oper, W_Callable): + raise NotCallable(oper) return oper.call(ctx, self.cdr) class W_Nil(W_Root): @@ -189,7 +199,8 @@ arg_lst = [] arg = lst while not isinstance(arg, W_Nil): - assert isinstance(arg, W_Pair) + if not isinstance(arg, W_Pair): + raise SchemeSyntaxError arg_lst.append(arg.car.eval(ctx)) arg = arg.cdr @@ -222,8 +233,10 @@ self.args.append(Formal(arg.to_string(), True)) break else: - assert isinstance(arg, W_Pair) - assert isinstance(arg.car, W_Identifier) + if not isinstance(arg, W_Pair): + raise SchemeSyntaxError + if not isinstance(arg.car, W_Identifier): + raise WrongArgType(arg.car, "Identifier") #list of argument names, not evaluated self.args.append(Formal(arg.car.to_string(), False)) arg = arg.cdr @@ -271,14 +284,18 @@ return self.default_result if len(lst) == 1: - return self.unary_oper(lst[0].eval(ctx)) + if not isinstance(lst[0], W_Number): + raise WrongArgType(lst[0], "Number") + return self.unary_oper(lst[0]) acc = None for arg in lst: + if not isinstance(arg, W_Number): + raise WrongArgType(arg, "Number") if acc is None: - acc = arg.eval(ctx) + acc = arg else: - acc = self.oper(acc, arg.eval(ctx)) + acc = self.oper(acc, arg) return acc @@ -339,8 +356,13 @@ return W_Boolean(True) prev = lst[0] + if not isinstance(prev, W_Number): + raise WrongArgType(prev, "Number") + for arg in lst[1:]: - assert isinstance(arg, W_Number) + if not isinstance(arg, W_Number): + raise WrongArgType(arg, "Number") + if prev.to_number() != arg.to_number(): return W_Boolean(False) prev = arg @@ -359,10 +381,11 @@ if len(lst) != 1: raise WrongArgsNumber - if not isinstance(lst[0], W_Number): - raise WrongArgType(1, 'Number') + w_obj = lst[0] + if not isinstance(w_obj, W_Number): + raise WrongArgType(w_obj, 'Number') - return W_Boolean(self.predicate(lst[0])) + return W_Boolean(self.predicate(w_obj)) class IntegerP(PredicateNumber): def predicate(self, w_obj): @@ -394,14 +417,14 @@ class OddP(PredicateNumber): def predicate(self, w_obj): if not w_obj.is_integer(): - raise WrongArgType(1, "Integer") + raise WrongArgType(w_obj, "Integer") return w_obj.round() % 2 != 0 class EvenP(PredicateNumber): def predicate(self, w_obj): if not w_obj.is_integer(): - raise WrongArgType(1, "Integer") + raise WrongArgType(w_obj, "Integer") return w_obj.round() % 2 == 0 @@ -410,9 +433,11 @@ ## class Define(W_Macro): def call(self, ctx, lst): - assert isinstance(lst, W_Pair) + if not isinstance(lst, W_Pair): + raise SchemeSyntaxError w_identifier = lst.car - assert isinstance(w_identifier, W_Identifier) + if not isinstance(w_identifier, W_Identifier): + raise WrongArgType(w_identifier, "Identifier") w_val = lst.cdr.car.eval(ctx) ctx.set(w_identifier.name, w_val) @@ -420,9 +445,11 @@ class Sete(W_Macro): def call(self, ctx, lst): - assert isinstance(lst, W_Pair) + if not isinstance(lst, W_Pair): + raise SchemeSyntaxError w_identifier = lst.car - assert isinstance(w_identifier, W_Identifier) + if not isinstance(w_identifier, W_Identifier): + raise WrongArgType(w_identifier, "Identifier") w_val = lst.cdr.car.eval(ctx) ctx.sete(w_identifier.name, w_val) @@ -430,7 +457,8 @@ class MacroIf(W_Macro): def call(self, ctx, lst): - assert isinstance(lst, W_Pair) + if not isinstance(lst, W_Pair): + raise SchemeSyntaxError w_condition = lst.car w_then = lst.cdr.car if isinstance(lst.cdr.cdr, W_Nil): @@ -454,13 +482,15 @@ class Car(W_Procedure): def procedure(self, ctx, lst): w_pair = lst[0] - assert isinstance(w_pair, W_Pair) + if not isinstance(w_pair, W_Pair): + raise WrongArgType(w_pair, "Pair") return w_pair.car class Cdr(W_Procedure): def procedure(self, ctx, lst): w_pair = lst[0] - assert isinstance(w_pair, W_Pair) + if not isinstance(w_pair, W_Pair): + raise WrongArgType(w_pair, "Pair") return w_pair.cdr class Quit(W_Procedure): @@ -475,7 +505,8 @@ class Let(W_Macro): def call(self, ctx, lst): - assert isinstance(lst, W_Pair) + if not isinstance(lst, W_Pair): + raise SchemeSyntaxError local_ctx = ctx.copy() w_formal = lst.car while not isinstance(w_formal, W_Nil): @@ -489,7 +520,8 @@ class Letrec(W_Macro): def call(self, ctx, lst): - assert isinstance(lst, W_Pair) + if not isinstance(lst, W_Pair): + raise SchemeSyntaxError local_ctx = ctx.copy() #bound variables @@ -514,7 +546,8 @@ class Quote(W_Macro): def call(self, ctx, lst): - assert isinstance(lst, W_Pair) + if not isinstance(lst, W_Pair): + raise SchemeSyntaxError return lst.car ## Modified: pypy/dist/pypy/lang/scheme/test/test_eval.py ============================================================================== --- pypy/dist/pypy/lang/scheme/test/test_eval.py (original) +++ pypy/dist/pypy/lang/scheme/test/test_eval.py Sat Jul 14 11:09:44 2007 @@ -48,6 +48,15 @@ w_num = eval_noctx("(- 4 -5 6.1)") assert w_num.to_number() == 4 - (-5) - 6.1 + py.test.raises(WrongArgType, eval_noctx, "(+ 'a)") + py.test.raises(WrongArgType, eval_noctx, "(+ 1 'a)") + py.test.raises(WrongArgType, eval_noctx, "(- 'a)") + py.test.raises(WrongArgType, eval_noctx, "(- 1 'a)") + py.test.raises(WrongArgType, eval_noctx, "(* 'a)") + py.test.raises(WrongArgType, eval_noctx, "(* 1 'a)") + py.test.raises(WrongArgType, eval_noctx, "(/ 'a)") + py.test.raises(WrongArgType, eval_noctx, "(/ 1 'a)") + def test_numerical_nested(): w_num = eval_noctx("(+ 4 (* (+ 5) 6) (+ 1 2))") assert w_num.to_number() == 37 @@ -184,6 +193,8 @@ w_bool = eval_noctx("(= 2.1 2.1 2.1 2)") assert w_bool.to_boolean() is False + py.test.raises(WrongArgType, eval_noctx, "(= 'a 1)") + def test_comparison_heteronums(): w_bool = eval_noctx("(= 1 1.0 1.1)") assert w_bool.to_boolean() is False From fijal at codespeak.net Sat Jul 14 11:16:32 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 14 Jul 2007 11:16:32 +0200 (CEST) Subject: [pypy-svn] r45052 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070714091632.E494B8195@code0.codespeak.net> Author: fijal Date: Sat Jul 14 11:16:32 2007 New Revision: 45052 Modified: pypy/dist/pypy/rpython/lltypesystem/rffi.py pypy/dist/pypy/rpython/lltypesystem/test/test_rffi.py Log: (simonb, fijal) Pass around include_dirs to be able to specify new extra includes in rffi.llexternal Modified: pypy/dist/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rffi.py Sat Jul 14 11:16:32 2007 @@ -18,12 +18,14 @@ def lltype(self): return self.TP -def llexternal(name, args, result, _callable=None, sources=[], includes=[], libraries=[]): +def llexternal(name, args, result, _callable=None, sources=[], includes=[], + libraries=[], include_dirs=[]): ext_type = lltype.FuncType(args, result) funcptr = lltype.functionptr(ext_type, name, external='C', sources=tuple(sources), includes=tuple(includes), libraries=tuple(libraries), + include_dirs=tuple(include_dirs), _callable=_callable) if _callable is None: from pypy.rpython.lltypesystem import ll2ctypes Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_rffi.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_rffi.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_rffi.py Sat Jul 14 11:16:32 2007 @@ -173,3 +173,25 @@ res = interpret(f, []) assert res == 3 + +def test_extra_include_dirs(): + udir.ensure("incl", dir=True) + udir.join("incl", "incl.h").write("#define C 3") + c_file = udir.join("test_extra_include_dirs.c") + c_source = """ + #include + int fun () + { + return (C); + } + """ + c_file.write(c_source) + z = llexternal('fun', [], Signed, sources=[str(c_file)], include_dirs= + [str(udir.join("incl"))]) + + def f(): + return z() + + res = compile(f, []) + assert res() == 3 + From fijal at codespeak.net Sat Jul 14 11:17:21 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 14 Jul 2007 11:17:21 +0200 (CEST) Subject: [pypy-svn] r45053 - pypy/dist/pypy/translator/c Message-ID: <20070714091721.673DB81B4@code0.codespeak.net> Author: fijal Date: Sat Jul 14 11:17:20 2007 New Revision: 45053 Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py Log: Support for extraincludes in the C backend Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Sat Jul 14 11:17:20 2007 @@ -133,21 +133,23 @@ from pypy.translator.c.symboltable import SymbolTable # XXX fix symboltable #self.symboltable = SymbolTable() - cfile, extra = gen_source(db, modulename, targetdir, - defines = defines, - exports = self.exports, - symboltable = self.symboltable) + cfile, extra, extraincludes = gen_source(db, modulename, targetdir, + defines = defines, + exports = self.exports, + symboltable = self.symboltable) else: if self.config.translation.instrument: defines['INSTRUMENT'] = 1 if CBuilder.have___thread: if not self.config.translation.no__thread: defines['USE___THREAD'] = 1 - cfile, extra = gen_source_standalone(db, modulename, targetdir, - entrypointname = pfname, - defines = defines) + cfile, extra, extraincludes = \ + gen_source_standalone(db, modulename, targetdir, + entrypointname = pfname, + defines = defines) self.c_source_filename = py.path.local(cfile) self.extrafiles = extra + self.extraincludes = extraincludes.keys() if self.standalone: self.gen_makefile(targetdir) return cfile @@ -176,9 +178,10 @@ def compile(self): assert self.c_source_filename assert not self._compiled + extra_includes = self.extraincludes compile_c_module([self.c_source_filename] + self.extrafiles, self.c_source_filename.purebasename, - include_dirs = [autopath.this_dir], + include_dirs = [autopath.this_dir] + extra_includes, libraries=self.libraries) self._compiled = True @@ -241,7 +244,8 @@ def compile(self): assert self.c_source_filename assert not self._compiled - compiler = self.getccompiler(extra_includes=[str(self.targetdir)]) + compiler = self.getccompiler(extra_includes=[str(self.targetdir)] + + self.extraincludes) if sys.platform == 'darwin': compiler.compile_extra.append('-mdynamic-no-pic') if self.config.translation.compilerflags: @@ -606,6 +610,7 @@ def extra_information(database): includes = {} sources = {} + include_dirs = {} for node in database.globalcontainers(): if hasattr(node, 'includes'): for include in node.includes: @@ -613,11 +618,14 @@ if hasattr(node, 'sources'): for source in node.sources: sources[source] = True + if hasattr(node, 'include_dirs'): + for include_dir in node.include_dirs: + include_dirs[include_dir] = True includes = includes.keys() includes.sort() sources = sources.keys() sources.sort() - return includes, sources + return includes, sources, include_dirs def gen_source_standalone(database, modulename, targetdir, entrypointname, defines={}): @@ -648,7 +656,7 @@ for line in database.gcpolicy.pre_gc_code(): print >> fi, line - includes, sources = extra_information(database) + includes, sources, include_dirs = extra_information(database) for include in includes: print >> fi, '#include <%s>' % (include,) fi.close() @@ -676,7 +684,7 @@ print >>fi, "#define INSTRUMENT_NCOUNTER %d" % n fi.close() - return filename, sg.getextrafiles() + sources + return filename, sg.getextrafiles() + sources, include_dirs def gen_source(database, modulename, targetdir, defines={}, exports={}, @@ -706,7 +714,7 @@ for line in database.gcpolicy.pre_gc_code(): print >> fi, line - includes, sources = extra_information(database) + includes, sources, include_dirs = extra_information(database) for include in includes: print >> fi, '#include <%s>' % (include,) fi.close() @@ -837,7 +845,7 @@ f.write(SETUP_PY % locals()) f.close() - return filename, sg.getextrafiles() + sources + return filename, sg.getextrafiles() + sources, include_dirs SETUP_PY = ''' Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Sat Jul 14 11:17:20 2007 @@ -613,6 +613,8 @@ db.namespace.uniquename('g_' + self.basename())) if hasattr(obj, 'libraries'): self.libraries = obj.libraries + if hasattr(obj, 'include_dirs'): + self.include_dirs = obj.include_dirs self.make_funcgens() #self.dependencies = {} self.typename = db.gettype(T) #, who_asks=self) From simonb at codespeak.net Sat Jul 14 11:26:10 2007 From: simonb at codespeak.net (simonb at codespeak.net) Date: Sat, 14 Jul 2007 11:26:10 +0200 (CEST) Subject: [pypy-svn] r45054 - in pypy/dist/pypy/rpython/tool: . test Message-ID: <20070714092610.2BEA881A1@code0.codespeak.net> Author: simonb Date: Sat Jul 14 11:26:09 2007 New Revision: 45054 Modified: pypy/dist/pypy/rpython/tool/mkrffi.py pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Log: (fijal, simonb) support includes, libraries and include_dirs in RffiSource Modified: pypy/dist/pypy/rpython/tool/mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/mkrffi.py Sat Jul 14 11:26:09 2007 @@ -34,7 +34,8 @@ } class RffiSource(object): - def __init__(self, structs=None, source=None): + def __init__(self, structs=None, source=None, includes=[], libraries=[], + include_dirs=[]): # set of ctypes structs if structs is None: self.structs = set() @@ -44,6 +45,11 @@ self.source = py.code.Source() else: self.source = source + includes = includes and "includes=%s, " % repr(tuple(includes)) or '' + libraries = libraries and "libraries=%s, " % repr(tuple(libraries)) or '' + include_dirs = include_dirs and \ + "include_dirs=%s, " % repr(tuple(include_dirs)) or '' + self.extra_args = includes+libraries+include_dirs def __str__(self): return str(self.source) @@ -90,10 +96,9 @@ print "proc_func", func name = func.__name__ src = py.code.Source(""" - %s = rffi.llexternal('%s', [%s], %s) - """%(name, name, ", ".join([self.proc_tp(arg) for - arg in func.argtypes]), - self.proc_tp(func.restype))) + %s = rffi.llexternal('%s', [%s], %s, %s) + """%(name, name, ", ".join([self.proc_tp(arg) for arg in func.argtypes]), + self.proc_tp(func.restype), self.extra_args)) self.source = py.code.Source(self.source, src) def proc_namespace(self, ns): Modified: pypy/dist/pypy/rpython/tool/test/test_mkrffi.py ============================================================================== --- pypy/dist/pypy/rpython/tool/test/test_mkrffi.py (original) +++ pypy/dist/pypy/rpython/tool/test/test_mkrffi.py Sat Jul 14 11:26:09 2007 @@ -40,7 +40,7 @@ src = RffiSource() src.proc_func(func) _src = py.code.Source(""" - int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], rffi.VOIDP) + int_to_void_p = rffi.llexternal('int_to_void_p', [rffi.INT], rffi.VOIDP, ) """) assert src.source == _src, str(src) + "\n" + str(_src) @@ -55,7 +55,7 @@ _src = py.code.Source(""" random_structure = lltype.Struct('random_structure', ('one', rffi.INT), ('two', lltype.Ptr(lltype.Array(rffi.INT, hints={'nolength': True}))), hints={'external':'C'}) - int_int_to_struct_p = rffi.llexternal('int_int_to_struct_p', [rffi.INT, rffi.INT], lltype.Ptr(random_structure)) + int_int_to_struct_p = rffi.llexternal('int_int_to_struct_p', [rffi.INT, rffi.INT], lltype.Ptr(random_structure), ) """) src = rffi_source.source assert src.strip() == _src.strip(), str(src) + "\n" + str(_src) From fijal at codespeak.net Sat Jul 14 11:37:20 2007 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 14 Jul 2007 11:37:20 +0200 (CEST) Subject: [pypy-svn] r45055 - pypy/dist/pypy/translator/llvm/test Message-ID: <20070714093720.00A5881A1@code0.codespeak.net> Author: fijal Date: Sat Jul 14 11:37:20 2007 New Revision: 45055 Modified: pypy/dist/pypy/translator/llvm/test/test_extfunc.py Log: Skip llvm extfunc tests as noone wants to fix those anyway Modified: pypy/dist/pypy/translator/llvm/test/test_extfunc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_extfunc.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_extfunc.py Sat Jul 14 11:37:20 2007 @@ -8,6 +8,8 @@ from pypy.rlib.rarithmetic import r_uint from pypy.rlib import ros +py.test.skip("Extfunc support in llvm needs refactoring") + from pypy.translator.llvm.test.runtest import * def test_external_function_ll_os_dup(): From alecu at codespeak.net Sat Jul 14 11:38:25 2007 From: alecu at codespeak.net (alecu at codespeak.net) Date: Sat, 14 Jul 2007 11:38:25 +0200 (CEST) Subject: [pypy-svn] r45056 - pypy/branch/flex-backend Message-ID: <20070714093825.B16BF81A1@code0.codespeak.net> Author: alecu Date: Sat Jul 14 11:38:25 2007 New Revision: 45056 Added: pypy/branch/flex-backend/ - copied from r45055, pypy/dist/pypy/ Log: Flex backend, Europython 2007 sprints From arigo at codespeak.net Sat Jul 14 11:40:37 2007 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jul 2007 11:40:37 +0200 (CEST) Subject: [pypy-svn] r45057 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20070714094037.A4A9981AA@code0.codespeak.net> Author: arigo Date: Sat Jul 14 11:40:36 2007 New Revision: 45057 Modified: pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Log: (lac, arigo) Make strchr() callable via rffi. Modified: pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/ll2ctypes.py Sat Jul 14 11:40:36 2007 @@ -2,6 +2,11 @@ import ctypes import ctypes.util from pypy.rpython.lltypesystem import lltype +from pypy.tool.uid import fixid + + +def uaddressof(obj): + return fixid(ctypes.addressof(obj)) _ctypes_cache = { @@ -48,6 +53,9 @@ cobj = lltype2ctypes(value) setattr(self, field_name, cobj) + def _eq(self, other): + return ctypes.addressof(self) == ctypes.addressof(other) + CStruct.__name__ = 'ctypes_%s' % (S,) if max_n is not None: CStruct._normalized_ctype = get_ctypes_type(S) @@ -75,13 +83,29 @@ return bigarray _malloc = classmethod(_malloc) - def _getitem(self, index): - cobj = self.items[index] + def _indexable(self, index): + PtrType = ctypes.POINTER((index+1) * ctypes_item) + p = ctypes.cast(ctypes.pointer(self.items), PtrType) + return p.contents + + def _getitem(self, index, boundscheck=True): + if boundscheck: + items = self.items + else: + items = self._indexable(index) + cobj = items[index] return ctypes2lltype(ITEM, cobj) - def _setitem(self, index, value): + def _setitem(self, index, value, boundscheck=True): + if boundscheck: + items = self.items + else: + items = self._indexable(index) cobj = lltype2ctypes(value) - self.items[index] = cobj + items[index] = cobj + + def _eq(self, other): + return ctypes.addressof(self) == ctypes.addressof(other) CArray.__name__ = 'ctypes_%s*%d' % (A, max_n) if max_n > 0: @@ -111,9 +135,14 @@ container._ctypes_storage = cstruct for field_name in STRUCT._names: field_value = getattr(container, field_name) - delattr(container, field_name) if not isinstance(field_value, lltype._uninitialized): setattr(cstruct, field_name, lltype2ctypes(field_value)) + remove_regular_struct_content(container) + +def remove_regular_struct_content(container): + STRUCT = container._TYPE + for field_name in STRUCT._names: + delattr(container, field_name) def convert_array(container): ARRAY = container._TYPE @@ -122,9 +151,31 @@ container._ctypes_storage = carray for i in range(container.getlength()): item_value = container.items[i] # fish fish - container.items[i] = None if not isinstance(item_value, lltype._uninitialized): carray.items[i] = lltype2ctypes(item_value) + remove_regular_array_content(container) + +def remove_regular_array_content(container): + for i in range(container.getlength()): + container.items[i] = None + +class _array_of_unknown_length(lltype._parentable): + _kind = "array" + + __slots__ = () + + def __repr__(self): + return '' % (uaddressof(self._ctypes_storage),) + + def getbounds(self): + # we have no clue, so we allow whatever index + return 0, sys.maxint + + def getitem(self, index, uninitialized_ok=False): + return self._ctypes_storage._getitem(index, boundscheck=False) + + def setitem(self, index, value): + self._ctypes_storage._setitem(index, value, boundscheck=False) # ____________________________________________________________ @@ -160,6 +211,21 @@ """ if T is lltype.Char: llobj = chr(cobj) + elif isinstance(T, lltype.Ptr): + if isinstance(T.TO, lltype.Struct): + # XXX var-sized structs + container = lltype._struct(T.TO) + container._ctypes_storage = cobj.contents + remove_regular_struct_content(container) + elif isinstance(T.TO, lltype.Array): + if T.TO._hints.get('nolength', False): + container = _array_of_unknown_length(T.TO) + container._ctypes_storage = cobj.contents + else: + raise NotImplementedError("array with an explicit length") + else: + raise NotImplementedError(T) + llobj = lltype._ptr(T, container, solid=True) else: llobj = cobj Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Sat Jul 14 11:40:36 2007 @@ -1248,6 +1248,18 @@ def _setup_extra_args(self): pass + def __eq__(self, other): + if not isinstance(other, _parentable): + return False + if (self._ctypes_storage is not None and + other._ctypes_storage is not None): + return self._ctypes_storage._eq(other._ctypes_storage) + else: + return self is other + + def __ne__(self, other): + return not (self == other) + def _struct_variety(flds, cache={}): flds = list(flds) flds.sort() Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Sat Jul 14 11:40:36 2007 @@ -81,6 +81,10 @@ res = strlen(s) rffi.free_charp(s) assert res == 3 + s = rffi.str2charp("") + res = strlen(s) + rffi.free_charp(s) + assert res == 0 def test_func_not_in_clib(): foobar = rffi.llexternal('I_really_dont_exist', [], lltype.Signed) @@ -97,3 +101,74 @@ foobar = rffi.llexternal('I_really_dont_exist', [], lltype.Signed, libraries=['I_really_dont_exist_either']) py.test.raises(NotImplementedError, foobar) + +def test_cstruct_to_ll(): + S = lltype.Struct('S', ('x', lltype.Signed), ('y', lltype.Signed)) + s = lltype.malloc(S, flavor='raw') + s2 = lltype.malloc(S, flavor='raw') + s.x = 123 + sc = lltype2ctypes(s) + t = ctypes2lltype(lltype.Ptr(S), sc) + assert lltype.typeOf(t) == lltype.Ptr(S) + assert s == t + assert not (s != t) + assert t == s + assert not (t != s) + assert t != lltype.nullptr(S) + assert not (t == lltype.nullptr(S)) + assert lltype.nullptr(S) != t + assert not (lltype.nullptr(S) == t) + assert t != s2 + assert not (t == s2) + assert s2 != t + assert not (s2 == t) + assert t.x == 123 + t.x += 1 + assert s.x == 124 + s.x += 1 + assert t.x == 125 + lltype.free(s, flavor='raw') + lltype.free(s2, flavor='raw') + +def test_carray_to_ll(): + A = lltype.Array(lltype.Signed, hints={'nolength': True}) + a = lltype.malloc(A, 10, flavor='raw') + a2 = lltype.malloc(A, 10, flavor='raw') + a[0] = 100 + a[1] = 101 + a[2] = 110 + ac = lltype2ctypes(a) + b = ctypes2lltype(lltype.Ptr(A), ac) + assert lltype.typeOf(b) == lltype.Ptr(A) + assert b == a + assert not (b != a) + assert a == b + assert not (a != b) + assert b != lltype.nullptr(A) + assert not (b == lltype.nullptr(A)) + assert lltype.nullptr(A) != b + assert not (lltype.nullptr(A) == b) + assert b != a2 + assert not (b == a2) + assert a2 != b + assert not (a2 == b) + assert b[2] == 110 + b[2] *= 2 + assert a[2] == 220 + a[2] *= 3 + assert b[2] == 660 + lltype.free(a, flavor='raw') + lltype.free(a2, flavor='raw') + +def test_strchr(): + # XXX int vs long issues + strchr = rffi.llexternal('strchr', [rffi.CCHARP, lltype.Signed], + rffi.CCHARP, + includes=['string.h']) + s = rffi.str2charp("hello world") + res = strchr(s, ord('r')) + assert res[0] == 'r' + assert res[1] == 'l' + assert res[2] == 'd' + assert res[3] == '\x00' + rffi.free_charp(s) From alecu at codespeak.net Sat Jul 14 11:58:11 2007 From: alecu at codespeak.net (alecu at codespeak.net) Date: Sat, 14 Jul 2007 11:58:11 +0200 (CEST) Subject: [pypy-svn] r45058 - in pypy/branch/flex-backend: bin translator translator/flex translator/flex/bin translator/flex/examples translator/flex/examples/bnb translator/flex/examples/bnb/data translator/flex/examples/bnb/data/images translator/flex/examples/console translator/flex/examples/console/data translator/flex/examples/console/test translator/flex/examples/data translator/flex/examples/djangoping translator/flex/examples/djangoping/templates translator/flex/examples/djangoping/test translator/flex/examples/test translator/flex/jssrc translator/flex/lib translator/flex/lib/test translator/flex/modules translator/flex/modules/pygame translator/flex/modules/test translator/flex/modules/test/html translator/flex/sandbox translator/flex/sandbox/py translator/flex/sandbox/py/pygame translator/flex/test translator/flex/tutorial Message-ID: <20070714095811.198A481B1@code0.codespeak.net> Author: alecu Date: Sat Jul 14 11:58:08 2007 New Revision: 45058 Added: pypy/branch/flex-backend/bin/flexcompile.py (contents, props changed) pypy/branch/flex-backend/translator/flex/ pypy/branch/flex-backend/translator/flex/__init__.py pypy/branch/flex-backend/translator/flex/_class.py pypy/branch/flex-backend/translator/flex/asmgen.py pypy/branch/flex-backend/translator/flex/autopath.py pypy/branch/flex-backend/translator/flex/bin/ pypy/branch/flex-backend/translator/flex/commproxy.py pypy/branch/flex-backend/translator/flex/conftest.py pypy/branch/flex-backend/translator/flex/database.py pypy/branch/flex-backend/translator/flex/examples/ pypy/branch/flex-backend/translator/flex/examples/__init__.py pypy/branch/flex-backend/translator/flex/examples/autopath.py pypy/branch/flex-backend/translator/flex/examples/bnb/ pypy/branch/flex-backend/translator/flex/examples/bnb/__init__.py pypy/branch/flex-backend/translator/flex/examples/bnb/autopath.py pypy/branch/flex-backend/translator/flex/examples/bnb/bnb.py pypy/branch/flex-backend/translator/flex/examples/bnb/data/ pypy/branch/flex-backend/translator/flex/examples/bnb/data/bnb.html pypy/branch/flex-backend/translator/flex/examples/bnb/data/images/ pypy/branch/flex-backend/translator/flex/examples/bnb/msgstruct.py pypy/branch/flex-backend/translator/flex/examples/bnb/servermessage.py pypy/branch/flex-backend/translator/flex/examples/bnb/start_bnb.py (contents, props changed) pypy/branch/flex-backend/translator/flex/examples/console/ pypy/branch/flex-backend/translator/flex/examples/console/__init__.py pypy/branch/flex-backend/translator/flex/examples/console/client.py pypy/branch/flex-backend/translator/flex/examples/console/console.py pypy/branch/flex-backend/translator/flex/examples/console/data/ pypy/branch/flex-backend/translator/flex/examples/console/data/console.html pypy/branch/flex-backend/translator/flex/examples/console/docloader.py pypy/branch/flex-backend/translator/flex/examples/console/ideas.txt pypy/branch/flex-backend/translator/flex/examples/console/play1_snippets.py pypy/branch/flex-backend/translator/flex/examples/console/session.py pypy/branch/flex-backend/translator/flex/examples/console/test/ pypy/branch/flex-backend/translator/flex/examples/console/test/test_console.py pypy/branch/flex-backend/translator/flex/examples/console/test/test_docloader.py pypy/branch/flex-backend/translator/flex/examples/console/test/test_session.py pypy/branch/flex-backend/translator/flex/examples/console/test/test_snippets.py pypy/branch/flex-backend/translator/flex/examples/data/ pypy/branch/flex-backend/translator/flex/examples/data/error.html pypy/branch/flex-backend/translator/flex/examples/data/index.html pypy/branch/flex-backend/translator/flex/examples/data/launcher.html pypy/branch/flex-backend/translator/flex/examples/data/py-web1.png (contents, props changed) pypy/branch/flex-backend/translator/flex/examples/data/style.css pypy/branch/flex-backend/translator/flex/examples/djangoping/ pypy/branch/flex-backend/translator/flex/examples/djangoping/__init__.py pypy/branch/flex-backend/translator/flex/examples/djangoping/client.py pypy/branch/flex-backend/translator/flex/examples/djangoping/manage.py (contents, props changed) pypy/branch/flex-backend/translator/flex/examples/djangoping/settings.py pypy/branch/flex-backend/translator/flex/examples/djangoping/templates/ pypy/branch/flex-backend/translator/flex/examples/djangoping/templates/index.html pypy/branch/flex-backend/translator/flex/examples/djangoping/test/ pypy/branch/flex-backend/translator/flex/examples/djangoping/test/test_build.py pypy/branch/flex-backend/translator/flex/examples/djangoping/urls.py pypy/branch/flex-backend/translator/flex/examples/djangoping/views.py pypy/branch/flex-backend/translator/flex/examples/guestbook.py (contents, props changed) pypy/branch/flex-backend/translator/flex/examples/guestbook_client.py pypy/branch/flex-backend/translator/flex/examples/over_client.py pypy/branch/flex-backend/translator/flex/examples/overmind.py (contents, props changed) pypy/branch/flex-backend/translator/flex/examples/pythonconsole.py (contents, props changed) pypy/branch/flex-backend/translator/flex/examples/serialise.py pypy/branch/flex-backend/translator/flex/examples/test/ pypy/branch/flex-backend/translator/flex/examples/test/__init__.py pypy/branch/flex-backend/translator/flex/examples/test/test_examples.py pypy/branch/flex-backend/translator/flex/function.py pypy/branch/flex-backend/translator/flex/helper.py pypy/branch/flex-backend/translator/flex/js.py pypy/branch/flex-backend/translator/flex/jsbuiltin.py pypy/branch/flex-backend/translator/flex/json.py pypy/branch/flex-backend/translator/flex/jssrc/ pypy/branch/flex-backend/translator/flex/jssrc/flex.mxml pypy/branch/flex-backend/translator/flex/jssrc/misc.as pypy/branch/flex-backend/translator/flex/jssrc/misc.js pypy/branch/flex-backend/translator/flex/jts.py pypy/branch/flex-backend/translator/flex/lib/ pypy/branch/flex-backend/translator/flex/lib/__init__.py pypy/branch/flex-backend/translator/flex/lib/server.py pypy/branch/flex-backend/translator/flex/lib/support.py pypy/branch/flex-backend/translator/flex/lib/test/ pypy/branch/flex-backend/translator/flex/lib/test/__init__.py pypy/branch/flex-backend/translator/flex/lib/test/test_server.py pypy/branch/flex-backend/translator/flex/lib/test/test_server_g.py pypy/branch/flex-backend/translator/flex/lib/test/test_support.py pypy/branch/flex-backend/translator/flex/lib/test/test_url.py pypy/branch/flex-backend/translator/flex/lib/url.py pypy/branch/flex-backend/translator/flex/log.py pypy/branch/flex-backend/translator/flex/main.py pypy/branch/flex-backend/translator/flex/metavm.py pypy/branch/flex-backend/translator/flex/modules/ pypy/branch/flex-backend/translator/flex/modules/__init__.py pypy/branch/flex-backend/translator/flex/modules/dom.py pypy/branch/flex-backend/translator/flex/modules/flex.py pypy/branch/flex-backend/translator/flex/modules/mochikit.py pypy/branch/flex-backend/translator/flex/modules/pygame/ pypy/branch/flex-backend/translator/flex/modules/pygame/__init__.py pypy/branch/flex-backend/translator/flex/modules/pygame/event.py pypy/branch/flex-backend/translator/flex/modules/pygame/font.py pypy/branch/flex-backend/translator/flex/modules/pygame/image.py pypy/branch/flex-backend/translator/flex/modules/pygame/locals.py pypy/branch/flex-backend/translator/flex/modules/pygame/mixer.py pypy/branch/flex-backend/translator/flex/modules/pygame/pygame.py pypy/branch/flex-backend/translator/flex/modules/pygame/sprite.py pypy/branch/flex-backend/translator/flex/modules/pygame/time.py pypy/branch/flex-backend/translator/flex/modules/test/ pypy/branch/flex-backend/translator/flex/modules/test/__init__.py pypy/branch/flex-backend/translator/flex/modules/test/html/ pypy/branch/flex-backend/translator/flex/modules/test/html/anim.html pypy/branch/flex-backend/translator/flex/modules/test/html/test.html pypy/branch/flex-backend/translator/flex/modules/test/test_dom.py pypy/branch/flex-backend/translator/flex/modules/test/test_mochikit.py pypy/branch/flex-backend/translator/flex/opcodes.py pypy/branch/flex-backend/translator/flex/sandbox/ pypy/branch/flex-backend/translator/flex/sandbox/MyFirst.html pypy/branch/flex-backend/translator/flex/sandbox/MyFirst.mxml pypy/branch/flex-backend/translator/flex/sandbox/chimp.py pypy/branch/flex-backend/translator/flex/sandbox/fun.py pypy/branch/flex-backend/translator/flex/sandbox/misc.as pypy/branch/flex-backend/translator/flex/sandbox/output.mxml pypy/branch/flex-backend/translator/flex/sandbox/output2.mxml pypy/branch/flex-backend/translator/flex/sandbox/py/ pypy/branch/flex-backend/translator/flex/sandbox/py/DictIter.as pypy/branch/flex-backend/translator/flex/sandbox/py/StringBuilder.as pypy/branch/flex-backend/translator/flex/sandbox/py/pygame/ pypy/branch/flex-backend/translator/flex/sandbox/py/pygame/Rect.as pypy/branch/flex-backend/translator/flex/support.py pypy/branch/flex-backend/translator/flex/test/ pypy/branch/flex-backend/translator/flex/test/__init__.py pypy/branch/flex-backend/translator/flex/test/browsertest.py pypy/branch/flex-backend/translator/flex/test/runtest.py pypy/branch/flex-backend/translator/flex/test/test_basicexternal.py pypy/branch/flex-backend/translator/flex/test/test_bltn.py pypy/branch/flex-backend/translator/flex/test/test_class.py pypy/branch/flex-backend/translator/flex/test/test_exc_operation.py pypy/branch/flex-backend/translator/flex/test/test_exception.py pypy/branch/flex-backend/translator/flex/test/test_extfunc.py pypy/branch/flex-backend/translator/flex/test/test_loops.py pypy/branch/flex-backend/translator/flex/test/test_main.py pypy/branch/flex-backend/translator/flex/test/test_rclass.py pypy/branch/flex-backend/translator/flex/test/test_rdict.py pypy/branch/flex-backend/translator/flex/test/test_rfloat.py pypy/branch/flex-backend/translator/flex/test/test_rlist.py pypy/branch/flex-backend/translator/flex/test/test_rpbc.py pypy/branch/flex-backend/translator/flex/test/test_rsnippet.py pypy/branch/flex-backend/translator/flex/test/test_rsnippet1.py pypy/branch/flex-backend/translator/flex/test/test_runtest.py pypy/branch/flex-backend/translator/flex/test/test_seq.py pypy/branch/flex-backend/translator/flex/test/test_snippet.py pypy/branch/flex-backend/translator/flex/test/test_str.py pypy/branch/flex-backend/translator/flex/test/test_typed.py pypy/branch/flex-backend/translator/flex/test/tgtest.py pypy/branch/flex-backend/translator/flex/tester.py pypy/branch/flex-backend/translator/flex/tutorial/ pypy/branch/flex-backend/translator/flex/tutorial/__init__.py pypy/branch/flex-backend/translator/flex/tutorial/step1.py (contents, props changed) pypy/branch/flex-backend/translator/flex/tutorial/step2.py (contents, props changed) pypy/branch/flex-backend/translator/flex/tutorial/step3.py (contents, props changed) Modified: pypy/branch/flex-backend/translator/driver.py Log: moving the prototype from googlecode to codespeak svn Added: pypy/branch/flex-backend/bin/flexcompile.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/bin/flexcompile.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,28 @@ +#!/usr/bin/env python +""" RPython to javascript compiler +Usage: jscompile module_to_compile [list of functions to export] + +- or - +jscompile --help to show list of options +""" + +import autopath +import sys, os + +from pypy.translator.flex.main import rpython2javascript_main, js_optiondescr + +from pypy.config.config import Config, to_optparse + +def process_options(): + jsconfig = Config(js_optiondescr) + parser = to_optparse(jsconfig, parserkwargs={"usage": __doc__}) + parser.disable_interspersed_args() + options, args = parser.parse_args() + return args, jsconfig + +if __name__ == '__main__': + args, jsconfig = process_options() + curdir = os.getcwd() + if curdir not in sys.path: + sys.path.insert(0, curdir) + rpython2javascript_main(args, jsconfig) Modified: pypy/branch/flex-backend/translator/driver.py ============================================================================== --- pypy/branch/flex-backend/translator/driver.py (original) +++ pypy/branch/flex-backend/translator/driver.py Sat Jul 14 11:58:08 2007 @@ -627,6 +627,27 @@ task_run_js = taskdef(task_run_js, ['compile_js'], 'Please manually run the generated code') + + def task_source_flex(self): + from pypy.translator.flex.js import JS + self.gen = JS(self.translator, functions=[self.entry_point], + stackless=self.config.translation.stackless) + filename = self.gen.write_source() + self.log.info("Wrote %s" % (filename,)) + task_source_flex = taskdef(task_source_flex, + [OOTYPE], + 'Generating Flex source') + + def task_compile_flex(self): + pass + task_compile_flex = taskdef(task_compile_flex, ['source_flex'], + 'Skipping Javascript compilation') + + def task_run_flex(self): + pass + task_run_flex = taskdef(task_run_flex, ['compile_flex'], + 'Please manually run the generated code') + def task_source_cli(self): from pypy.translator.cli.gencli import GenCli from pypy.translator.cli.entrypoint import get_entrypoint Added: pypy/branch/flex-backend/translator/flex/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/_class.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/_class.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,103 @@ + +""" genjs class definition +""" + +from pypy.translator.cli.node import Node +from pypy.translator.cli.cts import CTS + +import pypy.translator.flex.asmgen as asmgen + +class Class(Node): + def __init__(self, db, classdef): + self.db = db + self.cts = db.genoo.TypeSystem(db) + self.classdef = classdef + self.name = classdef._name.replace('.', '_')#[-1] + self.real_name = classdef._name + + if not self.is_root(classdef): + self.parent = self.db.pending_class(classdef._superclass) + self.order = self.parent.order + 1 + else: + self.order = 0 + + def __hash__(self): + return hash(self.classdef) + + def __eq__(self, other): + return self.classdef == other.classdef + + def __cmp__(self, other): + return cmp(self.order, other.order) + + def is_root(classdef): + return classdef._superclass is None + is_root = staticmethod(is_root) + + def get_name(self): + return self.name + + def render(self, ilasm): + if self.is_root(self.classdef) or self.name == 'Object': + return + + if self.db.class_name(self.classdef) is not None: + return # already rendered + + self.ilasm = ilasm + + old_codegenerator = ilasm.codegenerator + ilasm.codegenerator = asmgen.CodeGenerator(open("py/%s.as"%self.name, "w")) + if not self.is_root(self.classdef): + basename = self.basename(self.classdef._superclass._name) + if basename != 'Root': + ilasm.begin_class(self.name, basename) + else: + ilasm.begin_class(self.name) + else: + ilasm.begin_class(self.name) + + ilasm.begin_function(self.name, []) + # we need to copy here all the arguments + self.copy_class_attributes(ilasm) + ilasm.end_function() + + # begin to_String method + ilasm.begin_method("toString", self.name, []) + ilasm.load_str("'<%s object>'" % self.real_name) + ilasm.ret() + ilasm.end_function() + + #for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): + # cts_type = self.cts.lltype_to_cts(f_type) + #if cts_type != 'void': + # ilasm.field(f_name, cts_type) + + + + for m_name, m_meth in self.classdef._methods.iteritems(): + graph = getattr(m_meth, 'graph', None) + if graph: + f = self.db.genoo.Function(self.db, graph, m_name, is_method = True, _class = self.name) + f.render(ilasm) + else: + pass + # XXX: We want to implement an abstract method here + self.db.pending_abstract_function(m_name) + + self.db.record_class(self.classdef, self.name) + + ilasm.end_class() + ilasm.codegenerator._out.close() + ilasm.codegenerator = old_codegenerator + + def copy_class_attributes(self, ilasm): + default_values = self.classdef._fields.copy() + default_values.update(self.classdef._overridden_defaults) + for field_name, (field_type, field_value) in default_values.iteritems(): + ilasm.load_str("this") + self.db.load_const(field_type, field_value, ilasm) + ilasm.set_field(None, field_name) + + def basename(self, name): + return name.replace('.', '_')#[-1] Added: pypy/branch/flex-backend/translator/flex/asmgen.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/asmgen.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,317 @@ + +""" backend generator routines +""" + +from pypy.translator.flex.log import log + +from pypy.objspace.flow.model import Variable + +from StringIO import StringIO + +class CodeGenerator(object): + def __init__(self, out, indentstep = 4, startblock = '{', endblock = '}'): + self._out = out + self._indent = 0 + self._bol = True # begin of line + self._indentstep = indentstep + self._startblock = startblock + self._endblock = endblock + + def write(self, s, indent = 0): + indent = self._indent + (indent * self._indentstep) + + if self._bol: + self._out.write(' ' * indent) + + self._out.write(s) + self._bol = (s and s[-1] == '\n') + + def writeline(self, s=''): + self.write(s) + self.write('\n') + + def openblock(self): + self.writeline(self._startblock) + self._indent += self._indentstep + + def closeblock(self): + self._indent -= self._indentstep + self.writeline(self._endblock) + +class Queue(object): + def __init__(self, l, subst_table): + self.l = l[:] + self.subst_table = subst_table + + def pop(self): + el = self.l.pop() + return self.subst_table.get(el, el) + + def __getattr__(self,attr): + return getattr(self.l, attr) + + def __len__(self): + return len(self.l) + + def __nonzero__(self): + return len(self.l) > 0 + + def empty(self): + self.l = [] + + def __repr__(self): + return "" % (repr(self.l),) + +class AsmGen(object): + """ JS 'assembler' generator routines + """ + def __init__(self, outfile, name): + self.outfile = outfile + self.name = name + self.subst_table = {} + self.right_hand = Queue([], self.subst_table) + self.codegenerator = CodeGenerator(outfile) + + def close(self): + self.outfile.close() + + def begin_function(self, name, arglist): + args = ",".join([i[1] for i in arglist]) + self.codegenerator.write("function %s (%s) "%(name, args)) + self.codegenerator.openblock() + + def begin_method(self, name, _class, arglist): + args = ",".join(arglist) + self.codegenerator.write("%s.prototype.%s = function (%s)"%(_class, name, args)) + self.codegenerator.openblock() + + def end_function(self): + self.codegenerator.closeblock() + self.codegenerator.writeline("") + + def begin_class(self, name, base="Object"): + self.codegenerator.write("package py ") + self.codegenerator.openblock() + + self.codegenerator.write("dynamic public class %s extends %s "%(name, base)) + self.codegenerator.openblock() + + def end_class(self): + self.codegenerator.closeblock() + self.codegenerator.closeblock() + self.codegenerator.writeline("") + + def load_arg(self, v): + self.right_hand.append(v.name) + + def store_local(self, v): + self.store_name(v.name) + + def store_name(self, name): + name = self.subst_table.get(name, name) + element = self.right_hand.pop() + if element != name: + self.codegenerator.writeline("%s = %s;"%(name, element)) + + def load_local(self, v): + self.right_hand.append(v.name) + + def load_const(self, v): + self.right_hand.append(v) + + def ret(self): + self.codegenerator.writeline("return ( %s );"%self.right_hand.pop()) + + def emit(self, opcode, *args): + v1 = self.right_hand.pop() + v2 = self.right_hand.pop() + self.right_hand.append("(%s%s%s)"%(v2, opcode, v1)) + + def call(self, func): + func_name, args = func + l = [self.right_hand.pop() for i in xrange(len(args))] + l.reverse() + real_args = ",".join(l) + self.right_hand.append("%s ( %s )" % (func_name, real_args)) + + def branch_if(self, exitcase): + def mapping(exitc): + if exitc in ['True', 'False']: + return exitc.lower() + return exitc + + arg = self.right_hand.pop() + if hasattr(arg,'name'): + arg_name = self.subst_table.get(arg.name, arg.name) + else: + arg_name = arg + self.branch_if_string("%s == %s"%(arg_name, mapping(str(exitcase)))) + + def branch_if_string(self, arg): + self.codegenerator.writeline("if (%s)"%arg) + self.codegenerator.openblock() + + def branch_elsif_string(self, arg): + self.codegenerator.closeblock() + self.codegenerator.writeline("else if (%s)"%arg) + self.codegenerator.openblock() + + def branch_elsif(self, arg, exitcase): + self.codegenerator.closeblock() + self.branch_if(arg, exitcase, "else if") + + def branch_while(self, arg, exitcase): + def mapping(exitc): + if exitc in ['True', 'False']: + return exitc.lower() + return exitc + + arg_name = self.subst_table.get(arg.name, arg.name) + self.codegenerator.write("while ( %s == %s )"%(arg_name, mapping(str(exitcase)))) + self.codegenerator.openblock() + + def branch_while_true(self): + self.codegenerator.write("while (true)") + self.codegenerator.openblock() + + def branch_else(self): + self.right_hand.pop() + self.codegenerator.closeblock() + self.codegenerator.write("else") + self.codegenerator.openblock() + + def close_branch(self): + self.codegenerator.closeblock() + + def label(self, *args): + self.codegenerator.openblock() + + def change_name(self, from_name, to_name): + if isinstance(from_name,Variable) and isinstance(to_name,Variable): + self.subst_table[from_name.name] = to_name.name + + def cast_function(self, name, num): + # FIXME: redundancy with call + args = [self.right_hand.pop() for i in xrange(num)] + args.reverse() + arg_list = ",".join(args) + self.right_hand.append("%s ( %s )"%(name, arg_list)) + + def prefix_op(self, st): + self.right_hand.append("%s%s"%(st, self.right_hand.pop())) + + #def field(self, f_name, cts_type): + # pass + + def set_locals(self, loc_string): + if loc_string != '': + self.codegenerator.writeline("var %s;"%loc_string) + + def set_static_field(self, _type, namespace, _class, varname): + self.codegenerator.writeline("%s.prototype.%s = %s;"%(_class, varname, self.right_hand.pop())) + + def set_field(self, useless_parameter, name): + v = self.right_hand.pop() + self.codegenerator.writeline("%s.%s = %s;"%(self.right_hand.pop(), name, v)) + #self.right_hand.append(None) + + def call_method(self, obj, name, signature): + l = [self.right_hand.pop() for i in signature] + l.reverse() + args = ",".join(l) + self.right_hand.append("%s.%s(%s)"%(self.right_hand.pop(), name, args)) + + def get_field(self, name): + self.right_hand.append("%s.%s"%(self.right_hand.pop(), name)) + + def new(self, obj): + #log("New: %r"%obj) + self.right_hand.append("new %s()"%obj) + + def runtimenew(self): + self.right_hand.append("new %s()" % self.right_hand.pop()) + + def load_self(self): + self.right_hand.append("this") + + def store_void(self): + if not len(self.right_hand): + return + v = self.right_hand.pop() + if v is not None and v.find('('): + self.codegenerator.writeline(v+";") + + def begin_consts(self, name): + # load consts, maybe more try to use stack-based features? + self.codegenerator.writeline("var %s = {}"%name) + self.codegenerator.writeline("function __load_consts_flex() ") + self.codegenerator.openblock() + + def end_consts(self): + self.codegenerator.closeblock() + + def new_obj(self): + self.right_hand.append("{}") + + def new_list(self): + self.right_hand.append("[]") + + # FIXME: will refactor later + load_str = load_const + + def list_setitem(self): + item = self.right_hand.pop() + value = self.right_hand.pop() + lst = self.right_hand.pop() + self.right_hand.append("%s[%s]=%s"%(lst, item, value)) + + def list_getitem(self): + item = self.right_hand.pop() + lst = self.right_hand.pop() + self.right_hand.append("%s[%s]"%(lst, item)) + + def load_void(self): + self.right_hand.append("undefined") + + def load_void_obj(self): + self.right_hand.append("{}") + + def begin_try(self): + self.codegenerator.write("try ") + self.codegenerator.openblock() + + def catch(self): + self.codegenerator.closeblock() + self.codegenerator.write("catch (exc)") + self.codegenerator.openblock() + + def begin_for(self): + self.codegenerator.writeline("var block = 0;") + self.codegenerator.write("for(;;)") + self.codegenerator.openblock() + self.codegenerator.write("switch(block)") + self.codegenerator.openblock() + + def write_case(self, num): + self.codegenerator.writeline("case %d:"%num) + + def jump_block(self, num): + self.codegenerator.writeline("block = %d;"%num) + self.codegenerator.writeline("break;") + + def end_for(self): + self.codegenerator.closeblock() + self.codegenerator.closeblock() + + def inherits(self, subclass_name, parent_name): + self.codegenerator.writeline("inherits(%s,%s);"%(subclass_name, parent_name)) + + def throw(self, var): + self.throw_real(var.name) + + def throw_real(self, s): + # use throwit wrapper function... because of compiler weirdness with flex compiler. + self.codegenerator.writeline("throwit(%s);"%s) + + def clean_stack(self): + self.right_hand.empty() Added: pypy/branch/flex-backend/translator/flex/autopath.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/autopath.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,114 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories don't have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.realpath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.realpath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + break + else: + raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) + + pypy_root = os.path.join(head, '') + try: + sys.path.remove(head) + except ValueError: + pass + sys.path.insert(0, head) + + munged = {} + for name, mod in sys.modules.items(): + if '.' in name: + continue + fn = getattr(mod, '__file__', None) + if not isinstance(fn, str): + continue + newname = os.path.splitext(os.path.basename(fn))[0] + if not newname.startswith(part + '.'): + continue + path = os.path.join(os.path.dirname(os.path.realpath(fn)), '') + if path.startswith(pypy_root) and newname != part: + modpaths = os.path.normpath(path[len(pypy_root):]).split(os.sep) + if newname != '__init__': + modpaths.append(newname) + modpath = '.'.join(modpaths) + if modpath not in sys.modules: + munged[modpath] = mod + + for name, mod in munged.iteritems(): + if name not in sys.modules: + sys.modules[name] = mod + if '.' in name: + prename = name[:name.rfind('.')] + postname = name[len(prename)+1:] + if prename not in sys.modules: + __import__(prename) + if not hasattr(sys.modules[prename], postname): + setattr(sys.modules[prename], postname, mod) + + return partdir, this_dir + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Added: pypy/branch/flex-backend/translator/flex/commproxy.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/commproxy.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,148 @@ + +""" Communication proxy rendering +""" + + +from pypy.objspace.flow.model import Variable, Constant +from pypy.rpython.ootypesystem.bltregistry import ArgDesc + +GET_METHOD_BODY = """ +%(class)s.prototype.%(method)s = function ( %(args)s ) { + var data,str; + var x = new XMLHttpRequest(); + data = %(data)s; + str = "" + for(i in data) { + if (data[i]) { + if (str.length == 0) { + str += "?"; + } else { + str += "&"; + } + str += escape(i) + "=" + escape(data[i].toString()); + } + } + //logDebug('%(call)s'+str); + x.open("GET", '%(call)s' + str, true); + x.setRequestHeader("Content-type", "application/x-www-form-urlencoded"); + x.onreadystatechange = function () { %(real_callback)s(x, callback) }; + //x.setRequestHeader("Connection", "close"); + //x.send(data); + x.send(null); +} +""" + +POST_METHOD_BODY = """ +%(class)s.prototype.%(method)s = function ( %(args)s ) { + var data,str; + var x = new XMLHttpRequest(); + data = %(data)s; + str = "" + for(i in data) { + if (data[i]) { + if (str.length != 0) { + str += "&"; + } + str += escape(i) + "=" + escape(data[i].toString()); + } + } + //logDebug('%(call)s'+str); + x.open("POST", '%(call)s', true); + //x.setRequestHeader("Content-type", "application/x-www-form-urlencoded"); + x.onreadystatechange = function () { %(real_callback)s(x, callback) }; + //x.setRequestHeader("Connection", "close"); + //logDebug(str); + x.send(str); + //x.send(null); +} +""" + +CALLBACK_BODY = """ +function %(real_callback)s (x, cb) { + var d; + if (x.readyState == 4) { + if (x.responseText) { + eval ( "d = " + x.responseText ); + cb(d); + } else { + cb({}); + } + } +} +""" + +CALLBACK_XML_BODY = """ +function %(real_callback)s (x, cb) { + if (x.readyState == 4) { + if (x.responseXML) { + cb(x.responseXML.documentElement); + } else { + cb(null); + } + } +} +""" + +MOCHIKIT_BODY = """ +%(class)s.prototype.%(method)s = function ( %(args)s ) { + var data,str; + data = %(data)s; + loadJSONDoc('%(call)s', data).addCallback(callback); +} +""" + +USE_MOCHIKIT = True # FIXME: some option? + +class XmlHttp(object): + """ Class for rendering xmlhttp request communication + over normal js code + """ + def __init__(self, ext_obj, name, use_xml=False, base_url="", method="GET"): + self.ext_obj = ext_obj + self.name = name + self.use_xml = use_xml + self.base_url = base_url + obj = self.ext_obj._TYPE._class_ + if not base_url and hasattr(obj, '_render_base_path'): + self.base_url = obj._render_base_path + self.method = method + + def render(self, ilasm): + self.render_body(ilasm) + for method_name, method in self.ext_obj._TYPE._class_._methods.iteritems(): + self.render_method(method_name, method, ilasm) + + def render_body(self, ilasm): + ilasm.begin_function(self.name, []) + ilasm.end_function() + + def render_method(self, method_name, method, ilasm): + args, retval = method.args, method.retval.name + if len(args) == 0 or args[-1].name != 'callback': + args.append(ArgDesc('callback', lambda : None)) + real_args = list(arg.name for arg in args) + # FIXME: dirty JS here + data = "{%s}" % ",".join(["'%s':%s" % (i,i) for i in real_args if i != 'callback']) + real_callback = Variable("callback").name + if len(self.base_url) > 0 and not self.base_url.endswith("/"): + url = self.base_url + "/" +method_name + else: + url = self.base_url + method_name + + METHOD_BODY = globals()[self.method + "_METHOD_BODY"] + if USE_MOCHIKIT and self.use_xml: + assert 0, "Cannot use mochikit and xml requests at the same time" + if USE_MOCHIKIT and self.method == "POST": + assert 0, "Cannot use mochikit with POST method" + if USE_MOCHIKIT: + ilasm.codegenerator.write(MOCHIKIT_BODY % {'class':self.name, 'method':method_name,\ + 'args':','.join(real_args), 'data':data, 'call':url}) + else: + if not self.use_xml: + callback_body = CALLBACK_BODY + else: + callback_body = CALLBACK_XML_BODY + ilasm.codegenerator.write(callback_body % {'real_callback':real_callback}) + ilasm.codegenerator.write(METHOD_BODY % {'class':self.name, 'method':method_name,\ + 'args':",".join(real_args), 'data':data, 'call':url,\ + 'real_callback':real_callback}) Added: pypy/branch/flex-backend/translator/flex/conftest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/conftest.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ +import py + +Option = py.test.config.Option + +option = py.test.config.addoptions("pypy-ojs options", + Option('--use-browser', action="store", dest="browser", type="string", + default="", help="run Javascript tests in your default browser"), + + Option('--tg', action="store_true", dest="tg", default=False, + help="Use TurboGears machinery for testing") + ) Added: pypy/branch/flex-backend/translator/flex/database.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/database.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,504 @@ + +""" genjs constant database module +""" + +from pypy.rpython.ootypesystem import ootype +from pypy.translator.flex.opcodes import opcodes +from pypy.translator.flex.function import Function +from pypy.translator.flex.log import log +from pypy.translator.flex._class import Class +from pypy.translator.flex.support import JavascriptNameManager + +from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong, typeOf +from pypy.rpython.lltypesystem.lltype import Char, UniChar +from pypy.rpython.ootypesystem import ootype +from pypy.rpython.ootypesystem import bltregistry + +from pypy.objspace.flow.model import Variable, Constant +from pypy.translator.flex.modules import dom +from pypy.translator.flex.commproxy import XmlHttp + +try: + set +except NameError: + from sets import Set as set + +class LowLevelDatabase(object): + def __init__(self, genoo): + self._pending_nodes = set() + self.genoo = genoo + self._rendered_nodes = set() + self.classes = {} # classdef --> class_name + self.functions = {} # graph --> function_name + self.function_names = {} # graph --> real_name + self.methods = {} # graph --> method_name + self.consts = {} # value --> const_name + self.reverse_consts = {} + self.const_names = set() + self.rendered = set() + self.const_var = Variable("__consts") + self.name_manager = JavascriptNameManager(self) + self.pending_consts = [] + self.cts = self.genoo.TypeSystem(self) + self.proxies = [] + + def is_primitive(self, type_): + if type_ in [Void, Bool, Float, Signed, Unsigned, SignedLongLong, UnsignedLongLong, Char, UniChar, ootype.StringBuilder] or \ + isinstance(type_,ootype.StaticMethod): + return True + return False + + def pending_function(self, graph): + self.pending_node(self.genoo.Function(self, graph)) + + def pending_abstract_function(self, name): + pass + # XXX we want to implement it at some point (maybe...) + + def pending_class(self, classdef): + c = Class(self, classdef) + self.pending_node(c) + return c + + def pending_record(self, record): + r = Record(self, record) + self.pending_node(r) + return r.get_name() + + def pending_node(self, node): + if node in self._pending_nodes or node in self._rendered_nodes: + return + self._pending_nodes.add(node) + + def record_function(self, graph, name): + self.functions[graph] = name + + def get_uniquename(self, graph, name): + try: + return self.function_names[graph] + except KeyError: + real_name = self.name_manager.uniquename(name, lenmax=1111111) + self.function_names[graph] = real_name + return real_name + + def record_class(self, classdef, name): + self.classes[classdef] = name + + def register_comm_proxy(self, proxy_const, *args): + """ Register external object which should be rendered as + method call + """ + self.proxies.append(XmlHttp(proxy_const, *args)) + + def graph_name(self, graph): + return self.functions.get(graph, None) + + def class_name(self, classdef): + return self.classes.get(classdef, None) + + def record_const(self, value, type_ = None, retval='name'): + if type_ is None: + type_ = typeOf(value) + if self.is_primitive(type_): + return None + const = AbstractConst.make(self, value) + if not const: + return None + try: + if retval == 'name': + return self.consts[const] + else: + self.consts[const] + return self.reverse_consts[self.consts[const]] + except KeyError: + if self.genoo.config.translation.verbose: + log("New const:%r"%value) + if isinstance(value, ootype._string): + log(value._str) + else: + log.dot() + name = const.get_name() + if name in self.const_names: + name += '__%d' % len(self.consts) + self.consts[const] = name + self.reverse_consts[name] = const + self.const_names.add(name) + self.pending_consts.append((const,name)) + if retval == 'name': + return name + else: + return const + + def gen_constants(self, ilasm, pending): + try: + while True: + const,name = self.pending_consts.pop() + const.record_fields() + except IndexError: + pass + + if pending: + return + + if not self.rendered: + ilasm.begin_consts(self.const_var.name) + + def generate_constants(consts): + all_c = [const for const,name in consts.iteritems()] + dep_ok = set() + while len(all_c) > 0: + const = all_c.pop() + if const not in self.rendered: + to_render = True + if hasattr(const, 'depends_on') and const.depends_on: + for i in const.depends_on: + if i not in self.rendered and i not in dep_ok: + assert i.depends is None or const in i.depends + to_render = False + continue + + if to_render and (not hasattr(const, 'depends')) or (not const.depends) or const in dep_ok: + yield const,consts[const] + self.rendered.add(const) + else: + all_c.append(const) + for i in const.depends: + all_c.append(i) + dep_ok.add(const) + + # We need to keep track of fields to make sure + # our items appear earlier than us + to_init = [] + for const, name in generate_constants(self.consts): + if self.genoo.config.translation.verbose: + log("Recording %r %r"%(const,name)) + else: + log.dot() + ilasm.load_local(self.const_var) + const.init(ilasm) + ilasm.set_field(None, name) + ilasm.store_void() + to_init.append((const, name)) + #ilasm.field(name, const.get_type(), static=True) + for const, name in to_init: + const.init_fields(ilasm, self.const_var, name) + + + + def load_const(self, type_, value, ilasm): + if self.is_primitive(type_): + ilasm.load_const(self.cts.primitive_repr(type_, value)) + else: + try: + return self.consts[BuiltinConst(value)] + except KeyError: + name = self.record_const(value) + ilasm.load_local(self.const_var) + ilasm.get_field(name) + #assert False, 'Unknown constant %s' % const + + +class AbstractConst(object): + def __init__(self, db, const): + self.db = db + self.const = const + self.cts = db.genoo.TypeSystem(db) + self.depends = set() + self.depends_on = set() + + def __hash__(self): + return hash(self.get_key()) + + def __eq__(self, other): + return (other.__class__ is self.__class__ and + other.get_key() == self.get_key()) + + def __ne__(self, other): + return not (self == other) + + def make(db, const): + if isinstance(const, ootype._view): + static_type = const._TYPE + const = const._inst + else: + static_type = None + + if isinstance(const, ootype._instance): + return InstanceConst(db, const, static_type) + elif isinstance(const, ootype._list): + return ListConst(db, const) + elif isinstance(const, ootype._record): + return RecordConst(db, const) + elif isinstance(const, ootype._string): + return StringConst(db, const) + elif isinstance(const, ootype._dict): + return DictConst(db, const) + elif isinstance(const, bltregistry._external_inst): + return ExtObject(db, const) + elif isinstance(const, ootype._class): + if const._INSTANCE: + return ClassConst(db, const) + else: + return None + else: + assert False, 'Unknown constant: %s %r' % (const, typeOf(const)) + make = staticmethod(make) + + def get_name(self): + pass + + def get_type(self): + pass + + def init(self, ilasm): + pass + + def init_fields(self, ilasm, const_var, name): + pass + + def record_fields(self): + pass + +class InstanceConst(AbstractConst): + def __init__(self, db, obj, static_type): + self.depends = set() + self.depends_on = set() + self.db = db + self.cts = db.genoo.TypeSystem(db) + self.obj = obj + if static_type is None: + self.static_type = obj._TYPE + else: + self.static_type = static_type + self.cts.lltype_to_cts(obj._TYPE) # force scheduling of obj's class + + def get_key(self): + return self.obj + + def get_name(self): + return self.obj._TYPE._name.replace('.', '_') + + def get_type(self): + return self.cts.lltype_to_cts(self.static_type) + + def init(self, ilasm): + if not self.obj: + ilasm.load_void() + return + + classdef = self.obj._TYPE + try: + classdef._hints['_suggested_external'] + ilasm.new(classdef._name.split(".")[-1]) + except KeyError: + ilasm.new(classdef._name.replace(".", "_")) + + def record_fields(self): + if not self.obj: + return + INSTANCE = self.obj._TYPE + #while INSTANCE: + for i, (_type, val) in INSTANCE._allfields().items(): + if _type is not ootype.Void: + name = self.db.record_const(getattr(self.obj, i), _type, 'const') + if name is not None: + self.depends.add(name) + name.depends_on.add(self) + + def init_fields(self, ilasm, const_var, name): + if not self.obj: + return + + INSTANCE = self.obj._TYPE + #while INSTANCE: + for i, (_type, el) in INSTANCE._allfields().items(): + if _type is not ootype.Void: + ilasm.load_local(const_var) + self.db.load_const(_type, getattr(self.obj, i), ilasm) + ilasm.set_field(None, "%s.%s"%(name, i)) + ilasm.store_void() + +class RecordConst(AbstractConst): + def get_name(self): + return "const_tuple" + + def init(self, ilasm): + if not self.const: + ilasm.load_void() + else: + ilasm.new_obj() + + def record_fields(self): + if not self.const: + return + + for i in self.const._items: + name = self.db.record_const(self.const._items[i], None, 'const') + if name is not None: + self.depends.add(name) + name.depends_on.add(self) + + def get_key(self): + return self.const + + def init_fields(self, ilasm, const_var, name): + if not self.const: + return + + #for i in self.const.__dict__["_items"]: + for i in self.const._items: + ilasm.load_local(const_var) + el = self.const._items[i] + self.db.load_const(typeOf(el), el, ilasm) + ilasm.set_field(None, "%s.%s"%(name, i)) + ilasm.store_void() + +class ListConst(AbstractConst): + + def get_name(self): + return "const_list" + + def init(self, ilasm): + if not self.const: + ilasm.load_void() + else: + ilasm.new_list() + + def record_fields(self): + if not self.const: + return + + for i in self.const._list: + name = self.db.record_const(i, None, 'const') + if name is not None: + self.depends.add(name) + name.depends_on.add(self) + + def get_key(self): + return self.const + + def init_fields(self, ilasm, const_var, name): + if not self.const: + return + + for i in xrange(len(self.const._list)): + ilasm.load_str("%s.%s"%(const_var.name, name)) + el = self.const._list[i] + self.db.load_const(typeOf(el), el, ilasm) + self.db.load_const(typeOf(i), i, ilasm) + ilasm.list_setitem() + ilasm.store_void() + +class StringConst(AbstractConst): + def get_name(self): + return "const_str" + + def get_key(self): + return self.const._str + + def init(self, ilasm): + if self.const: + s = self.const._str + # do some escaping + #s = s.replace("\n", "\\n").replace('"', '\"') + #s = repr(s).replace("\"", "\\\"") + ilasm.load_str("%s" % repr(s)) + else: + ilasm.load_str("undefined") + + def init_fields(self, ilasm, const_var, name): + pass + +class ClassConst(AbstractConst): + def __init__(self, db, const): + super(ClassConst, self).__init__(db, const) + self.cts.lltype_to_cts(const._INSTANCE) # force scheduling of class + + def get_name(self): + return "const_class" + + def get_key(self): + return self.get_name() + + def get_name(self): + return self.const._INSTANCE._name.replace(".", "_") + + def init(self, ilasm): + ilasm.load_const("%s" % self.get_name()) + + #def init_fields(self, ilasm, const_var, name): + # pass + +class BuiltinConst(AbstractConst): + def __init__(self, name): + self.name = name + + def get_key(self): + return self.name + + def get_name(self): + return self.name + + def init_fields(self, *args): + pass + + def init(self, ilasm): + ilasm.load_str(self.name) + +class DictConst(RecordConst): + def record_const(self, co): + name = self.db.record_const(co, None, 'const') + if name is not None: + self.depends.add(name) + name.depends_on.add(self) + + def record_fields(self): + if not self.const: + return + + for i in self.const._dict: + self.record_const(i) + self.record_const(self.const._dict[i]) + + def init_fields(self, ilasm, const_var, name): + if not self.const: + return + + for i in self.const._dict: + ilasm.load_str("%s.%s"%(const_var.name, name)) + el = self.const._dict[i] + self.db.load_const(typeOf(el), el, ilasm) + self.db.load_const(typeOf(i), i, ilasm) + ilasm.list_setitem() + ilasm.store_void() + +class ExtObject(AbstractConst): + def __init__(self, db, const): + self.db = db + self.const = const + self.name = self.get_name() + self.depends = set() + self.depends_on = set() + + def get_key(self): + return self.name + + def get_name(self): + return self.const._TYPE._name.split('.')[-1][:-2] + + def init(self, ilasm): + _class = self.const._TYPE._class_ + if getattr(_class, '_render_xmlhttp', False): + use_xml = getattr(_class, '_use_xml', False) + base_url = getattr(_class, '_base_url', "") # XXX: should be + method = getattr(_class, '_use_method', 'GET') + # on per-method basis + self.db.register_comm_proxy(self.const, self.name, use_xml, base_url, method) + ilasm.new(self.get_name()) + else: + # Otherwise they just exist, or it's not implemented + if not hasattr(self.const.value, '_render_name'): + raise ValueError("Prebuilt constant %s has no attribute _render_name," + "don't know how to render" % self.const.value) + ilasm.load_str(self.const.value._render_name) Added: pypy/branch/flex-backend/translator/flex/examples/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/__init__.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1 @@ +# \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/examples/autopath.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/autopath.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,114 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories don't have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.realpath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.realpath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + break + else: + raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) + + pypy_root = os.path.join(head, '') + try: + sys.path.remove(head) + except ValueError: + pass + sys.path.insert(0, head) + + munged = {} + for name, mod in sys.modules.items(): + if '.' in name: + continue + fn = getattr(mod, '__file__', None) + if not isinstance(fn, str): + continue + newname = os.path.splitext(os.path.basename(fn))[0] + if not newname.startswith(part + '.'): + continue + path = os.path.join(os.path.dirname(os.path.realpath(fn)), '') + if path.startswith(pypy_root) and newname != part: + modpaths = os.path.normpath(path[len(pypy_root):]).split(os.sep) + if newname != '__init__': + modpaths.append(newname) + modpath = '.'.join(modpaths) + if modpath not in sys.modules: + munged[modpath] = mod + + for name, mod in munged.iteritems(): + if name not in sys.modules: + sys.modules[name] = mod + if '.' in name: + prename = name[:name.rfind('.')] + postname = name[len(prename)+1:] + if prename not in sys.modules: + __import__(prename) + if not hasattr(sys.modules[prename], postname): + setattr(sys.modules[prename], postname, mod) + + return partdir, this_dir + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Added: pypy/branch/flex-backend/translator/flex/examples/bnb/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/examples/bnb/autopath.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/bnb/autopath.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,114 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories don't have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.realpath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.realpath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + break + else: + raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) + + pypy_root = os.path.join(head, '') + try: + sys.path.remove(head) + except ValueError: + pass + sys.path.insert(0, head) + + munged = {} + for name, mod in sys.modules.items(): + if '.' in name: + continue + fn = getattr(mod, '__file__', None) + if not isinstance(fn, str): + continue + newname = os.path.splitext(os.path.basename(fn))[0] + if not newname.startswith(part + '.'): + continue + path = os.path.join(os.path.dirname(os.path.realpath(fn)), '') + if path.startswith(pypy_root) and newname != part: + modpaths = os.path.normpath(path[len(pypy_root):]).split(os.sep) + if newname != '__init__': + modpaths.append(newname) + modpath = '.'.join(modpaths) + if modpath not in sys.modules: + munged[modpath] = mod + + for name, mod in munged.iteritems(): + if name not in sys.modules: + sys.modules[name] = mod + if '.' in name: + prename = name[:name.rfind('.')] + postname = name[len(prename)+1:] + if prename not in sys.modules: + __import__(prename) + if not hasattr(sys.modules[prename], postname): + setattr(sys.modules[prename], postname, mod) + + return partdir, this_dir + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Added: pypy/branch/flex-backend/translator/flex/examples/bnb/bnb.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/bnb/bnb.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,267 @@ + +""" xmlhttp controllers, usefull for testing +""" + +import py +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc + +from pypy.translator.js.examples.bnb.servermessage import log, ServerMessage,\ + PMSG_INLINE_FRAME, PMSG_DEF_ICON +from pypy.translator.js.examples.bnb.msgstruct import * +from pypy.translator.js.lib.support import callback +from pypy.translator.js.lib import server + +import re, time, sys, os, urllib, socket, copy, md5, random + +class SpriteManager(object): + def __init__(self): + self.sprite_sets = {} + self.positions = {} + self.num = 0 + self.next_pos = {} + self.last_seen = set() + self.seen = set() + self.num_frame = 0 + self.z_index = {} + + def def_icon(self, icon_code): + self.sprite_sets[icon_code] = [] + + def get_frame_number(self): + self.num_frame += 1 + + def get_sprite(self, icon_code, x, y): + try: + to_ret = self.positions[(icon_code, x, y)] + del self.positions[(icon_code, x, y)] + self.next_pos[(icon_code, x, y)] = to_ret + self.seen.add((icon_code, to_ret)) + return "still", to_ret + except KeyError: + try: + try: + to_ret = self.sprite_sets[icon_code].pop() + except KeyError: + self.def_icon(icon_code) + raise IndexError + self.next_pos[(icon_code, x, y)] = to_ret + self.seen.add((icon_code, to_ret)) + return "move", to_ret + except IndexError: + next = self.num + self.num += 1 + self.next_pos[(icon_code, x, y)] = next + self.seen.add((icon_code, next)) + return "new", next + + def end_frame(self): + self.positions = copy.deepcopy(self.next_pos) + self.next_pos = {} + to_ret = [] + for ic, i in self.last_seen - self.seen: + self.sprite_sets[ic].append(i) + to_ret.append(i) + self.last_seen = self.seen + self.seen = set() + return to_ret + +class ExportedMethods(server.ExportedMethods): + _serverMessage = {} + _spriteManagers = {} + + host = 'localhost' + try: + port = re.findall('value=".*"', urllib.urlopen('http://%s:8000' % host).read())[0] + port = int(port[7:-1]) + except IOError: + log("ERROR: Can't connect to BnB server on %s:8000" % host) +# sys.exit() + except IndexError: + log("ERROR: Connected to BnB server but unable to detect a running game") +# sys.exit() + + #def _close(self, sessionid): + # if sessionid in self._serverMessage: + # sm = self.serverMessage() + # if sm.socket is not None: + # sm.socket.close() + # del self._serverMessage[sessionid] + def get_sprite_manager(self, sessionid): + return self._spriteManagers[sessionid] + + def sessionSocket(self, sessionid, close=False): + sm = self.serverMessage(sessionid) + if sm.socket is None: + player_id = 0 #XXX hardcoded for now + sm.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sm.socket.connect((self.host, self.port)) + sm.socket.send(message(CMSG_PROTO_VERSION, 2)) #, version a kuku + sm.socket.send(message(CMSG_ENABLE_SOUND, 0)) #, has_sound + sm.socket.send(message(CMSG_ENABLE_MUSIC, 0)) #, has_music + sm.socket.send(message(CMSG_UDP_PORT, "\\")) #, port + sm.socket.send(message(CMSG_PING)) #so server starts sending data + return sm.socket + + def _closeIdleConnections(self): + t = time.time() - 20.0 #20 seconds until considered idle + for sessionid, sm in self._serverMessage.items(): + if sm.last_active < t: + log("Close connection with sessionid %s because it was idle for %.1f seconds" % ( + sessionid, time.time() - sm.last_active)) + if sm.socket is not None: + sm.socket.close() + del self._serverMessage[sessionid] + + def serverMessage(self, sessionid): + self._closeIdleConnections() + if sessionid not in self._serverMessage: + self._serverMessage[sessionid] = ServerMessage('data/images') + return self._serverMessage[sessionid] + + @callback(retval=None) + def player_name(self, player_id=0, name="", sessionid=""): + log("Changing player #%s name to %s" % (player_id, name)) + socket = self.sessionSocket(sessionid) + socket.send(message(CMSG_PLAYER_NAME, int(player_id), name)) + + @callback(retval=None) + def add_player(self, player_id=0, sessionid=""): + log("Adding player " + player_id) + socket = self.sessionSocket(sessionid) + socket.send(message(CMSG_ADD_PLAYER, int(player_id))) + + @callback(retval=None) + def remove_player(self, player_id=0, sessionid=""): + log("Remove player " + player_id) + socket = self.sessionSocket(sessionid) + socket.send(message(CMSG_REMOVE_PLAYER, int(player_id))) + + @callback(retval=str) + def initialize_session(self): + sessionid = md5.md5(str(random.random())).hexdigest() + self._create_session(sessionid) + return sessionid + + def _create_session(self, sessionid): + sm = ServerMessage('data/images/') + self._serverMessage[sessionid] = sm + self._spriteManagers[sessionid] = SpriteManager() + return sessionid + + @callback(retval={str:[{str:str}]}) + def get_message(self, sessionid="", player_id=0, keys=""): + """ This one is long, ugly and obscure + """ + #XXX hangs if not first sending CMSG_PING! + try: + sm = self.serverMessage(sessionid) + except KeyError: + self._create_session(sessionid) + sm = self.serverMessage(sessionid) + data = sm.data + sock = self.sessionSocket(sessionid) + while True: + try: + data += sock.recv(4096, socket.MSG_DONTWAIT) + except: + break + while sm.n_header_lines > 0 and '\n' in data: + sm.n_header_lines -= 1 + header_line, data = data.split('\n',1) + #log('RECEIVED HEADER LINE: %s' % header_line) + + #log('RECEIVED DATA CONTAINS %d BYTES' % len(data)) + messages = [] + while data: + values, data = decodemessage(data) + if not values: + break # incomplete message + messageOutput = sm.dispatch(*values) + if messageOutput: + if type(messageOutput) is type([]): + messages += messageOutput + else: + messages.append(messageOutput) + sm.data = data + + len_before = len(messages) + inline_frames = [i for i,msg in enumerate(messages) if msg['type'] == PMSG_INLINE_FRAME] + for i in reversed(inline_frames[:-1]): + del messages[i] + + to_append = [] + sprite_manager = self.get_sprite_manager(sessionid) + + sm_restart = 0 + if player_id != -1: + if keys: + for i in keys.split(":"): + self.sessionSocket(sessionid).\ + send(message(CMSG_KEY, int(player_id), int(i))) + + def get_partial_frame(next, z_num): + new_sprite, s_num = sprite_manager.get_sprite(*next) + if new_sprite == 'new': + to_append.append({'type':'ns', 's':s_num, 'icon_code':str(next[0]), 'x':str(next[1]), 'y':str(next[2]), 'z':z_num}) + sprite_manager.z_index[s_num] = z_num + elif new_sprite == 'move': + to_append.append({'type':'sm', 's':str(s_num), 'x':str(next[1]), 'y':str(next[2]), 'z':z_num}) + sprite_manager.z_index[s_num] = z_num + else: + if sprite_manager.z_index[s_num] != z_num: + to_append.append({'type':'zindex', 's':s_num, 'z':z_num}) + sprite_manager.z_index[s_num] = z_num + return s_num + + z_num = 0 + for i, msg in enumerate(messages): + if msg['type'] == PMSG_INLINE_FRAME: + for next in msg['sprites']: + s_num = get_partial_frame(next, z_num) + z_num += 1 + del messages[i] + + empty_frame = False + if sprite_manager.seen == set([]): + empty_frame = True + + if not empty_frame: + for i in sprite_manager.end_frame(): + to_append.append({'type':'ds', 's':str(i)}) + messages += to_append + return dict(messages=messages, add_data=[{'n':sm.count(), 'sm_restart':sm_restart}]) + +exported_methods = ExportedMethods() + +class BnbHandler(server.Collection): + """ BnB server handler + """ + exported_methods = exported_methods + static_dir = py.path.local(__file__).dirpath().join("data") + + index = server.Static(static_dir.join("bnb.html")) + images = server.StaticDir("data/images", type="image/png") + + def source_js(self): + return "text/javascript", self.server.source + source_js.exposed = True + + MochiKit = server.StaticDir('MochiKit') + + #@turbogears.expose(format='json') + + #@turbogears.expose(format='json') + +## @turbogears.expose(format='json') +## def key(self, player_id, keynum): +## self.sessionSocket().send(message(CMSG_KEY, int(player_id), int(keynum))) +## return dict() + + #@turbogears.expose(format='json') + def close(self): + self._close() + return dict() + +class BnbRoot(server.NewHandler): + application = BnbHandler() + Added: pypy/branch/flex-backend/translator/flex/examples/bnb/data/bnb.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/bnb/data/bnb.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ + + + + + Bub'n'bros + + + + + + Added: pypy/branch/flex-backend/translator/flex/examples/bnb/msgstruct.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/bnb/msgstruct.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,78 @@ +from struct import pack, unpack, calcsize + +try: + from localmsg import PORTS +except ImportError: + PORTS = {} +try: + from localmsg import HOSTNAME +except ImportError: + from socket import gethostname + HOSTNAME = gethostname() + + +MSG_WELCOME = "Welcome to gamesrv.py(3) !\n" +MSG_BROADCAST_PORT= "*" +MSG_DEF_PLAYFIELD = "p" +MSG_DEF_KEY = "k" +MSG_DEF_ICON = "r" +MSG_DEF_BITMAP = "m" +MSG_DEF_SAMPLE = "w" +MSG_DEF_MUSIC = "z" +MSG_PLAY_MUSIC = "Z" +MSG_FADEOUT = "f" +MSG_PLAYER_JOIN = "+" +MSG_PLAYER_KILL = "-" +MSG_PLAYER_ICON = "i" +MSG_PING = "g" +MSG_PONG = "G" +MSG_INLINE_FRAME = "\\" +MSG_PATCH_FILE = MSG_DEF_MUSIC +MSG_ZPATCH_FILE = "P" +MSG_MD5_FILE = "M" +MSG_RECORDED = "\x00" + +CMSG_PROTO_VERSION= "v" +CMSG_KEY = "k" +CMSG_ADD_PLAYER = "+" +CMSG_REMOVE_PLAYER= "-" +CMSG_UDP_PORT = "<" +CMSG_ENABLE_SOUND = "s" +CMSG_ENABLE_MUSIC = "m" +CMSG_PING = "g" +CMSG_PONG = "G" +CMSG_DATA_REQUEST = "M" +CMSG_PLAYER_NAME = "n" + +BROADCAST_MESSAGE = "game!" # less than 6 bytes + + +def message(tp, *values): + strtype = type('') + typecodes = [''] + for v in values: + if type(v) is strtype: + typecodes.append('%ds' % len(v)) + elif 0 <= v < 256: + typecodes.append('B') + else: + typecodes.append('l') + typecodes = ''.join(typecodes) + assert len(typecodes) < 256 + return pack(("!B%dsc" % len(typecodes)) + typecodes, + len(typecodes), typecodes, tp, *values) + +def decodemessage(data): + if data: + limit = ord(data[0]) + 1 + if len(data) >= limit: + typecodes = "!c" + data[1:limit] + try: + end = limit + calcsize(typecodes) + except TypeError: + return None, '' + if len(data) >= end: + return unpack(typecodes, data[limit:end]), data[end:] + elif end > 1000000: + raise OverflowError + return None, data Added: pypy/branch/flex-backend/translator/flex/examples/bnb/servermessage.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/bnb/servermessage.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,289 @@ +from msgstruct import * +from zlib import decompressobj, decompress +from urllib import quote +from os.path import exists +from struct import unpack +from time import time +import md5 + + +debug = True +def log(msg): + if debug: + print msg + + +class BitmapCreationException(Exception): + pass + + +#proxy messages +#PMSG_PING = "ping" #server wants to hear from client +#PMSG_PONG = "pong" #server responds to client's ping +PMSG_DEF_PLAYFIELD = "def_playfield" +PMSG_DEF_ICON = "def_icon" +PMSG_PLAYER_ICON = "player_icon" +PMSG_PLAYER_JOIN = "player_join" +PMSG_PLAYER_KILL = "player_kill" +PMSG_DEF_KEY = "def_key" +PMSG_INLINE_FRAME = "inline_frame" + + +# convert server messages to proxy messages in json format +class ServerMessage: + + _md5_file = {} + _bitmap2hexdigits={} + _def_icon_queue = {} + base_gfx_dir = 'data/images/' + base_gfx_url = '/images/' + gfx_extension = 'png' + + def __init__(self, base_gfx_dir = None): + if base_gfx_dir: + self.base_gfx_dir = base_gfx_dir + if not self.base_gfx_dir.endswith('/'): + self.base_gfx_dir += '/' + self.socket = None + self.data = '' + self.n_header_lines = 2 + self.gfx_dir = self.base_gfx_dir #gets overwritten depending on playfield FnDesc + self.gfx_url = self.base_gfx_url + self.decompressobj = decompressobj().decompress + self.last_active = time() + self._count = 0 + + def count(self): + self._count += 1 + return self._count + + def dispatch(self, *values): + #log('RECEIVED:%s(%d)' % (values[0], len(values[1:]))) + self.last_active = time() + fn = self.MESSAGES.get(values[0]) + if fn: + try: + return fn(self, *values[1:]) + except BitmapCreationException, e: + log(str(e)) + return dict() + else: + log("UNKNOWN:%s" % str(values)) + return dict(type='unknown', values=values) + + #server message handlers... + def ignore(self, *values): + #log('ignore %s' % str(values)) + return + + def def_playfield(self, width, height, backcolor, FnDesc): + #log('def_playfield width=%s, height=%s, backcolor=%s, FnDesc=%s' % (\ + # width, height, backcolor, FnDesc)) + self.gfx_dir = self.base_gfx_dir + self.gfx_url = self.base_gfx_url + return dict(type=PMSG_DEF_PLAYFIELD, width=width, height=height, + backcolor=backcolor, FnDesc=FnDesc) + + def def_bitmap(self, bitmap_code, data_or_fileid, *rest): + if type(data_or_fileid) is type(0): + fn = self.def_bitmap2 + else: + fn = self.def_bitmap1 + return fn(bitmap_code, data_or_fileid, *rest) + + def def_bitmap1(self, bitmap_code, data, *rest): + import PIL.Image + if len(rest) == 0: + colorkey = None + else: + c = rest[0] + colorkey = (c & 255, (c >> 8) & 255, (c >> 16) & 255) + #log('def_bitmap1 bitmap_code=%d, data=%d bytes, colorkey=%s' % ( + # bitmap_code, len(data), colorkey)) + + try: + decompressed_data = decompress(data) + except Exception, e: + raise BitmapCreationException('ERROR UNCOMPRESSING DATA FOR %s (%s)' % ( + bitmap_filename, str(e))) + hexdigits = md5.new(decompressed_data).hexdigest() + self._bitmap2hexdigits[bitmap_code] = hexdigits + + gfx_bitmap_filename = '%s%s.%s' % (self.gfx_dir, hexdigits, self.gfx_extension) + if exists(gfx_bitmap_filename): + #log('CACHED:%s' % gfx_bitmap_filename) + pass + else: + bitmap_filename = '%s%s.ppm' % (self.gfx_dir, hexdigits) + f = open(bitmap_filename, 'wb') + f.write(decompressed_data) + f.close() + #TODO: use in memory (don't save ppm first) + try: + bitmap = PIL.Image.open(bitmap_filename) + except IOError, e: + raise BitmapCreationException('ERROR LOADING %s (%s)' % ( + bitmap_filename, str(e))) + + #create alpha layer (PIL export this correctly with png but not with gif) + if colorkey is not None: + bitmap = bitmap.convert("RGBA") + data = bitmap.getdata() + c = (colorkey[0], colorkey[1], colorkey[2], 255) + width, height = bitmap.size + for y in range(height): #this is slowish but gfx are cached, so... + for x in range(width): + p = data.getpixel((x,y)) + if p == c: + data.putpixel((x,y), (0,0,0,0)) + + try: + bitmap.save(gfx_bitmap_filename) + log('SAVED:%s' % gfx_bitmap_filename) + except IOError: + raise BitmapCreationException('ERROR SAVING %s (%s)' % ( + gfx_bitmap_filename, str(e))) + + def def_bitmap2(self, bitmap_code, fileid, *rest): + #log('def_bitmap2: bitmap_code=%d, fileid=%d, colorkey=%s' % (bitmap_code, fileid, rest)) + hexdigits = self._md5_file[fileid]['hexdigits'] + self._bitmap2hexdigits[bitmap_code] = hexdigits + gfx_bitmap_filename = '%s%s.%s' % (self.gfx_dir, hexdigits, self.gfx_extension) + if exists(gfx_bitmap_filename): + #log('SKIP DATA_REQUEST:%s' % gfx_bitmap_filename) + pass + else: + self._md5_file[fileid]['bitmap_code'] = bitmap_code + self._md5_file[fileid]['colorkey'] = rest + position = self._md5_file[fileid]['offset'] + size = self._md5_file[fileid]['len_data'] + msg = message(CMSG_DATA_REQUEST, fileid, position, size) + self.socket.send(msg) + log('DATA_REQUEST:fileid=%d(pos=%d,size=%d):%s' % ( + fileid, position, size, gfx_bitmap_filename)) + + def def_icon(self, bitmap_code, icon_code, x,y,w,h, *rest): + import PIL.Image + + #log('def_icon bitmap_code=%s, icon_code=%s, x=%s, y=%s, w=%s, h=%s, alpha=%s' %\ + # (bitmap_code, icon_code, x,y,w,h, rest) #ignore alpha (bubbles) + + hexdigits = self._bitmap2hexdigits[bitmap_code] + bitmap_filename = '%s%s.%s' % (self.gfx_dir, hexdigits, self.gfx_extension) + icon_filename = '%s%s_%d_%d_%d_%d.%s' % ( + self.gfx_dir, hexdigits, x, y, w, h, self.gfx_extension) + if exists(icon_filename): + #log('CACHED:%s' % icon_filename) + pass + elif exists(bitmap_filename): + #TODO: use in memory (don't save ppm first) + icon = PIL.Image.open(bitmap_filename) + box = (x, y, x+w, y+h) + region = icon.crop(box) + region.save(icon_filename) + log('SAVED:%s' % icon_filename) + else: #bitmap is not available yet (protocol 2) + #log('%s NOT AVAILABLE FOR %s' % (bitmap_filename, icon_filename)) + if bitmap_code not in self._def_icon_queue: + self._def_icon_queue[bitmap_code] = [] + self._def_icon_queue[bitmap_code].append((icon_code, x, y, w, h, rest)) + return + + filename = '%s%s_%d_%d_%d_%d.%s' % ( + self.gfx_url, hexdigits, x, y, w, h, self.gfx_extension) + return dict(type=PMSG_DEF_ICON, icon_code=icon_code, filename=filename, width=w, height=h) + + def zpatch_file(self, fileid, position, data): #response to CMSG_DATA_REQUEST + #log('zpatch_file fileid=%d, position=%d, len(data)=%d' % (fileid, position, len(data))) + + bitmap_code = self._md5_file[fileid]['bitmap_code'] + colorkey = self._md5_file[fileid]['colorkey'] + try: + t = self.def_bitmap(bitmap_code, data, *colorkey) + except BitmapCreationException, e: + log(str(e)) + return #i.e. not attempting to create icons + messages = [] + if bitmap_code in self._def_icon_queue: + #log('%d icons queued for bitmap %d' % ( + # len(self._def_icon_queue[bitmap_code]), bitmap_code)) + for t in self._def_icon_queue[bitmap_code]: + icon_code, x, y, w, h, rest = t + messages.append(self.def_icon(bitmap_code, icon_code, x, y, w, h, *rest)) + del self._def_icon_queue[bitmap_code] + return messages + + def player_icon(self, player_id, icon_code): + #log('player_icon player_id=%d, icon_code=%d' % (player_id, icon_code)) + return dict(type=PMSG_PLAYER_ICON, player_id=player_id, icon_code=icon_code) + + def player_join(self, player_id, client_is_self): + #log('player_join player_id=%d, client_is_self=%d' % (player_id, client_is_self)) + return dict(type=PMSG_PLAYER_JOIN, player_id=player_id, client_is_self=client_is_self) + + def player_kill(self, player_id): + #log('player_kill player_id=%d' % player_id) + return dict(type=PMSG_PLAYER_KILL, player_id=player_id) + + def def_key(self, keyname, num, *icon_codes): + #log('def_key keyname=%s, num=%d, icon_codes=%s' % (keyname, num, str(icon_codes))) + return dict(type=PMSG_DEF_KEY, keyname=keyname, num=num, icon_codes=icon_codes) + + def md5_file(self, fileid, protofilepath, offset, len_data, checksum): + hd = '0123456789abcdef' + hexdigits = '' + for c in checksum: + i = ord(c) + hexdigits = hexdigits + hd[i >> 4] + hd[i & 15] + #log('md5_file fileid=%d, protofilepath=%s, offset=%d, len_data=%d, hexdigits=%s' % ( + # fileid, protofilepath, offset, len_data, hexdigits)) + self._md5_file[fileid] = { + 'protofilepath' : protofilepath, + 'offset' : offset, + 'len_data' : len_data, + 'checksum' : checksum, + 'hexdigits' : hexdigits, + } + + def inline_frame(self, data): + decompressed_data = d = self.decompressobj(data) + #log('inline_frame len(data)=%d, len(decompressed_data)=%d' % ( + # len(data), len(d))) + + return_raw_data = False + if return_raw_data: + return dict(type=PMSG_INLINE_FRAME, data=decompressed_data) + + #note: we are not returning the raw data here but we could let the Javascript + # handle it. If this is working we can convert this to RPython and move it + # to the client instead. (based on BnB update_sprites in display/pclient.py) + sounds, sprites = [], [] + + base = 0 + while d[base+4:base+6] == '\xFF\xFF': + key, lvol, rvol = struct.unpack("!hBB", udpdata[base:base+4]) + sounds.append((key, lvol, rvol)) + base += 6 + + for j in range(base, len(d)-5, 6): + info = d[j:j+6] + x, y, icon_code = unpack("!hhh", info[:6]) + sprites.append((icon_code, x, y)) + + return dict(type=PMSG_INLINE_FRAME, sounds=sounds, sprites=sprites) + + MESSAGES = { + MSG_BROADCAST_PORT : ignore, + MSG_PING : ignore, + MSG_PONG : ignore, + MSG_DEF_PLAYFIELD : def_playfield, + MSG_DEF_BITMAP : def_bitmap, + MSG_ZPATCH_FILE : zpatch_file, + MSG_DEF_ICON : def_icon, + MSG_PLAYER_ICON : player_icon, + MSG_PLAYER_JOIN : player_join, + MSG_PLAYER_KILL : player_kill, + MSG_DEF_KEY : def_key, + MSG_MD5_FILE : md5_file, + MSG_INLINE_FRAME : inline_frame, + } Added: pypy/branch/flex-backend/translator/flex/examples/bnb/start_bnb.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/bnb/start_bnb.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,283 @@ +#!/usr/bin/env python +""" bub-n-bros testing utility +""" + +import autopath + +import py + +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js.modules.dom import document, window +from pypy.translator.js.modules.mochikit import log, logWarning,\ + createLoggingPane, logDebug, connect +from pypy.translator.js.examples.bnb.bnb import exported_methods +from pypy.translator.js import commproxy + +commproxy.USE_MOCHIKIT = True + +import time +import os +import sys + +def logKey(msg): + pass + +class Stats(object): + """ Class containing some statistics + """ + def __init__(self): + self.n_received_inline_frames = 0 + self.n_rendered_inline_frames = 0 + self.n_rendered_dynamic_sprites = 0 + self.fps = 0 + self.starttime = 0.0 + self.n_sprites = 0 + + def register_frame(self): + self.n_rendered_inline_frames += 1 + if self.n_rendered_inline_frames >= 10: + next_time = time.time() + self.fps = 10000/(next_time - self.starttime) + self.n_rendered_inline_frames = 0 + self.starttime = next_time + +stats = Stats() + +class Player(object): + def __init__(self): + self.id = -1 + self.prev_count = 0 + self.sessionid = "" + +player = Player() + +class SpriteManager(object): + def __init__(self): + self.sprites = {} + self.filenames = {} + self.all_sprites = {} + self.frames = [] + + def add_icon(self, icon_code, filename): + self.filenames[icon_code] = filename + #self.sprite_queues[icon_code] = [] + #self.used[icon_code] = [] + # FIXME: Need to write down DictIterator once... + #self.icon_codes.append(icon_code) + + def add_sprite(self, s, icon_code, x, y): + #try: + # img = self.sprite_queues[icon_code].pop() + #except IndexError: + stats.n_sprites += 1 + img = document.createElement("img") + img.src = self.filenames[icon_code] + img.style.position = 'absolute' + img.style.left = x + 'px' + img.style.top = y + 'px' + img.style.visibility = 'visible' + document.getElementById("playfield").appendChild(img) + try: + self.sprites[s].style.visibility = "hidden" + # FIXME: We should delete it + except KeyError: + self.sprites[s] = img + return img + + def move_sprite(self, s, x, y): + i = self.sprites[s] + i.style.top = y + 'px' + i.style.left = x + 'px' + i.style.visibility = 'visible' + + def hide_sprite(self, s): + i = self.sprites[s] + i.style.visibility = "hidden" + + def start_clean_sprites(self): + self.all_sprites = {} + + def show_sprite(self, s, icon_code, x, y): + self.all_sprites[s] = 1 + try: + self.move_sprite(s, x, y) + except KeyError: + self.add_sprite(s, icon_code, x, y) + + def end_clean_sprites(self): + for i in self.sprites: + try: + self.all_sprites[i] + except KeyError: + self.hide_sprite(i) + + def set_z_index(self, s_num, z): + self.sprites[s_num].style.zIndex = z + + #def show_sprite(self, s): + # i = self.sprites[s] + # i.style.visibility = "visible" + +sm = SpriteManager() + +class KeyManager(object): + def __init__(self): + self.keymappings = {ord('D'):'right', ord('S'):'fire', ord('A'):'left', ord('W'):'up'} + self.key_to_bnb_down = {'right':0, 'left':1, 'fire':3, 'up':2} + self.key_to_bnb_up = {'right':4, 'left':5, 'fire':7, 'up':6} + self.queue = [] + + def add_key_up(self, key): + self.queue.append(self.key_to_bnb_up[key]) + + def add_key_down(self, key): + self.queue.append(self.key_to_bnb_down[key]) + + def get_keys(self): + retval = self.queue + self.queue = [] + return retval + +km = KeyManager() + +def appendPlayfield(msg): + body = document.getElementsByTagName('body')[0] + bgcolor = '#000' + body.style.backgroundColor = bgcolor + div = document.createElement("div") + div.id = 'playfield' + div.style.width = msg['width'] + div.style.height = msg['height'] + div.style.position = 'absolute' + div.style.top = '0px' + div.style.left = '0px' + div.appendChild(document.createTextNode('foobar?')) + + #document.body.childNodes.insert(0, div) + body.appendChild(div) + +def appendPlayfieldXXX(): + bgcolor = '#000000' + document.body.setAttribute('bgcolor', bgcolor) + div = document.createElement("div") + div.id = 'playfield' + div.style.width = 500 + div.style.height = 250 + div.style.position = 'absolute' + div.style.top = '0px' + div.style.left = '0px' + document.body.appendChild(div) + +def process_message(msg): + if msg['type'] == 'def_playfield': + appendPlayfield(msg) + elif msg['type'] == 'def_icon': + sm.add_icon(msg['icon_code'], msg['filename']) + elif msg['type'] == 'ns': + sm.add_sprite(msg['s'], msg['icon_code'], msg['x'], msg['y']) + sm.set_z_index(msg['s'], msg['z']) + elif msg['type'] == 'sm': + sm.move_sprite(msg['s'], msg['x'], msg['y']) + sm.set_z_index(msg['s'], msg['z']) + elif msg['type'] == 'ds': + sm.hide_sprite(msg['s']) + elif msg['type'] == 'begin_clean_sprites': + sm.start_clean_sprites() + elif msg['type'] == 'clean_sprites': + sm.end_clean_sprites() + elif msg['type'] == 'show_sprite': + sm.show_sprite(msg['s'], msg['icon_code'], msg['x'], msg['y']) + elif msg['type'] == 'zindex': + sm.set_z_index(msg['s'], msg['z']) + #elif msg['type'] == 'ss': + # sm.show_sprite(msg['s']) + elif msg['type'] == 'player_icon' or msg['type'] == 'def_key' or \ + msg['type'] == 'player_join' or msg['type'] == 'player_kill': + pass #ignore + else: + logWarning('unknown message type: ' + msg['type']) + + +def ignore(arg): + pass +ignore._annspecialcase_ = 'specialize:argtype(0)' + +def addPlayer(player_id): + name = "player no. " + str(player_id) + #name = "player no. %d" % player_id + #File "/Users/eric/projects/pypy-dist/pypy/translator/js/jts.py", line 52, in lltype_to_cts + # raise NotImplementedError("Type %r" % (t,)) + # NotImplementedError: Type + prev_player_id = player.id + if player.id >= 0: + exported_methods.remove_player(player.id, player.sessionid, ignore) + player.id = -1 + if player_id != prev_player_id: + exported_methods.player_name(player_id, name, player.sessionid, ignore) + exported_methods.add_player(player_id, player.sessionid, ignore) + player.id = player_id + + +def keydown(key): + #c = chr(int(key.keyCode)).lower() + #c = int(key.keyCode) + key = key._event + try: + c = key.keyCode + if c > ord('0') and c < ord('9'): + addPlayer(int(chr(c))) + #for i in km.keymappings: + # log(str(i)) + if c in km.keymappings: + km.add_key_down(km.keymappings[c]) + #else: + except Exception, e: + log(str(e)) + +def keyup(key): + key = key._event + c = key.keyCode + if c > ord('0') and c < ord('9'): + pass #don't print warning + elif c in km.keymappings: + km.add_key_up(km.keymappings[c]) + else: + logWarning('unknown keyup: ' + str(c)) + +#def ignore_dispatcher(msgs): +# pass + +def bnb_dispatcher(msgs): + s = ":".join([str(i) for i in km.get_keys()]) + exported_methods.get_message(player.sessionid, player.id, s, + bnb_dispatcher) + render_frame(msgs) + +def render_frame(msgs): + for msg in msgs['messages']: + process_message(msg) + stats.register_frame() + document.title = str(stats.n_sprites) + " sprites " + str(stats.fps) + +def session_dispatcher(sessionid): + player.sessionid = sessionid + connect(document, 'onkeydown', keydown) + connect(document, 'onkeyup', keyup) + exported_methods.get_message(player.sessionid, player.id, "", + bnb_dispatcher) + +def bnb(): + createLoggingPane(True) + log("keys: [0-9] to select player, [wsad] to walk around") + exported_methods.initialize_session(session_dispatcher) + +def run_bnb(): + from pypy.translator.js.examples.bnb.bnb import BnbRoot + from pypy.translator.js.lib import server + addr = ('', 7070) + httpd = server.create_server(handler=BnbRoot, server_address=addr) + httpd.source = rpython2javascript(sys.modules[__name__], ['bnb']) + httpd.serve_forever() + +if __name__ == '__main__': + run_bnb() Added: pypy/branch/flex-backend/translator/flex/examples/console/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/examples/console/client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/client.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,154 @@ + +from pypy.translator.js.modules import dom +from pypy.translator.js.modules.mochikit import log, createLoggingPane +from pypy.translator.js.examples.console.console import exported_methods + +class Glob(object): + def __init__(self): + self.console_running = False + self.next_console = "" + self.text_to_show = [] + self.pss = [] + +glob = Glob() + +def add_text_to_dom(txt): + data_elem = dom.document.getElementById("data") + if data_elem.childNodes: + data = data_elem.childNodes[0].nodeValue + txt + else: + data = txt + while data_elem.childNodes: + data_elem.removeChild(data_elem.childNodes[0]) + data_elem.appendChild(dom.document.createTextNode(data)) + + +def add_text(txt, server_flag, fn=add_text_to_dom): + if not server_flag: + if txt.find("\n") != len(txt) - 1: + if txt[-1] == '\n': + txt = txt[:-1] + lst = txt.split("\n") + add_text_to_dom(lst[0] + "\n") + glob.text_to_show += lst[1:] + else: + add_text_to_dom(txt) + else: + for ps in glob.pss: + if glob.text_to_show: + num = txt.find(ps) + if txt.startswith(ps): + txt = txt[len(ps):] + add_text_to_dom(ps + glob.text_to_show.pop(0) + "\n") + add_text(txt, True) + return + if txt.startswith("\n" + ps): + txt = txt[len(ps) + 1:] + add_text_to_dom(ps + glob.text_to_show.pop(0) + "\n") + add_text(txt, True) + return + add_text_to_dom(txt) + +def create_text(txt): + return dom.document.createTextNode(txt) + +def set_text(txt): + data_elem = dom.document.getElementById("data") + while data_elem.childNodes: + data_elem.removeChild(data_elem.childNodes[0]) + data_elem.appendChild(dom.document.createTextNode(txt)) + +def refresh_console(msg): + inp_elem = dom.document.getElementById("inp") + #inp_elem.disabled = False + if msg[0] == "refresh": + data = msg[1] + if data: + inp_elem.scrollIntoView() + inp_elem.focus() + exported_methods.refresh_empty(glob.sess_id, refresh_console) + add_text(data, True) + elif msg[0] == 'disconnected': + inp_elem.disabled = True + name_bar = dom.document.getElementById("namebar") + name_bar.style.color = "red" + text = name_bar.lastChild.nodeValue + name_bar.removeChild(name_bar.lastChild) + name_bar.appendChild(create_text(text + " [DEFUNCT]")) + glob.console_running = False + if glob.next_console: + next = glob.next_console + glob.next_console = "" + load_console(next) + +def set_sessid(data): + sessid = int(data[0]) + help_msg = data[1] + glob.pss = data[2:] + glob.sess_id = sessid + inp_elem = dom.document.getElementById("inp") + inp_elem.disabled = False + name_bar = dom.document.getElementById("namebar") + name_bar.style.color = "black" + name_bar.removeChild(name_bar.lastChild) + name_bar.appendChild(create_text("Python console")) + dom.document.getElementById("helpcontents").innerHTML = help_msg + exported_methods.refresh_empty(sessid, refresh_console) + +def empty_callback(msg): + inp_elem = dom.document.getElementById("inp") + #inp_elem.disabled = False + inp_elem.scrollIntoView() + +def keypressed(key): + kc = key.keyCode + if kc == ord("\r"): + inp_elem = dom.document.getElementById("inp") + cmd = inp_elem.value + inp_elem.value = '' + add_text(cmd + "\n", False) + #if not cmd: + # exported_methods.refresh(glob.sess_id, cmd, empty_callback) + #else: + exported_methods.refresh(glob.sess_id, cmd + "\n", refresh_console) + +def nothing(msg): + pass + +def nothing2(msg): + pass + +def cleanup_console(): + inp_elem = dom.document.getElementById("inp") + inp_elem.disabled = True + set_text("") + glob.text_to_show = [] # better safe than sorry + exported_methods.kill_console(glob.sess_id, nothing2) + +def load_console(python="python"): + if glob.console_running: + cleanup_console() + glob.next_console = python + return + inp_elem = dom.document.getElementById("inp") + main = dom.document.getElementById("main") + main.style.visibility = "visible" + inp_elem.disabled = False + inp_elem.focus() + glob.console_running = True + exported_methods.get_console(python, set_sessid) + +def add_snippet(snippet): + add_text(snippet, False) + exported_methods.refresh(glob.sess_id, snippet, refresh_console) + +def execute_snippet(name='python', number=3): + exported_methods.execute_snippet(name, number, add_snippet) + +def console_onload(): + #createLoggingPane(True) + #inp_elem = dom.document.getElementById("inp") + #inp_elem.focus() + dom.document.onkeypress = keypressed + #exported_methods.get_console("python", set_sessid) + Added: pypy/branch/flex-backend/translator/flex/examples/console/console.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/console.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,155 @@ +import py + +from pypy.translator.js.lib import server +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js.lib.support import callback +from pypy.translator.js import commproxy +from pypy.translator.js.examples.console.session import Interpreter, Killed +from pypy.translator.js.examples.console.docloader import DocLoader +from py.__.green.server.httpserver import GreenHTTPServer +from py.__.green.greensock2 import ConnexionClosed + +commproxy.USE_MOCHIKIT = True + +FUNCTION_LIST = ["load_console", "console_onload", "execute_snippet"] + +class Ignore(Exception): + pass + +def js_source(): + import client + return rpython2javascript(client, FUNCTION_LIST) + +def line_split(ret, max_len): + to_ret = [] + for line in ret.split("\n"): + if len(line) > max_len: + to_ret += [line[i*max_len:(i+1)*max_len] for i in + range(len(line)/max_len)] + i = len(line)/max_len + else: + i = 0 + to_ret.append(line[i*max_len:]) + return "\n".join(to_ret) + + +STATIC_DIR = py.path.local(__file__) +for x in range(6): + STATIC_DIR = STATIC_DIR.dirpath() +STATIC_DIR = STATIC_DIR.join("compiled") +DOCDIR = STATIC_DIR.dirpath().join("pypy", "doc", "play1") +CONSOLES = ['python', 'pypy-c', 'pypy-c-thunk', 'pypy-c-taint', 'pypy-cli', 'pyrolog-c', 'pypy-c-jit'] + +class Sessions(object): + def __init__(self): + self.sessions = {} + self.updating = {} + testfile = py.path.local(__file__).dirpath().join("play1_snippets.py") + self.docloader = DocLoader(docdir=DOCDIR, consoles=CONSOLES, + testfile=testfile) + + def new_session(self, python="python"): + if not py.path.local().sysfind(python): + python = str(STATIC_DIR.join(python)) + ip = Interpreter(python) + self.sessions[ip.pid] = ip + self.updating[ip.pid] = False + return ip.pid + + def update_session(self, pid, to_write=None): + ip = self.sessions[pid] + if self.updating[pid]: + ip.write_only(to_write) + raise Ignore() + self.updating[pid] = True + ret = ip.interact(to_write) + self.updating[pid] = False + if not ret: + return "" + MAX_LEN = 80 + return line_split(ret, MAX_LEN) + + def kill_session(self, pid): + ip = self.sessions[pid] + ip.pipe.stdin.close() + del self.sessions[pid] + del self.updating[pid] + + def get_ps(self, python): + if python == 'python': + return ['>>> ', '... '] + if python.startswith('pypy'): + return ['>>>> ', '.... '] + if python == 'pyrolog-c': + return ['>?-'] + return [] + +# We hack here, cause in exposed methods we don't have global 'server' +# state +sessions = Sessions() + +class ExportedMethods(server.ExportedMethods): + @callback(retval=[str]) + def get_console(self, python="python"): + retval = sessions.new_session(python) + return [str(retval), sessions.docloader.get_html(python)] +\ + sessions.get_ps(python) + + def _refresh(self, pid, to_write): + try: + return ["refresh", sessions.update_session(int(pid), to_write)] + except (KeyError, IOError, Killed, ConnexionClosed): + return ["disconnected"] + except Ignore: + return ["ignore"] + + @callback(retval=[str]) + def refresh(self, pid=0, to_write=""): + return self._refresh(pid, to_write) + + @callback(retval=[str]) + def refresh_empty(self, pid=0): + #print "Empty refresh %d" % int(pid) + return self._refresh(pid, None) + + @callback(retval=str) + def execute_snippet(self, name='aaa', number=3): + return sessions.docloader.get_snippet(name, int(number)) + "\n" + + @callback() + def kill_console(self, pid=0): + try: + sessions.kill_session(int(pid)) + except KeyError: + pass + +exported_methods = ExportedMethods() + +static_dir = py.path.local(__file__).dirpath().join("data") + +class Root(server.Collection): + exported_methods = exported_methods + index = server.FsFile(static_dir.join("console.html")) + style_css = server.FsFile(static_dir.dirpath().dirpath().join("data"). + join("style.css")) + MochiKit = server.StaticDir('MochiKit') + + def source_js(self): + if hasattr(self.server, 'source_console'): + source = self.server.source_console + else: + source = js_source() + self.server.source_console = source + return "text/javascript", source + source_js.exposed = True + +class Handler(server.NewHandler): + application = Root() + application.some = Root() + application.other = Root() + +if __name__ == '__main__': + addr = ('', 8007) + httpd = server.create_server(server_address=addr, handler=Handler, + server=GreenHTTPServer) + httpd.serve_forever() Added: pypy/branch/flex-backend/translator/flex/examples/console/data/console.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/data/console.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,43 @@ + + + + + + + Console + + + PyPy + + + + + Added: pypy/branch/flex-backend/translator/flex/examples/console/docloader.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/docloader.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,43 @@ + +""" Simple module for loading documentation of various +pypy-cs from doc directory +""" + +import py + +class DocLoader(object): + def __init__(self, consoles, docdir, testfile): + self.consoles = consoles + self.docdir = py.path.local(docdir) + assert self.docdir.check(dir=1) + self.testfile = testfile + assert self.testfile.check() + self.htmls = {} + self.snippets = {} + self.load() + + def get_html(self, console): + return self.htmls[console] + + def get_snippet(self, console, num): + return str(self.snippets[console][num]) + + def load(self): + def mangle_name(name): + return name.replace("-", "_").replace(".", "_") + + def mangle(source): + source = source.strip() + del source.lines[0] + return source.deindent() + + testmod = self.testfile.pyimport() + for console in self.consoles: + html = self.docdir.join(console + '.html').read() + snip_class = getattr(testmod, 'AppTest_' + mangle_name(console)) + snippets = [mangle(py.code.Source(getattr(snip_class, name))) + for name in + dir(snip_class) if name.startswith("test_snippet")] + self.snippets[console] = snippets + self.htmls[console] = html % tuple([str(i) for i in snippets]) + Added: pypy/branch/flex-backend/translator/flex/examples/console/ideas.txt ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/ideas.txt Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ +* Thunk objspace - lazy evaluation in demo/fibonacci2 or sth + +* Taint objspace - examples in the documentation + +* Transparent proxy + +* Stackless - examples in a docs of stackless/WP7 + +* __pypy__.internal_repr - --faassen, ropes + +* changing grammar at the runtime Added: pypy/branch/flex-backend/translator/flex/examples/console/play1_snippets.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/play1_snippets.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,82 @@ +from pypy.conftest import gettestobjspace + +class AppTest_pypy_c(object): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withtproxy": True, + "usemodules":("_stackless",)}) + + def test_snippet_1(self): + from tputil import make_proxy + history = [] + def recorder(operation): + history.append(operation) + return operation.delegate() + + l = make_proxy(recorder, obj=[]) + +class AppTest_pypy_c_thunk(object): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.name": 'thunk'}) + + def test_snippet_1(self): + from __pypy__ import thunk + def f(): + print 'computing...' + return 6*7 + + x = thunk(f) + +class AppTest_pypy_c_taint(object): + def setup_class(cls): + cls.space = gettestobjspace(**{'objspace.name': 'taint'}) + + def test_snippet_1(self): + from __pypy__ import taint + x = taint(6) + +class AppTest_pypy_cli(object): + def setup_class(cls): + cls.space = gettestobjspace(**{'usemodules': 'clr'}) + + def test_snippet_1(self): + import clr + ArrayList = clr.load_cli_class('System.Collections', 'ArrayList') + obj = ArrayList() + obj.Add(1) + +class AppTest_pyrolog_c(object): + pass + +class AppTest_python(object): + pass + +class AppTest_pypy_c_jit(object): + def setup_class(cls): + cls.space = gettestobjspace(**{'usemodules':('pypyjit',)}) + + def test_snippet_1(self): + import time + + def f1(n): + "Arbitrary test function." + i = 0 + x = 1 + while i%s") + code = tmpdir.ensure("test_snippets2.py").write(str(py.code.Source(''' + class AppTest_one(object): + def test_snippet_1(self): + x = 1 + ''')) + '\n') + ld = DocLoader(docdir=tmpdir, consoles=['one'], testfile=tmpdir.join('test_snippets2.py')) + assert ld.get_html('one') == "x = 1" + assert ld.get_snippet('one', 0) == 'x = 1' Added: pypy/branch/flex-backend/translator/flex/examples/console/test/test_session.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/test/test_session.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,71 @@ + +""" Various tests around interpreter in a subprocess +""" + +import py + +from py.__.green.greensock2 import allof, sleep +from py.__.green.pipe.fd import FDInput +from pypy.translator.js.examples.console.session import Interpreter, Killed + +import sys +if sys.platform == 'nt': + py.test.skip("Those tests doesn't run on windows (yet)") + +def test_greensock_reader_timeouter(): + i = Interpreter("python", timeout=3) + while not i.interact().endswith(">>> "): + pass + assert i.interact() is None + assert i.interact("a\n") is not None + +def test_two_interpreters(): + i = Interpreter("python", timeout=3) + i2 = Interpreter("python", timeout=3) + while not i.interact().endswith(">>> "): + pass + while not i2.interact().endswith(">>> "): + pass + + def f(): + return i.interact("import time;time.sleep(1)\n") + def g(): + return i2.interact("a\n") + + one, two = allof(g, f) + assert two.startswith(">>") + assert one.startswith("Traceback") + +def test_multiline_command(): + i = Interpreter("python", timeout=3) + while not i.interact().endswith(">>> "): + pass + val = i.interact("def f(x):\n y = x + 3\n return y\n\nf(8)\n") + while val is not None: + assert 'Traceback' not in val + assert 'Syntax' not in val + print val + val = i.interact() + +def test_kill_timeout(): + py.test.skip("XXX") + i = Interpreter("python", kill_timeout=1, timeout=3) + while not i.interact().endswith(">>> "): + pass + i.interact() + +def test_kill_timeout_outside(): + py.test.skip("XXX") + i = Interpreter("python", kill_timeout=1, timeout=3) + while not i.interact().endswith(">>> "): + pass + sleep(8) + i.interact() + +def test_does_not_die(): + i = Interpreter("python", kill_timeout=1, timeout=3) + while not i.interact().endswith(">>> "): + pass + i.interact("import time\n") + for _ in range(6): + i.interact("time.sleep(.2)\n") Added: pypy/branch/flex-backend/translator/flex/examples/console/test/test_snippets.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/console/test/test_snippets.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,3 @@ +from pypy.translator.js.examples.console.play1_snippets import * + +del AppTest_pypy_cli Added: pypy/branch/flex-backend/translator/flex/examples/data/error.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/data/error.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,19 @@ + + + %(message)s + + + + + PyPy + +
+

%(message)s

+

%(explain)s

+
+ Added: pypy/branch/flex-backend/translator/flex/examples/data/index.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/data/index.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,39 @@ + + + pypy.js various demos + + + + + PyPy + + + +
+

This site presents various demos created with + PyPy's compiler toolchain + to generate JavaScript code out of + RPython.

+ +

It also allows you to play with PyPy's unique features on consoles + presenting a PyPy standard interpreter translated to C or + CLI (.NET, running Mono), forwarded from the server. + On each console there is + a little help box which explains the features. +

+ +

The demos should work under Firefox, Mozilla and Opera. Support for + other browsers is uncertain. The Bub'n'Bros demo is only known to work + with Firefox so far.

+ +

Note: these demos are set up on a replicable Xen instance, but + please do not vandalise the machine, otherwise we will be forced to + take it down.

+
+ + Added: pypy/branch/flex-backend/translator/flex/examples/data/launcher.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/data/launcher.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,9 @@ + + + Python console + + + +

Console Launching, please wait...

+ + \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/examples/data/py-web1.png ============================================================================== Binary file. No diff available. Added: pypy/branch/flex-backend/translator/flex/examples/data/style.css ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/data/style.css Sat Jul 14 11:58:08 2007 @@ -0,0 +1,56 @@ +body,body.editor,body.body { + font: 90% "Times New Roman", Arial, Verdana, Helvetica, serif; + background: White; + color: Black; +} + +div#main { + position: absolute; + top: 130px; + left: 170px; + margin-right: 5px; +} + +div#contents { + float: left; + width: 160px; +} + +#inp { + font: 90% fixed; +} + +#help { + background: #EEE; +} + +.menubar { + font-weight: bold; + float: left; + font: 120% Verdana, Helvetica, Arial, sans-serif; + text-decoration: none; + padding-right: 0.3em; + +} + +.title{ + /*border-spacing: 20px;*/ + font: 160% Verdana, Helvetica, Arial, sans-serif; + color: #3ba6ec; + vertical-align: center; + padding-bottom: 0.3em; +} + +div#metaspace { + position: absolute; + top: 20px; + left: 170px; +} + +.main { +} + +.error { + color: red; + font-weight: bold; +} Added: pypy/branch/flex-backend/translator/flex/examples/djangoping/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/examples/djangoping/client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/djangoping/client.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,51 @@ +"""rpython javascript code""" + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.translator.js.lib.support import callback + +from pypy.translator.js.modules import mochikit +from pypy.translator.js.modules import dom + +class PingHandler(BasicExternal): + """Server side code which handles javascript calls""" + _render_xmlhttp = True + + @callback(retval={str:str}) + def ping(self, ping_str="aa"): + """Simply returns the string prefixed with a PONG""" + return dict(response="PONG: %s" % ping_str) + +ping_handler = PingHandler() + +def jsping(): + mochikit.logDebug("pinging") + ping_handler.ping("PING", callback) + +def callback(data): + mochikit.logDebug("Got response: " + data["response"]) + log = dom.document.getElementById("log") + mochikit.logDebug("got log element") + try: + s = "

" + data["response"] + "

" + except KeyError: + mochikit.logDebug("Can't find data") + s = "

" + "Error" + "

" + mochikit.logDebug("Adding: " + s) + log.innerHTML += s + mochikit.logDebug("added message") + +def doping_onclick(event): + mochikit.logDebug("calling pinger") + jsping() + +def ping_init(): + mochikit.createLoggingPane(True) + button = dom.document.getElementById("doping") + button.onclick = doping_onclick + mochikit.logDebug("Ping button setup") + +if __name__ == "__main__": + # circular import + from pypy.translator.js.examples.djangoping import client + from pypy.translator.js.main import rpython2javascript + print rpython2javascript(client, ["ping_init"]) Added: pypy/branch/flex-backend/translator/flex/examples/djangoping/manage.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/djangoping/manage.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ +#!/usr/bin/env python +from django.core.management import execute_manager +try: + import settings # Assumed to be in the same directory. +except ImportError: + import sys + sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__) + sys.exit(1) + +if __name__ == "__main__": + execute_manager(settings) Added: pypy/branch/flex-backend/translator/flex/examples/djangoping/settings.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/djangoping/settings.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,76 @@ +# Django settings for djangoping project. + +DEBUG = True +TEMPLATE_DEBUG = DEBUG + +ADMINS = ( + # ('Your Name', 'your_email at domain.com'), +) + +MANAGERS = ADMINS + +DATABASE_ENGINE = '' # 'postgresql', 'mysql', 'sqlite3' or 'ado_mssql'. +DATABASE_NAME = '' # Or path to database file if using sqlite3. +DATABASE_USER = '' # Not used with sqlite3. +DATABASE_PASSWORD = '' # Not used with sqlite3. +DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3. +DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3. + +# Local time zone for this installation. All choices can be found here: +# http://www.postgresql.org/docs/current/static/datetime-keywords.html#DATETIME-TIMEZONE-SET-TABLE +TIME_ZONE = 'America/Chicago' + +# Language code for this installation. All choices can be found here: +# http://www.w3.org/TR/REC-html40/struct/dirlang.html#langcodes +# http://blogs.law.harvard.edu/tech/stories/storyReader$15 +LANGUAGE_CODE = 'en-us' + +SITE_ID = 1 + +# Absolute path to the directory that holds media. +# Example: "/home/media/media.lawrence.com/" +MEDIA_ROOT = '' + +# URL that handles the media served from MEDIA_ROOT. +# Example: "http://media.lawrence.com" +MEDIA_URL = '' + +# URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a +# trailing slash. +# Examples: "http://foo.com/media/", "/media/". +ADMIN_MEDIA_PREFIX = '/media/' + +# Make this unique, and don't share it with anybody. +SECRET_KEY = 'z-n46o5v2gd^n2!tqlher5)w5!l at 0em+d_o+-3qhdulg7qc_lk' + +# List of callables that know how to import templates from various sources. +TEMPLATE_LOADERS = ( + 'django.template.loaders.filesystem.load_template_source', + 'django.template.loaders.app_directories.load_template_source', +# 'django.template.loaders.eggs.load_template_source', +) + +MIDDLEWARE_CLASSES = ( + 'django.middleware.common.CommonMiddleware', + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.middleware.doc.XViewMiddleware', +) + +ROOT_URLCONF = 'pypy.translator.js.examples.djangoping.urls' + +import os +# grab a sister package to get its __file__ +from pypy.translator.js.examples.djangoping import client +TEMPLATE_DIRS = ( + # Put strings here, like "/home/html/django_templates". + # Always use forward slashes, even on Windows. + os.path.join(os.path.dirname(client.__file__), "templates"), +) + +INSTALLED_APPS = ( + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.sites', +) Added: pypy/branch/flex-backend/translator/flex/examples/djangoping/templates/index.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/djangoping/templates/index.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,14 @@ + + + + + Python pinger + + + +

Guestbook

+
+ +
+
+ name:
+ message:
+
+ +
+ + + """ + return 'text/html', html + index.exposed = True + + # the (generated) javascript + def guestbook_js(self): + if hasattr(self.server, 'source'): + source = self.server.source + else: + source = guestbook_client() + self.server.source = source + return "text/javascript", source + guestbook_js.exposed = True + +if __name__ == '__main__': + addr = ('', 8008) + httpd = server.create_server(server_address=addr, handler=Handler) + httpd.serve_forever() Added: pypy/branch/flex-backend/translator/flex/examples/guestbook_client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/guestbook_client.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,35 @@ +""" rpython guestbook client-side code + + this code can be tested in CPython, but will also be converted to + JavaScript to provide the client-side functionality for the guestbook + example +""" + +from pypy.translator.js.modules import dom +from pypy.translator.js.examples.guestbook import exported_methods + +def _get_messages_callback(messages): + for message in messages: + add_html_message(message) + +def init_guestbook(): + exported_methods.get_messages(_get_messages_callback) + +def _add_message_callback(message): + add_html_message(message) + +def add_message(): + doc = dom.window.document + name = doc.getElementById('name').value + message = doc.getElementById('message').value + exported_methods.add_message(name, message, _add_message_callback) + +def add_html_message(text=''): + doc = dom.window.document + div = doc.getElementById('messages') + msgdiv = doc.createElement('div') + msgdiv.style.border = '1px solid black' + msgdiv.style.margin = '1em' + msgdiv.appendChild(doc.createTextNode(text)) + div.appendChild(msgdiv) + Added: pypy/branch/flex-backend/translator/flex/examples/over_client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/over_client.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,14 @@ + +""" Client side of overmind.py +""" + +from pypy.translator.js.modules import dom + +def callback(port): + hname = dom.window.location.hostname + dom.window.location.assign("http://%s:%d" % (hname, port)) + +def bnb_redirect(): + loc = dom.window.location + new_loc = loc.protocol + "//" + loc.hostname + ":7070" + loc.assign(new_loc) Added: pypy/branch/flex-backend/translator/flex/examples/overmind.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/overmind.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,83 @@ +#!/usr/bin/env python +""" This is script which collects all the demos and +run them when needed +""" + +import autopath + +from pypy.translator.js.lib import server +from pypy.translator.js.lib.support import callback +from pypy.rpython.ootypesystem.bltregistry import described +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js.examples.console import console +from py.__.green.server.httpserver import GreenHTTPServer + +import os +import py + +FUNCTION_LIST = ['bnb_redirect'] +TIMEOUT = 300 +pids = [] + +def js_source(function_list): + import over_client + return rpython2javascript(over_client, FUNCTION_LIST) + +static_dir = py.path.local(__file__).dirpath().join("data") + +class Root(server.Collection): + index = server.FsFile(static_dir.join("index.html")) + style_css = server.FsFile(static_dir.join("style.css"), + content_type="text/css") + terminal = server.Static(static_dir.join("terminal.html")) + console = console.Root() + py_web1_png = server.FsFile(static_dir.join("py-web1.png"), + content_type="image/png") + + def source_js(self): + if hasattr(self.server, 'source'): + source = self.server.source + else: + source = js_source(FUNCTION_LIST) + self.server.source = source + return "text/javascript", source + source_js.exposed = True + + def bnb(self): + return ''' + + + + + + + ''' + bnb.exposed = True + + def handle_error(self, exc, e_value, tb): + import traceback + tb_formatted = '\n'.join(traceback.format_tb(tb)) + \ + "%s: %s" % (exc, e_value) + log_file = open("/tmp/play1_error_log", "a") + log_file.write(tb_formatted) + log_file.close() + print tb_formatted + +class Handler(server.NewHandler): + application = Root() + + error_message_format = static_dir.join('error.html').read() + #console = server.Static(os.path.join(static_dir, "launcher.html")) + +if __name__ == '__main__': + try: + addr = ('', 8008) + httpd = server.create_server(server_address=addr, handler=Handler, + server=GreenHTTPServer) + httpd.serve_forever() + except KeyboardInterrupt: + for pid in pids: + # eventually os.kill stuff + os.kill(pid, 15) + os.waitpid(pid, 0) + Added: pypy/branch/flex-backend/translator/flex/examples/pythonconsole.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/pythonconsole.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,207 @@ +#!/usr/bin/env python +""" A basic Python console from your browser. + +This depends on MochiKit for proper quoting. +You need to provide the files as ..jsdemo/MochiKit/*.js. +(Symlinks are your friends.) + +Try to type: import time; time.sleep(2); print 'hi' +""" + +import autopath + +import sys, os, cStringIO +from cgi import parse_qs +from pypy.translator.js.modules.dom import setTimeout, document, window +from pypy.translator.js.modules.mochikit import connect, disconnect +from pypy.rpython.ootypesystem.bltregistry import MethodDesc, BasicExternal +from pypy.translator.js import commproxy +from pypy.rpython.extfunc import genericcallable + +from pypy.translator.js.lib import support +from pypy.translator.js.lib import server + +commproxy.USE_MOCHIKIT = True + +from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer +from SocketServer import ThreadingMixIn +import time + +HTML_PAGE = """ + + + Example + + + + +

Console

+

Note that a default timeout for the console is 5 minutes, after that time +console just dies and stops responding

+ +
+ + + + + +""" + +httpd = None + +def callback(data): + inp_elem = document.getElementById("inp") + inp_elem.disabled = False + answer = data.get('answer', '') + add_text(answer) + inp_elem.focus() + +def add_text(text): + data_elem = document.getElementById("data") + lines = text.split('\n') + lines.pop() + for line in lines: + pre = document.createElement('pre') + pre.style.margin = '0px' + pre.appendChild(document.createTextNode(line)) + data_elem.appendChild(pre) + +class Storage(object): + def __init__(self): + self.level = 0 + self.cmd = "" + +storage = Storage() + +def keypressed(key): + kc = key._event.keyCode + if kc == ord("\r"): + inp_elem = document.getElementById("inp") + cmd = inp_elem.value + if storage.level == 0: + add_text(">>> %s\n" % (cmd,)) + else: + add_text("... %s\n" % (cmd,)) + inp_elem.value = '' + if cmd: + storage.cmd += cmd + "\n" + if cmd.endswith(':'): + storage.level = 1 + elif storage.level == 0 or cmd == "": + if (not storage.level) or (not cmd): + inp_elem.disabled = True + httpd.some_callback(storage.cmd, callback) + storage.cmd = "" + storage.level = 0 + +def setup_page(): + connect(document, 'onkeypress', keypressed) + document.getElementById("inp").focus() + +class Server(HTTPServer, BasicExternal): + # Methods and signatures how they are rendered for JS + _methods = { + 'some_callback' : MethodDesc([('cmd', str), + ('callback', genericcallable([{str:str}]))], + {str:str}) + } + + _render_xmlhttp = True + + def __init__(self, *args, **kwargs): + HTTPServer.__init__(self, *args, **kwargs) + self.source = None + self.locals = {} + +class RequestHandler(BaseHTTPRequestHandler): + def do_GET(self): + path = self.path + if '?' in path: + i = path.index('?') + else: + i = len(path) + kwds = parse_qs(path[i+1:]) + path = path[:i].split("/") + if not path[0]: + del path[0] + if not path: + path = [''] + cmd = path[0] + args = path[1:] + method_to_call = getattr(self, "run_" + cmd, None) + if method_to_call is None: + self.send_error(404, "File %r not found" % (self.path,)) + else: + method_to_call(*args, **kwds) + + do_POST = do_GET + + def run_(self): + self.run_index() + + def run_index(self): + self.serve_data("text/html", HTML_PAGE) + + def run_MochiKit(self, filename): + assert filename in os.listdir('MochiKit') + pathname = os.path.join('MochiKit', filename) + f = open(pathname, 'r') + data = f.read() + f.close() + self.serve_data("text/javascript", data) + + def run_some_callback(self, cmd=[""]): + cmd = cmd[0] + self.server.last_activity = time.time() + if cmd: + buf = cStringIO.StringIO() + out1 = sys.stdout + err1 = sys.stderr + try: + sys.stdout = sys.stderr = buf + try: + exec compile(cmd, '?', 'single') in self.server.locals + except Exception: + import traceback + traceback.print_exc() + finally: + sys.stdout = out1 + sys.stderr = err1 + answer = buf.getvalue() + else: + answer = "" + self.serve_data("text/json", repr({'answer': answer})) + + def run_jssource(self): + if self.server.source: + source = self.server.source + else: + source = support.js_source([setup_page]) + self.server.source = source + self.serve_data("text/javascript", source) + + def serve_data(self, content_type, data): + self.send_response(200) + self.send_header("Content-type", content_type) + self.send_header("Content-length", len(data)) + self.send_header('Expires', 'Mon, 26 Jul 1997 05:00:00 GMT') + self.send_header('Last-Modified', + time.strftime("%a, %d %b %Y %H:%M:%S GMT")) + self.send_header('Cache-Control', 'no-cache, must-revalidate') + self.send_header('Cache-Control', 'post-check=0, pre-check=0') + self.send_header('Pragma', 'no-cache') + self.end_headers() + self.wfile.write(data) + + +def build_http_server(server_address=('', 8001)): + global httpd + httpd = Server(server_address, RequestHandler) + print 'http://127.0.0.1:%d' % (server_address[1],) + +def _main(address=('', 8001)): + build_http_server(server_address=address) + httpd.serve_forever() + +if __name__ == '__main__': + _main() Added: pypy/branch/flex-backend/translator/flex/examples/serialise.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/serialise.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,15 @@ + +""" example of (very simple) serialiser +""" + +def serialise(obj): + if isinstance(obj, str): + return "S" + obj + elif isinstance(obj, int): + return "I" + str(obj) + return "?" + +serialise._annspecialcase_ = "specialize:argtype(0)" + +def serialisetest(): + return serialise("aaa") + serialise(3) + serialise(None) Added: pypy/branch/flex-backend/translator/flex/examples/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/examples/test/test_examples.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/examples/test/test_examples.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,38 @@ + +""" Various build tests +""" +import py +from pypy.translator.js.main import rpython2javascript + +def test_console_build(): + from pypy.translator.js.examples import pythonconsole + httpd = pythonconsole.Server(('', 0), pythonconsole.RequestHandler) + pythonconsole.httpd = httpd + # XXX obscure hack + assert rpython2javascript(pythonconsole, ['setup_page'], use_pdb=False) + +def test_bnb_build(): + from pypy.translator.js.examples.bnb import start_bnb + assert rpython2javascript(start_bnb, ['bnb'], use_pdb=False) + +def test_overmind_build(): + from pypy.translator.js.examples import overmind, over_client + assert rpython2javascript(over_client, overmind.FUNCTION_LIST, + use_pdb=False) + +def test_guestbook_build(): + from pypy.translator.js.examples import guestbook, guestbook_client + assert rpython2javascript(guestbook_client, guestbook.FUNCTION_LIST, + use_pdb=False) + + +def test_console_2_build(): + from pypy.translator.js.examples.console import console, client + assert rpython2javascript(client, console.FUNCTION_LIST, + use_pdb=True) + +def test_ping_play1(): + from urllib import URLopener + u = URLopener() + text = "pypy.js various demos" + assert u.open("http://play1.pypy.org/").read().find(text) != -1 Added: pypy/branch/flex-backend/translator/flex/function.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/function.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,233 @@ +try: + set +except NameError: + from sets import Set as set + +from pypy.objspace.flow import model as flowmodel +from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong +from pypy.rpython.ootypesystem import ootype +from pypy.translator.oosupport.metavm import Generator,InstructionList +from pypy.translator.oosupport import function + +from pypy.translator.flex.log import log +from types import FunctionType + +import re + +class BaseGenerator(object): + def load(self, v): + if isinstance(v, flowmodel.Variable): + if v.name in self.argset: + selftype, selfname = self.args[0] + if self.is_method and v.name == selfname: + self.ilasm.load_self() + else: + self.ilasm.load_arg(v) + else: + self.ilasm.load_local(v) + elif isinstance(v, flowmodel.Constant): + self.db.load_const(v.concretetype, v.value, self.ilasm) + elif isinstance(v, str): + self.ilasm.load_const("'" + v + "'") + else: + assert False + + def store(self, v): + assert isinstance(v, flowmodel.Variable) + if v.concretetype is not Void: + self.ilasm.store_local(v) + else: + self.ilasm.store_void() + + def change_name(self, name, to_name): + self.ilasm.change_name(name, to_name) + + def add_comment(self, text): + pass + + def function_signature(self, graph): + return self.cts.graph_to_signature(graph, False) + + def class_name(self, ooinstance): + return ooinstance._name + + def emit(self, instr, *args): + self.ilasm.emit(instr, *args) + + def call_graph(self, graph): + self.db.pending_function(graph) + func_sig = self.function_signature(graph) + self.ilasm.call(func_sig) + + def call_external(self, name, args): + self.ilasm.call((name, args)) + + #def call_signature(self, signature): + # self.ilasm.call(signature) + + def cast_to(self, lltype): + cts_type = self.cts.lltype_to_cts(lltype, False) + self.ilasm.castclass(cts_type) + + def new(self, obj): + self.ilasm.new(self.cts.obj_name(obj)) + + def set_field(self, obj, name): + self.ilasm.set_field(obj, name) + #self.ilasm.set_field(self.field_name(obj,name)) + + def get_field(self, useless_stuff, name): + self.ilasm.get_field(name) + + def call_method(self, obj, name): + func_name, signature = self.cts.method_signature(obj, name) + self.ilasm.call_method(obj, name, signature) + + def call_external_method(self, name, arg_len): + self.ilasm.call_method(None, name, [0]*arg_len) + + def instantiate(self): + self.ilasm.runtimenew() + + def downcast(self, TYPE): + pass + + def load_special(self, v): + # special case for loading value + # when setting builtin field we need to load function instead of None + # FIXME: we cheat here + if isinstance(v, flowmodel.Constant) and v.concretetype is ootype.Void and isinstance(v.value, FunctionType): + graph = self.db.translator.annotator.bookkeeper.getdesc(v.value).cachedgraph(None) + self.db.pending_function(graph) + name = graph.name + self.ilasm.load_str(name) + else: + self.load(v) + + def cast_function(self, name, num): + self.ilasm.cast_function(name, num) + + def prefix_op(self, st): + self.ilasm.prefix_op(st) + + def load_str(self, s): + self.ilasm.load_str(s) + + def load_void(self): + self.ilasm.load_void() + + def list_setitem(self, base_obj, item, val): + self.load(base_obj) + self.load(val) + self.load(item) + self.ilasm.list_setitem() + + def list_getitem(self, base_obj, item): + self.load(base_obj) + self.load(item) + self.ilasm.list_getitem() + + def push_primitive_constant(self, TYPE, value): + self.db.load_const(TYPE, value, self.ilasm) + + def branch_unconditionally(self, target_label): + self.ilasm.jump_block(self.block_map[target_label]) + + def branch_conditionally(self, exitcase, target_label): + self.ilasm.branch_if(exitcase) + self.ilasm.jump_block(self.block_map[target_label]) + self.ilasm.close_branch() + +class Function(function.Function, BaseGenerator): + def __init__(self, db, graph, name=None, is_method=False, + is_entrypoint=False, _class=None): + self._class = _class + super(Function, self).__init__(db, graph, name, is_method, is_entrypoint) + self._set_args() + self._set_locals() + self.order = 0 + self.name = name or self.db.get_uniquename(self.graph, self.graph.name) + + def _setup_link(self, link, is_exc_link = False): + target = link.target + for to_load, to_store in zip(link.args, target.inputargs): + if to_load.concretetype is not Void: + if is_exc_link and isinstance(to_load, flowmodel.Variable) and re.match("last_exc_value", to_load.name): + self.ilasm.load_str("exc") + else: + self.load(to_load) + self.store(to_store) + + def _create_generator(self, ilasm): + return self + + def begin_render(self): + block_map = {} + for blocknum, block in enumerate(self.graph.iterblocks()): + block_map[self._get_block_name(block)] = blocknum + self.block_map = block_map + + if self.is_method: + args = self.args[1:] # self is implicit + else: + args = self.args + if self.is_method: + self.ilasm.begin_method(self.name, self._class, [i[1] for i in args]) + else: + self.ilasm.begin_function(self.name, args) + self.ilasm.set_locals(",".join([i[1] for i in self.locals])) + self.ilasm.begin_for() + + def render_return_block(self, block): + return_var = block.inputargs[0] + if return_var.concretetype is not Void: + self.load(return_var) + self.ilasm.ret() + else: + self.ilasm.load_void() + self.ilasm.ret() + + def end_render(self): + self.ilasm.end_for() + self.ilasm.end_function() + + def render_raise_block(self, block): + self.ilasm.throw(block.inputargs[1]) + + def end_try(self, target_label): + self.ilasm.jump_block(self.block_map[target_label]) + self.ilasm.catch() + #self.ilasm.close_branch() + + def record_ll_meta_exc(self, ll_meta_exc): + pass + + def begin_catch(self, llexitcase): + real_name = self.cts.lltype_to_cts(llexitcase._inst.class_._INSTANCE) + s = "isinstanceof(exc, %s)"%real_name + self.ilasm.branch_if_string(s) + + def end_catch(self, target_label): + """ Ends the catch block, and branchs to the given target_label as the + last item in the catch block """ + self.ilasm.close_branch() + + def store_exception_and_link(self, link): + self._setup_link(link, True) + self.ilasm.jump_block(self.block_map[self._get_block_name(link.target)]) + + def after_except_block(self): + #self.ilasm.close_branch() + self.ilasm.throw_real("exc") + self.ilasm.close_branch() + + def set_label(self, label): + self.ilasm.write_case(self.block_map[label]) + #self.ilasm.label(label) + + def begin_try(self): + self.ilasm.begin_try() + + def clean_stack(self): + self.ilasm.clean_stack() Added: pypy/branch/flex-backend/translator/flex/helper.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/helper.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,41 @@ + +""" Some helpers +""" + +from pypy.translator.flex.modules.dom import document + +def escape(s): + #return s.replace("&", "&").replace("<", "<").replace(">", ">"). \ + # replace("'", "\\'").replace(" ", " ").replace("\n", "
") + return s + +def create_debug_div(): + debug_div = document.createElement("div") + debug_div.setAttribute("id", "debug_div") + # XXX attach it somewhere... + #body = document.getElementsByTagName('body')[0] + document.childNodes[0].childNodes[1].appendChild(debug_div) + return debug_div + +def __show_traceback(tb, exc): + debug_div = document.getElementById("debug_div") + if not debug_div: + # create div here + debug_div = create_debug_div() + + pre_div = document.createElement("pre") + pre_div.style.color = "#FF0000" + debug_div.appendChild(pre_div) + txt = document.createTextNode("") + pre_div.appendChild(txt) + for tb_entry in tb[1:]: + # list of tuples... + fun_name, args, filename, lineno = tb_entry + # some source maybe? or so? + line1 = escape("%s %s" % (fun_name, args)) + line2 = escape(" %s: %s\n" % (filename, lineno)) + txt.nodeValue += line1 + '\n' + line2 + + txt.nodeValue += str(exc) + +__show_traceback.explicit_traceback = True Added: pypy/branch/flex-backend/translator/flex/js.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/js.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,133 @@ +''' +reference material: + http://webreference.com/javascript/reference/core_ref/ + http://webreference.com/programming/javascript/ + http://mochikit.com/ + http://www.mozilla.org/js/spidermonkey/ + svn co http://codespeak.net/svn/kupu/trunk/ecmaunit +''' + +import py +import os + +from pypy.rpython.rmodel import inputconst +from pypy.rpython.typesystem import getfunctionptr +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.ootypesystem import ootype +from pypy.tool.udir import udir +from pypy.translator.flex.log import log + +from pypy.translator.flex.asmgen import AsmGen +from pypy.translator.flex.jts import JTS +from pypy.translator.flex.opcodes import opcodes +from pypy.translator.flex.function import Function +from pypy.translator.flex.database import LowLevelDatabase + +from pypy.translator.oosupport.genoo import GenOO + +from heapq import heappush, heappop + +from StringIO import StringIO + +def _path_join(root_path, *paths): + path = root_path + for p in paths: + path = os.path.join(path, p) + return path + +class Tee(object): + def __init__(self, *args): + self.outfiles = args + + def write(self, s): + for outfile in self.outfiles: + outfile.write(s) + + def close(self): + for outfile in self.outfiles: + if outfile is not sys.stdout: + outfile.close() + +class JS(GenOO): + TypeSystem = JTS + opcodes = opcodes + Function = Function + Database = LowLevelDatabase + + def __init__(self, translator, functions=[], stackless=False, compress=False, \ + logging=False, use_debug=False): + if not isinstance(functions, list): + functions = [functions] + GenOO.__init__(self, udir, translator, None) + + pending_graphs = [translator.annotator.bookkeeper.getdesc(f).cachedgraph(None) for f in functions ] + for graph in pending_graphs: + self.db.pending_function(graph) + + self.db.translator = translator + self.use_debug = use_debug + self.assembly_name = self.translator.graphs[0].name + self.tmpfile = udir.join(self.assembly_name + '.js') + + def gen_pendings(self): + while self.db._pending_nodes: + node = self.db._pending_nodes.pop() + to_render = [] + nparent = node + while nparent.order != 0: + nparent = nparent.parent + to_render.append(nparent) + to_render.reverse() + for i in to_render: + i.render(self.ilasm) + + node.render(self.ilasm) + + def generate_communication_proxy(self): + """ Render necessary stuff aroundc communication + proxies + """ + for proxy in self.db.proxies: + proxy.render(self.ilasm) + + + def create_assembler(self): + out = self.tmpfile.open('w') + return AsmGen(out, self.assembly_name) + + def generate_source(self): + self.ilasm = self.create_assembler() + self.fix_names() + self.gen_entrypoint() + while self.db._pending_nodes: + self.gen_pendings() + self.db.gen_constants(self.ilasm, self.db._pending_nodes) + + self.ilasm.end_consts() + self.ilasm.close() + assert len(self.ilasm.right_hand) == 0 + return self.tmpfile.strpath + + def write_source(self): + + # write down additional functions + # FIXME: when used with browser option it should probably + # not be used as inlined, rather another script to load + # this is just workaround + self.generate_source() + sio = StringIO() + data = self.tmpfile.open().read() + + src_filename = _path_join(os.path.dirname(__file__), 'jssrc', 'misc.as') + flex_filename = _path_join(os.path.dirname(__file__), 'jssrc', 'flex.mxml') + f = self.tmpfile.open("w") + lib = open(src_filename).read() + flex = open(flex_filename).read() + self.ilasm = AsmGen(sio, self.assembly_name ) + self.generate_communication_proxy() + f.write(flex%(lib, data)) + f.close() + + self.filename = self.tmpfile + + return self.tmpfile Added: pypy/branch/flex-backend/translator/flex/jsbuiltin.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/jsbuiltin.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,84 @@ +""" JavaScript builtin mappings +""" + +from pypy.translator.oosupport.metavm import InstructionList, PushAllArgs,\ + _PushAllArgs +from pypy.translator.flex.metavm import SetBuiltinField, ListGetitem, ListSetitem, \ + GetBuiltinField, CallBuiltin, Call, SetTimeout, ListContains,\ + NewBuiltin, SetOnEvent, ListRemove, CallBuiltinMethod, _GetPredefinedField,\ + _SetPredefinedField + +from pypy.rpython.ootypesystem import ootype + +class _Builtins(object): + def __init__(self): + list_resize = _SetPredefinedField('length') + + self.builtin_map = { + 'll_js_jseval' : CallBuiltin('eval'), + 'set_on_keydown' : SetOnEvent('onkeydown'), + 'set_on_keyup' : SetOnEvent('onkeyup'), + 'setTimeout' : SetTimeout, + 'll_int_str' : CallBuiltinMethod('toString', [2]), + 'll_strconcat' : InstructionList([_PushAllArgs(slice(1, None)), '+']), + 'll_int' : CallBuiltin('parseInt'), + #'alert' : CallBuiltin('alert'), + 'seval' : CallBuiltin('seval'), + 'date': NewBuiltin('Date'), + 'll_math.ll_math_fmod' : InstructionList([_PushAllArgs(slice(1, None)), '%']), + 'll_time_time' : CallBuiltin('time'), + 'll_time_clock' : CallBuiltin('clock'), + 'll_os_write' : CallBuiltin('print'), + } + self.builtin_obj_map = { + ootype.String.__class__: { + 'll_strconcat' : InstructionList([_PushAllArgs(slice(1, None)), '+']), + 'll_strlen' : _GetPredefinedField('length'), + 'll_stritem_nonneg' : CallBuiltinMethod('charAt', slice(1,None)), + 'll_streq' : InstructionList([_PushAllArgs(slice(1, None)), '==']), + 'll_strcmp' : CallBuiltin('strcmp'), + 'll_startswith' : CallBuiltin('startswith'), + 'll_endswith' : CallBuiltin('endswith'), + 'll_split_chr' : CallBuiltin('splitchr'), + 'll_substring' : CallBuiltin('substring'), + 'll_lower' : CallBuiltinMethod('toLowerCase', slice(1, None)), + 'll_upper' : CallBuiltinMethod('toUpperCase', slice(1, None)), + 'll_find' : CallBuiltin('findIndexOf'), + 'll_find_char' : CallBuiltin('findIndexOf'), + 'll_contains' : CallBuiltin('findIndexOfTrue'), + 'll_replace_chr_chr' : CallBuiltinMethod('replace', slice(1, None), ['g']), + 'll_count_char' : CallBuiltin('countCharOf'), + 'll_count' : CallBuiltin('countOf'), + }, + ootype.List: { + 'll_setitem_fast' : ListSetitem, + 'll_getitem_fast' : ListGetitem, + '_ll_resize' : list_resize, + '_ll_resize_ge' : list_resize, + '_ll_resize_le' : list_resize, + 'll_length' : _GetPredefinedField('length'), + }, + ootype.Dict: { + 'll_get' : ListGetitem, + 'll_set' : ListSetitem, + 'll_contains' : ListContains, + 'll_get_items_iterator' : CallBuiltin('dict_items_iterator'), + 'll_length' : CallBuiltin('get_dict_len'), + 'll_remove' : ListRemove, + 'll_clear': CallBuiltin('clear_dict'), + }, + ootype.Record: { + 'll_get' : ListGetitem, + 'll_set' : ListSetitem, + 'll_contains' : ListContains, + } + } + self.fix_opcodes() + + def fix_opcodes(self): + from pypy.translator.flex.metavm import fix_opcodes + #fix_opcodes(self.builtin_map) + #for value in self.builtin_obj_map.values(): + # fix_opcodes(value) + +Builtins = _Builtins() Added: pypy/branch/flex-backend/translator/flex/json.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/json.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,310 @@ +import string +import types + +## json.py implements a JSON (http://json.org) reader and writer. +## Copyright (C) 2005 Patrick D. Logan +## Contact mailto:patrickdlogan at stardecisions.com +## +## This library is free software; you can redistribute it and/or +## modify it under the terms of the GNU Lesser General Public +## License as published by the Free Software Foundation; either +## version 2.1 of the License, or (at your option) any later version. +## +## This library is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +## Lesser General Public License for more details. +## +## You should have received a copy of the GNU Lesser General Public +## License along with this library; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +class _StringGenerator(object): + def __init__(self, string): + self.string = string + self.index = -1 + def peek(self): + i = self.index + 1 + if i < len(self.string): + return self.string[i] + else: + return None + def next(self): + self.index += 1 + if self.index < len(self.string): + return self.string[self.index] + else: + raise StopIteration + def all(self): + return self.string + +class WriteException(Exception): + pass + +class ReadException(Exception): + pass + +class JsonReader(object): + hex_digits = {'A': 10,'B': 11,'C': 12,'D': 13,'E': 14,'F':15} + escapes = {'t':'\t','n':'\n','f':'\f','r':'\r','b':'\b'} + + def read(self, s): + self._generator = _StringGenerator(s) + result = self._read() + return result + + def _read(self): + self._eatWhitespace() + peek = self._peek() + if peek is None: + raise ReadException, "Nothing to read: '%s'" % self._generator.all() + if peek == '{': + return self._readObject() + elif peek == '[': + return self._readArray() + elif peek == '"': + return self._readString() + elif peek == '-' or peek.isdigit(): + return self._readNumber() + elif peek == 't': + return self._readTrue() + elif peek == 'f': + return self._readFalse() + elif peek == 'n': + return self._readNull() + elif peek == '/': + self._readComment() + return self._read() + else: + raise ReadException, "Input is not valid JSON: '%s'" % self._generator.all() + + def _readTrue(self): + self._assertNext('t', "true") + self._assertNext('r', "true") + self._assertNext('u', "true") + self._assertNext('e', "true") + return True + + def _readFalse(self): + self._assertNext('f', "false") + self._assertNext('a', "false") + self._assertNext('l', "false") + self._assertNext('s', "false") + self._assertNext('e', "false") + return False + + def _readNull(self): + self._assertNext('n', "null") + self._assertNext('u', "null") + self._assertNext('l', "null") + self._assertNext('l', "null") + return None + + def _assertNext(self, ch, target): + if self._next() != ch: + raise ReadException, "Trying to read %s: '%s'" % (target, self._generator.all()) + + def _readNumber(self): + isfloat = False + result = self._next() + peek = self._peek() + while peek is not None and (peek.isdigit() or peek == "."): + isfloat = isfloat or peek == "." + result = result + self._next() + peek = self._peek() + try: + if isfloat: + return float(result) + else: + return int(result) + except ValueError: + raise ReadException, "Not a valid JSON number: '%s'" % result + + def _readString(self): + result = "" + assert self._next() == '"' + try: + while self._peek() != '"': + ch = self._next() + if ch == "\\": + ch = self._next() + if ch in 'brnft': + ch = self.escapes[ch] + elif ch == "u": + ch4096 = self._next() + ch256 = self._next() + ch16 = self._next() + ch1 = self._next() + n = 4096 * self._hexDigitToInt(ch4096) + n += 256 * self._hexDigitToInt(ch256) + n += 16 * self._hexDigitToInt(ch16) + n += self._hexDigitToInt(ch1) + ch = unichr(n) + elif ch not in '"/\\': + raise ReadException, "Not a valid escaped JSON character: '%s' in %s" % (ch, self._generator.all()) + result = result + ch + except StopIteration: + raise ReadException, "Not a valid JSON string: '%s'" % self._generator.all() + assert self._next() == '"' + return result + + def _hexDigitToInt(self, ch): + try: + result = self.hex_digits[ch.upper()] + except KeyError: + try: + result = int(ch) + except ValueError: + raise ReadException, "The character %s is not a hex digit." % ch + return result + + def _readComment(self): + assert self._next() == "/" + second = self._next() + if second == "/": + self._readDoubleSolidusComment() + elif second == '*': + self._readCStyleComment() + else: + raise ReadException, "Not a valid JSON comment: %s" % self._generator.all() + + def _readCStyleComment(self): + try: + done = False + while not done: + ch = self._next() + done = (ch == "*" and self._peek() == "/") + if not done and ch == "/" and self._peek() == "*": + raise ReadException, "Not a valid JSON comment: %s, '/*' cannot be embedded in the comment." % self._generator.all() + self._next() + except StopIteration: + raise ReadException, "Not a valid JSON comment: %s, expected */" % self._generator.all() + + def _readDoubleSolidusComment(self): + try: + ch = self._next() + while ch != "\r" and ch != "\n": + ch = self._next() + except StopIteration: + pass + + def _readArray(self): + result = [] + assert self._next() == '[' + done = self._peek() == ']' + while not done: + item = self._read() + result.append(item) + self._eatWhitespace() + done = self._peek() == ']' + if not done: + ch = self._next() + if ch != ",": + raise ReadException, "Not a valid JSON array: '%s' due to: '%s'" % (self._generator.all(), ch) + assert ']' == self._next() + return result + + def _readObject(self): + result = {} + assert self._next() == '{' + done = self._peek() == '}' + while not done: + key = self._read() + if type(key) is not types.StringType: + raise ReadException, "Not a valid JSON object key (should be a string): %s" % key + self._eatWhitespace() + ch = self._next() + if ch != ":": + raise ReadException, "Not a valid JSON object: '%s' due to: '%s'" % (self._generator.all(), ch) + self._eatWhitespace() + val = self._read() + result[key] = val + self._eatWhitespace() + done = self._peek() == '}' + if not done: + ch = self._next() + if ch != ",": + raise ReadException, "Not a valid JSON array: '%s' due to: '%s'" % (self._generator.all(), ch) + assert self._next() == "}" + return result + + def _eatWhitespace(self): + p = self._peek() + while p is not None and p in string.whitespace or p == '/': + if p == '/': + self._readComment() + else: + self._next() + p = self._peek() + + def _peek(self): + return self._generator.peek() + + def _next(self): + return self._generator.next() + +class JsonWriter(object): + + def _append(self, s): + self._results.append(s) + + def write(self, obj, escaped_forward_slash=False): + self._escaped_forward_slash = escaped_forward_slash + self._results = [] + self._write(obj) + return "".join(self._results) + + def _write(self, obj): + ty = type(obj) + if ty is types.DictType: + n = len(obj) + self._append("{") + for k, v in obj.items(): + self._write(k) + self._append(":") + self._write(v) + n = n - 1 + if n > 0: + self._append(",") + self._append("}") + elif ty is types.ListType or ty is types.TupleType: + n = len(obj) + self._append("[") + for item in obj: + self._write(item) + n = n - 1 + if n > 0: + self._append(",") + self._append("]") + elif ty is types.StringType or ty is types.UnicodeType: + self._append('"') + obj = obj.replace('\\', r'\\') + if self._escaped_forward_slash: + obj = obj.replace('/', r'\/') + obj = obj.replace('"', r'\"') + obj = obj.replace('\b', r'\b') + obj = obj.replace('\f', r'\f') + obj = obj.replace('\n', r'\n') + obj = obj.replace('\r', r'\r') + obj = obj.replace('\t', r'\t') + self._append(obj) + self._append('"') + elif ty is types.IntType or ty is types.LongType: + self._append(str(obj)) + elif ty is types.FloatType: + self._append("%f" % obj) + elif obj is True: + self._append("true") + elif obj is False: + self._append("false") + elif obj is None: + self._append("null") + else: + raise WriteException, "Cannot write in JSON: %s" % repr(obj) + +def write(obj, escaped_forward_slash=False): + return JsonWriter().write(obj, escaped_forward_slash) + +def read(s): + return JsonReader().read(s) Added: pypy/branch/flex-backend/translator/flex/jssrc/flex.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/jssrc/flex.mxml Sat Jul 14 11:58:08 2007 @@ -0,0 +1,18 @@ + + + + + + + + Added: pypy/branch/flex-backend/translator/flex/jssrc/misc.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/jssrc/misc.as Sat Jul 14 11:58:08 2007 @@ -0,0 +1,241 @@ + + +// we should put the imports here. +import mx.controls.Button; + + +import py.*; +//import py.f.DictIter; + + +// starts hand written code +var MALLOC_ZERO_FILLED = 0; + +var print = trace; + + + + +Function.prototype.method = function (name, func) { + this.prototype[name] = func; + return this; +}; + +function inherits(child, parent) { + child.parent = parent; + for (var i in parent.prototype) { + if (!child.prototype[i]) { + child.prototype[i] = parent.prototype[i]; + } + } +} + +function isinstanceof(self, what) { + + //return self instanceof what; + + + if (!self) { + return (false); + } + var t = self.constructor; + while ( t ) { + if (t == what) { + return (true); + } + t = t.parent; + } + return (false); +} + +/*function delitem(fn, l, i) { + for(var j = i; j < l.length-1; ++j) { + l[j] = l[j+1]; + } + l.length--; +}*/ + +function strcmp(s1, s2) { + if ( s1 < s2 ) { + return ( -1 ); + } else if ( s1 == s2 ) { + return ( 0 ); + } + return (1); +} + +function startswith(s1, s2) { + if (s1.length < s2.length) { + return(false); + } + for (var i = 0; i < s2.length; ++i){ + if (s1.charAt(i) != s2.charAt(i)) { + return(false); + } + } + return(true); +} + +function endswith(s1, s2) { + if (s2.length > s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} + + +function __flash_main() { + + + var aaa = new Button(); + aaa.label = "before consts"; + addChild ( aaa ); + + + + __load_consts_flex() + + aaa.label = "after consts"; + flash_main(1) +} + + +public function flexTrace( text:String ):void { + var myUrl:URLRequest = new URLRequest("javascript:console.log('" + text + "');void(0);"); + sendToURL(myUrl); +} + +// wrapper for throw, because flex compiler weirdness. +function throwit(e) { + throw(e); +} + + +// ends hand written code + Added: pypy/branch/flex-backend/translator/flex/jssrc/misc.js ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/jssrc/misc.js Sat Jul 14 11:58:08 2007 @@ -0,0 +1,229 @@ +// starts hand written code +MALLOC_ZERO_FILLED = 0 + +try { + log; + print = log; +} catch(e) { +} + +Function.prototype.method = function (name, func) { + this.prototype[name] = func; + return this; +}; + +function inherits(child, parent) { + child.parent = parent; + for (i in parent.prototype) { + if (!child.prototype[i]) { + child.prototype[i] = parent.prototype[i]; + } + } +} + +function isinstanceof(self, what) { + if (!self) { + return (false); + } + t = self.constructor; + while ( t ) { + if (t == what) { + return (true); + } + t = t.parent; + } + return (false); +} + +/*function delitem(fn, l, i) { + for(j = i; j < l.length-1; ++j) { + l[j] = l[j+1]; + } + l.length--; +}*/ + +function strcmp(s1, s2) { + if ( s1 < s2 ) { + return ( -1 ); + } else if ( s1 == s2 ) { + return ( 0 ); + } + return (1); +} + +function startswith(s1, s2) { + if (s1.length < s2.length) { + return(false); + } + for (i = 0; i < s2.length; ++i){ + if (s1.charAt(i) != s2.charAt(i)) { + return(false); + } + } + return(true); +} + +function endswith(s1, s2) { + if (s2.length > s1.length) { + return(false); + } + for (i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} +// ends hand written code Added: pypy/branch/flex-backend/translator/flex/jts.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/jts.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,158 @@ +""" JavaScript type system +""" + +from pypy.rpython.ootypesystem import ootype +from pypy.rpython.lltypesystem import lltype +from pypy.translator.cli import oopspec + +from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong, Primitive +from pypy.rpython.lltypesystem.lltype import Char, UniChar +from pypy.rpython.ootypesystem.ootype import String, _string, List, StaticMethod +from pypy.rlib.objectmodel import Symbolic + +from pypy.translator.flex.log import log + +from types import FunctionType +from pypy.rpython.extfunc import is_external + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc, ExternalType + +try: + set +except NameError: + from sets import Set as set + +class JTS(object): + """ Class implementing JavaScript type system + calls with mapping similiar to cts + """ + def __init__(self, db): + self.db = db + + #def __class(self, name): + # return name.replace(".", "_") + + def escape_name(self, name): + return name.replace('.', '_') + + def llvar_to_cts(self, var): + return 'var ', var.name + + def lltype_to_cts(self, t): + if isinstance(t, ootype.Instance): + self.db.pending_class(t) + return self.escape_name(t._name) + elif isinstance(t, ootype.List): + return "Array" + elif isinstance(t, lltype.Primitive): + return "var" + elif isinstance(t, ootype.Record): + return "Object" + elif isinstance(t, ootype.String.__class__): + return '""' + elif isinstance(t, ootype.Dict): + return "Object" + elif isinstance(t, ootype.DictItemsIterator): + return "Object" + elif t is ootype.StringBuilder: + return "StringBuilder" + #return "var" + raise NotImplementedError("Type %r" % (t,)) + + def graph_to_signature(self, graph, is_method = False, func_name = None): + func_name = func_name or self.db.get_uniquename(graph,graph.name) + + args = [arg for arg in graph.getargs() if + arg.concretetype is not ootype.Void] + if is_method: + args = args[1:] + + return func_name,args + + def method_signature(self, obj, name): + # TODO: use callvirt only when strictly necessary + if isinstance(obj, ootype.Instance): + owner, meth = obj._lookup(name) + METH = meth._TYPE + return obj._name, METH.ARGS + elif isinstance(obj, ootype.BuiltinType): + meth = oopspec.get_method(obj, name) + class_name = self.lltype_to_cts(obj) + return class_name,meth.ARGS + else: + assert False + + def obj_name(self, obj): + #import pdb + #pdb.set_trace() + if isinstance(obj, ExternalType): + basic_external = obj._class_ + if not hasattr(basic_external, "_render_class"): + raise TypeError("This class %s can not be instantiated." % (basic_external,)) + return basic_external._render_class + return self.lltype_to_cts(obj) + + def primitive_repr(self, _type, v): + if _type is Bool: + if v == False: + val = 'false' + else: + val = 'true' + elif _type is Void: + val = 'undefined' + elif isinstance(_type,String.__class__): + val = '%r'%v._str + elif isinstance(_type,List): + # FIXME: It's not ok to use always empty list + val = "[]" + elif isinstance(_type,StaticMethod): + if hasattr(v, 'graph') and not is_external(v): + self.db.pending_function(v.graph) + else: + self.db.pending_abstract_function(v) + val = v._name + val = val.replace('.', '_') + if val == '?': + val = 'undefined' + elif _type is UniChar or _type is Char: + #log("Constant %r"%v) + s = repr(v) + if s.startswith('u'): + s = s[1:] + if s != "'\''": + s.replace("'", '"') + val = s + elif isinstance(v, Symbolic): + val = v.expr + elif isinstance(_type, Primitive): + #log("Type: %r"%_type) + val = str(v) + else: + assert False, "Unknown constant %r"%_type + val = str(v) + return val + + #def lltype_to_cts(self, t, include_class=True): + # return 'var' +## if isinstance(t, ootype.Instance): +## self.db.pending_class(t) +## return self.__class(t._name, include_class) +## elif isinstance(t, ootype.Record): +## name = self.db.pending_record(t) +## return self.__class(name, include_class) +## elif isinstance(t, ootype.StaticMethod): +## return 'void' # TODO: is it correct to ignore StaticMethod? +## elif isinstance(t, ootype.List): +## item_type = self.lltype_to_cts(t._ITEMTYPE) +## return self.__class(PYPY_LIST % item_type, include_class) +## elif isinstance(t, ootype.Dict): +## key_type = self.lltype_to_cts(t._KEYTYPE) +## value_type = self.lltype_to_cts(t._VALUETYPE) +## return self.__class(PYPY_DICT % (key_type, value_type), include_class) +## elif isinstance(t, ootype.DictItemsIterator): +## key_type = self.lltype_to_cts(t._KEYTYPE) +## value_type = self.lltype_to_cts(t._VALUETYPE) +## return self.__class(PYPY_DICT_ITEMS_ITERATOR % (key_type, value_type), include_class) +## +## return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Added: pypy/branch/flex-backend/translator/flex/lib/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/lib/server.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/server.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,396 @@ + +""" This is example of totally basic server for XMLHttp request +built on top of BaseHTTPServer. + +Construction is like that: + +you take your own implementation of Handler and subclass it +to provide whatever you like. Each request is checked first for +apropriate method in handler (with dots replaced as _) and this method +needs to have set attribute exposed + +If method is not there, we instead try to search exported_methods (attribute +of handler) for apropriate JSON call. We write down a JSON which we get as +a return value (note that right now arguments could be only strings) and +pass them to caller +""" + +import traceback + +HTTP_STATUS_MESSAGES = { + 200: 'OK', + 204: 'No Content', + 301: 'Moved permanently', + 302: 'Found', + 304: 'Not modified', + 401: 'Unauthorized', + 403: 'Forbidden', + 404: 'Not found', + 500: 'Server error', + 501: 'Not implemented', +} + +class HTTPError(Exception): + """ raised on HTTP errors """ + def __init__(self, status, data=None): + self.status = status + self.message = HTTP_STATUS_MESSAGES[status] + self.data = data + + def __str__(self): + data = '' + if self.data: + data = ' (%s)' % (self.data,) + return '' % (self.status, self.message, data) + +from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler + +import re +import time +import random +import os +import sys + +import py +from pypy.translator.js.lib.url import parse_url + +from pypy.translator.js import json + +from pypy.rpython.ootypesystem.bltregistry import MethodDesc, BasicExternal,\ + described +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js import commproxy + +commproxy.USE_MOCHIKIT = False + +class Collection(object): + """ an HTTP collection + + essentially this is a container object that has a path that ends on a + slash, and support for PATH_INFO (so can have (virtual or not) + children) + + children are callable attributes of ourselves that have an 'exposed' + attribute themselves, that accept 3 arguments: 'handler', a reference + to the BaseHTTPHandler that handles the request (XXX should be + abstracted?), 'path', the requested path to the object, and 'query', + the (unparsed!) GET query string (without a preceding ?) + """ + + def traverse(self, path, orgpath): + """ traverse path relative to self + + 'path' is the path requested by the client, split on '/', but + relative from the current object: parent Collection items may have + removed items (they will have, actually, unless 'self' is the root + of the website) from the beginning on traversal to 'self' + + path is split on '/', the first item is removed and used to do + a lookup on self, if that fails a 404 is raised, if successful + the item is used to continue traversal (if the object found is + a Collection type) or to handle the request (if the object found + is a callable with .exposed set to True) + + if path equals '', a lookup for 'index' is done + + can be overridden in subclasses to implement different path + handling (PATH_INFO-like stuff) + """ + name = path.pop(0) + if name == '': + name = 'index' + name = name.replace(".", "_") + resource = getattr(self, name, None) + if (resource is None or (not isinstance(resource, Collection) and + (not callable(resource) or + not getattr(resource, 'exposed', True)))): + raise HTTPError(404) + if path: + if not isinstance(resource, Collection): + raise HTTPError(500) # no PATH_INFO allowed for non-Collection + return resource.traverse(path, orgpath) + else: + if isinstance(resource, Collection): + # targeting a collection directly: redirect to its 'index' + raise HTTPError(301, orgpath + '/') + if not getattr(resource, 'exposed', False): + # don't reveal what is not accessible... + raise HTTPError(404) + return resource + +class ExportedMethods(BasicExternal, Collection): + _render_base_path = "exported_methods" + def traverse(self, path, orgpath): + """ traverse path relative to self + + 'path' is the path requested by the client, split on '/', but + relative from the current object: parent Collection items may have + removed items (they will have, actually, unless 'self' is the root + of the website) from the beginning on traversal to 'self' + + path is split on '/', the first item is removed and used to do + a lookup on self, if that fails a 404 is raised, if successful + the item is used to continue traversal (if the object found is + a Collection type) or to handle the request (if the object found + is a callable with .exposed set to True) + + if path equals '', a lookup for 'index' is done + + can be overridden in subclasses to implement different path + handling (PATH_INFO-like stuff) + """ + name = path.pop(0) + name = name.replace(".", "_") + resource = getattr(self, name, None) + if not resource: + raise HTTPError(404) + return lambda **args : ('text/json', json.write(resource(**args))) + _render_xmlhttp = True + +exported_methods = ExportedMethods() + +def patch_handler(handler_class): + """ This function takes care of adding necessary + attributed to Static objects + """ + for name, value in handler_class.__dict__.iteritems(): + if isinstance(value, Static) and value.path is None: + assert hasattr(handler_class, "static_dir") + value.path = os.path.join(str(handler_class.static_dir), + name + ".html") + +class TestHandler(BaseHTTPRequestHandler): + exported_methods = exported_methods + + def do_GET(self): + path, args = parse_url(self.path) + if not path: + path = ["index"] + name_path = path[0].replace(".", "_") + if len(path) > 1: + rest = os.path.sep.join(path[1:]) + else: + rest = None + method_to_call = getattr(self, name_path, None) + if method_to_call is None or not getattr(method_to_call, 'exposed', None): + exec_meth = getattr(self.exported_methods, name_path, None) + if exec_meth is None: + self.send_error(404, "File %s not found" % path) + else: + self.serve_data('text/json', json.write(exec_meth(**args)), + True) + else: + if rest: + outp = method_to_call(rest, **args) + else: + outp = method_to_call(**args) + if isinstance(outp, (str, unicode)): + self.serve_data('text/html', outp) + elif isinstance(outp, tuple): + self.serve_data(*outp) + else: + raise ValueError("Don't know how to serve %s" % (outp,)) + + def log_message(self, format, *args): + # XXX just discard it + pass + + do_POST = do_GET + + def serve_data(self, content_type, data, nocache=False): + self.send_response(200) + self.send_header("Content-type", content_type) + self.send_header("Content-length", len(data)) + if nocache: + self.send_nocache_headers() + self.end_headers() + self.wfile.write(data) + + def send_nocache_headers(self): + self.send_header('Expires', 'Mon, 26 Jul 1997 05:00:00 GMT') + self.send_header('Last-Modified', + time.strftime("%a, %d %b %Y %H:%M:%S GMT")) + self.send_header('Cache-Control', 'no-cache, must-revalidate') + self.send_header('Cache-Control', 'post-check=0, pre-check=0') + self.send_header('Pragma', 'no-cache') + +class Static(object): + exposed = True + + def __init__(self, path=None): + self.path = path + + def __call__(self): + return open(str(self.path)).read() + +class FsFile(object): + exposed = True + debug = False + def __init__(self, path, content_type="text/html"): + self._path = path + self._content_type = content_type + + _data = None + def __call__(self): + if self._data is None or self.debug: + self._data = self._path.read() + return ({'Content-Type': self._content_type}, self._data) + +class StaticDir(Collection): + exposed = True + + def __init__(self, path, type=None): + self.path = path + self.type = type + + def traverse(self, path, orgpath): + data = open(os.path.join(str(self.path), *path)).read() + if self.type: + return lambda : (self.type, data) + return lambda : data + +def create_server(server_address = ('', 8000), handler=TestHandler, + server=HTTPServer): + """ Parameters: + spawn - create new thread and return (by default it doesn't return) + fork - do a real fork + timeout - kill process after X seconds (actually doesn't work for threads) + port_file - function to be called with port number + """ + patch_handler(handler) + httpd = server(server_address, handler) + httpd.last_activity = time.time() + print "Server started, listening on %s:%s" %\ + (httpd.server_address[0],httpd.server_port) + return httpd + +def start_server_in_new_thread(server): + import thread + thread.start_new_thread(server.serve_forever, ()) + +def start_server_in_new_process(server, timeout=None): + pid = os.fork() + if not pid: + if timeout: + def f(httpd): + while 1: + time.sleep(.3) + if time.time() - httpd.last_activity > timeout: + httpd.server_close() + import os + os.kill(os.getpid(), 15) + import thread + thread.start_new_thread(f, (server,)) + + server.serve_forever() + os._exit(0) + return pid + +Handler = TestHandler +# deprecate TestHandler name + +class NewHandler(BaseHTTPRequestHandler): + """ BaseHTTPRequestHandler that does object publishing + """ + + application = None # attach web root (Collection object) here!! + bufsize = 1024 + + def do_GET(self, send_body=True): + """ perform a request """ + path, query = self.process_path(self.path) + _, args = parse_url("?" + query) + try: + resource = self.find_resource(path) + # XXX strange hack + if hasattr(resource, 'im_self'): + resource.im_self.server = self.server + retval = resource(**args) + if isinstance(retval, str): + headers = {'Content-Type': 'text/html'} + data = retval + else: + headers, data = retval + if isinstance(headers, str): + headers = {'Content-Type': headers} + except HTTPError, e: + status = e.status + headers, data = self.process_http_error(e) + except: + exc, e, tb = sys.exc_info() + tb_formatted = '\n'.join(traceback.format_tb(tb)) + status = 200 + data = 'An error has occurred: %s - %s\n\n%s' % (exc, e, + tb_formatted) + headers = {'Content-Type': 'text/plain'} + if hasattr(self.application, 'handle_error'): + self.application.handle_error(exc, e, tb) + else: + status = 200 + if not 'content-type' in [k.lower() for k in headers]: + headers['Content-Type'] = 'text/html; charset=UTF-8' + self.response(status, headers, data, send_body) + + do_POST = do_GET + + def do_HEAD(self): + return self.do_GET(False) + + def process_path(self, path): + """ split the path in a path and a query part# + + returns a tuple (path, query), where path is a string and + query a dictionary containing the GET vars (URL decoded and such) + """ + path = path.split('?') + if len(path) > 2: + raise ValueError('illegal path %s' % (path,)) + p = path[0] + q = len(path) > 1 and path[1] or '' + return p, q + + def find_resource(self, path): + """ find the resource for a given path + """ + if not path: + raise HTTPError(301, '/') + assert path.startswith('/') + chunks = path.split('/') + chunks.pop(0) # empty item + return self.application.traverse(chunks, path) + + def process_http_error(self, e): + """ create the response body and headers for errors + """ + headers = {'Content-Type': 'text/html'} # XXX need more headers here? + if e.status in [301, 302]: + headers['Location'] = e.data + body = 'Redirecting to %s' % (e.data,) + else: + message, explain = self.responses[e.status] + body = self.error_message_format % {'code': e.status, 'message': message, + 'explain': explain} + return headers, body + + def response(self, status, headers, body, send_body=True): + """ generate the HTTP response and send it to the client + """ + self.send_response(status) + if (isinstance(body, str) and + not 'content-length' in [k.lower() for k in headers]): + headers['Content-Length'] = len(body) + for keyword, value in headers.iteritems(): + self.send_header(keyword, value) + self.end_headers() + if not send_body: + return + if isinstance(body, str): + self.wfile.write(body) + elif hasattr(body, 'read'): + while 1: + data = body.read(self.bufsize) + if data == '': + break + self.wfile.write(data) + else: + raise ValueError('body is not a plain string or file-like object') Added: pypy/branch/flex-backend/translator/flex/lib/support.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/support.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,48 @@ + +""" Various simple support functions +""" + +from pypy.rpython.ootypesystem.bltregistry import described, load_dict_args,\ + MethodDesc + +from pypy.rpython.extfunc import genericcallable + +def callback(retval=None, args={}): + """ Variant of described decorator, which flows + an additional argument with return value of decorated + function, used specifically for callbacks + """ + def decorator(func): + defs = func.func_defaults + if defs is None: + defs = () + vars = func.func_code.co_varnames[:func.func_code.co_argcount] + if isinstance(args, dict): + arg_list = load_dict_args(vars, defs, args) + else: + arg_list = args + arg_list.append(("callback", genericcallable(args=[retval]))) + func._method = (func.__name__, MethodDesc(arg_list, retval)) + return func + + return decorator + +import sys, new +from pypy.translator.js.main import rpython2javascript + +def js_source(functions, use_pdb=True): + mod = new.module('_js_src') + function_names = [] + for func in functions: + name = func.__name__ + if hasattr(mod, name): + raise ValueError("exported function name %r is duplicated" + % (name,)) + mod.__dict__[name] = func + function_names.append(name) + sys.modules['_js_src'] = mod + try: + return rpython2javascript(mod, function_names, use_pdb=use_pdb) + finally: + del sys.modules['_js_src'] + Added: pypy/branch/flex-backend/translator/flex/lib/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/lib/test/test_server.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/test/test_server.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,77 @@ + +""" Server testing +""" + +import py +from pypy.translator.js.lib import server +from urllib import URLopener +import os + +class Handler(server.TestHandler): + def index(self): + return "xxx" + index.exposed = True + +def test_basic_startup(): + import thread + # XXX: how to do this without threads? + httpd = server.HTTPServer(('127.0.0.1', 21210), Handler) + thread.start_new_thread(httpd.serve_forever, ()) + assert URLopener().open("http://127.0.0.1:21210/index").read() == "xxx" + assert URLopener().open("http://127.0.0.1:21210/").read() == "xxx" + +def test_own_startup(): + httpd = server.create_server(server_address=('127.0.0.1', 21211), + handler=Handler) + server.start_server_in_new_thread(httpd) + assert URLopener().open("http://127.0.0.1:21210/index").read() == "xxx" + +def test_static_page(): + import thread + tmpdir = py.test.ensuretemp("server_static_page") + tmpdir.ensure("test.html").write("") + + class StaticHandler(server.TestHandler): + static_dir = str(tmpdir) + index = server.Static(os.path.join(static_dir, "test.html")) + + httpd = server.HTTPServer(('127.0.0.1', 21212), StaticHandler) + thread.start_new_thread(httpd.serve_forever, ()) + assert URLopener().open("http://127.0.0.1:21212/index").read() == \ + "" + +def test_static_page_implicit(): + import thread + tmpdir = py.test.ensuretemp("server_static_page_implicit") + tmpdir.ensure("index.html").write("") + + class StaticHandler(server.TestHandler): + static_dir = str(tmpdir) + index = server.Static() + + server.patch_handler(StaticHandler) + httpd = server.HTTPServer(('127.0.0.1', 21213), StaticHandler) + thread.start_new_thread(httpd.serve_forever, ()) + assert URLopener().open("http://127.0.0.1:21213/index").read() == \ + "" + + +def test_static_directory(): + py.test.skip("Fails") + import thread + tmpdir = py.test.ensuretemp("server_static_dir") + tmpdir.ensure("a", dir=1) + tmpdir.join("a").ensure("a.txt").write("aaa") + tmpdir.join("a").ensure("b.txt").write("bbb") + + class StaticDir(server.Handler): + static_dir = tmpdir + a_dir = server.StaticDir(tmpdir.join("a")) + + httpd = server.HTTPServer(('127.0.0.1', 0), StaticDir) + port = httpd.server_port + thread.start_new_thread(httpd.serve_forever, ()) + addr = "http://127.0.0.1:%d/" % port + assert URLopener().open(addr + "a_dir/a.txt").read() == "aaa" + assert URLopener().open(addr + "a_dir/b.txt").read() == "bbb" + Added: pypy/branch/flex-backend/translator/flex/lib/test/test_server_g.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/test/test_server_g.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,132 @@ +import py +from pypy.translator.js.lib.server import * + +class NonInitHandler(NewHandler): + request_version = '1.0' + + def __init__(self): + pass + + def log_request(self, code='-', size='-'): + pass + +class SomePage(object): + """ test resource """ + exposed = True + def __call__(self, handler, path, query): + return ('text/plain', 'foo') + +def raising_page(handler, path, query): + xxx +raising_page.exposed = True + +def build_app_structure(): + app = Collection() + app.sub = Collection() + app.sub = Collection() + app.sub.index = SomePage() + app.error = raising_page + return app + +class TestCollection(object): + def test_traverse(self): + app = build_app_structure() + assert app.traverse(['sub', 'index'], '/sub/index') is app.sub.index + assert app.traverse(['sub', ''], '/sub/') is app.sub.index + try: + app.traverse(['sub'], '/sub') + except HTTPError, e: + assert e.status == 301 + assert e.data == '/sub/' + else: + py.test.fail('should have redirected') + # 404 errors (first -> no index) + py.test.raises(HTTPError, "app.traverse([''], '/')") + py.test.raises(HTTPError, "app.traverse(['other', ''], '/other/')") + +class TestResource(object): + pass + +class TestHandler(object): + def setup_method(self, method): + self.handler = NonInitHandler() + self.handler.wfile = self.wfile = py.std.StringIO.StringIO() + + def test_process_path(self): + path, query = self.handler.process_path('') + assert path == '' + assert query == '' + + path, query = self.handler.process_path('/foo') + assert path == '/foo' + assert query == '' + + path, query = self.handler.process_path('/foo?bar') + assert path == '/foo' + assert query == 'bar' + + py.test.raises(ValueError, "self.handler.process_path('/foo?bar?baz')") + + def test_find_resource(self): + app = build_app_structure() + self.handler.application = app + assert self.handler.find_resource('/sub/index') is app.sub.index + assert self.handler.find_resource('/sub/') is app.sub.index + try: + self.handler.find_resource('/sub') + except HTTPError, e: + assert e.status == 301 + assert e.data == '/sub/' + else: + py.test.raises('should have raised a redirect') + try: + self.handler.find_resource('') + except HTTPError, e: + assert e.status == 301 + assert e.data == '/' + else: + py.test.raises('should have raised a redirect') + py.test.raises(HTTPError, "self.handler.find_resource('/foo/')") + + def test_response(self): + self.handler.response(200, {'Content-Type': 'text/plain'}, 'foo') + response = self.wfile.getvalue() + assert response.startswith('HTTP/1.0 200 OK') + assert 'Content-Type: text/plain\r\n' in response + assert 'Content-Length: 3\r\n' in response + assert response.endswith('\r\n\r\nfoo') + + def test_get_response_file(self): + rfile = py.std.StringIO.StringIO() + rfile.write('foo\nbar\nbaz') + rfile.seek(0) + self.handler.response(200, {'Content-Type': 'text/plain'}, rfile) + response = self.wfile.getvalue() + assert response.endswith('\r\n\r\nfoo\nbar\nbaz') + + def test_get_response_wrong_body(self): + py.test.raises(ValueError, "self.handler.response(200, {}, u'xxx')") + + def test_handle_error(self): + l = [] + def f(*args): + l.append(args) + app = build_app_structure() + self.handler.application = app + self.handler.path = '/error' + self.handler.do_GET() + self.handler.application.handle_error = f + self.handler.do_GET() + assert len(l) == 1 + +class TestFsFile(object): + def test_call(self): + temp = py.test.ensuretemp('TestStaticResource.test_handle') + foo = temp.ensure('foo.txt') + foo.write('foo') + r = FsFile(foo, 'text/plain') + ret = r()#None, '/bar/foo.txt', '') + assert ret[0] == {'Content-Type': 'text/plain'} + assert ret[1] == 'foo' + + Added: pypy/branch/flex-backend/translator/flex/lib/test/test_support.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/test/test_support.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,15 @@ + +""" Tests for support module +""" + +from pypy.translator.js.lib import support + +def test_callback(): + @support.callback(retval=int) + def f(self, a=8, b=3.2): + pass + + methdesc = f._method[1] + assert len(methdesc.args) == 3 + assert methdesc.args[2].name == 'callback' + Added: pypy/branch/flex-backend/translator/flex/lib/test/test_url.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/test/test_url.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ + +from pypy.translator.js.lib.url import parse_url + +def test_url(): + assert parse_url("path") == (["path"], {}) + assert parse_url("a/b/c/d") == (["a", "b", "c", "d"], {}) + assert parse_url("/a/b") == (["a", "b"], {}) + assert parse_url("/a/b/c/") == (["a", "b", "c"], {}) + assert parse_url("a/b?q=a&c=z") == (["a","b"], {"q":"a", "c":"z"}) + got = parse_url('/get_message?sid=2ed&pid=-1') + assert got == (["get_message"], {'pid':'-1', 'sid':'2ed'}) Added: pypy/branch/flex-backend/translator/flex/lib/url.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/lib/url.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,45 @@ + +""" Some support files for mapping urls, mostly bindings +for existing cgi stuff +""" + +import cgi +import urllib + +class URL(object): + def __init__(self, path, vars): + self.path = path + self.vars = vars + + def __eq__(self, other): + if isinstance(other, URL): + return self.path == other.path and self.vars == other.vars + if isinstance(other, tuple): + if len(other) != 2: + return False + return self.path, self.vars == other + return False + + def __ne__(self, other): + return not self == other + + def __iter__(self): + return iter((self.path, self.vars)) + +def parse_url(path): + """ Parse a/b/c?q=a into ('a', 'b', 'c') {'q':'a'} + """ + if '?' in path: + path, var_str = path.split("?") + vars_orig = cgi.parse_qs(var_str) + # if vars has a list inside... + vars = {} + for i, v in vars_orig.items(): + if isinstance(v, list): + vars[i] = v[0] + else: + vars[i] = v + else: + vars = {} + parts = [urllib.unquote(i) for i in path.split("/") if i] + return URL(parts, vars) Added: pypy/branch/flex-backend/translator/flex/log.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/log.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,4 @@ +import py +from pypy.tool.ansi_print import ansi_log +log = py.log.Producer("flex") +py.log.setconsumer("flex", ansi_log) Added: pypy/branch/flex-backend/translator/flex/main.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/main.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,166 @@ +"""Contains high level javascript compilation function +""" + +import autopath + +#from pypy.translator.flex.test.runtest import compile_function +#from pypy.translator.translator import TranslationContext +from pypy.translator.driver import TranslationDriver +from pypy.translator.flex.js import JS +from pypy.tool.error import AnnotatorError, FlowingError, debug +from pypy.rlib.nonconst import NonConstant +from pypy.annotation.policy import AnnotatorPolicy +from py.compat import optparse +from pypy.config.config import OptionDescription, BoolOption, StrOption +from pypy.config.config import Config, to_optparse +import py +import sys + +js_optiondescr = OptionDescription("jscompile", "", [ + BoolOption("view", "View flow graphs", + default=False, cmdline="--view"), + BoolOption("use_pdb", "Use debugger", + default=False, cmdline="--pdb"), + StrOption("output", "File to save results (default output.mxml)", + default="output.mxml", cmdline="--output")]) + + +class FunctionNotFound(Exception): + pass + +class BadSignature(Exception): + pass + +class JsPolicy(AnnotatorPolicy): + allow_someobjects = False + +def get_args(func_data): + l = [] + for i in xrange(func_data.func_code.co_argcount): + l.append("NonConstant(%s)" % repr(func_data.func_defaults[i])) + return ",".join(l) + +def get_arg_names(func_data): + return ",".join(func_data.func_code.co_varnames\ + [:func_data.func_code.co_argcount]) + +def rpython2javascript_main(argv, jsconfig): + if len(argv) == 0: + print "usage: module " + sys.exit(0) + module_name = argv[0] + if not module_name.endswith('.py'): + module_name += ".py" + mod = py.path.local(module_name).pyimport() + if len(argv) == 1: + function_names = [] + for function_name in dir(mod): + function = getattr(mod, function_name) + if callable(function) and getattr(function, '_client', False): + function_names.append( function_name ) + if not function_names: + print "Cannot find any function with _client=True in %s"\ + % module_name + sys.exit(1) + else: + function_names = argv[1:] + source = rpython2javascript(mod, function_names, jsconfig=jsconfig) + if not source: + print "Exiting, source not generated" + sys.exit(1) + open(jsconfig.output, "w").write(source) + print "Written file %s" % jsconfig.output + +# some strange function source +source_ssf_base = """ + +import %(module_name)s +from pypy.translator.flex.helper import __show_traceback +from pypy.rlib.nonconst import NonConstant as NonConst + +%(function_defs)s + +def some_strange_function_which_will_never_be_called(): + +%(functions)s +""" + +wrapped_function_def_base = """ +def %(fun_name)s(%(arg_names)s): + try: + traceback_handler.enter(NonConst("entrypoint"), NonConst("()"), NonConst(""), NonConst(0)) + %(module_name)s.%(fun_name)s(%(arg_names)s) + traceback_handler.leave(NonConst("entrypoint")) + except Exception, e: + new_tb = traceback_handler.tb[:] + __show_traceback(new_tb, str(e)) + +%(fun_name)s.explicit_traceback = True +""" + +function_base = "%(module_name)s.%(fun_name)s(%(args)s)" +wrapped_function_base = "%(fun_name)s(%(args)s)" + +def get_source_ssf(mod, module_name, function_names): + #source_ssf = "\n".join(["import %s" % module_name, "def some_strange_function_which_will_never_be_called():"] + [" "+\ + # module_name+"."+fun_name+get_args(mod.__dict__[fun_name]) for fun_name in function_names]) + function_list = [] + function_def_list = [] + for fun_name in function_names: + args = get_args(mod.__dict__[fun_name]) + arg_names = get_arg_names(mod.__dict__[fun_name]) + base = function_base + function_list.append(py.code.Source(base % locals())) + function_defs = "\n\n".join([str(i) for i in function_def_list]) + functions = "\n".join([str(i.indent()) for i in function_list]) + retval = source_ssf_base % locals() + print retval + return retval + +def rpython2javascript(mod, function_names, jsconfig=None, use_pdb=False): + if isinstance(function_names, str): + function_names = [function_names] + # avoid confusion + if mod is None: + # this means actual module, which is quite hairy to get in python, + # so we cheat + import sys + mod = sys.modules[sys._getframe(1).f_globals['__name__']] + + if jsconfig is None: + jsconfig = Config(js_optiondescr) + if use_pdb: + jsconfig.use_pdb = True + module_name = mod.__name__ + if not function_names and 'main' in mod.__dict__: + function_names.append('main') + for func_name in function_names: + if func_name not in mod.__dict__: + raise FunctionNotFound("function %r was not found in module %r" % (func_name, module_name)) + func_code = mod.__dict__[func_name] + if func_code.func_defaults: + lgt = len(func_code.func_defaults) + else: + lgt = 0 + if func_code.func_code.co_argcount > 0 and func_code.func_code. \ + co_argcount != lgt: + raise BadSignature("Function %s does not have default arguments" % func_name) + source_ssf = get_source_ssf(mod, module_name, function_names) + exec(source_ssf) in globals() + # now we gonna just cut off not needed function + # XXX: Really do that + #options = optparse.Values(defaults=DEFAULT_OPTIONS) + from pypy.config.pypyoption import get_pypy_config + config = get_pypy_config(translating=True) + driver = TranslationDriver(config=config) + try: + driver.setup(some_strange_function_which_will_never_be_called, [], policy = JsPolicy()) + driver.proceed(["compile_flex"]) + if jsconfig.view: + driver.translator.view() + return driver.gen.tmpfile.open().read() + # XXX: Add some possibility to write down selected file + except Exception, e: + # do something nice with it + raise + debug(driver, use_pdb) Added: pypy/branch/flex-backend/translator/flex/metavm.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/metavm.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,297 @@ +""" Opcode meaning objects, descendants of MicroInstruction +""" + +#from pypy.translator.js.jsbuiltin import Builtins +from pypy.translator.oosupport.metavm import PushArg, PushAllArgs, StoreResult,\ + InstructionList, New, GetField, MicroInstruction + +from pypy.translator.flex.log import log +from pypy.rpython.ootypesystem import ootype +from types import FunctionType +from pypy.objspace.flow.model import Constant + +class NewBuiltin(MicroInstruction): + def __init__(self, arg): + self.arg = arg + + def render(self, generator, op): + generator.ilasm.new(self.arg) + +class _ListSetitem(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[1]) + generator.load(op.args[3]) + generator.load(op.args[2]) + generator.ilasm.list_setitem() +ListSetitem = _ListSetitem() + +class _ListGetitem(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[1]) + generator.load(op.args[2]) + generator.ilasm.list_getitem() +ListGetitem = _ListGetitem() + +class _ListContains(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[1]) + generator.load(op.args[2]) + generator.ilasm.list_getitem() + generator.ilasm.load_void() + generator.emit('!=') +ListContains = _ListContains() + +class _Call(MicroInstruction): + def render(self, generator, op): + graph = op.args[0].value.graph + self._render_function(generator, graph, op.args) + + def _render_builtin(self, generator, builtin, args): + for func_arg in args[1:]: # push parameters + generator.load(func_arg) + generator.call_external(builtin, args[1:]) + + def _render_builtin_prepared_args(self, generator, builtin, args): + for func_arg in args: + generator.load_str(func_arg) + generator.call_external(builtin, args) + + def _render_builtin_method(self, generator, builtin, args): + for func_arg in args: + generator.load_special(func_arg) + generator.call_external_method(builtin, len(args)-1) + + def _render_function(self, generator, graph, args): + for func_arg in args[1:]: # push parameters + if func_arg.concretetype is not ootype.Void: + generator.load(func_arg) + generator.call_graph(graph) + + def _render_method(self, generator, method_name, args): + this = args[0] + for arg in args: # push parametes + generator.load(arg) + generator.call_method(this.concretetype, method_name) + +Call = _Call() + +class CallBuiltin(_Call): + def __init__(self, builtin): + self.builtin = builtin + + def render(self, generator, op): + self._render_builtin(generator, self.builtin, op.args) + +class CallBuiltinMethod(_Call): + def __init__(self, builtin, slice=None, additional_args=[]): + self.builtin = builtin + self.slice = slice + self.additional_args = additional_args + + def render(self, generator, op): + if self.slice is not None: + args = op.args[self.slice] + else: + args = op.args + args += self.additional_args + self._render_builtin_method(generator, self.builtin, args) + +class _SameAs(MicroInstruction): + def render(self, generator, op): + generator.change_name(op.result, op.args[0]) + +class _CastFun(MicroInstruction): + def __init__(self, name, num): + self.name = name + self.num = num + + def render(self, generator, op): + log("Args: %r"%op.args) + generator.cast_function(self.name, self.num) + +class _Prefix(MicroInstruction): + def __init__(self, st): + self.st = st + + def render(self, generator, op): + generator.prefix_op(self.st) + +class _NotImplemented(MicroInstruction): + def __init__(self, reason): + self.reason = reason + + def render(self, generator, op): + raise NotImplementedError(self.reason) + +class _CastMethod(MicroInstruction): + def __init__(self, method_name, num=0): + self.method_name = method_name + self.num = num + + def render(self, generator, op): + generator.call_external_method(self.method_name, self.num) + +class _LoadConst(MicroInstruction): + def __init__(self, value): + self.value = value + + def render(self, generator, op): + generator.load(Constant(self.value, ootype.typeOf(self.value))) + +class _GetBuiltinField(MicroInstruction): + def render(self, generator, op): + this = op.args[0] + field = op.args[1].value[1:] + generator.load(this) + generator.get_field(None, field) + +class _GetPredefinedField(MicroInstruction): + def __init__(self, field): + self.field = field + + def render(self, generator, op): + this = op.args[1] + generator.load(this) + generator.get_field(None, self.field) + +GetBuiltinField = _GetBuiltinField() + +class _SetBuiltinField(MicroInstruction): + def render(self, generator, op): + this = op.args[0] + field = op.args[1].value + if not field.startswith('o'): + generator.load_void() + else: + value = op.args[2] + field_name = field[1:] + self.run_it(generator, this, field_name, value) + + def run_it(self, generator, this, field_name, value): + generator.load(this) + generator.load_special(value) + generator.set_field(None, field_name) + +class _SetPredefinedField(_SetBuiltinField): + def __init__(self, field): + self.field = field + + def render(self, generator, op): + value = op.args[2] + this = op.args[1] + self.run_it(generator, this, self.field, value) + +class _SetExternalField(_SetBuiltinField): + def render(self, generator, op): + self.run_it(generator, op.args[0], op.args[1].value, op.args[2]) + +SetBuiltinField = _SetBuiltinField() +SetExternalField = _SetExternalField() + +class _CallMethod(_Call): + def render(self, generator, op): + method = op.args[0] + self._render_method(generator, method.value, op.args[1:]) + +class _CallBuiltinObject(_Call): + def render(self, generator, op): + this = op.args[1].concretetype + method = op.args[0] + method_name = this._methods[method.value]._name[1:] + generator.load(op.args[1]) + self._render_builtin_method(generator, method_name, op.args[1:]) + +class _CallExternalObject(_Call): + def render(self, generator, op): + this = op.args[1].concretetype + method = op.args[0] + method_name = method.value + #generator.load(op.args[1]) + self._render_builtin_method(generator, method_name, op.args[1:]) + +CallBuiltinObject = _CallBuiltinObject() +CallExternalObject = _CallExternalObject() + +class _IsInstance(MicroInstruction): + def render(self, generator, op): + # FIXME: just temporary hack + generator.load(op.args[0]) + generator.ilasm.load_const(op.args[1].value._name.replace('.', '_'))#[-1]) + generator.cast_function("isinstanceof", 2) + +class _IndirectCall(MicroInstruction): + def render(self, generator, op): + for func_arg in op.args[1:]: # push parameters + generator.load(func_arg) + generator.call_external(op.args[0].name, op.args[1:]) + +class _SetTimeout(MicroInstruction): + # FIXME: Dirty hack for javascript callback stuff + def render(self, generator, op): + val = op.args[1].value + assert(isinstance(val, ootype._static_meth)) + #if isinstance(val, ootype.StaticMethod): + real_name = val._name + generator.db.pending_function(val.graph) + #generator.db.pending_function(val.graph) + #else: + # concrete = val.concretize() + # real_name = concrete.value._name + # generator.db.pending_function(concrete.value.graph) + generator.load_str("'%s()'" % real_name) + generator.load(op.args[2]) + generator.call_external('setTimeout',[0]*2) + +class _DiscardStack(MicroInstruction): + def render(self, generator, op): + generator.clean_stack() + +class SetOnEvent(MicroInstruction): + def __init__(self, field): + self.field = field + + # FIXME: Dirty hack for javascript callback stuff + def render(self, generator, op): + val = op.args[1].value + val = val.concretize().value + assert(isinstance(val, ootype._static_meth)) + real_name = val._name + generator.db.pending_function(val.graph) + generator.load_str("document") + generator.load_str(real_name) + generator.set_field(None, self.field) + +class _CheckLength(MicroInstruction): + def render(self, generator, op): + assert not generator.ilasm.right_hand + +class _ListRemove(MicroInstruction): + def render(self, generator, op): + generator.list_getitem(op.args[1], op.args[2]) + generator.call_external('delete', [0]) + +ListRemove = _ListRemove() +CheckLength = _CheckLength() +SetTimeout = _SetTimeout() +IndirectCall = _IndirectCall() +IsInstance = _IsInstance() +CallMethod = _CallMethod() +CopyName = [PushAllArgs, _SameAs ()] +CastString = _CastFun("convertToString", 1) +SameAs = CopyName +DiscardStack = _DiscardStack() + +def fix_opcodes(opcodes): + for key, value in opcodes.iteritems(): + if type(value) is str: + value = InstructionList([PushAllArgs, value, StoreResult, CheckLength]) + elif value == []: + value = InstructionList([CheckLength]) + elif value is not None: + if StoreResult not in value: + value.append(StoreResult) + if CheckLength not in value: + value.append(CheckLength) + value = InstructionList(value) + + opcodes[key] = value Added: pypy/branch/flex-backend/translator/flex/modules/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/__init__.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,3 @@ + +""" Support classes needed for javascript to translate +""" Added: pypy/branch/flex-backend/translator/flex/modules/dom.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/dom.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,815 @@ + +"""Document Object Model support + + this provides a mock browser API, both the standard DOM level 2 stuff as + the browser-specific additions + + in addition this provides the necessary descriptions that allow rpython + code that calls the browser DOM API to be translated + + note that the API is not and will not be complete: more exotic features + will most probably not behave as expected, or are not implemented at all + + http://www.w3.org/DOM/ - main standard + http://www.w3schools.com/dhtml/dhtml_dom.asp - more informal stuff + http://developer.mozilla.org/en/docs/Gecko_DOM_Reference - Gecko reference +""" + +import time +import re +import urllib +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.rlib.nonconst import NonConstant + +from pypy.rpython.extfunc import genericcallable, register_external +from xml.dom import minidom + +from pypy.annotation.signature import annotation +from pypy.annotation import model as annmodel + +# EventTarget is the base class for Nodes and Window +class EventTarget(BasicExternal): + def addEventListener(self, type, listener, useCapture): + if not hasattr(self._original, '_events'): + self._original._events = [] + # XXX note that useCapture is ignored... + self._original._events.append((type, listener, useCapture)) + + def dispatchEvent(self, event): + if event._cancelled: + return + event.currentTarget = self + if event.target is None: + event.target = self + if event.originalTarget is None: + event.originalTarget = self + if hasattr(self._original, '_events'): + for etype, handler, capture in self._original._events: + if etype == event.type: + handler(event) + if event._cancelled or event.cancelBubble: + return + parent = getattr(self, 'parentNode', None) + if parent is not None: + parent.dispatchEvent(event) + + def removeEventListener(self, type, listener, useCapture): + if not hasattr(self._original, '_events'): + raise ValueError('no registration for listener') + filtered = [] + for data in self._original._events: + if data != (type, listener, useCapture): + filtered.append(data) + if filtered == self._original._events: + raise ValueError('no registration for listener') + self._original._events = filtered + +# XML node (level 2 basically) implementation +# the following classes are mostly wrappers around minidom nodes that try to +# mimic HTML DOM behaviour by implementing browser API and changing the +# behaviour a bit + +class Node(EventTarget): + """base class of all node types""" + _original = None + + def __init__(self, node=None): + self._original = node + + def __getattr__(self, name): + """attribute access gets proxied to the contained minidom node + + all returned minidom nodes are wrapped as Nodes + """ + try: + return super(Node, self).__getattr__(name) + except AttributeError: + pass + if (name not in self._fields and + (not hasattr(self, '_methods') or name not in self._methods)): + raise NameError, name + value = getattr(self._original, name) + return _wrap(value) + + def __setattr__(self, name, value): + """set an attribute on the wrapped node""" + if name in dir(self) or name.startswith('_'): + return super(Node, self).__setattr__(name, value) + if name not in self._fields: + raise NameError, name + setattr(self._original, name, value) + + def __eq__(self, other): + original = getattr(other, '_original', other) + return original is self._original + + def __ne__(self, other): + original = getattr(other, '_original', other) + return original is not self._original + + def getElementsByTagName(self, name): + name = name.lower() + return self.__getattr__('getElementsByTagName')(name) + + def __repr__(self): + return '<%s %s>' % (self.__class__.__name__, self.nodeName) + + def _getClassName(self): + return self.getAttribute('class') + + def _setClassName(self, name): + self.setAttribute('class', name) + + className = property(_getClassName, _setClassName) + + def _getId(self): + return self.getAttribute('id') + + def _setId(self, id): + self.setAttribute('id', id) + + id = property(_getId, _setId) + +class Element(Node): + nodeType = 1 + style = None + + def _style(self): + style = getattr(self._original, '_style', None) + if style is not None: + return style + styles = {} + if self._original.hasAttribute('style'): + for t in self._original.getAttribute('style').split(';'): + name, value = t.split(':') + dashcharpairs = re.findall('-\w', name) + for p in dashcharpairs: + name = name.replace(p, p[1].upper()) + styles[name.strip()] = value.strip() + style = Style(styles) + self._original._style = style + return style + style = property(_style) + + def _nodeName(self): + return self._original.nodeName.upper() + nodeName = property(_nodeName) + + def _get_innerHTML(self): + ret = [] + for child in self.childNodes: + ret.append(_serialize_html(child)) + return ''.join(ret) + + def _set_innerHTML(self, html): + dom = minidom.parseString('%s' % (html,)) + while self.childNodes: + self.removeChild(self.lastChild) + for child in dom.documentElement.childNodes: + child = self.ownerDocument.importNode(child, True) + self._original.appendChild(child) + del dom + + innerHTML = property(_get_innerHTML, _set_innerHTML) + + def scrollIntoView(self): + pass + +class Attribute(Node): + nodeType = 2 + +class Text(Node): + nodeType = 3 + +class Comment(Node): + nodeType = 8 + +class Document(Node): + nodeType = 9 + + def createEvent(self, group=''): + """create an event + + note that the group argument is ignored + """ + if group in ('KeyboardEvent', 'KeyboardEvents'): + return KeyEvent() + elif group in ('MouseEvent', 'MouseEvents'): + return MouseEvent() + return Event() + + def getElementById(self, id): + nodes = self.getElementsByTagName('*') + for node in nodes: + if node.getAttribute('id') == id: + return node + +# the standard DOM stuff that doesn't directly deal with XML +# note that we're mimicking the standard (Mozilla) APIs, so things tested +# against this code may not work in Internet Explorer + +# XXX note that we store the events on the wrapped minidom node to avoid losing +# them on re-wrapping +class Event(BasicExternal): + def initEvent(self, type, bubbles, cancelable): + self.type = type + self.cancelBubble = not bubbles + self.cancelable = cancelable + self.target = None + self.currentTarget = None + self.originalTarget = None + self._cancelled = False + + def preventDefault(self): + if not self.cancelable: + raise TypeError('event can not be canceled') + self._cancelled = True + + def stopPropagation(self): + self.cancelBubble = True + +class KeyEvent(Event): + pass + +class MouseEvent(Event): + pass + +class Style(BasicExternal): + def __init__(self, styles={}): + for name, value in styles.iteritems(): + setattr(self, name, value) + + def __getattr__(self, name): + if name not in self._fields: + raise AttributeError, name + return None + + def _tostring(self): + ret = [] + for name in sorted(self._fields): + value = getattr(self, name, None) + if value is not None: + ret.append(' ') + for char in name: + if char.upper() == char: + char = '-%s' % (char.lower(),) + ret.append(char) + ret.append(': %s;' % (value,)) + return ''.join(ret[1:]) + +# non-DOM ('DOM level 0') stuff + +# Window is the main environment, the root node of the JS object tree + +class Location(BasicExternal): + _fields = { + 'hostname' : str, + 'href' : str, + 'hash' : str, + 'host' : str, + 'pathname' : str, + 'port' : str, + 'protocol' : str, + 'search' : str, + } + _methods = { + 'assign' : MethodDesc([str]), + 'reload' : MethodDesc([bool]), + 'replace' : MethodDesc([str]), + 'toString' : MethodDesc([], str), + } + +class Navigator(BasicExternal): + def __init__(self): + self.appName = 'Netscape' + +class Window(EventTarget): + def __init__(self, html=('Untitled document' + ''), parent=None): + super(Window, self).__init__() + self._html = html + self.document = Document(minidom.parseString(html)) + + # references to windows + self.content = self + self.self = self + self.window = self + self.parent = parent or self + self.top = self.parent + while 1: + if self.top.parent is self.top: + break + self.top = self.top.parent + + # other properties + self.closed = True + self._location = 'about:blank' + + self._original = self # for EventTarget interface (XXX a bit nasty) + + self.navigator = Navigator() + + def __getattr__(self, name): + return globals()[name] + + def _getLocation(self): + return self._location + + def _setLocation(self, newloc): + url = urllib.urlopen(newloc) + html = url.read() + self.document = Document(minidom.parseString(html)) + + location = property(_getLocation, _setLocation) + +scrollX = 0 +scrollMaxX = 0 +scrollY = 0 +scrollMaxY = 0 + +def some_fun(): + pass + +def setTimeout(func, delay): + pass +register_external(setTimeout, args=[genericcallable([]), int], result=None) + +window = Window() +document = window.document +Window._render_name = 'window' +Document._render_name = 'document' + +# rtyper stuff + +EventTarget._fields = { + 'onabort' : genericcallable([Event]), + 'onblur' : genericcallable([Event]), + 'onchange' : genericcallable([Event]), + 'onclick' : genericcallable([MouseEvent]), + 'onclose' : genericcallable([MouseEvent]), + 'ondblclick' : genericcallable([MouseEvent]), + 'ondragdrop' : genericcallable([MouseEvent]), + 'onerror' : genericcallable([MouseEvent]), + 'onfocus' : genericcallable([Event]), + 'onkeydown' : genericcallable([KeyEvent]), + 'onkeypress' : genericcallable([KeyEvent]), + 'onkeyup' : genericcallable([KeyEvent]), + 'onload' : genericcallable([KeyEvent]), + 'onmousedown' : genericcallable([MouseEvent]), + 'onmousemove' : genericcallable([MouseEvent]), + 'onmouseup' : genericcallable([MouseEvent]), + 'onmouseover' : genericcallable([MouseEvent]), + 'onresize' : genericcallable([Event]), + 'onscroll' : genericcallable([MouseEvent]), + 'onselect' : genericcallable([MouseEvent]), + 'onsubmit' : genericcallable([MouseEvent]), + 'onunload' : genericcallable([Event]), +} + +lambda_returning_true = genericcallable([Event]) + +EventTarget._methods = { + 'addEventListener' : MethodDesc([str, lambda_returning_true, bool]), + 'dispatchEvent' : MethodDesc([str], bool), + 'removeEventListener' : MethodDesc([str, lambda_returning_true, bool]), +} + +Node._fields = EventTarget._fields.copy() +Node._fields.update({ + 'childNodes' : [Element], + 'firstChild' : Element, + 'lastChild' : Element, + 'localName' : str, + 'name' : str, + 'namespaceURI' : str, + 'nextSibling' : Element, + 'nodeName' : str, + 'nodeType' : int, + 'nodeValue' : str, + 'ownerDocument' : Document, + 'parentNode' : Element, + 'prefix' : str, + 'previousSibling': Element, + 'tagName' : str, + 'textContent' : str, +}) + +Node._methods = EventTarget._methods.copy() +Node._methods.update({ + 'appendChild' : MethodDesc([Element]), + 'cloneNode' : MethodDesc([int], Element), + 'getElementsByTagName' : MethodDesc([str], [Element]), + 'hasChildNodes' : MethodDesc([], bool), + 'insertBefore' : MethodDesc([Element], Element), + 'normalize' : MethodDesc([]), + 'removeChild' : MethodDesc([Element]), + 'replaceChild' : MethodDesc([Element], Element), +}) + +Element._fields = Node._fields.copy() +Element._fields.update({ + 'attributes' : [Attribute], + 'className' : str, + 'clientHeight' : int, + 'clientWidth' : int, + 'clientLeft' : int, + 'clientTop' : int, + 'dir' : str, + 'innerHTML' : str, + 'id' : str, + 'lang' : str, + 'offsetHeight' : int, + 'offsetLeft' : int, + 'offsetParent' : int, + 'offsetTop' : int, + 'offsetWidth' : int, + 'scrollHeight' : int, + 'scrollLeft' : int, + 'scrollTop' : int, + 'scrollWidth' : int, + 'disabled': bool, + # HTML specific + 'style' : Style, + 'tabIndex' : int, + # XXX: From HTMLInputElement to make pythonconsole work. + 'value': str, + 'checked': bool, + # IMG specific + 'src': str, +}) + +Element._methods = Node._methods.copy() +Element._methods.update({ + 'getAttribute' : MethodDesc([str], str), + 'getAttributeNS' : MethodDesc([str], str), + 'getAttributeNode' : MethodDesc([str], Element), + 'getAttributeNodeNS' : MethodDesc([str], Element), + 'hasAttribute' : MethodDesc([str], bool), + 'hasAttributeNS' : MethodDesc([str], bool), + 'hasAttributes' : MethodDesc([], bool), + 'removeAttribute' : MethodDesc([str]), + 'removeAttributeNS' : MethodDesc([str]), + 'removeAttributeNode' : MethodDesc([Element], str), + 'setAttribute' : MethodDesc([str, str]), + 'setAttributeNS' : MethodDesc([str]), + 'setAttributeNode' : MethodDesc([Element], Element), + 'setAttributeNodeNS' : MethodDesc([str, Element], Element), + # HTML specific + 'blur' : MethodDesc([]), + 'click' : MethodDesc([]), + 'focus' : MethodDesc([]), + 'scrollIntoView' : MethodDesc([]), + 'supports' : MethodDesc([str, float]), +}) + +Document._fields = Node._fields.copy() +Document._fields.update({ + 'characterSet' : str, + # 'contentWindow' : Window(), XXX doesn't exist, only on iframe + 'doctype' : str, + 'documentElement' : Element, + 'styleSheets' : [Style], + 'alinkColor' : str, + 'bgColor' : str, + 'body' : Element, + 'cookie' : str, + 'defaultView' : Window, + 'domain' : str, + 'embeds' : [Element], + 'fgColor' : str, + 'forms' : [Element], + 'height' : int, + 'images' : [Element], + 'lastModified' : str, + 'linkColor' : str, + 'links' : [Element], + 'referrer' : str, + 'title' : str, + 'URL' : str, + 'vlinkColor' : str, + 'width' : int, +}) + +Document._methods = Node._methods.copy() +Document._methods.update({ + 'createAttribute' : MethodDesc([str], Element), + 'createDocumentFragment' : MethodDesc([], Element), + 'createElement' : MethodDesc([str], Element), + 'createElementNS' : MethodDesc([str], Element), + 'createEvent' : MethodDesc([str], Event), + 'createTextNode' : MethodDesc([str], Element), + #'createRange' : MethodDesc(["aa"], Range()) - don't know what to do here + 'getElementById' : MethodDesc([str], Element), + 'getElementsByName' : MethodDesc([str], [Element]), + 'importNode' : MethodDesc([Element, bool], Element), + 'clear' : MethodDesc([]), + 'close' : MethodDesc([]), + 'open' : MethodDesc([]), + 'write' : MethodDesc([str]), + 'writeln' : MethodDesc([str]), +}) + +Window._fields = EventTarget._fields.copy() +Window._fields.update({ + 'content' : Window, + 'closed' : bool, + # 'crypto' : Crypto() - not implemented in Gecko, leave alone + 'defaultStatus' : str, + 'document' : Document, + # 'frameElement' : - leave alone + 'frames' : [Window], + 'history' : [str], + 'innerHeight' : int, + 'innerWidth' : int, + 'length' : int, + 'location' : Location, + 'name' : str, + # 'preference' : # denied in gecko + 'opener' : Window, + 'outerHeight' : int, + 'outerWidth' : int, + 'pageXOffset' : int, + 'pageYOffset' : int, + 'parent' : Window, + # 'personalbar' : - disallowed + # 'screen' : Screen() - not part of the standard, allow it if you want + 'screenX' : int, + 'screenY' : int, + 'scrollMaxX' : int, + 'scrollMaxY' : int, + 'scrollX' : int, + 'scrollY' : int, + 'self' : Window, + 'status' : str, + 'top' : Window, + 'window' : Window, + 'navigator': Navigator, +}) + +Window._methods = Node._methods.copy() +Window._methods.update({ + 'alert' : MethodDesc([str]), + 'atob' : MethodDesc([str], str), + 'back' : MethodDesc([]), + 'blur' : MethodDesc([]), + 'btoa' : MethodDesc([str], str), + 'close' : MethodDesc([]), + 'confirm' : MethodDesc([str], bool), + 'dump' : MethodDesc([str]), + 'escape' : MethodDesc([str], str), + #'find' : MethodDesc(["aa"], - gecko only + 'focus' : MethodDesc([]), + 'forward' : MethodDesc([]), + 'getComputedStyle' : MethodDesc([Element, str], Style), + 'home' : MethodDesc([]), + 'open' : MethodDesc([str]), +}) + +Style._fields = { + 'azimuth' : str, + 'background' : str, + 'backgroundAttachment' : str, + 'backgroundColor' : str, + 'backgroundImage' : str, + 'backgroundPosition' : str, + 'backgroundRepeat' : str, + 'border' : str, + 'borderBottom' : str, + 'borderBottomColor' : str, + 'borderBottomStyle' : str, + 'borderBottomWidth' : str, + 'borderCollapse' : str, + 'borderColor' : str, + 'borderLeft' : str, + 'borderLeftColor' : str, + 'borderLeftStyle' : str, + 'borderLeftWidth' : str, + 'borderRight' : str, + 'borderRightColor' : str, + 'borderRightStyle' : str, + 'borderRightWidth' : str, + 'borderSpacing' : str, + 'borderStyle' : str, + 'borderTop' : str, + 'borderTopColor' : str, + 'borderTopStyle' : str, + 'borderTopWidth' : str, + 'borderWidth' : str, + 'bottom' : str, + 'captionSide' : str, + 'clear' : str, + 'clip' : str, + 'color' : str, + 'content' : str, + 'counterIncrement' : str, + 'counterReset' : str, + 'cssFloat' : str, + 'cssText' : str, + 'cue' : str, + 'cueAfter' : str, + 'onBefore' : str, + 'cursor' : str, + 'direction' : str, + 'displays' : str, + 'elevation' : str, + 'emptyCells' : str, + 'font' : str, + 'fontFamily' : str, + 'fontSize' : str, + 'fontSizeAdjust' : str, + 'fontStretch' : str, + 'fontStyle' : str, + 'fontVariant' : str, + 'fontWeight' : str, + 'height' : str, + 'left' : str, + 'length' : str, + 'letterSpacing' : str, + 'lineHeight' : str, + 'listStyle' : str, + 'listStyleImage' : str, + 'listStylePosition' : str, + 'listStyleType' : str, + 'margin' : str, + 'marginBottom' : str, + 'marginLeft' : str, + 'marginRight' : str, + 'marginTop' : str, + 'markerOffset' : str, + 'marks' : str, + 'maxHeight' : str, + 'maxWidth' : str, + 'minHeight' : str, + 'minWidth' : str, + 'MozBinding' : str, + 'MozOpacity' : str, + 'orphans' : str, + 'outline' : str, + 'outlineColor' : str, + 'outlineStyle' : str, + 'outlineWidth' : str, + 'overflow' : str, + 'padding' : str, + 'paddingBottom' : str, + 'paddingLeft' : str, + 'paddingRight' : str, + 'paddingTop' : str, + 'page' : str, + 'pageBreakAfter' : str, + 'pageBreakBefore' : str, + 'pageBreakInside' : str, + 'parentRule' : str, + 'pause' : str, + 'pauseAfter' : str, + 'pauseBefore' : str, + 'pitch' : str, + 'pitchRange' : str, + 'playDuring' : str, + 'position' : str, + 'quotes' : str, + 'richness' : str, + 'right' : str, + 'size' : str, + 'speak' : str, + 'speakHeader' : str, + 'speakNumeral' : str, + 'speakPunctuation' : str, + 'speechRate' : str, + 'stress' : str, + 'tableLayout' : str, + 'textAlign' : str, + 'textDecoration' : str, + 'textIndent' : str, + 'textShadow' : str, + 'textTransform' : str, + 'top' : str, + 'unicodeBidi' : str, + 'verticalAlign' : str, + 'visibility' : str, + 'voiceFamily' : str, + 'volume' : str, + 'whiteSpace' : str, + 'widows' : str, + 'width' : str, + 'wordSpacing' : str, + 'zIndex' : str, +} + +Event._fields = { + 'bubbles': bool, + 'cancelBubble': bool, + 'cancelable': bool, + 'currentTarget': Element, + 'detail': int, + 'relatedTarget': Element, + 'target': Element, + 'type': str, + 'returnValue': bool, + 'which': int, + 'keyCode' : int, + 'charCode': int, + 'altKey' : bool, + 'ctrlKey' : bool, + 'shiftKey': bool, +} + +Event._methods = { + 'initEvent': MethodDesc([str, bool, bool]), + 'preventDefault': MethodDesc([]), + 'stopPropagation': MethodDesc([]), +} + +KeyEvent._methods = Event._methods.copy() + +KeyEvent._fields = Event._fields.copy() + +Navigator._methods = { +} +Navigator._fields = { + 'appName': str, +} + +class _FunctionWrapper(object): + """makes sure function return values are wrapped if appropriate""" + def __init__(self, callable): + self._original = callable + + def __call__(self, *args, **kwargs): + args = list(args) + for i, arg in enumerate(args): + if isinstance(arg, Node): + args[i] = arg._original + for name, arg in kwargs.iteritems(): + if isinstance(arg, Node): + kwargs[arg] = arg._original + value = self._original(*args, **kwargs) + return _wrap(value) + +_typetoclass = { + 1: Element, + 2: Attribute, + 3: Text, + 8: Comment, + 9: Document, +} +def _wrap(value): + if isinstance(value, minidom.Node): + nodeclass = _typetoclass[value.nodeType] + return nodeclass(value) + elif callable(value): + return _FunctionWrapper(value) + # nothing fancier in minidom, i hope... + # XXX and please don't add anything fancier either ;) + elif isinstance(value, list): + return [_wrap(x) for x in value] + return value + +# some helper functions + +def _quote_html(text): + for char, e in [('&', 'amp'), ('<', 'lt'), ('>', 'gt'), ('"', 'quot'), + ("'", 'apos')]: + text = text.replace(char, '&%s;' % (e,)) + return text + +_singletons = ['link', 'meta'] +def _serialize_html(node): + ret = [] + if node.nodeType in [3, 8]: + return node.nodeValue + elif node.nodeType == 1: + original = getattr(node, '_original', node) + nodeName = original.nodeName + ret += ['<', nodeName] + if len(node.attributes): + for aname in node.attributes.keys(): + if aname == 'style': + continue + attr = node.attributes[aname] + ret.append(' %s="%s"' % (attr.nodeName, + _quote_html(attr.nodeValue))) + styles = getattr(original, '_style', None) + if styles: + ret.append(' style="%s"' % (_quote_html(styles._tostring()),)) + if len(node.childNodes) or nodeName not in _singletons: + ret.append('>') + for child in node.childNodes: + if child.nodeType == 1: + ret.append(_serialize_html(child)) + else: + ret.append(_quote_html(child.nodeValue)) + ret += [''] + else: + ret.append(' />') + else: + raise ValueError('unsupported node type %s' % (node.nodeType,)) + return ''.join(ret) + +def alert(msg): + window.alert(msg) + +# initialization + +# set the global 'window' instance to an empty HTML document, override using +# dom.window = Window(html) (this will also set dom.document) + Added: pypy/branch/flex-backend/translator/flex/modules/flex.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/flex.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,19 @@ +from pypy.rpython.extfunc import genericcallable, register_external +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc + + +class Button( BasicExternal ): + + _fields = { + 'label' : str, + '_x' : int, + '_y' : int, + } + _render_class = 'Button' + +def addChild(e): + pass + +register_external(addChild, args=[Button]) +#addChild._annspecialcase_ = 'specialize:argtype(0)' + Added: pypy/branch/flex-backend/translator/flex/modules/mochikit.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/mochikit.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,75 @@ + +""" mochikit wrappers +""" + +from pypy.rpython.extfunc import genericcallable, register_external +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.translator.js.modules import dom + +# MochiKit.LoggingPane + +def createLoggingPane(var): + pass +register_external(createLoggingPane, args=[bool]) + +# MochiKit.Logging + +def log(data): + print data +register_external(log, args=None) + +def logDebug(data): + print "D:", data +register_external(logDebug, args=None) + +def logWarning(data): + print "Warning:", data +register_external(logWarning, args=None) + +def logError(data): + print "ERROR:", data +register_external(logError, args=None) + +def logFatal(data): + print "FATAL:", data +register_external(logFatal, args=None) + +# MochiKit.DOM + +def escapeHTML(data): + return data +register_external(escapeHTML, args=[str], result=str) + +# MochiKit.Base + +def serializeJSON(data): + pass +register_external(serializeJSON, args=None, result=str) + +# MochiKit.Signal + +class Event(BasicExternal): + pass + +Event._fields = { + '_event': dom.Event, +} + +Event._methods = { + 'preventDefault': MethodDesc([]), +} + + +def connect(src, signal, dest): + print 'connecting signal %s' % (signal,) +register_external(connect, args=[dom.EventTarget, str, genericcallable([Event])], + result=int) + +def disconnect(id): + pass +register_external(disconnect, args=[int]) + +def disconnectAll(src, signal): + print 'disconnecting all handlers for signal: %s' % (signal,) +register_external(disconnectAll, args=[dom.EventTarget, str]) + Added: pypy/branch/flex-backend/translator/flex/modules/pygame/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/__init__.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ +from pygame import * + +import mixer +import sprite +import font +import time +import event +import image + +class error(Exception): + pass \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/pygame/event.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/event.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,7 @@ +def get(): + return [Event() for x in range(2) ] + +class Event: + def __init__(self): + self.type = 1 + \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/pygame/font.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/font.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,9 @@ +import pygame + +class Font: + def __init__(self, fontfile, size): + self.size = size + + + def render(self, text, alpha, color): + return pygame.Surface( (len(text)*self.size, self.size) ) \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/pygame/image.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/image.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,4 @@ +import pygame + +def load(filename): + return pygame.Surface((10,10)) \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/pygame/locals.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/locals.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,227 @@ +ACTIVEEVENT = 1 +ANYFORMAT = 268435456 +ASYNCBLIT = 4 +AUDIO_S16 = 32784 +AUDIO_S16LSB = 32784 +AUDIO_S16MSB = 36880 +AUDIO_S16SYS = 36880 +AUDIO_S8 = 32776 +AUDIO_U16 = 16 +AUDIO_U16LSB = 16 +AUDIO_U16MSB = 4112 +AUDIO_U16SYS = 4112 +AUDIO_U8 = 8 +DOUBLEBUF = 1073741824 +FULLSCREEN = -2147483648 +GL_ACCUM_ALPHA_SIZE = 11 +GL_ACCUM_BLUE_SIZE = 10 +GL_ACCUM_GREEN_SIZE = 9 +GL_ACCUM_RED_SIZE = 8 +GL_ALPHA_SIZE = 3 +GL_BLUE_SIZE = 2 +GL_BUFFER_SIZE = 4 +GL_DEPTH_SIZE = 6 +GL_DOUBLEBUFFER = 5 +GL_GREEN_SIZE = 1 +GL_MULTISAMPLEBUFFERS = 13 +GL_MULTISAMPLESAMPLES = 14 +GL_RED_SIZE = 0 +GL_STENCIL_SIZE = 7 +GL_STEREO = 12 +HAT_CENTERED = 0 +HAT_DOWN = 4 +HAT_LEFT = 8 +HAT_LEFTDOWN = 12 +HAT_LEFTUP = 9 +HAT_RIGHT = 2 +HAT_RIGHTDOWN = 6 +HAT_RIGHTUP = 3 +HAT_UP = 1 +HWACCEL = 256 +HWPALETTE = 536870912 +HWSURFACE = 1 +IYUV_OVERLAY = 1448433993 +JOYAXISMOTION = 7 +JOYBALLMOTION = 8 +JOYBUTTONDOWN = 10 +JOYBUTTONUP = 11 +JOYHATMOTION = 9 +KEYDOWN = 2 +KEYUP = 3 +KMOD_ALT = 768 +KMOD_CAPS = 8192 +KMOD_CTRL = 192 +KMOD_LALT = 256 +KMOD_LCTRL = 64 +KMOD_LMETA = 1024 +KMOD_LSHIFT = 1 +KMOD_META = 3072 +KMOD_MODE = 16384 +KMOD_NONE = 0 +KMOD_NUM = 4096 +KMOD_RALT = 512 +KMOD_RCTRL = 128 +KMOD_RMETA = 2048 +KMOD_RSHIFT = 2 +KMOD_SHIFT = 3 +K_0 = 48 +K_1 = 49 +K_2 = 50 +K_3 = 51 +K_4 = 52 +K_5 = 53 +K_6 = 54 +K_7 = 55 +K_8 = 56 +K_9 = 57 +K_AMPERSAND = 38 +K_ASTERISK = 42 +K_AT = 64 +K_BACKQUOTE = 96 +K_BACKSLASH = 92 +K_BACKSPACE = 8 +K_BREAK = 318 +K_CAPSLOCK = 301 +K_CARET = 94 +K_CLEAR = 12 +K_COLON = 58 +K_COMMA = 44 +K_DELETE = 127 +K_DOLLAR = 36 +K_DOWN = 274 +K_END = 279 +K_EQUALS = 61 +K_ESCAPE = 27 +K_EURO = 321 +K_EXCLAIM = 33 +K_F1 = 282 +K_F10 = 291 +K_F11 = 292 +K_F12 = 293 +K_F13 = 294 +K_F14 = 295 +K_F15 = 296 +K_F2 = 283 +K_F3 = 284 +K_F4 = 285 +K_F5 = 286 +K_F6 = 287 +K_F7 = 288 +K_F8 = 289 +K_F9 = 290 +K_FIRST = 0 +K_GREATER = 62 +K_HASH = 35 +K_HELP = 315 +K_HOME = 278 +K_INSERT = 277 +K_KP0 = 256 +K_KP1 = 257 +K_KP2 = 258 +K_KP3 = 259 +K_KP4 = 260 +K_KP5 = 261 +K_KP6 = 262 +K_KP7 = 263 +K_KP8 = 264 +K_KP9 = 265 +K_KP_DIVIDE = 267 +K_KP_ENTER = 271 +K_KP_EQUALS = 272 +K_KP_MINUS = 269 +K_KP_MULTIPLY = 268 +K_KP_PERIOD = 266 +K_KP_PLUS = 270 +K_LALT = 308 +K_LAST = 323 +K_LCTRL = 306 +K_LEFT = 276 +K_LEFTBRACKET = 91 +K_LEFTPAREN = 40 +K_LESS = 60 +K_LMETA = 310 +K_LSHIFT = 304 +K_LSUPER = 311 +K_MENU = 319 +K_MINUS = 45 +K_MODE = 313 +K_NUMLOCK = 300 +K_PAGEDOWN = 281 +K_PAGEUP = 280 +K_PAUSE = 19 +K_PERIOD = 46 +K_PLUS = 43 +K_POWER = 320 +K_PRINT = 316 +K_QUESTION = 63 +K_QUOTE = 39 +K_QUOTEDBL = 34 +K_RALT = 307 +K_RCTRL = 305 +K_RETURN = 13 +K_RIGHT = 275 +K_RIGHTBRACKET = 93 +K_RIGHTPAREN = 41 +K_RMETA = 309 +K_RSHIFT = 303 +K_RSUPER = 312 +K_SCROLLOCK = 302 +K_SEMICOLON = 59 +K_SLASH = 47 +K_SPACE = 32 +K_SYSREQ = 317 +K_TAB = 9 +K_UNDERSCORE = 95 +K_UNKNOWN = 0 +K_UP = 273 +K_a = 97 +K_b = 98 +K_c = 99 +K_d = 100 +K_e = 101 +K_f = 102 +K_g = 103 +K_h = 104 +K_i = 105 +K_j = 106 +K_k = 107 +K_l = 108 +K_m = 109 +K_n = 110 +K_o = 111 +K_p = 112 +K_q = 113 +K_r = 114 +K_s = 115 +K_t = 116 +K_u = 117 +K_v = 118 +K_w = 119 +K_x = 120 +K_y = 121 +K_z = 122 +MOUSEBUTTONDOWN = 5 +MOUSEBUTTONUP = 6 +MOUSEMOTION = 4 +NOEVENT = 0 +NOFRAME = 32 +NUMEVENTS = 32 +OPENGL = 2 +OPENGLBLIT = 10 +PREALLOC = 16777216 +QUIT = 12 +RESIZABLE = 16 +RLEACCEL = 16384 +RLEACCELOK = 8192 +SRCALPHA = 65536 +SRCCOLORKEY = 4096 +SWSURFACE = 0 +SYSWMEVENT = 13 +TIMER_RESOLUTION = 10 +USEREVENT = 24 +UYVY_OVERLAY = 1498831189 +VIDEOEXPOSE = 17 +VIDEORESIZE = 16 +YUY2_OVERLAY = 844715353 +YV12_OVERLAY = 842094169 +YVYU_OVERLAY = 1431918169 Added: pypy/branch/flex-backend/translator/flex/modules/pygame/mixer.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/mixer.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,6 @@ +def get_init(): + return True + +class Sound: + def __init__(self, filename): + self.filename = filename \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/pygame/pygame.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/pygame.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,65 @@ + +def init(): + pass + +class Rect: + pass + +class Surface: + def __init__(self, size): + self.size = size + + def convert(self): + return Surface(self.size) + + def fill(self, color): + pass + + def blit(self, surface, position): + pass + blit._annspecialcase_ = 'specialize:argtype(2)' + + def get_size(self): + return self.size + + def get_rect(self, centerx=1): + return Rect() + + def get_width(self): + return self.size[0] + + def get_at(self, position): + return (100,100,100) + + def set_colorkey(self, colorkey, options): + pass + + + + + + +class Display: + def get_surface(self): + return Surface((10,10)) + + def set_mode(self, (x, y)): + return Surface((x,y)) + + def set_caption(self, caption): + pass + + def flip(self): + pass + + def get_rect(self): + pass + +display = Display() + +class Mouse: + def set_visible(self, bool): + pass + +mouse = Mouse() + Added: pypy/branch/flex-backend/translator/flex/modules/pygame/sprite.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/sprite.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,14 @@ + +class Sprite: + def __init__(self): + pass + +class RenderPlain: + def __init__(self, sprites): + pass + + def update(self): + pass + + def draw(self, surface): + pass \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/pygame/time.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/pygame/time.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,3 @@ +class Clock: + def tick(self, delta): + pass \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/modules/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/modules/test/html/anim.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/test/html/anim.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,36 @@ + + + + + + + +
+ + +
+ + Added: pypy/branch/flex-backend/translator/flex/modules/test/html/test.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/test/html/test.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,35 @@ + + + + + +

Blah

+
+ + +
+ + Added: pypy/branch/flex-backend/translator/flex/modules/test/test_dom.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/test/test_dom.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,385 @@ + +# -*- encoding: utf-8 -*- +import py +from pypy.translator.js.modules import dom +from pypy.translator.js.main import rpython2javascript +from xml.dom.minidom import parseString +import sys + +TRANSLATING = False +USE_PDB = False +# XXX: How to get it from config.option??? + +class handler: + called = False + event = dom.Event() + def __call__(self, e): + self.called = True + self.event = e + e.stopPropagation() + +def get_window(): + if TRANSLATING: + return dom.window + else: + return dom.Window() + +def test_quote_html(): + assert dom._quote_html('foo&bar') == 'foo&bar' + assert dom._quote_html('foo"&bar') == 'foo"&bar' + +def test_serialize_html(): + def roundtrip(html): + return dom._serialize_html(parseString(html).documentElement) + html = '
content
' + assert roundtrip(html) == html + html = '' + assert roundtrip(html) == html + html = '' + assert roundtrip(html) == '' + html = '
\n
foo
\n
' + assert roundtrip(html) == html + html = '
foo&bar
' + assert roundtrip(html) == html + +def code_init(): + window = get_window() + nodeType = window.document.nodeType + docel = window.document.documentElement.nodeName + children = len(window.document.documentElement.childNodes) + + return nodeType, docel, children + +def test_init(): + nodeType, docel, children = code_init() + assert nodeType == 9 + assert docel == 'HTML' + assert children == 2 + +def test_init_failing(): + py.test.raises(py.std.xml.parsers.expat.ExpatError, + 'dom.Window(html="")') + +def code_wrap(): + window = get_window() + document = window.document + div = document.createElement('div') + document.documentElement.appendChild(div) + return document, div + +def test_wrap(): + document, div = code_wrap() + assert isinstance(div, dom.Element) # wrapped node + assert div.nodeType == 1 + assert document.documentElement.childNodes[-1]._original is div._original + +def code_nodeValue(): + window = get_window() + document = window.document + td = document.createElement('td') + td.appendChild(document.createTextNode('foo')) + td.childNodes[0].nodeValue = 'bar' + return td.childNodes[0].nodeValue + +def test_nodeValue(): + nodevalue = code_nodeValue() + assert nodevalue == 'bar' + +def code_node_eq(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + firstref = body.parentNode.lastChild + secondref = window.document.createElement('body') + return body, firstref, secondref + +def test_node_eq(): + body, ref1, ref2 = code_node_eq() + assert ref1 == body + assert ref2 != body + +def code_get_element_by_id(): + window = get_window() + document = window.document + div = document.createElement('div') + div.id = 'foo' + window.document.getElementsByTagName('body')[0].appendChild(div) + div = window.document.getElementById('foo') + return div + +def test_get_element_by_id(): + div = code_get_element_by_id() + assert div.nodeName == 'DIV' + +def code_element_style(): + window = get_window() + document = window.document + div = document.createElement('div') + div.style.backgroundColor = 'green' + return div + +def test_element_style(): + div = code_element_style() + assert div.style + assert not div.style.color + assert div.style.backgroundColor == 'green' + py.test.raises(AttributeError, 'div.style.nonExistent') + +def code_get_elements_by_tag_name(): + window = get_window() + document = window.document + div1 = document.createElement('div') + div1.appendChild(document.createTextNode('foo')) + div2 = document.createElement('div') + div2.appendChild(document.createTextNode('bar')) + body = document.getElementsByTagName('body')[0] + body.appendChild(div1) + body.appendChild(div2) + return document + +def test_get_elements_by_tag_name(): + document = code_get_elements_by_tag_name() + divs = document.getElementsByTagName('div') + assert len(divs) == 2 + divs = document.getElementsByTagName('DIV') + assert len(divs) == 2 + +def test_window_references(): + window = dom.Window() + assert window is window.window + assert window is window.self + assert window is window.parent + assert window is window.top + + window2 = dom.Window(parent=window) + assert window2.parent is window + assert window2.top is window + + window3 = dom.Window(parent=window2) + assert window3.parent is window2 + assert window3.top is window + +def code_read_innerHTML(): + window = get_window() + document = window.document + h1 = document.createElement('h1') + h1.appendChild(document.createTextNode('some document')) + p = document.createElement('p') + p.id = 'content' + p.appendChild(document.createTextNode('some content')) + body = document.getElementsByTagName('body')[0] + body.appendChild(h1) + body.appendChild(p) + return document.documentElement.innerHTML + +def test_read_innerHTML(): + html = code_read_innerHTML() + assert html == ('Untitled document' + '

some document

' + '

some content

') + +def code_read_innerHTML_singletons(): + window = get_window() + document = window.document + head = document.getElementsByTagName('head')[0] + meta = document.createElement('meta') + meta.setAttribute('name', 'foo') + meta.setAttribute('content', 'bar') + head.appendChild(meta) + headhtml = window.document.getElementsByTagName('head')[0].innerHTML + return headhtml + +def test_read_innerHTML_singletons(): + headhtml = code_read_innerHTML_singletons() + assert py.std.re.search(']*\/>', headhtml) + +def code_set_innerHTML(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.innerHTML = '
some content
' + return body + +def test_set_innerHTML(): + body = code_set_innerHTML() + assert body.innerHTML == '
some content
' + assert body.childNodes[0].nodeName == 'DIV' + div = body.childNodes[0] + html = div.innerHTML + assert html == 'some content' + +def code_set_innerHTML_empty(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.innerHTML = '' + body.appendChild(window.document.createTextNode('foobar')) + return body + +def test_set_innerHTML_empty(): + body = code_set_innerHTML_empty() + html = body.innerHTML + assert html == 'foobar' + +def code_event_init_1(): + window = get_window() + e = window.document.createEvent() + e.initEvent('click', True, True) + return window, e + +def code_event_init_2(): + window, event = code_event_init_1() + body = window.document.getElementsByTagName('body')[0] + body.dispatchEvent(event) + return body, event + +def test_event_init(): + window, e = code_event_init_1() + assert e.cancelable == True + assert e.target == None + body, e = code_event_init_2() + assert e.target is body + +def code_event_handling(): + h = handler() + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', True, True) + body.dispatchEvent(e) + return h + +def test_event_handling(): + h = code_event_handling() + assert h.called == True + +def code_event_bubbling_1(): + h = handler() + window = get_window() + body = window.document.getElementsByTagName('body')[0] + div = window.document.createElement('div') + body.appendChild(div) + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', False, True) + div.dispatchEvent(e) + return div, h + +def code_event_bubbling_2(): + div, h = code_event_bubbling_1() + e = div.ownerDocument.createEvent() + e.initEvent('click', True, True) + div.dispatchEvent(e) + return h + +def test_event_bubbling(): + div, h = code_event_bubbling_1() + assert not h.called + h = code_event_bubbling_2() + assert h.called == True + +def code_remove_event_listener(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + div = window.document.createElement('div') + body.appendChild(div) + h = handler() + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', True, True) + body.dispatchEvent(e) + return body, h + +def code_remove_event_listener_2(): + body, h = code_remove_event_listener() + h.called = False + body.removeEventListener('click', h, False) + e = body.ownerDocument.createEvent() + e.initEvent('click', True, True) + body.dispatchEvent(e) + return h + +def test_remove_event_listener(): + body, h = code_remove_event_listener() + assert h.called == True + h = code_remove_event_listener_2() + assert h.called == False + +def code_event_vars(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + div = window.document.createElement('div') + body.appendChild(div) + h = handler() + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', True, True) + div.dispatchEvent(e) + return body, div, h + +def test_event_vars(): + body, div, h = code_event_vars() + assert h.event.target == div + assert h.event.originalTarget == div + assert h.event.currentTarget == body + +def code_class_name(): + window = get_window() + document = window.document + body = document.getElementsByTagName('body')[0] + div = document.createElement('div') + div.appendChild(document.createTextNode('foo')) + div.setAttribute('class', 'foo') + body.appendChild(div) + div = window.document.getElementsByTagName('div')[0] + return body, div + +def test_class_name(): + body, div = code_class_name() + assert div.className == 'foo' + assert not body.className + div.className = 'bar' + assert div.className == 'bar' + assert body.innerHTML == '
foo
' + +def code_read_styles(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.innerHTML = ('
foo' + '
') + return body.childNodes[0].style + +def test_read_styles(): + style = code_read_styles() + assert style.color == 'red' + bgcolor = style.backgroundColor + assert bgcolor == 'green' + +def code_write_styles(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.style.color = 'green' + body.style.backgroundColor = 'red' + return body + +def test_write_styles(): + body = code_write_styles() + assert dom._serialize_html(body) == ('') + +def test_document_location(): + py.test.skip("To be written") + +def test_build(): + py.test.skip('B?rken') + global TRANSLATING + TRANSLATING = True + for var in globals(): + if var.startswith('code_'): + # just build it + #def f(): + assert rpython2javascript(sys.modules[__name__], [var], use_pdb=False) + + TRANSLATING = False + +def test_navigator(): + window = get_window() + assert window.navigator.appName == 'Netscape' Added: pypy/branch/flex-backend/translator/flex/modules/test/test_mochikit.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/modules/test/test_mochikit.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,14 @@ + +from pypy.translator.js.lib.support import js_source +from pypy.translator.js.modules.mochikit import * + +class TestRender(object): + def test_escape_html(self): + def x(): + escapeHTML("xxx") + "xxx" + assert js_source([x], use_pdb=False).find("escapeHTML (") != -1 + + def test_serialize_json(self): + def x(): + return serializeJSON(3) + serializeJSON("aaa") + serializeJSON([1]) + assert js_source([x]) Added: pypy/branch/flex-backend/translator/flex/opcodes.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/opcodes.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,158 @@ +""" opcode definitions +""" + +from pypy.translator.oosupport.metavm import PushArg, PushAllArgs, StoreResult,\ + InstructionList, New, GetField, MicroInstruction, RuntimeNew, PushPrimitive + +from pypy.translator.oosupport.metavm import _GetFieldDispatcher, _SetFieldDispatcher, \ + _CallDispatcher, _MethodDispatcher, SetField + +from pypy.translator.flex.metavm import IsInstance, Call, CallMethod,\ + CopyName, CastString, _Prefix, _CastFun, _NotImplemented, CallBuiltin,\ + CallBuiltinObject, GetBuiltinField, SetBuiltinField, IndirectCall,\ + CallExternalObject, SetExternalField, _CastMethod, _LoadConst,\ + DiscardStack, CheckLength, fix_opcodes + +from pypy.translator.flex.jsbuiltin import Builtins +from pypy.rpython.ootypesystem import ootype + +DoNothing = [] + +from pypy.translator.flex.log import log + +class_map = { 'Call' : Call, + 'CallMethod' : CallMethod, + 'CallBuiltinObject' : CallBuiltinObject, + 'CallBuiltin' : CallBuiltin, + 'GetBuiltinField' : GetBuiltinField, + 'GetField' : GetField, + 'SetField' : SetField, + 'SetBuiltinField' : SetBuiltinField, + 'CallExternalObject' : CallExternalObject, + 'SetExternalField' : SetExternalField, +} + +opcodes = {'int_mul': '*', + 'int_add': '+', + 'int_sub': '-', + 'int_sub_ovf': '-', # XXX overflow + 'int_floordiv': '/', + 'int_mod': '%', + 'int_mod_ovf': '%', # XXX: what's that? + 'int_mod_zer': '%', # XXX: fix zero stuff + 'int_and': '&', + 'int_or': '|', + 'int_xor': '^', + 'int_lshift': '<<', + 'int_lshift_ovf': '<<', # XXX overflow + 'int_rshift': '>>', + 'int_rshift_ovf': '>>', # XXX overflow + 'int_lt': '<', + 'int_le': '<=', + 'int_eq': '==', + 'int_ne': '!=', + 'int_ge': '>=', + 'int_gt': '>', + + 'uint_mul': '*', + 'uint_add': '+', + 'uint_sub': '-', + 'uint_floordiv': '/', + 'uint_mod': '%', + 'uint_and': '&', + 'uint_or': '|', + 'uint_xor': '^', + 'uint_lshift': '<<', + 'uint_rshift': '>>', + 'uint_lt': '<', + 'uint_le': '<=', + 'uint_eq': '==', + 'uint_ne': '!=', + 'uint_ge': '>=', + 'uint_gt': '>', + + 'unichar_lt': '<', + 'unichar_le': '<=', + 'unichar_eq': '==', + 'unichar_ne': '!=', + 'unichar_ge': '>=', + 'unichar_gt': '>', + + 'char_lt': '<', + 'char_le': '<=', + 'char_eq': '==', + 'char_ne': '!=', + 'char_ge': '>=', + 'char_gt': '>', + + 'float_mul': '*', + 'float_add': '+', + 'float_sub': '-', + 'float_truediv': '/', + 'float_lt': '<', + 'float_le': '<=', + 'float_eq': '==', + 'float_ne': '!=', + 'float_ge': '>=', + 'float_gt': '>', + + 'ptr_eq': '==', + 'ptr_ne': '!=', + + 'bool_not': [PushAllArgs,_Prefix('!')], + 'int_neg': [PushAllArgs,_Prefix('-')], + 'int_invert': [PushAllArgs,_Prefix('~')], + 'float_neg': [PushAllArgs,_Prefix('-')], + + 'float_pow': [PushAllArgs,_CastFun('Math.pow',2)], + 'int_abs': [PushAllArgs,_CastFun('Math.abs',1)], + 'float_abs': [PushAllArgs,_CastFun('Math.abs',1)], + + 'int_is_true': [PushAllArgs,_Prefix('!!')], + 'uint_is_true': [PushAllArgs,_Prefix('!!')], + 'float_is_true': [PushAllArgs,_Prefix('!!')], + 'is_true': [PushAllArgs,_Prefix('!!')], + + 'direct_call' : [_CallDispatcher(Builtins, class_map)], + 'indirect_call' : [IndirectCall], + 'same_as' : CopyName, + 'new' : [New], + 'runtimenew' : [RuntimeNew], + 'instanceof' : [IsInstance], + #'subclassof' : [IsSubclassOf], + + # objects + + 'oosetfield' : [_SetFieldDispatcher(Builtins, class_map)], + 'oogetfield' : [_GetFieldDispatcher(Builtins, class_map)], + 'oosend' : [_MethodDispatcher(Builtins, class_map)], + 'ooupcast' : CopyName, + 'oodowncast' : CopyName, + 'oononnull' : [PushAllArgs,_Prefix('!!')], + 'oostring' : [PushArg(0),CastString], + 'ooparse_int' : [PushAllArgs,_CastFun("parseInt",2)], + 'ooparse_float' : [PushAllArgs,_CastFun("parseFloat",1)], + 'oois' : '===', + 'cast_bool_to_int': CopyName, + 'cast_bool_to_uint': CopyName, + 'cast_bool_to_float': CopyName, + 'cast_char_to_int': [PushAllArgs,_LoadConst(0),_CastMethod("charCodeAt",1)], + 'cast_unichar_to_int': [PushAllArgs,_LoadConst(0),_CastMethod("charCodeAt",1)], + 'cast_int_to_char': [PushAllArgs,_CastFun("String.fromCharCode",1)], + 'cast_int_to_unichar': [PushAllArgs,_CastFun("String.fromCharCode",1)], + 'cast_int_to_uint': CopyName, + 'cast_int_to_float': CopyName, + 'cast_int_to_longlong': CopyName, + 'cast_uint_to_int': CopyName, + 'cast_uint_to_float': CopyName, + 'cast_float_to_int': [PushAllArgs,_CastFun("Math.floor",1)], + 'cast_float_to_uint': [PushAllArgs,_CastFun("Math.floor",1)], + 'cast_float_to_longlong': [PushAllArgs,_CastFun("Math.floor",1)], + 'truncate_longlong_to_int': CopyName, + + 'debug_assert' : DoNothing, + 'resume_point' : DoNothing, + 'is_early_constant': [PushPrimitive(ootype.Bool, False)], +} + +fix_opcodes(opcodes) Added: pypy/branch/flex-backend/translator/flex/sandbox/MyFirst.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/MyFirst.html Sat Jul 14 11:58:08 2007 @@ -0,0 +1,11 @@ + + + + + + + Added: pypy/branch/flex-backend/translator/flex/sandbox/MyFirst.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/MyFirst.mxml Sat Jul 14 11:58:08 2007 @@ -0,0 +1,74 @@ + + + + + + + + + + Added: pypy/branch/flex-backend/translator/flex/sandbox/chimp.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/chimp.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,6 @@ +#/usr/bin/env python """ This simple example is used for the line-by-line tutorial that comes with pygame. It is based on a 'popular' web banner. Note there are comments here, but for the full explanation, follow along in the tutorial. """ #Import Modules import os +import pypy.translator.flex.modules.pygame as pygame from pypy.translator.flex.modules.pygame.locals import * if not pygame.font: print 'Warning, fonts disabled' if not pygame.mixer: print 'Warning, sound disabled' #functions to create our resources def load_image(name, colorkey=None): fullname = os.path.join('data', name) try: image = pygame.image.load(fullname) except pygame.error, message: print 'Cannot load image:', fullname raise SystemExit, message image = image.convert() if colorkey is not None: if colorkey is -1: colorkey = image.get_at((0,0)) image.set_colorkey(colorkey, RLEACCEL) return image, image.get_rect() + +class NoneSound: def play(self): pass + def load_sound(name): if not pygame.mixer or not pygame.mixer.get_init(): return NoneSound() fullname = os.path.join('data', name) try: sound = pygame.mixer.Sound(fullname) except pygame.error, message: print 'Cannot load sound:', fullname raise SystemExit, message return sound #classes for our game objects class Fist(pygame.sprite.Sprite): """moves a clenched fist on the screen, following the mouse""" def __init__(self): pygame.sprite.Sprite.__init__(self) #call Sprite initializer self.image, self.rect = load_image('fist.bmp', -1) self.punching = 0 def update(self): "move the fist based on the mouse position" pos = pygame.mouse.get_pos() self.rect.midtop = pos if self.punching: self.rect.move_ip(5, 10) def punch(self, target): "returns true if the fist collides with the target" if not self.punching: self.punching = 1 hitbox = self.rect.inflate(-5, -5) return hitbox.colliderect(target.rect) def unpunch(self): "called to pull the fist back" self.punching = 0 class Chimp(pygame.sprite.Sprite): """moves a monkey critter across the screen. it can spin the monkey when it is punched.""" def __init__(self): pygame.sprite.Sprite.__init__(self) #call Sprite intializer self.image, self.rect = load_image('chimp.bmp', -1) screen = pygame.display.get_surface() self.area = screen.get_rect() self.rect.topleft = 10, 10 self.move = 9 self.dizzy = 0 def update(self): "walk or spin, depending on the monkeys state" if self.dizzy: self._spin() else: self._walk() def _walk(self): "move the monkey across the screen, and turn at the ends" newpos = self.rect.move((self.move, 0)) if self.rect.left < self.area.left or \ self.rect.right > self.area.right: self.move = -self.move newpos = self.rect.move((self.move, 0)) self.image = pygame.transform.flip(self.image, 1, 0) self.rect = newpos def _spin(self): "spin the monkey image" center = self.rect.center self.dizzy = self.dizzy + 12 if self.dizzy >= 360: self.dizzy = 0 self.image = self.original else: rotate = pygame.transform.rotate self.image = rotate(self.original, self.dizzy) self.rect = self.image.get_rect(center=center) def punched(self): "this will cause the monkey to start spinning" if not self.dizzy: self.dizzy = 1 self.original = self.image def flash_main(): """this function is called when the program starts. it initializes everything it needs, then runs in a loop until the function returns.""" #Initialize Everything pygame.init() screen = pygame.display.set_mode((468, 60)) pygame.display.set_caption('Monkey Fever') pygame.mouse.set_visible(0) #Create The Backgound background = pygame.Surface(screen.get_size()) background = background.convert() background.fill((250, 250, 250)) #Put Text On The Background, Centered if pygame.font: font = pygame.font.Font(None, 36) text = font.render("Pummel The Chimp, And Win $$$", 1, (10, 10, 10)) + textpos = text.get_rect(centerx=background.get_width()/2) background.blit(text, textpos) #Display The Background screen.blit(background, (0, 0)) pygame.display.flip() #Prepare Game Objects clock = pygame.time.Clock() whiff_sound = load_sound('whiff.wav') punch_sound = load_sound('punch.wav') chimp = Chimp() fist = Fist() allsprites = pygame.sprite.RenderPlain((fist, chimp)) #Main Loop while 1: clock.tick(60) #Handle Input Events for event in pygame.event.get(): if event.type == QUIT: return elif event.type == KEYDOWN and event.key == K_ESCAPE: return elif event.type == MOUSEBUTTONDOWN: if fist.punch(chimp): punch_sound.play() #punch chimp.punched() else: whiff_sound.play() #miss elif event.type is MOUSEBUTTONUP: fist.unpunch() allsprites.update() #Draw Everything screen.blit(background, (0, 0)) allsprites.draw(screen) pygame.display.flip() #Game Over #this calls the 'main' function when this script is executed if __name__ == '__main__': flash_main() \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/sandbox/fun.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/fun.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,17 @@ +from pypy.translator.flex.modules.flex import * + + +def flash_main(a=1): + #abutton = Button() + #abutton.label = "I'm a button!" + #addChild(abutton) + + for x in range(20): + abutton = Button() + abutton.label = "I'm button :" + str(x) + addChild(abutton) + + return a+1 + + + Added: pypy/branch/flex-backend/translator/flex/sandbox/misc.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/misc.as Sat Jul 14 11:58:08 2007 @@ -0,0 +1,236 @@ + + import py.*; + //import py.f.DictIter; + + +// starts hand written code +var MALLOC_ZERO_FILLED = 0; + +var print = trace; + + + + +Function.prototype.method = function (name, func) { + this.prototype[name] = func; + return this; +}; + +function inherits(child, parent) { + child.parent = parent; + for (var i in parent.prototype) { + if (!child.prototype[i]) { + child.prototype[i] = parent.prototype[i]; + } + } +} + +function isinstanceof(self, what) { + if (!self) { + return (false); + } + var t = self.constructor; + while ( t ) { + if (t == what) { + return (true); + } + t = t.parent; + } + return (false); +} + +/*function delitem(fn, l, i) { + for(var j = i; j < l.length-1; ++j) { + l[j] = l[j+1]; + } + l.length--; +}*/ + +function strcmp(s1, s2) { + if ( s1 < s2 ) { + return ( -1 ); + } else if ( s1 == s2 ) { + return ( 0 ); + } + return (1); +} + +function startswith(s1, s2) { + if (s1.length < s2.length) { + return(false); + } + for (var i = 0; i < s2.length; ++i){ + if (s1.charAt(i) != s2.charAt(i)) { + return(false); + } + } + return(true); +} + +function endswith(s1, s2) { + if (s2.length > s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} +// ends hand written code +function some_strange_function_which_will_never_be_called () { + var v1,v2; + var block = 0; + for(;;){ + switch(block){ + case 0: + v1 = 2; + v2 = xx ( v1 ); + block = 1; + break; + case 1: + return ( undefined ); + } + } +} + +function xx (y_0) { + var v3,v4; + var block = 0; + for(;;){ + switch(block){ + case 0: + v4 = (y_0+1); + v3 = v4; + block = 1; + break; + case 1: + return ( v3 ); + } + } +} + + +var __consts_0 = {}; Added: pypy/branch/flex-backend/translator/flex/sandbox/output.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/output.mxml Sat Jul 14 11:58:08 2007 @@ -0,0 +1,462 @@ + + + + + + s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} + + +function __flash_main() { + + + var aaa = new Button(); + aaa.label = "before consts"; + addChild ( aaa ); + + + + __load_consts_flex() + + aaa.label = "after consts"; + flash_main(1) +} + + +public function flexTrace( text:String ):void { + var myUrl:URLRequest = new URLRequest("javascript:console.log('" + text + "');void(0);"); + sendToURL(myUrl); +} + +// wrapper for throw, because flex compiler weirdness. +function throwit(e) { + throw(e); +} + + +// ends hand written code + + + +function some_strange_function_which_will_never_be_called () { + var v1,v2; + var block = 0; + for(;;){ + switch(block){ + case 0: + v1 = 1; + v2 = flash_main ( v1 ); + block = 1; + break; + case 1: + return ( undefined ); + } + } +} + +function flash_main (a_0) { + var v3,v4,v5,a_1,v6,v7,last_exc_value_0,a_2,x_0,v8,v9,v10,v11,v14,v15; + var block = 0; + for(;;){ + switch(block){ + case 0: + v4 = ll_newrange__Record_start__Signed__stop__Si_Signed_Signed ( 0,20 ); + v5 = ll_rangeiter__Record_stop__Signed__next__Sig_Record_start__Signed__stop__Si ( v4 ); + a_1 = a_0; + v6 = v5; + block = 1; + break; + case 1: + try { + v7 = ll_rangenext_up__Record_stop__Signed__next__Sig_Signed ( v6,1 ); + a_2 = a_1; + x_0 = v7; + v8 = v6; + block = 2; + break; + } + catch (exc){ + if (isinstanceof(exc, exceptions_StopIteration)) + { + v14 = a_1; + block = 3; + break; + } + throwit(exc); + } + case 2: + v9 = new Button(); + v10 = ll_int_str__IntegerR_SignedConst_Signed ( x_0 ); + v11 = ll_strconcat__String_String ( __consts_0.const_str,v10 ); + v9.label = v11; + addChild ( v9 ); + a_1 = a_2; + v6 = v8; + block = 1; + break; + case 3: + v15 = (v14+1); + v3 = v15; + block = 4; + break; + case 4: + return ( v3 ); + } + } +} + +function ll_strconcat__String_String (obj_0,arg0_0) { + var v39,v40,v41; + var block = 0; + for(;;){ + switch(block){ + case 0: + v41 = (obj_0+arg0_0); + v39 = v41; + block = 1; + break; + case 1: + return ( v39 ); + } + } +} + +function ll_newrange__Record_start__Signed__stop__Si_Signed_Signed (start_0,stop_0) { + var v16,v17; + var block = 0; + for(;;){ + switch(block){ + case 0: + v17 = new Object(); + v17.start = start_0; + v17.stop = stop_0; + v16 = v17; + block = 1; + break; + case 1: + return ( v16 ); + } + } +} + +function ll_rangeiter__Record_stop__Signed__next__Sig_Record_start__Signed__stop__Si (rng_0) { + var v20,v21,v22,v24; + var block = 0; + for(;;){ + switch(block){ + case 0: + v21 = new Object(); + v22 = rng_0.start; + v21.next = v22; + v24 = rng_0.stop; + v21.stop = v24; + v20 = v21; + block = 1; + break; + case 1: + return ( v20 ); + } + } +} + +function ll_int_str__IntegerR_SignedConst_Signed (i_0) { + var v37,v38; + var block = 0; + for(;;){ + switch(block){ + case 0: + v38 = ll_int2dec__Signed ( i_0 ); + v37 = v38; + block = 1; + break; + case 1: + return ( v37 ); + } + } +} + +function ll_rangenext_up__Record_stop__Signed__next__Sig_Signed (iter_0,step_0) { + var v26,v27,v28,v29,v30,iter_1,next_0,v31,v32,v34,v35,v36,etype_0,evalue_0; + var block = 0; + for(;;){ + switch(block){ + case 0: + v27 = iter_0.next; + v28 = iter_0.stop; + v29 = (v27>=v28); + iter_1 = iter_0; + next_0 = v27; + v31 = step_0; + if (v29 == false) + { + block = 1; + break; + } + block = 3; + break; + case 1: + v32 = (next_0+v31); + iter_1.next = v32; + v26 = next_0; + block = 2; + break; + case 3: + v34 = __consts_0.exceptions_StopIteration; + v35 = v34.meta; + etype_0 = v35; + evalue_0 = v34; + block = 4; + break; + case 4: + throwit(evalue_0); + case 2: + return ( v26 ); + } + } +} + +function ll_int2dec__Signed (i_1) { + var v42,v43; + var block = 0; + for(;;){ + switch(block){ + case 0: + v43 = convertToString ( i_1 ); + v42 = v43; + block = 1; + break; + case 1: + return ( v42 ); + } + } +} + +var __consts_0 = {} +function __load_consts_flex() +{ + __consts_0.exceptions_StopIteration__3 = exceptions_StopIteration; + __consts_0.exceptions_StopIteration_meta = new exceptions_StopIteration_meta(); + __consts_0.exceptions_StopIteration = new exceptions_StopIteration(); + __consts_0.const_str = "I'm button :"; + __consts_0.exceptions_StopIteration_meta.class_ = __consts_0.exceptions_StopIteration__3; + __consts_0.exceptions_StopIteration.meta = __consts_0.exceptions_StopIteration_meta; +} + + ]]> + + Added: pypy/branch/flex-backend/translator/flex/sandbox/output2.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/output2.mxml Sat Jul 14 11:58:08 2007 @@ -0,0 +1,260 @@ + + + + + + s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} +// ends hand written code + + +function some_strange_function_which_will_never_be_called () { + var v1,v2; + var block = 0; + for(;;){ + switch(block){ + case 0: + v1 = 1; + v2 = flash_main ( v1 ); + block = 1; + break; + case 1: + return ( undefined ); + } + } +} + +function flash_main (a_0) { + var v3,v4,v6; + var block = 0; + for(;;){ + switch(block){ + case 0: + var v4 = new Button(); + v4.label = __consts_0.const_str; + addChild (v4); + v6 = (a_0+1); + v3 = v6; + block = 1; + break; + case 1: + return ( v3 ); + } + } +} + + +var __consts_0 = new Constants(); +//__consts_0.const_str = "I'm a button!"; +//__consts_0['const_str'] = "I'm a button!"; + + ]]> + + Added: pypy/branch/flex-backend/translator/flex/sandbox/py/DictIter.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/py/DictIter.as Sat Jul 14 11:58:08 2007 @@ -0,0 +1,31 @@ + +package py { + +public class DictIter { + + var current_key; + var l; + + public function DictIter() { + this.current_key = null; + } + + function ll_go_next() { + var ret = this.l.length != 0; + this.current_key = this.l.pop(); + return ret; + } + + function ll_current_key() { + if (this.current_key) { + return this.current_key; + } else { + return null; + } + } + +} + + + +} Added: pypy/branch/flex-backend/translator/flex/sandbox/py/StringBuilder.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/py/StringBuilder.as Sat Jul 14 11:58:08 2007 @@ -0,0 +1,33 @@ +package py { + + +public class StringBuilder { + + public function StringBuilder() { + this.l = []; + } + + function ll_append_char(s) { + this.l.length += 1; + this.l[this.l.length - 1] = s; + } + + function ll_append(s) { + this.l.push(s); + } + + function ll_allocate(t) { + } + + function ll_build() { + var s; + s = ""; + for (var i in this.l) { + s += this.l[i]; + } + return (s); + } + +} + +} Added: pypy/branch/flex-backend/translator/flex/sandbox/py/pygame/Rect.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/sandbox/py/pygame/Rect.as Sat Jul 14 11:58:08 2007 @@ -0,0 +1,45 @@ +class Rect { + public var x : Float; + public var y : Float; + public var w : Float; + public var h : Float; + + public var centerx(get_centerx,set_centerx) : Float; + public var centery(get_centery,set_centery) : Float; + public var top(get_top,set_top): Float; + public var right(get_right,set_right): Float; + public var bottom(get_bottom,set_bottom): Float; + public var left(get_left,set_left): Float; + + public function new(x_:Float,y_:Float,w_:Float,h_:Float) { + x = x_; + y = y_; + w = w_; + h = h_; + } + + public function colliderect(b:Rect) { + var a = this; + if (a.bottom < b.top) { return false; } + if (a.left > b.right) { return false; } + if (a.top > b.bottom) { return false; } + if (a.right < b.left) { return false; } + return true; + } + + private function get_centerx() { return x+w/2; } + private function set_centerx(v:Float) { x = v-w/2; return x+w/2; } + private function get_centery() { return y+h/2; } + private function set_centery(v:Float) { y = v-h/2; return y+h/2; } + private function get_top() { return y; } + private function set_top(v:Float) { y = v; return y; } + private function get_right() { return x+w; } + private function set_right(v:Float) { x = v-w; return x+w; } + private function get_bottom() { return y+h; } + private function set_bottom(v:Float) { y = v-h; return y+h; } + private function get_left() { return x; } + private function set_left(v:Float) { x = v; return x; } +} + + + \ No newline at end of file Added: pypy/branch/flex-backend/translator/flex/support.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/support.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,64 @@ +from pypy.translator.gensupp import NameManager +#from pypy.translator.js.optimize import is_optimized_function + +class JavascriptNameManager(NameManager): + def __init__(self, db): + NameManager.__init__(self) + self.db = db + self.reserved = {} + + #http://javascript.about.com/library/blreserved.htm + reserved_words = ''' + abstract as boolean break byte case catch + char class continue const debugger default delete + do double else enum export extends false + final finally float for function goto if implements + import in instanceof int interface is long + namespace native new null package private protected + public return short static super switch synchronized + this throw throws transient true try typeof + use var void volatile while with alert + ''' + for name in reserved_words.split(): + self.reserved[name] = True + + #http://javascript.about.com/library/blclassobj.htm + # XXX WAAAHHH!!! IE alert :( there are a lot of objects here that are + # _not_ in standard JS, see + # http://devedge-temp.mozilla.org/library/manuals/2000/javascript/1.5/reference/ + predefined_classes_and_objects = ''' + Anchor anchors Applet applets Area Array Body + Button Checkbox Date document Error EvalError FileUpload + Form forms frame frames Function Hidden History + history Image images Link links location Math + MimeType mimetypes navigator Number Object Option options + Password Plugin plugins Radio RangeError ReferenceError RegExp + Reset screen Script Select String Style StyleSheet + Submit SyntaxError Text Textarea TypeError URIError window + ''' + for name in predefined_classes_and_objects.split(): + self.reserved[name] = True + + #http://javascript.about.com/library/blglobal.htm + global_properties_and_methods = ''' + _content closed Components controllers crypto defaultstatus directories + document frames history innerHeight innerWidth length location + locationbar menubar name navigator opener outerHeight outerWidth + pageXOffset pageYOffset parent personalbar pkcs11 prompter screen + screenX screenY scrollbars scrollX scrollY self statusbar + toolbar top window + ''' + for name in global_properties_and_methods.split(): + self.reserved[name] = True + + self.make_reserved_names(' '.join(self.reserved)) + + self.predefined = set(predefined_classes_and_objects) + + #def uniquename(self, name, lenmax=0): + # return NameManager.uniquename(self, , lenmax) + + def ensure_non_reserved(self, name): + while name in self.reserved: + name += '_' + return name Added: pypy/branch/flex-backend/translator/flex/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/test/browsertest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/browsertest.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,177 @@ +from BaseHTTPServer import HTTPServer as BaseHTTPServer, BaseHTTPRequestHandler +import py +from os import system +from cgi import parse_qs +from sys import platform +from time import sleep +import webbrowser +from pypy.translator.js.log import log +log = log.browsertest + +class HTTPServer(BaseHTTPServer): + allow_reuse_address = True + +class config: + http_port = 10001 + + html_page = """ + + + + + %(jsfilename)s +
+ +
+
+ +""" + + refresh_page = """ + + + + +
+// testcase: %(jstestcase)s
+%(jscode)s
+
+ +""" + + +class TestCase(object): + def __init__(self, jsfilename, jstestcase): + self.jsfilename = jsfilename + self.jscode = open(jsfilename).read() + self.jstestcase = jstestcase + self.result = None + + +class TestHandler(BaseHTTPRequestHandler): + """The HTTP handler class that provides the tests and handles results""" + + def do_GET(self): + global do_status + if self.path != "/test.html": + self.send_error(404, "File /test.html not found") + return + jsfilename = jstest.jsfilename + jstestcase = jstest.jstestcase + jscode = jstest.jscode + if self.server.html_page: + if self.server.is_interactive: + isinteractive = '' + else: + isinteractive = 'resultform.submit();' + try: + html_page = open(self.server.html_page).read() % locals() + except IOError: + log("HTML FILE WAS NOT FOUND!!!!") + self.send_error(404, "File %s not found" % self.server.html_page) + return + else: + html_page = config.html_page % locals() + + open("html_page.html", "w").write(html_page) + self.serve_data('text/html', html_page) + do_status = 'do_GET' + + def do_POST(self): + global do_status + if self.path != "/test.html": + self.send_error(404, "File /test.html not found") + return + form = parse_qs(self.rfile.read(int(self.headers['content-length']))) + if self.server.is_interactive: + if not form.has_key('ok'): + jstest.result = 'Not clicked OK' + else: + jstest.result = 'OK' + #assert False, "Clicked not ok" + else: + jstest.result = form['result'][0] + + #we force a page refresh here because of two reason: + # 1. we don't have the next testcase ready yet + # 2. browser should ask again when we do have a test + jsfilename = jstest.jsfilename + jstestcase = jstest.jstestcase + jscode = jstest.jscode + refresh_page = config.refresh_page % locals() + self.serve_data('text/html', refresh_page) + do_status = 'do_POST' + + def serve_data(self, content_type, data): + self.send_response(200) + self.send_header("Content-type", content_type) + self.send_header("Content-length", len(data)) + self.end_headers() + self.wfile.write(data) + + +class BrowserTest(object): + """The browser driver""" + + def start_server(self, port, html_page, is_interactive): + server_address = ('', port) + self.httpd = HTTPServer(server_address, TestHandler) + self.httpd.is_interactive = is_interactive + self.httpd.html_page = html_page + + def get_result(self): + global do_status + do_status = None + while do_status != 'do_GET': + self.httpd.handle_request() + while do_status != 'do_POST': + self.httpd.handle_request() + return jstest.result + + +def jstest(jsfilename, jstestcase, browser_to_use, html_page = None, is_interactive = False): + global driver, jstest + jstest = TestCase(str(jsfilename), str(jstestcase)) + + try: + driver.httpd.html_page = html_page + driver.httpd.is_interactive = is_interactive + except: + driver = BrowserTest() + driver.start_server(config.http_port, html_page, is_interactive) + if browser_to_use == 'default': + browser_to_use = None + if browser_to_use != 'none': + webbrowser.get(browser_to_use).open('http://localhost:%d/test.html' % config.http_port) + + result = driver.get_result() + return result Added: pypy/branch/flex-backend/translator/flex/test/runtest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/runtest.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,215 @@ + +import py, os, re, subprocess +from pypy.translator.translator import TranslationContext +from pypy.translator.backendopt.all import backend_optimizations +from pypy.translator.js.js import JS +from pypy.translator.js.test.browsertest import jstest +from pypy.translator.js import conftest +from pypy.translator.js.log import log +from pypy.conftest import option +from pypy.rpython.test.tool import BaseRtypingTest, OORtypeMixin +from pypy.rlib.nonconst import NonConstant +from pypy.rpython.ootypesystem import ootype + +from pypy.rpython.llinterp import LLException + +log = log.runtest +use_browsertest = conftest.option.browser +use_tg = conftest.option.tg + +port = 8080 + +class JSException(LLException): + pass + +def _CLI_is_on_path(): + if py.path.local.sysfind('js') is None: #we recommend Spidermonkey + return False + return True + +class compile_function(object): + def __init__(self, function, annotations, stackless=False, view=False, html=None, is_interactive=False, root = None, run_browser = True, policy = None): + if not use_browsertest and not _CLI_is_on_path(): + py.test.skip('Javascript CLI (js) not found') + + self.html = html + self.is_interactive = is_interactive + t = TranslationContext() + + if policy is None: + from pypy.annotation.policy import AnnotatorPolicy + policy = AnnotatorPolicy() + policy.allow_someobjects = False + + ann = t.buildannotator(policy=policy) + ann.build_types(function, annotations) + if view or option.view: + t.view() + t.buildrtyper(type_system="ootype").specialize() + + if view or option.view: + t.view() + #self.js = JS(t, [function, callback_function], stackless) + self.js = JS(t, function, stackless) + self.js.write_source() + if root is None and use_tg: + from pypy.translator.js.demo.jsdemo.controllers import Root + self.root = Root + else: + self.root = root + self.run_browser = run_browser + self.function_calls = [] + + def source(self): + return self.js.tmpfile.open().read() + + def _conv(self, v): + if isinstance(v, str): + return repr(v) + return str(v).lower() + + def __call__(self, *kwds): + return self.call(None, kwds) + + def call(self, entry_function, kwds): + args = ', '.join([self._conv(kw) for kw in kwds]) #lowerstr for (py)False->(js)false, etc. + + if entry_function is None: + entry_function = self.js.translator.graphs[0].name + else: + entry_function = self.js.translator.annotator.bookkeeper.getdesc(entry_function).cached_graph(None) + function_call = "%s(%s)" % (entry_function, args) + self.function_calls.append(function_call) + #if self.js.stackless: + # function_call = "slp_entry_point('%s')" % function_call + + if use_browsertest: + if not use_tg: + log("Used html: %r" % self.html) + output = jstest(self.js.filename, function_call, use_browsertest, self.html, self.is_interactive) + else: + global port + from pypy.translator.js.test.tgtest import run_tgtest + out = run_tgtest(self, tg_root = self.root, port=port, run_browser=self.run_browser).results + assert out[1] == 'undefined' or out[1] == "" + output = out[0] + port += 1 + else: +# cmd = 'echo "load(\'%s\'); print(%s)" | js 2>&1' % (self.js.filename, function_call) +# log(cmd) +# output = os.popen(cmd).read().strip() + js = subprocess.Popen(["js"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + input = "load(%r);\n" % self.js.filename.strpath + for call in self.function_calls[:-1]: + input += "%s;\n" % call + input += "print(\"'\" + %s + \"'\");\n" % self.function_calls[-1] + js.stdin.write(input) + stdout, stderr = js.communicate() + output = (stderr + stdout).strip() + for s in output.split('\n'): + log(s) + + m = re.match("'(.*)'", output, re.DOTALL) + if not m: + log("Error: %s" % output) + raise JSException(output) + return self.reinterpret(m.group(1)) + + def reinterpret(cls, s): + #while s.startswith(" "): + # s = s[1:] # :-) quite inneficient, but who cares + if s == 'false': + res = False + elif s == 'true': + res = True + elif s == 'undefined': + res = None + elif s == 'inf': + res = 1e300 * 1e300 + elif s == 'NaN': + res = (1e300 * 1e300) / (1e300 * 1e300) + elif s.startswith('[') or s.startswith('('): + l = s[1:-1].split(',') + res = [cls.reinterpret(i) for i in l] + else: + try: + res = float(s) + if float(int(res)) == res: + return int(res) + except ValueError: + res = str(s) + return res + reinterpret = classmethod(reinterpret) + +class JsTest(BaseRtypingTest, OORtypeMixin): + def _compile(self, _fn, args, policy=None): + argnames = _fn.func_code.co_varnames[:_fn.func_code.co_argcount] + func_name = _fn.func_name + if func_name == '': + func_name = 'func' + source = py.code.Source(""" + def %s(): + from pypy.rlib.nonconst import NonConstant + res = _fn(%s) + if isinstance(res, type(None)): + return None + else: + return str(res)""" + % (func_name, ",".join(["%s=NonConstant(%r)" % (name, i) for + name, i in zip(argnames, args)]))) + exec source.compile() in locals() + return compile_function(locals()[func_name], [], policy=policy) + + def string_to_ll(self, s): + return s + + def interpret(self, fn, args, policy=None): + f = self._compile(fn, args, policy) + res = f(*args) + return res + + def interpret_raises(self, exception, fn, args): + #import exceptions # needed by eval + #try: + #import pdb; pdb.set_trace() + try: + res = self.interpret(fn, args) + except JSException, e: + s = e.args[0] + assert s.startswith('uncaught exception:') + assert re.search(exception.__name__, s) + else: + raise AssertionError("Did not raise, returned %s" % res) + #except ExceptionWrapper, ex: + # assert issubclass(eval(ex.class_name), exception) + #else: + # assert False, 'function did raise no exception at all' + + def ll_to_string(self, s): + return str(s) + + def ll_to_list(self, l): + return l + + def ll_unpack_tuple(self, t, length): + assert len(t) == length + return tuple(t) + + def class_name(self, value): + return value[:-8].split('.')[-1] + + def is_of_instance_type(self, val): + m = re.match("^<.* object>$", val) + return bool(m) + + def read_attr(self, obj, name): + py.test.skip('read_attr not supported on genjs tests') + +def check_source_contains(compiled_function, pattern): + import re + + source = compiled_function.js.tmpfile.open().read() + return re.search(pattern, source) Added: pypy/branch/flex-backend/translator/flex/test/test_basicexternal.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_basicexternal.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,129 @@ + +""" BasicExternal testers +""" + +import py + +from pypy.rpython.ootypesystem.bltregistry import MethodDesc, BasicExternal, described +from pypy.translator.js.test.runtest import compile_function, check_source_contains +from pypy.translator.js.tester import schedule_callbacks + +class A(BasicExternal): + @described(retval=int) + def some_code(self, var="aa"): + pass + +a = A() +a._render_name = 'a' + +class B(object): + pass + +def test_decorator(): + def dec_fun(): + a.some_code("aa") + + fun = compile_function(dec_fun, []) + assert check_source_contains(fun, "\.some_code") + +def test_basicexternal_element(): + def be_fun(): + b = B() + b.a = a + b.a.some_code("aa") + + fun = compile_function(be_fun, []) + assert check_source_contains(fun, "\.some_code") + +##def test_basicexternal_raise(): +## #py.test.skip("Constant BasicExternals not implemented") +## def raising_fun(): +## try: +## b = B() +## except: +## pass +## else: +## return 3 +## +## fun = compile_function(raising_fun, []) +## assert fun() == 3 + +class EE(BasicExternal): + @described(retval=int) + def bb(self): + pass + +ee = EE() +ee._render_name = 'ee' + +def test_prebuild_basicexternal(): + def tt_fun(): + ee.bb() + + fun = compile_function(tt_fun, []) + assert check_source_contains(fun, "EE = ee") + +class C(BasicExternal): + @described(retval=int) + def f(self): + pass + +c = C() +c._render_name = 'c' + +def test_basicexternal_raise_method_call(): + def raising_method_call(): + try: + c.f() + except: + pass + + fun = compile_function(raising_method_call, []) + assert len(C._methods) == 1 + assert 'f' in C._methods + +class D(BasicExternal): + _fields = { + 'a': {str:str}, + 'b': [str], + } + +D._fields['c'] = [D] + +d = D() +d._render_name = 'd' + +def test_basicexternal_list(): + def getaa(item): + return d.c[item] + + def return_list(i): + one = getaa(i) + if one: + two = getaa(i + 3) + return two + return one + + fun2 = compile_function(return_list, [int]) + +def test_basicextenal_dict(): + def return_dict(): + return d.a + + fun1 = compile_function(return_dict, []) + +def test_method_call(): + class Meth(BasicExternal): + @described(retval=int) + def meth(self): + return 8 + + l = [] + + def callback(i): + l.append(i) + + meth = Meth() + meth.meth(callback) + schedule_callbacks(meth) + assert l[0] == 8 Added: pypy/branch/flex-backend/translator/flex/test/test_bltn.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_bltn.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,79 @@ +""" blttest - some tests of builtin stuff +""" + +import py + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.translator.js.test.runtest import compile_function, check_source_contains +from pypy.rpython.extfunc import genericcallable +from pypy.translator.js import commproxy + +# check rendering dom.document +def test_simple_builtin(): + from pypy.translator.js.modules.dom import document + def test_document_call(): + return document.getElementById("some_id") + + fn = compile_function(test_document_call, []) + assert check_source_contains(fn, "= document") + assert check_source_contains(fn, "\.getElementById") + +# check rendering transparent proxy +class SomeProxy(BasicExternal): + _render_xmlhttp = True + + _methods = { + 'some_method' : MethodDesc([], int), + } + +class SomeNode(BasicExternal): + _fields = { + 'some_callback' : genericcallable([int], int), + } + +SomeProxyInstance = SomeProxy() + +def test_simple_proxy(): + try: + oldproxy = commproxy.USE_MOCHIKIT + commproxy.USE_MOCHIKIT = True + def simple_proxy(): + retval = SomeProxyInstance.some_method() + + fn = compile_function(simple_proxy, []) + assert check_source_contains(fn, "loadJSONDoc\('some_method'") + finally: + commproxy.USE_MOCHIKIT = oldproxy + +SomeNodeInstance = SomeNode() +SomeNodeInstance._render_name = 's' + +# next will try out the callback +def test_callback(): + def callback(a): + return a + + def callback_stuff(): + SomeNodeInstance.some_callback = callback + + fn = compile_function(callback_stuff, []) + assert check_source_contains(fn, "\.some_callback = callback") + +def test_get_elements(): + from pypy.translator.js.modules import dom + + def getaa(tname): + return dom.document.getElementsByTagName(tname)[0].nodeValue + + def some_stuff(): + one = getaa("some") + if one: + two = getaa("other") + return two + # if two: + # return one + two + # else: + # return one + return one + + fn = compile_function(some_stuff, []) Added: pypy/branch/flex-backend/translator/flex/test/test_class.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_class.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,146 @@ +from __future__ import division +import py + +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.llvm.test import llvmsnippet + +class TestClass(object): + def test_classsimple(self): + f = compile_function(llvmsnippet.class_simple, []) + assert f() == 14 + + def test_classsimple1(self): + f = compile_function(llvmsnippet.class_simple1, [int]) + assert f(2) == 10 + + def test_id_int(self): + #py.test.skip("Full list support not implemented") + f = compile_function(llvmsnippet.id_int, [int]) + for i in range(1, 20): + assert f(i) == i + + def test_classsimple2(self): + f = compile_function(llvmsnippet.class_simple2, [int]) + assert f(2) == 10 + + def test_inherit1(self): + f = compile_function(llvmsnippet.class_inherit1, []) + assert f() == 11 + + def test_inherit2(self): + #py.test.skip("Inheritance not implemented") + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + f = compile_function(llvmsnippet.class_inherit2, []) + assert f() == 1 + + def test_method_of_base_class(self): + #py.test.skip("Inheritance not implemented") + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + f = compile_function(llvmsnippet.method_of_base_class, []) + assert f() == 14 + + def test_attribute_from_base_class(self): + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.attribute_from_base_class, []) + assert f() == 4 + + def test_direct_call_of_virtual_method(self): + #py.test.skip("Inheritance not implemented") + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + f = compile_function(llvmsnippet.direct_call_of_virtual_method, []) + assert f() == 14 + + def test_flow_type(self): + #py.test.skip("isinstanceof not implemented") + f = compile_function(llvmsnippet.flow_type, []) + assert f() == 16 + + def test_merge_class(self): + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.merge_classes, [bool]) + assert f(True) == 1 + assert f(False) == 2 + + def test_attribute_instance(self): + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.attribute_instance, [bool]) + assert f(True) == 1 + assert f(False) == 2 + + def test_global_instance(self): #issue we restart every test with a fresh set of globals + f = compile_function(llvmsnippet.global_instance, [int]) + assert f(-1) == llvmsnippet.global_instance(-1) + for i in range(20): + x = f(i) + y = llvmsnippet.global_instance(i) + assert x == y + + def test_getset(self): + #py.test.skip("Basic arguments of const objects not implemented") + f = compile_function(llvmsnippet.testgetset, [int]) + assert f(15) == 25 + + def test_call_degrading_func(self): + #py.test.skip("isinstanceof not implemented") + f = compile_function(llvmsnippet.call_degrading_func, [bool]) + assert f(True) == llvmsnippet.call_degrading_func(True) + assert f(False) == llvmsnippet.call_degrading_func(False) + + def test_circular_classdef(self): + #py.test.skip("Problems with constant names") + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.circular_classdef, []) + assert f() == 10 + +class A(object): + def __init__(self, a): + self.a = a + if a is None: + self.b = 0 + else: + self.b = a.b + 3 + +a = A(None) + +def test_circular_class(): + def circular_class(): + b = A(a) + return b.b + + fn = compile_function(circular_class, []) + assert fn() == 3 + +INIT_VAL = 0 + +class B(object): + def __init__(self): + self.a = [INIT_VAL] * 30 + +b = B() + +def test_init_list(): + def init_list(i): + if i == 8: + b.a = [1,1,1] + return b.a[2] + + fn = compile_function(init_list, [int]) + assert fn(3) == INIT_VAL + assert fn(8) == 1 + +class C(object): + pass + +def test_instance_str(): + def instance_str(): + return str(C()) + + fn = compile_function(instance_str, []) + assert fn() == '' + +def test_instance_ret(): + def instance_ret(): + return str(C()) + + fn = compile_function(instance_ret, []) + assert fn() == '' Added: pypy/branch/flex-backend/translator/flex/test/test_exc_operation.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_exc_operation.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,195 @@ +import py +import sys +from pypy.translator.js.test.runtest import compile_function +from pypy.rlib.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift +from pypy.translator.test import snippet + +py.test.skip("Exception work in progress") + +def test_zerodiv_int(): + def zerodiv_int(n): + try: + r=100/n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodiv_int, [int]) + for i in (-50,0,50): + assert f(i) == zerodiv_int(i) + +def test_zerodiv_uint(): + def zerodiv_uint(n): + try: + r=100/n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodiv_uint, [r_uint]) + for i in (0,50,100): + assert f(i) == zerodiv_uint(i) + +def test_zerodivrem_int(): + def zerodivrem_int(n): + try: + r=100%n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodivrem_int, [int]) + for i in (-50,0,50): + assert f(i) == zerodivrem_int(i) + +def test_zerodivrem_uint(): + def zerodivrem_uint(n): + try: + r=100%n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodivrem_uint, [r_uint]) + for i in (0,50,100): + assert f(i) == zerodivrem_uint(i) + +def DONTtest_neg_int_ovf(): #issue with Javascript Number() having a larger range + def neg_int_ovf(n): + try: + r=ovfcheck(-n) + except OverflowError: + return 123 + return r + f = compile_function(neg_int_ovf, [int]) + for i in (-sys.maxint-1, -sys.maxint, 0, sys.maxint-1, sys.maxint): + assert f(i) == neg_int_ovf(i) + +def DONTtest_abs_int_ovf(): #issue with Javascript Number() having a larger range + def abs_int_ovf(n): + try: + r=ovfcheck(abs(n)) + except OverflowError: + return 123 + return r + f = compile_function(abs_int_ovf, [int]) + for i in (-sys.maxint-1, -sys.maxint, 0, sys.maxint-1, sys.maxint): + assert f(i) == abs_int_ovf(i) + +############################ + +#raises(...) fails because we do'nt reraise javascript exceptions on the python level + +def DONTtest_int_ovf(): #issue with Javascript Number() having a larger range + def int_ovf_fn(i): + try: + return snippet.add_func(i) + except OverflowError: + return 123 + except: + return 1234 + fn = compile_function(int_ovf_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == int_ovf_fn(i) + +def test_int_div_ovf_zer(): + def int_div_ovf_zer_fn(i): + try: + return snippet.div_func(i) + except OverflowError: + return 123 + except ZeroDivisionError: + return 1234 + except: + return 12345 + fn = compile_function(int_div_ovf_zer_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == int_div_ovf_zer_fn(i) + +def DONTtest_int_mod_ovf_zer(): #issue with Javascript Number() having a larger range + def int_mod_ovf_zer_fn(i): + try: + return snippet.mod_func(i) + except OverflowError: + return 123 + except ZeroDivisionError: + return 1234 + except: + return 12345 + fn = compile_function(int_mod_ovf_zer_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == int_mod_ovf_zer_fn(i) + +def DONTtest_int_rshift_val(): #issue with Javascript Number() having a larger range + def rshift_fn(i): + try: + return snippet.rshift_func(i) + except ValueError: + return 123 + except: + return 1234 + fn = compile_function(rshift_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == rshift_fn(i) + +def test_int_lshift_ovf_val(): + def lshift_fn(i): + try: + return snippet.lshift_func(i) + except ValueError: + return 123 + except OverflowError: + return 1234 + except: + return 12345 + fn = compile_function(lshift_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == lshift_fn(i) + +def test_uint_arith(): + py.test.skip("unsigned number becomes negative because we should cast to unsigned when necessary") + def fn(i): + try: + return ~(i*(i+1))/(i-1) + except ZeroDivisionError: + return r_uint(91872331) + f = compile_function(fn, [r_uint]) + for value in range(15): + i = r_uint(value) + assert f(i) == fn(i) + +def test_int_add_ovf(): + def add_func(i): + try: + return ovfcheck(i + 1) + except OverflowError: + return 123 + f = compile_function(add_func, [int]) + assert f(0) == add_func(0) + assert f(0) == 1 + assert f(sys.maxint) == add_func(sys.maxint) + assert f(sys.maxint) == 123 + +def test_int_sub_ovf(): + def sub_func(i): + try: + return ovfcheck(i - 1) + except OverflowError: + return 123 + n_ovf = 0 + f = compile_function(sub_func, [int]) + for i in (-1, 0, 1, sys.maxint, -sys.maxint, -sys.maxint-1): + result = f(i) + assert result == sub_func(i) + n_ovf += result == 123 + assert n_ovf == 1 + +#As JavaScript uses floating-point numbers the accuracy is only assured +#for integers between: -9007199254740992 (-2^53) and 9007199254740992 (2^53) +def DONTtest_shift_with_overflow(): #issue with Javascript Number() having a larger range + def shiftleft(x, y): + return x << y + def shiftright(x, y): + return x >> y + shl = compile_function(shiftleft , [int, int]) + shr = compile_function(shiftright, [int, int]) + for i in [1, 2, 3, 100000, 2000000, sys.maxint - 1]: + for j in [1, 2, 3, 100000, 2000000, sys.maxint - 1]: + assert shl(i, j) == i << j + assert shr(i, j) == i >> j Added: pypy/branch/flex-backend/translator/flex/test/test_exception.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_exception.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,287 @@ +import py + +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.test.snippet import try_raise_choose +from pypy.rlib.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift + +import sys + +#py.test.skip("Exception work in progress") + +class TestException(Exception): + pass + +class MyException(Exception): + def __init__(self, n): + self.n = n + +def getitem(l, i): #LookupError, KeyError + if not isinstance(i, int): + raise TypeError + if i < 0: + i = len(l) - i + if i>= len(l): + raise IndexError + return l[i] + +def test_simple1(): + def raise_(i): + if i: + raise TestException() + else: + return 3 + def fn(i): + try: + a = raise_(i) + 11 + b = raise_(i) + 12 + c = raise_(i) + 13 + return a+b+c + except TestException: + return 7 + else: + return 3 + f = compile_function(fn, [int]) + assert f(0) == fn(0) + assert f(1) == fn(1) + +def test_simple2(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except: + return 2 + return 4 + f = compile_function(fn, [int]) + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == fn(10) + +def test_simple3(): + def raise_(i): + if i == 0: + raise TestException() + elif i == 1: + raise MyException(42) + else: + return 3 + def fn(i): + try: + a = raise_(i) + 11 + b = raise_(i) + 12 + c = raise_(i) + 13 + return a+b+c + except TestException: + return 7 + except MyException: + return 123 + except: + return 22 + return 66 + f = compile_function(fn, [int]) + assert f(0) == fn(0) + assert f(1) == fn(1) + assert f(2) == fn(2) + +def test_pass_exc(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except: + pass + return 4 + f = compile_function(fn, [int]) + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == fn(10) + +def test_reraise1(): + def fn2(n): + lst = range(10) + try: + getitem(lst,n) + except: + raise + return 4 + def fn(n): + try: + return fn2(n) + except: + return 123 + f = compile_function(fn, [int]) + assert f(-1) == 123 + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == 123 + assert f(10) == fn(10) + +def test_reraise2(): + def fn2(n): + lst = range(10) + try: + getitem(lst,n) + except Exception, e: + raise e + return 4 + def fn(n): + try: + return fn2(n) + except: + return 123 + f = compile_function(fn, [int]) + assert f(-1) == 123 + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == 123 + assert f(10) == fn(10) + +def test_simple_exception(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except IndexError: + return 2 + return 4 + f = compile_function(fn, [int]) + for i in range(10): + assert f(i) == fn(i) + for i in range(10, 20): + assert f(i) == fn(i) + +def test_two_exceptionsA(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except IndexError: + return 2 + except KeyError: + return 3 + return 4 + f = compile_function(fn, [int]) + for i in range(10): + assert f(i) == fn(i) + for i in range(10, 20): + assert f(i) == fn(i) + +def test_catch_base_exception(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except LookupError: + return 2 + return 4 + f = compile_function(fn, [int]) + for i in range(10): + assert f(i) == fn(i) + for i in range(10, 20): + assert f(i) == fn(i) + +def test_catches(): + def raises(i): + if i == 3: + raise MyException, 12 + if i == 4: + raise IndexError + if i > 5: + raise MyException(i) + return 1 + def fn(i): + try: + return raises(i) + except MyException, e: + return e.n + except: + return 123 + f = compile_function(fn, [int]) + assert f(1) == fn(1) + assert f(2) == fn(2) + assert f(3) == fn(3) + #py.test.raises(RuntimeError, "f(4)") #XXX would like to test: py.test.not_raises(....) + assert f(5) == fn(5) + assert f(6) == fn(6) + assert f(13) == fn(13) + +def test_try_raise_choose(): + f = compile_function(try_raise_choose, [int]) + for i in [-1, 0, 1, 2]: + assert f(i) == i + +def test_two_exceptionsB(): + def fn1(): + raise Exception + def fn2(): + return 10 + def two_exceptionsB(): + r = 50 + try: + fn1() + r += 1 + except: + r += 100 + try: + r += fn2() + except: + r += 300 + r += fn2() + return r + f = compile_function(two_exceptionsB, []) + assert f() == two_exceptionsB() + +def test_raise_outside_testfn(): + def raiser(n): + if n < 0: + raise ValueError("hello") + else: + raise MyException("world") + + def intermediate(n): + raiser(n) + + def testfn(n): + try: + intermediate(n) + except ValueError: + return 1 + except Exception: + return 2 + return 0 + + saved = no_magic() + try: + f = compile_function(testfn, [int]) + assert f(1) == testfn(1) + assert f(-1) == testfn(-1) + finally: + restore_magic(saved) + +def no_magic(): + import __builtin__ + try: + py.magic.revert(__builtin__, 'AssertionError') + return True + except ValueError: + return False + +def restore_magic(saved): + if saved: + py.magic.invoke(assertion=True) + +def test_always_raise(): + def function_raise2(i): + if i == 3: + raise IndexError() + else: + pass + + def function_raise1(i): + try: + function_raise2(i) + except Exception, e: + return str(e) + + fn = compile_function(function_raise1, [int]) + fn(3) Added: pypy/branch/flex-backend/translator/flex/test/test_extfunc.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_extfunc.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,17 @@ + +""" Some external functions tests +""" + +from pypy.translator.js.test.runtest import compile_function, check_source_contains + +def test_set_timeout(): + from pypy.translator.js.modules.dom import setTimeout + + def to_timeout(): + pass + + def s_timeout_call(): + setTimeout(to_timeout, 300) + + c = compile_function(s_timeout_call, []) + assert check_source_contains(c, "setTimeout \( 'to_timeout\(\)',300 \)") Added: pypy/branch/flex-backend/translator/flex/test/test_loops.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_loops.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,66 @@ +import py + +from pypy.translator.js.test.runtest import compile_function + +import sys + +def test_if_simple(): + def f(x): + if x == 1: + return 1 + else: + return 2 + fn = compile_function(f,[int]) + assert fn(1) == f(1) + assert fn(2) == f(2) + +def test_if_call(): + def check(x): + if x == 1: + return True + return False + + def f(x): + if check(x): + return 1 + else: + return 2 + + fn = compile_function(f,[int]) + assert fn(1) == f(1) + assert fn(2) == f(2) + +def test_while(): + def f(s): + while s > 1: + s -= 1 + return s + + fn = compile_function(f,[int]) + assert fn(38) == f(38) + assert fn(5) == f(5) + +def test_while_break(): + def f(s): + while s > 1: + if s == 8: + break + s -= 1 + return s + + fn = compile_function(f,[int]) + assert fn(38) == f(38) + assert fn(5) == f(5) + +def test_while_if(): + def f(x): + while x > 1: + if x%2 == 0: + x -= 7 + else: + x -= 3 + return x + + fn = compile_function(f,[int]) + assert fn(38) == f(38) + assert fn(5) == f(5) Added: pypy/branch/flex-backend/translator/flex/test/test_main.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_main.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,69 @@ + +""" tests of rpython2javascript function +""" + +from pypy.translator.js.main import rpython2javascript, rpython2javascript_main,\ + js_optiondescr +from pypy.translator.js import conftest +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, described +import py +import sys +from pypy.tool.udir import udir +from pypy.config.config import Config, to_optparse + + +#if not conftest.option.browser: +# py.test.skip("Works only in browser (right now?)") + +def setup_module(mod): + mod.jsconfig = Config(js_optiondescr) + +class A(BasicExternal): + def method(self, a={'a':'a'}): + pass + method = described(retval={str:str})(method) + +a = A() +a._render_name = 'a' + +def fun(x='3'): + return a.method({'a':x})['a'] + +def fff(): + pass + +def test_bookkeeper_cleanup(): + assert rpython2javascript(sys.modules[__name__], ["fun"]) + assert rpython2javascript(sys.modules[__name__], ["fun"]) + +def test_module_none(): + assert rpython2javascript(None, "fff") + +class TestJsMain(object): + def _test_not_raises(self, mod_file, args_rest=[]): + try: + rpython2javascript_main([str(mod_file)] + args_rest, + jsconfig) + except SystemExit: + py.test.fail("Exited") + + def _test_raises(self, mod_file, args_rest): + py.test.raises(SystemExit, rpython2javascript_main, + [str(mod_file)] + args_rest, jsconfig) + + def test_main_one(self): + udir.ensure("js_one.py").write(py.code.Source(""" + def f(): + pass + f._client = True + """)) + self._test_not_raises(udir.join("js_one.py")) + + def test_main_two(self): + udir.ensure("js_two.py").write(py.code.Source(""" + def f(): + pass + """)) + self._test_not_raises(udir.join("js_two.py"), ["f"]) + self._test_raises(udir.join("js_two.py"), []) + Added: pypy/branch/flex-backend/translator/flex/test/test_rclass.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rclass.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,77 @@ +import py +from pypy.translator.js.test.runtest import JsTest +from pypy.rpython.test.test_exception import BaseTestException +from pypy.rpython.test.test_rclass import BaseTestRclass +from pypy.rpython.test.test_rtuple import BaseTestRtuple +from pypy.rpython.test.test_rstr import BaseTestRstr + +class TestJsException(JsTest, BaseTestException): + pass + +# ====> ../../../rpython/test/test_rclass.py + +class TestJsClass(JsTest, BaseTestRclass): + def test___class___attribute(self): + class Base(object): pass + class A(Base): pass + class B(Base): pass + class C(A): pass + def seelater(): + C() + def f(n): + if n == 1: + x = A() + else: + x = B() + y = B() + result = x.__class__, y.__class__ + seelater() + return result + def g(): + cls1, cls2 = f(1) + return cls1 is A, cls2 is B + + res = self.interpret(g, []) + assert res[0] + assert res[1] + + def test_mixin(self): + class Mixin(object): + _mixin_ = True + + def m(self, v): + return v + + class Base(object): + pass + + class A(Base, Mixin): + pass + + class B(Base, Mixin): + pass + + class C(B): + pass + + def f(): + a = A() + v0 = a.m(2) + b = B() + v1 = b.m('x') + c = C() + v2 = c.m('y') + return v0, v1, v2 + + res = self.interpret(f, []) + assert isinstance(res[0], int) + + def test_hash_preservation(self): + py.test.skip("WIP") + + def test_issubclass_type(self): + py.test.skip("WIP") + + #def test_isinstance(self): + # py.test.skip("WIP") + Added: pypy/branch/flex-backend/translator/flex/test/test_rdict.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rdict.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,45 @@ + +import py +from pypy.rpython.test.test_rdict import BaseTestRdict +from pypy.translator.js.test.runtest import JsTest + +# ====> ../../../rpython/test/test_rdict.py + +class TestJsDict(JsTest, BaseTestRdict): + def test_tuple_dict(self): + py.test.skip("rdict not implemented") + + def test_dict_itermethods(self): + py.test.skip("itermethods not implemented") + + def test_dict_copy(self): + py.test.skip("itermethods not implemented") + + def test_dict_update(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_iterkeys(self): + py.test.skip("itermethods not implemented") + + def test_dict_values(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_values(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_itervalues(self): + py.test.skip("itermethods not implemented") + + def test_dict_items(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_items(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_iteritems(self): + py.test.skip("itermethods not implemented") + + + def test_specific_obscure_bug(self): + py.test.skip("rdict not implemented") + Added: pypy/branch/flex-backend/translator/flex/test/test_rfloat.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rfloat.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,24 @@ + +import py +from pypy.translator.js.test.runtest import JsTest +from pypy.rpython.test.test_rfloat import BaseTestRfloat +from pypy.rlib.rarithmetic import r_uint, r_longlong + +class TestJsFloat(JsTest, BaseTestRfloat): + def test_from_r_uint(self): + py.test.skip("Not implemented") + + def test_longlong_conversion(self): + def fn(f): + return r_longlong(f) + + res = self.interpret(fn, [1.0]) + assert res == 1 + #assert self.is_of_type(res, r_longlong) + res = self.interpret(fn, [2.34]) + assert res == fn(2.34) + big = float(0x7fffffffffffffff) + x = big - 1.e10 + assert x != big + y = fn(x) + assert fn(x) == 9223372026854775808 Added: pypy/branch/flex-backend/translator/flex/test/test_rlist.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rlist.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,85 @@ + +import py +from pypy.rpython.test.test_rlist import BaseTestRlist +from pypy.translator.js.test.runtest import JsTest + +class Foo: + pass + +class Bar(Foo): + pass + +# ====> ../../../rpython/test/test_rlist.py + +class TestJsList(JsTest, BaseTestRlist): + def test_append(self): + def dummyfn(): + l = [] + l.append(50) + l.append(60) + l.append(70) + l.append(80) + l.append(90) + return len(l), l[0], l[-1] + res = self.interpret(dummyfn, []) + assert res == [5, 50, 90] + + def test_slice(self): + py.test.skip("Imperfect testing machinery") + def dummyfn(): + l = [5, 6, 7, 8, 9] + return l[:2], l[1:4], l[3:] + res = self.interpret(dummyfn, []) + + def dummyfn(): + l = [5, 6, 7, 8] + l.append(9) + return l[:2], l[1:4], l[3:] + res = self.interpret(dummyfn, []) + assert res == ([5, 6], [6, 7, 8], [8, 9]) + + def test_setslice(self): + def dummyfn(): + l = [10, 9, 8, 7] + l[:2] = [6, 5] + return l[0], l[1], l[2], l[3] + res = self.interpret(dummyfn, ()) + assert res == [6, 5, 8, 7] + + def test_insert_bug(self): + def dummyfn(n): + l = [1] + l = l[:] + l.pop(0) + if n < 0: + l.insert(0, 42) + else: + l.insert(n, 42) + return l + res = self.interpret(dummyfn, [0]) + assert len(res) == 1 + assert res[0] == 42 + res = self.interpret(dummyfn, [-1]) + assert len(res) == 1 + assert res[0] == 42 + + def test_list_str(self): + pass + + def test_inst_list(self): + def fn(): + l = [None] + l[0] = Foo() + l.append(Bar()) + l2 = [l[1], l[0], l[0]] + l.extend(l2) + for x in l2: + l.append(x) + x = l.pop() + x = l.pop() + x = l.pop() + x = l2.pop() + return str(x)+";"+str(l) + res = self.ll_to_string(self.interpret(fn, [])) + res = res.replace('pypy.translator.js.test.test_rlist.', '') + assert res == ';[, , , , ]' Added: pypy/branch/flex-backend/translator/flex/test/test_rpbc.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rpbc.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,36 @@ + +import py +from pypy.translator.js.test.runtest import JsTest +from pypy.rpython.test.test_rpbc import BaseTestRPBC + +# ====> ../../../rpython/test/test_rpbc.py + +class TestJsPBC(JsTest, BaseTestRPBC): + def test_single_pbc_getattr(self): + class C: + def __init__(self, v1, v2): + self.v1 = v1 + self.v2 = v2 + def _freeze_(self): + return True + c1 = C(11, lambda: "hello") + c2 = C(22, lambda: 623) + def f1(l, c): + l.append(c.v1) + def f2(c): + return c.v2 + def f3(c): + return c.v2 + def g(): + l = [] + f1(l, c1) + f1(l, c2) + return f2(c1)(), f3(c2)() + + res = self.interpret(g, []) + assert res[0] == "hello" + assert res[1] == 623 + + def test_call_memoized_function_with_bools(self): + py.test.skip("WIP") + Added: pypy/branch/flex-backend/translator/flex/test/test_rsnippet.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rsnippet.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,453 @@ +from __future__ import division + +import sys +import py + +from pypy.rlib.rarithmetic import r_uint +from pypy.translator.js.test.runtest import compile_function + +def test_simple_function_pointer(): + #py.test.skip("ootypesystem problems with lists") + def f1(x): + return x + 1 + def f2(x): + return x + 2 + + l = [f1, f2] + + def pointersimple(i): + return l[i](i) + + f = compile_function(pointersimple, [int]) + assert f(0) == pointersimple(0) + assert f(1) == pointersimple(1) + +def test_return1(): + def simple1(): + return 1 + f = compile_function(simple1, []) + assert f() == 1 + +def test_invoke_function_pointer(): + #py.test.skip("ootypesystem problems with lists") + def f1(x): + return x + 1 + def f2(x): + return x + 2 + + l = [f1, f2] + + def invokepointer(i): + try: + return l[i](i) + except: + return 123 + + f = compile_function(invokepointer, [int]) + assert f(0) == invokepointer(0) + assert f(1) == invokepointer(1) + +def test_simple_branching(): + def simple5(b): + if b: + x = 12 + else: + x = 13 + return x + f = compile_function(simple5, [bool]) + assert f(True) == 12 + assert f(False) == 13 + +def test_int_ops(): + def ops(i): + x = 0 + x += i < i + x += i <= i + x += i == i + x += i != i + x += i >= i + x += i > i + x += x % i + x += i + 1 * i // i - 1 + return int(x+0.0001) + f = compile_function(ops, [int]) + assert f(1) == ops(1) + assert f(2) == ops(2) + +def test_uint_ops(): + def ops(i): + x = 0 + x += i < i + x += i <= i + x += i == i + x += i != i + x += i >= i + x += i > i + x += x % i + x += i + 1 * i // i - 1 + return int(x+0.0001) + f = compile_function(ops, [r_uint]) + assert f(1) == ops(1) + assert f(2) == ops(2) + +def test_float_ops(): + def ops(flt): + x = 0 + x += flt < flt + x += flt <= flt + x += flt == flt + x += flt != flt + x += flt >= flt + x += flt > flt + x += int(flt + 1 * flt / flt - 1) + return x + f = compile_function(ops, [float]) + assert f(1) == ops(1) + assert f(2) == ops(2) + +def test_while_loop(): + def factorial(i): + r = 1 + while i>1: + r *= i + i -= 1 + return r + f = compile_function(factorial, [int]) + assert factorial(4) == 24 + assert factorial(5) == 120 + f = compile_function(factorial, [float]) + assert factorial(4.) == 24. + assert factorial(5.) == 120. + +def test_return_void(): + def return_void(i): + return None + def call_return_void(i): + return_void(i) + return 1 + f = compile_function(call_return_void, [int]) + assert f(10) == 1 + +def test_break_while_loop(): + def factorial(i): + r = 1 + while 1: + if i<=1: + break + r *= i + i -= 1 + return r + f = compile_function(factorial, [int]) + assert factorial(4) == 24 + assert factorial(5) == 120 + +def test_primitive_is_true(): + def var_is_true(v): + return bool(v) + f = compile_function(var_is_true, [int]) + assert f(256) + assert not f(0) + f = compile_function(var_is_true, [r_uint]) + assert f(r_uint(256)) + assert not f(r_uint(0)) + f = compile_function(var_is_true, [float]) + assert f(256.0) + assert not f(0.0) + +def test_function_call(): + def callee(): + return 1 + def caller(): + return 3 + callee() + f = compile_function(caller, []) + assert f() == 4 + +def test_recursive_call(): + def call_ackermann(n, m): + return ackermann(n, m) + def ackermann(n, m): + if n == 0: + return m + 1 + if m == 0: + return ackermann(n - 1, 1) + return ackermann(n - 1, ackermann(n, m - 1)) + f = compile_function(call_ackermann, [int, int]) + assert f(0, 2) == 3 + +def test_tuple_getitem(): + def tuple_getitem(i): + l = (4,5,i) + return l[1] + f = compile_function(tuple_getitem, [int]) + assert f(1) == tuple_getitem(1) + +def test_nested_tuple(): + def nested_tuple(i): + l = (1,(1,2,i),i) + return l[1][2] + f = compile_function(nested_tuple, [int]) + assert f(4) == 4 + +def test_prebuilt_tuples(): + t1 = (1,2,3,4) + t2 = (5,6,7,8) + def callee_tuple(t): + return t[0] + def caller_tuple(i): + if i: + return callee_tuple(t1) + i + else: + return callee_tuple(t2) + i + f = compile_function(caller_tuple, [int]) + assert f(0) == 5 + assert f(1) == 2 + +def test_pbc_fns(): + #py.test.skip("Indirect call not implemented") + def f2(x): + return x+1 + def f3(x): + return x+2 + def g(y): + if y < 0: + f = f2 + else: + f = f3 + return f(y+3) + f = compile_function(g, [int]) + assert f(-1) == 3 + assert f(0) == 5 + +def test_simple_chars(): #XXX test this also without optimize_call(...) + #py.test.skip("Method mapping not implemented") + def char_constant2(s): + s = s + s + s + return len(s + '.') + def char_constant(): + return char_constant2("kk") + f = compile_function(char_constant, []) + assert f() == 7 + +def test_list_getitem(): + #py.test.skip("List suuport") + def list_getitem(i): + l = [1,2,i+1] + return l[i] + f = compile_function(list_getitem, [int]) + assert f(0) == 1 + assert f(1) == 2 + assert f(2) == 3 + +def test_list_list_getitem(): + #py.test.skip("List support") + def list_list_getitem(): + l = [[1]] + return l[0][0] + f = compile_function(list_list_getitem, []) + assert f() == 1 + +def test_list_getitem_pbc(): + #py.test.skip("pbc support") + l = [1,2] + def list_getitem_pbc(i): + return l[i] + f = compile_function(list_getitem_pbc, [int]) + assert f(0) == 1 + assert f(1) == 2 + +def test_list_list_getitem_pbc(): + #py.test.skip("pbc support") + l = [[0, 1], [0, 1]] + def list_list_getitem_pbc(i): + return l[i][i] + f = compile_function(list_list_getitem_pbc, [int]) + assert f(0) == 0 + assert f(1) == 1 + +def test_list_basic_ops(): + #py.test.skip("List support") + def list_basic_ops(i, j): + l = [1,2,3] + l.insert(0, 42) + del l[1] + l.append(i) + listlen = len(l) + l.extend(l) + del l[listlen:] + l += [5,6] + l[1] = i + return l[j] + f = compile_function(list_basic_ops, [int, int]) + for i in range(6): + for j in range(6): + assert f(i,j) == list_basic_ops(i,j) + +def test_string_simple(): + #py.test.skip("ord semantics") + def string_simple(i): + return ord(str(i)) + f = compile_function(string_simple, [int]) + assert f(3) == string_simple(3) + +def test_string_simple_ops(): + #py.test.skip("ord semantics") + def string_simple_ops(i): + res = 0 + s = str(i) + s2 = s + s + s + s + s3 = s + s + s + s + res += s != s2 + res += s2 == s3 + res += ord(s) + return res + f = compile_function(string_simple_ops, [int]) + assert f(5) == ord('5') + 2 + +def test_string_getitem1(): + l = "Hello, World" + def string_getitem1(i): + return l[i] + f = compile_function(string_getitem1, [int]) + assert f(0) == "H" + +def test_string_getitem2(): + def string_test(i): + l = "Hello, World" + return l[i] + f = compile_function(string_test, [int]) + assert f(0) == "H" + +def test_list_of_string(): + a = ["hello", "world"] + def string_simple(i, j, k, l): + s = a[i][j] + a[k][l] + return s[0] + s[1] + f = compile_function(string_simple, [int, int, int, int]) + assert f(0, 0, 1, 4) == 'hd' + +def test_attrs_class(): + class MyBase: + pass + def attrs_class(a): + obj = MyBase() + obj.z = a + return obj.z * 4 + f = compile_function(attrs_class, [int]) + assert f(4) == 16 + +def test_attrs_class_pbc(): +# py.test.skip("pbc support") + class MyBase: + pass + obj = MyBase() + obj.z = 4 + def attrs_class_pbc(): + return obj.z * 4 + f = compile_function(attrs_class_pbc, []) + assert f() == 16 + +def test_method_call(): + class MyBase: + def m(self): return self.z + obj = MyBase() + obj.z = 4 + def method_call(): + return obj.m() + f = compile_function(method_call, []) + assert f() == 4 + +def test_dict_creation(): + d = {'hello' : 23, + 'world' : 21} + l = ["hello", "world"] + def createdict(i, j): + return d[l[i]] + d[l[j]] + assert createdict(0,1) == 44 + f = compile_function(createdict, [int, int]) + assert f(0,1) == createdict(0,1) + +def test_closure(): + class A: + def set(self, x): + self.x = x + def get(self): + return self.x + class B(A): + def get(self): + return A.get(self) * 2 + a = A() + b = B() + def createfn(y): + z = 42 + def fn(x): + return x + y + z + a.get() + b.get() + f2 = lambda: A.get(b) + def getfn(x): + if x % 2: + f = A.get + else: + f = B.get + return f + def proxy(s): + f1 = s.get + t = 0 + for ii in range(5): + f3 = getfn(ii) + t += f1() + t += f2() + t += f3(s) + return t + setattr(B, "f2", proxy) + return fn + fn = createfn(10) + def testf(x): + a.set(10) + b.set(25) + return fn(x) + b.get() + b.f2() + f = compile_function(testf, [int]) + assert f(1) == testf(1) + assert f(2) == testf(2) + +def test_cast_str(): + def cast_str(x): + return str(x)+str(x)+'px' + + f = compile_function(cast_str, [int]) + assert f(1) == cast_str(1) + assert f(10) == cast_str(10) + +class A(object): + pass + +def some_fun(): + return 3 + +def test_me(): + def some_test(): + a = A() + a.some_fun = some_fun + return a.some_fun() + + fn = compile_function(some_test, []) + assert fn() == some_test() + +def test_symbolic(): + from pypy.rlib.objectmodel import malloc_zero_filled + + def symbolic1(): + if malloc_zero_filled: + return 3 + return 2 + + fn = compile_function(symbolic1, []) + assert fn() == 2 + +def test_is_early_constant(): + from pypy.rlib import jit + def f(x): + if jit._is_early_constant(x): + return 42 + return 0 + fn = compile_function(f, [int]) + res = fn(5) + assert res == 0 Added: pypy/branch/flex-backend/translator/flex/test/test_rsnippet1.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_rsnippet1.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,91 @@ +from __future__ import division +import sys + +import py + +from pypy.objspace.flow.model import Constant, Variable +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.llvm.test import llvmsnippet + +class TestGenLLVM(object): + def test_simple1(self): + f = compile_function(llvmsnippet.simple1, []) + assert f() == 1 + + def test_simple2(self): + f = compile_function(llvmsnippet.simple2, []) + assert f() == False + + def test_simple4(self): + f = compile_function(llvmsnippet.simple4, []) + assert f() == 4 + + def test_simple5(self): + f = compile_function(llvmsnippet.simple5, [int]) + assert f(1) == 12 + assert f(0) == 13 + + def test_ackermann(self): + f = compile_function(llvmsnippet.ackermann, [int, int]) + for i in range(4): # (otherwise too much recursion) max 4 in Safari, max 7 in Firefox, IE allows more recursion + assert f(0, i) == i + 1 + assert f(1, i) == i + 2 + assert f(2, i) == 2 * i + 3 + assert f(3, i) == 2 ** (i + 3) - 3 + + def test_calling(self): + f = compile_function(llvmsnippet.calling1, [int]) + assert f(10) == 1 + + def test_call_default_arguments(self): + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.call_default_arguments, [int, int]) + for i in range(3): + assert f(i + 3, i) == llvmsnippet.call_default_arguments(i + 3, i) + + def DONTtest_call_list_default_argument(self): #issue we restart every test with a fresh set of globals + f = compile_function(llvmsnippet.call_list_default_argument, [int]) + for i in range(20): + assert f(i) == llvmsnippet.call_list_default_argument(i) + + def test_shift(self): + shl = compile_function(llvmsnippet.shiftleft, [int, int]) + shr = compile_function(llvmsnippet.shiftright, [int, int]) + assert shl(42,2) == llvmsnippet.shiftleft(42, 2) + assert shr(42,2) == llvmsnippet.shiftright(42,2) + +class TestFloat(object): + def test_float_f1(self): + f = compile_function(llvmsnippet.float_f1, [float]) + assert f(1.0) == 2.2 + + def test_float_int_bool(self): + f = compile_function(llvmsnippet.float_int_bool, [float]) + assert f(3.0) == 9.0 + + +class TestString(object): + def test_f2(self): + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.string_f2, [int, int]) + assert f(1, 0) == "a" + + +class TestPBC(object): + #py.test.skip("pbc not implemented") + def test_pbc_function1(self): + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.pbc_function1, [int]) + assert f(0) == 2 + assert f(1) == 4 + assert f(2) == 6 + assert f(3) == 8 + + def test_pbc_function2(self): + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.pbc_function2, [int]) + assert f(0) == 13 + assert f(1) == 15 + assert f(2) == 17 + assert f(3) == 19 Added: pypy/branch/flex-backend/translator/flex/test/test_runtest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_runtest.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,75 @@ +import py +from pypy.translator.js.test.runtest import compile_function + +#test returntypes +def test_bool_return(): + def bool_return_False(): + return False + def bool_return_True(): + return True + f_false = compile_function(bool_return_False, []) + assert f_false() == bool_return_False() + f_true = compile_function(bool_return_True , []) + assert f_true() == bool_return_True() + +def test_int_return(): + def int_return(): + return 42 + f = compile_function(int_return, []) + assert f() == int_return() + +def test_float_return(): + def float_return(): + return 42.5 + f = compile_function(float_return, []) + assert f() == float_return() + +#test paramtypes +def test_bool_param(): + def bool_param(b): + return b + f = compile_function(bool_param, [bool]) + assert f(False) == bool_param(False) + assert f(True ) == bool_param(True ) + +def test_int_param(): + def int_param(n): + return n * 2 + f = compile_function(int_param, [int]) + assert f(42) == int_param(42) + +def test_float_param(): + def float_param(f): + return f * 3.0 + f = compile_function(float_param, [float]) + assert f(12.5) == float_param(12.5) + +def test_combined_params(): + def combined_params(b, n, f): + return int(int(b) * 5 + n * 2 + f * 3.0) + f = compile_function(combined_params, [bool,int,float]) + assert f(False, 13, 12.5) == combined_params(False, 13, 12.5) + assert f(True , 13, 12.5) == combined_params(True , 13, 12.5) + +def test_return_function(): + rp = compile_function.reinterpret + assert rp('[a,b]') == ["a", "b"] + #assert rp('(true,[a,b])') == [True, ["a", "b"]] + +def test_return_newline(): + def fun_newline(): + return "\n" + fun = compile_function(fun_newline, []) + assert fun() == "\n" + +##def test_multiple_function(): +## def one(): +## return 1 +## +## def two(x): +## return x +## +## f = compile_function([one, two], [[], [int]]) +## assert f.call(one) == 1 +## assert f.call(two, 3) == 3 +## Added: pypy/branch/flex-backend/translator/flex/test/test_seq.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_seq.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,138 @@ +from __future__ import division +import py + +from pypy.objspace.flow.model import Constant, Variable +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.llvm.test import llvmsnippet + +class TestList(object): + def test_normal_list(self): + def normal_list(): + l = [1,2,3] + return l[1] + + fn = compile_function(normal_list, []) + assert fn() == 2 + + def test_var_list(self): + def var_list(): + l = [] + for i in xrange(10): + l.append(i) + return l[8] + + fn = compile_function(var_list, []) + assert fn() == 8 + + def test_slice(self): + def l_test(): + l = [] + for i in xrange(10): + l.append(i) + return len(l[3:8]) + + fn = compile_function(l_test, []) + assert fn() == l_test() + + def test_init_0(self): + def l_init(): + l = [0] * 100 + return l[38] + + fn = compile_function(l_init, []) + assert fn() == 0 + +class TestDict(object): + def test_dict_iter(self): + def dict_iter(): + sum = 0 + d = {'a':3, 'b':4, 'c':8} + for k in d: + sum += d[k] + return sum + + fn = compile_function(dict_iter, []) + assert fn() == dict_iter() + + def test_const_dict(self): + c = {'aa':'asd', 'ab':'ds', 'ac':'asddsa', 'ad':'sadsad', 'ae':'sadsa'} + + def const_dict(ind): + return c[ind] + + fn = compile_function(const_dict, [str]) + for i in c.keys(): + assert fn(i) == const_dict(i) + + def test_sophisticated_const(self): + c = {'aa':1, 'bb':2, 'cc':3, 'dd':4} + c1 = {'fff' : 'aa', 'zzz' : 'bb', 'xxx' : 'cc', 'bbb' : 'dd'} + c2 = {'xxx' : 'aa', 'kkk' : 'bb', 'aaaa' : 'cc', 'www' : 'dd'} + def soph_const(i1, i2): + return c1[i1] + c2[i2] + + fn = compile_function(soph_const, [str, str]) + for x in c1.keys(): + for y in c2.keys(): + assert fn(x, y) == soph_const(x, y) + + def test_dict_iterator(self): + c = {'aa':1, 'bb':2, 'cc':3, 'dd':4} + def dict_iter(): + sum = 0 + for i in c: + sum += c[i] + return sum + + fn = compile_function(dict_iter, []) + assert fn() == dict_iter() + +class TestTuple(object): + def test_f1(self): + f = compile_function(llvmsnippet.tuple_f1, [int]) + assert f(10) == 10 + assert f(15) == 15 + + def test_f3(self): + f = compile_function(llvmsnippet.tuple_f3, [int]) + assert f(10) == 10 + assert f(15) == 15 + assert f(30) == 30 + + def test_constant_tuple(self): + f = compile_function(llvmsnippet.constant_tuple, [int]) + for i in range(3, 7): + assert f(i) == i + 3 + +class TestString(object): + def test_upperlower(self): + def upperlower(): + s = "aBaF" + return s.upper() + s.lower() + + fn = compile_function(upperlower, []) + assert fn() == "ABAFabaf" + + def test_slice(self): + def one_slice(s): + return s[1:] + + def two_slice(s): + return s[2:] + + fn = compile_function(one_slice, [str]) + assert fn("dupa") == "upa" + fn = compile_function(two_slice, [str]) + assert fn("kupa") == "pa" + +def test_simple_seq(): + def fun(i): + if i: + a = [("ab", "cd"), ("ef", "xy")] + else: + a = [("xz", "pr"), ("as", "fg")] + return ",".join(["%s : %s" % (i, j) for i,j in a]) + + fn = compile_function(fun, [int]) + assert fn(0) == fun(0) + assert fn(1) == fun(1) Added: pypy/branch/flex-backend/translator/flex/test/test_snippet.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_snippet.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,94 @@ +from __future__ import division + +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.test import snippet as test + +import py + +class TestSnippet(object): + def test_if_then_else(self): + f = compile_function(test.if_then_else, [int, int, int]) + assert f(0, 12, 13) == 13 + assert f(13, 12, 13) == 12 + + def test_my_gcd(self): + f = compile_function(test.my_gcd, [int, int]) + assert f(15, 5) == 5 + assert f(18, 42) == 6 + + def test_is_perfect_number(self): + f = compile_function(test.is_perfect_number, [int]) + assert f(28) == 1 + assert f(123) == 0 + assert f(496) == 1 + + def test_my_bool(self): + f = compile_function(test.my_bool, [int]) + assert f(10) == 1 + assert f(1) == 1 + assert f(0) == 0 + + def test_two_plus_two(self): + f = compile_function(test.two_plus_two, []) + assert f() == 4 + + def test_sieve_of_eratosthenes(self): + #py.test.skip("Loop error in loop detection software") + f = compile_function(test.sieve_of_eratosthenes, []) + assert f() == 1028 + + def test_nested_whiles(self): + #py.test.skip("Loop error in loop detection software") + f = compile_function(test.nested_whiles, [int, int]) + assert test.nested_whiles(3,2) == f(3,2) + + def test_simple_func(self): + f = compile_function(test.simple_func, [int]) + assert f(1027) == 1028 + + def test_while_func(self): + while_func = compile_function(test.while_func, [int]) + assert while_func(10) == 55 + + def test_time_waster(self): + #py.test.skip("Loop error in loop detection software") + f = compile_function(test.time_waster, [int]) + assert f(1) == 1 + assert f(2) == 2 + assert f(3) == 6 + assert f(4) == 12 + + def test_int_id(self): + f = compile_function(test.int_id, [int]) + assert f(1027) == 1027 + + def test_factorial2(self): + #py.test.skip("unknown error") + factorial2 = compile_function(test.factorial2, [int]) + assert factorial2(5) == 120 + + def test_factorial(self): + #py.test.skip("unknown error") + factorial = compile_function(test.factorial, [int]) + assert factorial(5) == 120 + + def test_set_attr(self): + set_attr = compile_function(test.set_attr, []) + assert set_attr() == 2 + + def test_merge_setattr(self): + merge_setattr = compile_function(test.merge_setattr, [bool]) + assert merge_setattr(1) == 1 + + def test_simple_method(self): + simple_method = compile_function(test.simple_method, [int]) + assert simple_method(65) == 65 + + def test_with_init(self): + with_init = compile_function(test.with_init, [int]) + assert with_init(42) == 42 + + def test_with_more_init(self): + with_more_init = compile_function(test.with_more_init, [int, bool]) + assert with_more_init(42, True) == 42 + assert with_more_init(42, False) == -42 Added: pypy/branch/flex-backend/translator/flex/test/test_str.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_str.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,72 @@ + +import py +from pypy.translator.js.test.runtest import JsTest +import pypy.translator.oosupport.test_template.string as oostring + + +# ====> ../../../rpython/test/test_rstr.py + +class TestJsString(JsTest, oostring.BaseTestString): + def test_char_unichar_eq(self): + py.test.skip("Cannot test it yet") + + def test_char_unichar_eq_2(self): + py.test.skip("Cannot test it yet") + + def test_unichar_const(self): + py.test.skip("Cannot test it yet") + + def test_unichar_eq(self): + py.test.skip("Cannot test it yet") + + def test_unichar_ord(self): + py.test.skip("Cannot test it yet") + + def test_unichar_hash(self): + py.test.skip("Cannot test it yet") + + def test_rfind(self): + py.test.skip("Not implemented") + + def test_rfind_empty_string(self): + py.test.skip("Not implemented") + + def test_find_char(self): + py.test.skip("Not implemented") + + def test_strip(self): + py.test.skip("Not implemented") + + def test_upper(self): + #XXX Testing machinery is quite confused by js print + strings = ['', ' ', 'upper', 'UpPeR', ',uppEr,'] + #for i in range(256): + # if chr(i) != "\x00" and chr(i) != '(' and chr(i) != '[': + # strings.append(chr(i)) + def fn(i): + return strings[i].upper() + for i in range(len(strings)): + res = self.interpret(fn, [i]) + assert self.ll_to_string(res) == fn(i) + + def test_lower(self): + #XXX Testing machinery is quite confused by js print + strings = ['', ' ', 'lower', 'LoWeR', ',lowEr,'] + #for i in range(256): strings.append(chr(i)) + def fn(i): + return strings[i].lower() + for i in range(len(strings)): + res = self.interpret(fn, [i]) + assert self.ll_to_string(res) == fn(i) + + def test_strformat(self): + py.test.skip("string formatting not implemented for base different than 10") + + def test_float(self): + py.test.skip("returning NaN instead of raising ValueError") + + def test_hash(self): + py.test.skip("Not implemented") + + def test_hash_value(self): + py.test.skip("Not implemented") Added: pypy/branch/flex-backend/translator/flex/test/test_typed.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/test_typed.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,331 @@ +import sys +import py +from py.test import raises +from pypy.translator.test import snippet +from pypy.rlib.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift + +from pypy.translator.js.test.runtest import compile_function + +def test_call_five(): + def wrapper(): + lst = snippet.call_five() + res = list((len(lst), lst[0])) + expected = [1, 5] + return res == expected + fn = compile_function(wrapper, []) + result = fn() + assert result == wrapper() + +def test_get_set_del_slice(): + def get_set_del_nonneg_slice(): # no neg slices for now! + l = [ord('a'), ord('b'), ord('c'), ord('d'), ord('e'), ord('f'), ord('g'), ord('h'), ord('i'), ord('j')] + del l[:1] + bound = len(l)-1 + if bound >= 0: + del l[bound:] + del l[2:4] + #l[:1] = [3] + #bound = len(l)-1 + #assert bound >= 0 + #l[bound:] = [9] no setting slice into lists for now + #l[2:4] = [8,11] + l[0], l[-1], l[2], l[3] = 3, 9, 8, 11 + + list_3_c = l[:2] + list_9 = l[5:] + list_11_h = l[3:5] + return list((len(l), l[0], l[1], l[2], l[3], l[4], l[5], + len(list_3_c), list_3_c[0], list_3_c[1], + len(list_9), list_9[0], + len(list_11_h), list_11_h[0], list_11_h[1])) + + def wrapper(): + res = get_set_del_nonneg_slice() + expected = [6, 3, ord('c'), 8, 11, ord('h'), 9, + 2, 3, ord('c'), + 1, 9, + 2, 11, ord('h')] + + return res == expected + + fn = compile_function(wrapper, []) + result = fn() + assert result + +def test_is(): + def testfn(): + l1 = [] + return l1 is l1 + fn = compile_function(testfn, []) + result = fn() + assert result == True + def testfn(): + l1 = [] + return l1 is None + fn = compile_function(testfn, []) + result = fn() + assert result == False + +def test_nones(): + a = [None] * 4 + def nones(): + a.append(None) + return len(a) + fn = compile_function(nones, []) + result = fn() + assert result == 4 + +def test_str_compare(): + def testfn_eq(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] == s2[j] + fn = compile_function(testfn_eq, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_eq(i, j) + + def testfn_ne(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] != s2[j] + fn = compile_function(testfn_ne, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_ne(i, j) + + def testfn_lt(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] < s2[j] + fn = compile_function(testfn_lt, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_lt(i, j) + + def testfn_le(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] <= s2[j] + fn = compile_function(testfn_le, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_le(i, j) + + def testfn_gt(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] > s2[j] + fn = compile_function(testfn_gt, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_gt(i, j) + + def testfn_ge(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] >= s2[j] + fn = compile_function(testfn_ge, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_ge(i, j) + +def test_str_methods(): + def testfn_startswith(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'ne', 'e', 'twos', 'foobar', 'fortytwo'] + return s1[i].startswith(s2[j]) + fn = compile_function(testfn_startswith, [int, int]) + for i in range(2): + for j in range(9): + res = fn(i, j) + assert res == testfn_startswith(i, j) + + def testfn_endswith(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'ne', 'e', 'twos', 'foobar', 'fortytwo'] + return s1[i].endswith(s2[j]) + fn = compile_function(testfn_endswith, [int, int]) + for i in range(2): + for j in range(9): + res = fn(i, j) + assert res == testfn_endswith(i, j) + +def test_str_join(): + #py.test.skip("issue with malloc_varsize of varsized struct (rpystring here)") + #py.test.skip("stringBuilder support") + def testfn(i, j): + s1 = [ '', ',', ' and '] + s2 = [ [], ['foo'], ['bar', 'baz', 'bazz']] + return len(s1[i].join(s2[j])) + + fn = compile_function(testfn, [int, int]) + for i in range(3): + for j in range(3): + res = fn(i, j) + assert res == testfn(i, j) + +def test_unichr_eq(): + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i, j): + return l[i] == l[j] + fn = compile_function(f, [int, int]) + for i in range(len(l)): + for j in range(len(l)): + res = fn(i, j) + assert res == f(i,j) + +def test_unichr_ne(): + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i, j): + return l[i] != l[j] + fn = compile_function(f, [int, int]) + for i in range(len(l)): + for j in range(len(l)): + res = fn(i, j) + assert res == f(i, j) + +def test_unichr_ord(): + #py.test.skip("ord semantics") + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i): + return ord(l[i]) + fn = compile_function(f, [int]) + for i in range(len(l)): + res = fn(i) + assert res == f(i) + +def test_unichr_unichr(): + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i, j): + return l[i] == unichr(j) + fn = compile_function(f, [int, int]) + for i in range(len(l)): + for j in range(len(l)): + res = fn(i, ord(l[j])) + assert res == f(i, ord(l[j])) + +# floats +def test_float_operations(): + #py.test.skip("issue with ll_math_fmod calling itself") + import math + def func(x, y): + z = x + y / 2.1 * x + z = math.fmod(z, 60.0) + z = pow(z, 2) + z = -z + return int(z) + + fn = compile_function(func, [float, float]) + r1 = fn(5.0, 6.0) + r2 = func(5.0, 6.0) + assert r1 == r2-1 #-1 for stupid spidermonkey rounding error + +def test_rpbc_bound_method_static_call(): + class R: + def meth(self): + return 0 + r = R() + m = r.meth + def fn(): + return m() + res = compile_function(fn, [])() + assert res == 0 + +def test_constant_return_disagreement(): + class R: + def meth(self): + return 0 + r = R() + def fn(): + return r.meth() + res = compile_function(fn, [])() + assert res == 0 + +def test_stringformatting(): + #py.test.skip("StringBuilder not implemented") + def fn(i): + return "you said %d, you did" % i + def wrapper(i): + res = fn(i) + return res == "you said 42, you did" + + f = compile_function(wrapper, [int]) + assert f(42) + +def test_int2str(): + def fn(i): + return str(i) + def wrapper(i): + res = fn(i) + return res == "42" + + f = compile_function(wrapper, [int]) + assert f(42) + +def test_int_invert(): + def fn(i): + return ~i + f = compile_function(fn, [int]) + for i in range(-15, 15): + assert f(i) == fn(i) + +def DONTtest_uint_invert(): #issue with Javascript Number() having a larger range + def fn(i): + inverted = ~i + inverted -= sys.maxint + return inverted + f = compile_function(fn, [r_uint]) + for value in range(1, 15): + i = r_uint(value) + assert str(f(i)) == str(fn(i)) + s = 0xfffffff + for value in range(s, s+1024, 64): + i = r_uint(value) + assert str(f(i)) == str(fn(i)) + +def test_int_abs(): + def int_abs_(n): + return abs(n) + f = compile_function(int_abs_, [int]) + for i in (-25, 0, 75): + assert f(i) == int_abs_(i) + +def test_float_abs(): + def float_abs_(n): + return abs(n) + f = compile_function(float_abs_, [float]) + for i in (-100.1 -50.2, -0.0, 0.0, 25.3, 50.4): + assert f(i) == float_abs_(i) + +def test_cast_to_int(): + def casting(v): + return int(ord(chr(v))) + f = compile_function(casting, [int]) + for ii in range(255): + assert f(ii) == ii + +def test_char_comparisons(): + #py.test.skip("chr/ord semantics") + def comps(v): + x = chr(v) + res = 0 + res += x < chr(0) + res += x > chr(1) + res += x >= chr(127) + res += x < chr(128) + res += x < chr(250) + return res + f = compile_function(comps, [int]) + for ii in range(255): + assert f(ii) == comps(ii) Added: pypy/branch/flex-backend/translator/flex/test/tgtest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/test/tgtest.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,42 @@ + +""" TurboGears browser testing utility +""" + +import thread + +import pkg_resources +pkg_resources.require("TurboGears") + +import cherrypy +import os +import sys +import webbrowser + +from pypy.translator.js.demo.jsdemo import controllers + +conf_file = os.path.join(os.path.dirname(controllers.__file__), "..", "dev.cfg") + +class run_tgtest(object): + def __init__(self, compiled_fun, tg_root = None, port = 8080, run_browser=True): + def cont(): + cherrypy.server.wait() + if run_browser: + webbrowser.open("http://localhost:%d/" % port) + cherrypy.root.wait_for_results() + self.results = cherrypy.root.results + cherrypy.server.stop() + cherrypy.server.interrupt = SystemExit() + + cherrypy.config.update(file=conf_file) + cherrypy.config.update({'global':{'server.socketPort':port}}) + + if tg_root is None: + cherrypy.root = controllers.Root() + else: + cherrypy.root = tg_root() + cherrypy.root.jssource = compiled_fun.js.tmpfile.open().read() + cherrypy.root.jsname = compiled_fun.js.translator.graphs[0].name + + thread.start_new_thread(cont, ()) + sys.path.insert(1, os.path.join(os.path.dirname(controllers.__file__), "..")) + cherrypy.server.start() Added: pypy/branch/flex-backend/translator/flex/tester.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/tester.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,34 @@ + +""" tester - support module for testing js code inside python + +Needs to be imported in case one wants tests involving calling BasicExternal +methods +""" + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal + +def __getattr__(self, attr): + val = super(BasicExternal, self).__getattribute__(attr) + if not callable(val) or attr not in self._methods: + return val # we don't do anything special + # otherwise.... + def wrapper(*args, **kwargs): + args = list(args) + # do this only if last arg is callable + if not (len(args) > 0 and callable(args[-1])): + return val(*args, **kwargs) + callback = args.pop() + res = val(*args, **kwargs) + if not hasattr(self, '__callbacks'): + self.__callbacks = [] + self.__callbacks.append((callback, res)) + wrapper.func_name = attr + return wrapper + +BasicExternal.__getattribute__ = __getattr__ + +def schedule_callbacks(*args): + for arg in args: + if hasattr(arg, '__callbacks'): + for callback, res in arg.__callbacks: + callback(res) Added: pypy/branch/flex-backend/translator/flex/tutorial/__init__.py ============================================================================== Added: pypy/branch/flex-backend/translator/flex/tutorial/step1.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/tutorial/step1.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,27 @@ +#!/usr/bin/env python +""" + +This is simple all-in-one self-containing server, +which just shows almost-empty HTML page + +""" + +# here we import server, which is derivative of +# BaseHTTPServer from python standard library +from pypy.translator.js.lib import server + +# We create handler, which will handle all our requests +class Handler(server.TestHandler): + + def index(self): + # provide some html contents + return "

Something

" + # this line is necessary to make server show something, + # otherwise method is considered private-only + index.exposed = True + +if __name__ == '__main__': + # let's start our server, + # this will create running server instance on port 8000 by default, + # which will run until we press Control-C + server.create_server(handler=Handler).serve_forever() Added: pypy/branch/flex-backend/translator/flex/tutorial/step2.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/translator/flex/tutorial/step2.py Sat Jul 14 11:58:08 2007 @@ -0,0 +1,52 @@ +#!/usr/bin/env python +""" +In this example, we'll show how to add javascript to our simple +server from previous example +""" + +from pypy.translator.js.lib import server +import sys + +# here we have virtual script "source.js" which we generate +# on-the-fly when requested +HTML = """ + + + + + + + + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/bnb/msgstruct.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/bnb/msgstruct.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,78 @@ +from struct import pack, unpack, calcsize + +try: + from localmsg import PORTS +except ImportError: + PORTS = {} +try: + from localmsg import HOSTNAME +except ImportError: + from socket import gethostname + HOSTNAME = gethostname() + + +MSG_WELCOME = "Welcome to gamesrv.py(3) !\n" +MSG_BROADCAST_PORT= "*" +MSG_DEF_PLAYFIELD = "p" +MSG_DEF_KEY = "k" +MSG_DEF_ICON = "r" +MSG_DEF_BITMAP = "m" +MSG_DEF_SAMPLE = "w" +MSG_DEF_MUSIC = "z" +MSG_PLAY_MUSIC = "Z" +MSG_FADEOUT = "f" +MSG_PLAYER_JOIN = "+" +MSG_PLAYER_KILL = "-" +MSG_PLAYER_ICON = "i" +MSG_PING = "g" +MSG_PONG = "G" +MSG_INLINE_FRAME = "\\" +MSG_PATCH_FILE = MSG_DEF_MUSIC +MSG_ZPATCH_FILE = "P" +MSG_MD5_FILE = "M" +MSG_RECORDED = "\x00" + +CMSG_PROTO_VERSION= "v" +CMSG_KEY = "k" +CMSG_ADD_PLAYER = "+" +CMSG_REMOVE_PLAYER= "-" +CMSG_UDP_PORT = "<" +CMSG_ENABLE_SOUND = "s" +CMSG_ENABLE_MUSIC = "m" +CMSG_PING = "g" +CMSG_PONG = "G" +CMSG_DATA_REQUEST = "M" +CMSG_PLAYER_NAME = "n" + +BROADCAST_MESSAGE = "game!" # less than 6 bytes + + +def message(tp, *values): + strtype = type('') + typecodes = [''] + for v in values: + if type(v) is strtype: + typecodes.append('%ds' % len(v)) + elif 0 <= v < 256: + typecodes.append('B') + else: + typecodes.append('l') + typecodes = ''.join(typecodes) + assert len(typecodes) < 256 + return pack(("!B%dsc" % len(typecodes)) + typecodes, + len(typecodes), typecodes, tp, *values) + +def decodemessage(data): + if data: + limit = ord(data[0]) + 1 + if len(data) >= limit: + typecodes = "!c" + data[1:limit] + try: + end = limit + calcsize(typecodes) + except TypeError: + return None, '' + if len(data) >= end: + return unpack(typecodes, data[limit:end]), data[end:] + elif end > 1000000: + raise OverflowError + return None, data Added: pypy/branch/flex-backend/pypy/translator/flex/examples/bnb/servermessage.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/bnb/servermessage.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,289 @@ +from msgstruct import * +from zlib import decompressobj, decompress +from urllib import quote +from os.path import exists +from struct import unpack +from time import time +import md5 + + +debug = True +def log(msg): + if debug: + print msg + + +class BitmapCreationException(Exception): + pass + + +#proxy messages +#PMSG_PING = "ping" #server wants to hear from client +#PMSG_PONG = "pong" #server responds to client's ping +PMSG_DEF_PLAYFIELD = "def_playfield" +PMSG_DEF_ICON = "def_icon" +PMSG_PLAYER_ICON = "player_icon" +PMSG_PLAYER_JOIN = "player_join" +PMSG_PLAYER_KILL = "player_kill" +PMSG_DEF_KEY = "def_key" +PMSG_INLINE_FRAME = "inline_frame" + + +# convert server messages to proxy messages in json format +class ServerMessage: + + _md5_file = {} + _bitmap2hexdigits={} + _def_icon_queue = {} + base_gfx_dir = 'data/images/' + base_gfx_url = '/images/' + gfx_extension = 'png' + + def __init__(self, base_gfx_dir = None): + if base_gfx_dir: + self.base_gfx_dir = base_gfx_dir + if not self.base_gfx_dir.endswith('/'): + self.base_gfx_dir += '/' + self.socket = None + self.data = '' + self.n_header_lines = 2 + self.gfx_dir = self.base_gfx_dir #gets overwritten depending on playfield FnDesc + self.gfx_url = self.base_gfx_url + self.decompressobj = decompressobj().decompress + self.last_active = time() + self._count = 0 + + def count(self): + self._count += 1 + return self._count + + def dispatch(self, *values): + #log('RECEIVED:%s(%d)' % (values[0], len(values[1:]))) + self.last_active = time() + fn = self.MESSAGES.get(values[0]) + if fn: + try: + return fn(self, *values[1:]) + except BitmapCreationException, e: + log(str(e)) + return dict() + else: + log("UNKNOWN:%s" % str(values)) + return dict(type='unknown', values=values) + + #server message handlers... + def ignore(self, *values): + #log('ignore %s' % str(values)) + return + + def def_playfield(self, width, height, backcolor, FnDesc): + #log('def_playfield width=%s, height=%s, backcolor=%s, FnDesc=%s' % (\ + # width, height, backcolor, FnDesc)) + self.gfx_dir = self.base_gfx_dir + self.gfx_url = self.base_gfx_url + return dict(type=PMSG_DEF_PLAYFIELD, width=width, height=height, + backcolor=backcolor, FnDesc=FnDesc) + + def def_bitmap(self, bitmap_code, data_or_fileid, *rest): + if type(data_or_fileid) is type(0): + fn = self.def_bitmap2 + else: + fn = self.def_bitmap1 + return fn(bitmap_code, data_or_fileid, *rest) + + def def_bitmap1(self, bitmap_code, data, *rest): + import PIL.Image + if len(rest) == 0: + colorkey = None + else: + c = rest[0] + colorkey = (c & 255, (c >> 8) & 255, (c >> 16) & 255) + #log('def_bitmap1 bitmap_code=%d, data=%d bytes, colorkey=%s' % ( + # bitmap_code, len(data), colorkey)) + + try: + decompressed_data = decompress(data) + except Exception, e: + raise BitmapCreationException('ERROR UNCOMPRESSING DATA FOR %s (%s)' % ( + bitmap_filename, str(e))) + hexdigits = md5.new(decompressed_data).hexdigest() + self._bitmap2hexdigits[bitmap_code] = hexdigits + + gfx_bitmap_filename = '%s%s.%s' % (self.gfx_dir, hexdigits, self.gfx_extension) + if exists(gfx_bitmap_filename): + #log('CACHED:%s' % gfx_bitmap_filename) + pass + else: + bitmap_filename = '%s%s.ppm' % (self.gfx_dir, hexdigits) + f = open(bitmap_filename, 'wb') + f.write(decompressed_data) + f.close() + #TODO: use in memory (don't save ppm first) + try: + bitmap = PIL.Image.open(bitmap_filename) + except IOError, e: + raise BitmapCreationException('ERROR LOADING %s (%s)' % ( + bitmap_filename, str(e))) + + #create alpha layer (PIL export this correctly with png but not with gif) + if colorkey is not None: + bitmap = bitmap.convert("RGBA") + data = bitmap.getdata() + c = (colorkey[0], colorkey[1], colorkey[2], 255) + width, height = bitmap.size + for y in range(height): #this is slowish but gfx are cached, so... + for x in range(width): + p = data.getpixel((x,y)) + if p == c: + data.putpixel((x,y), (0,0,0,0)) + + try: + bitmap.save(gfx_bitmap_filename) + log('SAVED:%s' % gfx_bitmap_filename) + except IOError: + raise BitmapCreationException('ERROR SAVING %s (%s)' % ( + gfx_bitmap_filename, str(e))) + + def def_bitmap2(self, bitmap_code, fileid, *rest): + #log('def_bitmap2: bitmap_code=%d, fileid=%d, colorkey=%s' % (bitmap_code, fileid, rest)) + hexdigits = self._md5_file[fileid]['hexdigits'] + self._bitmap2hexdigits[bitmap_code] = hexdigits + gfx_bitmap_filename = '%s%s.%s' % (self.gfx_dir, hexdigits, self.gfx_extension) + if exists(gfx_bitmap_filename): + #log('SKIP DATA_REQUEST:%s' % gfx_bitmap_filename) + pass + else: + self._md5_file[fileid]['bitmap_code'] = bitmap_code + self._md5_file[fileid]['colorkey'] = rest + position = self._md5_file[fileid]['offset'] + size = self._md5_file[fileid]['len_data'] + msg = message(CMSG_DATA_REQUEST, fileid, position, size) + self.socket.send(msg) + log('DATA_REQUEST:fileid=%d(pos=%d,size=%d):%s' % ( + fileid, position, size, gfx_bitmap_filename)) + + def def_icon(self, bitmap_code, icon_code, x,y,w,h, *rest): + import PIL.Image + + #log('def_icon bitmap_code=%s, icon_code=%s, x=%s, y=%s, w=%s, h=%s, alpha=%s' %\ + # (bitmap_code, icon_code, x,y,w,h, rest) #ignore alpha (bubbles) + + hexdigits = self._bitmap2hexdigits[bitmap_code] + bitmap_filename = '%s%s.%s' % (self.gfx_dir, hexdigits, self.gfx_extension) + icon_filename = '%s%s_%d_%d_%d_%d.%s' % ( + self.gfx_dir, hexdigits, x, y, w, h, self.gfx_extension) + if exists(icon_filename): + #log('CACHED:%s' % icon_filename) + pass + elif exists(bitmap_filename): + #TODO: use in memory (don't save ppm first) + icon = PIL.Image.open(bitmap_filename) + box = (x, y, x+w, y+h) + region = icon.crop(box) + region.save(icon_filename) + log('SAVED:%s' % icon_filename) + else: #bitmap is not available yet (protocol 2) + #log('%s NOT AVAILABLE FOR %s' % (bitmap_filename, icon_filename)) + if bitmap_code not in self._def_icon_queue: + self._def_icon_queue[bitmap_code] = [] + self._def_icon_queue[bitmap_code].append((icon_code, x, y, w, h, rest)) + return + + filename = '%s%s_%d_%d_%d_%d.%s' % ( + self.gfx_url, hexdigits, x, y, w, h, self.gfx_extension) + return dict(type=PMSG_DEF_ICON, icon_code=icon_code, filename=filename, width=w, height=h) + + def zpatch_file(self, fileid, position, data): #response to CMSG_DATA_REQUEST + #log('zpatch_file fileid=%d, position=%d, len(data)=%d' % (fileid, position, len(data))) + + bitmap_code = self._md5_file[fileid]['bitmap_code'] + colorkey = self._md5_file[fileid]['colorkey'] + try: + t = self.def_bitmap(bitmap_code, data, *colorkey) + except BitmapCreationException, e: + log(str(e)) + return #i.e. not attempting to create icons + messages = [] + if bitmap_code in self._def_icon_queue: + #log('%d icons queued for bitmap %d' % ( + # len(self._def_icon_queue[bitmap_code]), bitmap_code)) + for t in self._def_icon_queue[bitmap_code]: + icon_code, x, y, w, h, rest = t + messages.append(self.def_icon(bitmap_code, icon_code, x, y, w, h, *rest)) + del self._def_icon_queue[bitmap_code] + return messages + + def player_icon(self, player_id, icon_code): + #log('player_icon player_id=%d, icon_code=%d' % (player_id, icon_code)) + return dict(type=PMSG_PLAYER_ICON, player_id=player_id, icon_code=icon_code) + + def player_join(self, player_id, client_is_self): + #log('player_join player_id=%d, client_is_self=%d' % (player_id, client_is_self)) + return dict(type=PMSG_PLAYER_JOIN, player_id=player_id, client_is_self=client_is_self) + + def player_kill(self, player_id): + #log('player_kill player_id=%d' % player_id) + return dict(type=PMSG_PLAYER_KILL, player_id=player_id) + + def def_key(self, keyname, num, *icon_codes): + #log('def_key keyname=%s, num=%d, icon_codes=%s' % (keyname, num, str(icon_codes))) + return dict(type=PMSG_DEF_KEY, keyname=keyname, num=num, icon_codes=icon_codes) + + def md5_file(self, fileid, protofilepath, offset, len_data, checksum): + hd = '0123456789abcdef' + hexdigits = '' + for c in checksum: + i = ord(c) + hexdigits = hexdigits + hd[i >> 4] + hd[i & 15] + #log('md5_file fileid=%d, protofilepath=%s, offset=%d, len_data=%d, hexdigits=%s' % ( + # fileid, protofilepath, offset, len_data, hexdigits)) + self._md5_file[fileid] = { + 'protofilepath' : protofilepath, + 'offset' : offset, + 'len_data' : len_data, + 'checksum' : checksum, + 'hexdigits' : hexdigits, + } + + def inline_frame(self, data): + decompressed_data = d = self.decompressobj(data) + #log('inline_frame len(data)=%d, len(decompressed_data)=%d' % ( + # len(data), len(d))) + + return_raw_data = False + if return_raw_data: + return dict(type=PMSG_INLINE_FRAME, data=decompressed_data) + + #note: we are not returning the raw data here but we could let the Javascript + # handle it. If this is working we can convert this to RPython and move it + # to the client instead. (based on BnB update_sprites in display/pclient.py) + sounds, sprites = [], [] + + base = 0 + while d[base+4:base+6] == '\xFF\xFF': + key, lvol, rvol = struct.unpack("!hBB", udpdata[base:base+4]) + sounds.append((key, lvol, rvol)) + base += 6 + + for j in range(base, len(d)-5, 6): + info = d[j:j+6] + x, y, icon_code = unpack("!hhh", info[:6]) + sprites.append((icon_code, x, y)) + + return dict(type=PMSG_INLINE_FRAME, sounds=sounds, sprites=sprites) + + MESSAGES = { + MSG_BROADCAST_PORT : ignore, + MSG_PING : ignore, + MSG_PONG : ignore, + MSG_DEF_PLAYFIELD : def_playfield, + MSG_DEF_BITMAP : def_bitmap, + MSG_ZPATCH_FILE : zpatch_file, + MSG_DEF_ICON : def_icon, + MSG_PLAYER_ICON : player_icon, + MSG_PLAYER_JOIN : player_join, + MSG_PLAYER_KILL : player_kill, + MSG_DEF_KEY : def_key, + MSG_MD5_FILE : md5_file, + MSG_INLINE_FRAME : inline_frame, + } Added: pypy/branch/flex-backend/pypy/translator/flex/examples/bnb/start_bnb.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/bnb/start_bnb.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,283 @@ +#!/usr/bin/env python +""" bub-n-bros testing utility +""" + +import autopath + +import py + +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js.modules.dom import document, window +from pypy.translator.js.modules.mochikit import log, logWarning,\ + createLoggingPane, logDebug, connect +from pypy.translator.js.examples.bnb.bnb import exported_methods +from pypy.translator.js import commproxy + +commproxy.USE_MOCHIKIT = True + +import time +import os +import sys + +def logKey(msg): + pass + +class Stats(object): + """ Class containing some statistics + """ + def __init__(self): + self.n_received_inline_frames = 0 + self.n_rendered_inline_frames = 0 + self.n_rendered_dynamic_sprites = 0 + self.fps = 0 + self.starttime = 0.0 + self.n_sprites = 0 + + def register_frame(self): + self.n_rendered_inline_frames += 1 + if self.n_rendered_inline_frames >= 10: + next_time = time.time() + self.fps = 10000/(next_time - self.starttime) + self.n_rendered_inline_frames = 0 + self.starttime = next_time + +stats = Stats() + +class Player(object): + def __init__(self): + self.id = -1 + self.prev_count = 0 + self.sessionid = "" + +player = Player() + +class SpriteManager(object): + def __init__(self): + self.sprites = {} + self.filenames = {} + self.all_sprites = {} + self.frames = [] + + def add_icon(self, icon_code, filename): + self.filenames[icon_code] = filename + #self.sprite_queues[icon_code] = [] + #self.used[icon_code] = [] + # FIXME: Need to write down DictIterator once... + #self.icon_codes.append(icon_code) + + def add_sprite(self, s, icon_code, x, y): + #try: + # img = self.sprite_queues[icon_code].pop() + #except IndexError: + stats.n_sprites += 1 + img = document.createElement("img") + img.src = self.filenames[icon_code] + img.style.position = 'absolute' + img.style.left = x + 'px' + img.style.top = y + 'px' + img.style.visibility = 'visible' + document.getElementById("playfield").appendChild(img) + try: + self.sprites[s].style.visibility = "hidden" + # FIXME: We should delete it + except KeyError: + self.sprites[s] = img + return img + + def move_sprite(self, s, x, y): + i = self.sprites[s] + i.style.top = y + 'px' + i.style.left = x + 'px' + i.style.visibility = 'visible' + + def hide_sprite(self, s): + i = self.sprites[s] + i.style.visibility = "hidden" + + def start_clean_sprites(self): + self.all_sprites = {} + + def show_sprite(self, s, icon_code, x, y): + self.all_sprites[s] = 1 + try: + self.move_sprite(s, x, y) + except KeyError: + self.add_sprite(s, icon_code, x, y) + + def end_clean_sprites(self): + for i in self.sprites: + try: + self.all_sprites[i] + except KeyError: + self.hide_sprite(i) + + def set_z_index(self, s_num, z): + self.sprites[s_num].style.zIndex = z + + #def show_sprite(self, s): + # i = self.sprites[s] + # i.style.visibility = "visible" + +sm = SpriteManager() + +class KeyManager(object): + def __init__(self): + self.keymappings = {ord('D'):'right', ord('S'):'fire', ord('A'):'left', ord('W'):'up'} + self.key_to_bnb_down = {'right':0, 'left':1, 'fire':3, 'up':2} + self.key_to_bnb_up = {'right':4, 'left':5, 'fire':7, 'up':6} + self.queue = [] + + def add_key_up(self, key): + self.queue.append(self.key_to_bnb_up[key]) + + def add_key_down(self, key): + self.queue.append(self.key_to_bnb_down[key]) + + def get_keys(self): + retval = self.queue + self.queue = [] + return retval + +km = KeyManager() + +def appendPlayfield(msg): + body = document.getElementsByTagName('body')[0] + bgcolor = '#000' + body.style.backgroundColor = bgcolor + div = document.createElement("div") + div.id = 'playfield' + div.style.width = msg['width'] + div.style.height = msg['height'] + div.style.position = 'absolute' + div.style.top = '0px' + div.style.left = '0px' + div.appendChild(document.createTextNode('foobar?')) + + #document.body.childNodes.insert(0, div) + body.appendChild(div) + +def appendPlayfieldXXX(): + bgcolor = '#000000' + document.body.setAttribute('bgcolor', bgcolor) + div = document.createElement("div") + div.id = 'playfield' + div.style.width = 500 + div.style.height = 250 + div.style.position = 'absolute' + div.style.top = '0px' + div.style.left = '0px' + document.body.appendChild(div) + +def process_message(msg): + if msg['type'] == 'def_playfield': + appendPlayfield(msg) + elif msg['type'] == 'def_icon': + sm.add_icon(msg['icon_code'], msg['filename']) + elif msg['type'] == 'ns': + sm.add_sprite(msg['s'], msg['icon_code'], msg['x'], msg['y']) + sm.set_z_index(msg['s'], msg['z']) + elif msg['type'] == 'sm': + sm.move_sprite(msg['s'], msg['x'], msg['y']) + sm.set_z_index(msg['s'], msg['z']) + elif msg['type'] == 'ds': + sm.hide_sprite(msg['s']) + elif msg['type'] == 'begin_clean_sprites': + sm.start_clean_sprites() + elif msg['type'] == 'clean_sprites': + sm.end_clean_sprites() + elif msg['type'] == 'show_sprite': + sm.show_sprite(msg['s'], msg['icon_code'], msg['x'], msg['y']) + elif msg['type'] == 'zindex': + sm.set_z_index(msg['s'], msg['z']) + #elif msg['type'] == 'ss': + # sm.show_sprite(msg['s']) + elif msg['type'] == 'player_icon' or msg['type'] == 'def_key' or \ + msg['type'] == 'player_join' or msg['type'] == 'player_kill': + pass #ignore + else: + logWarning('unknown message type: ' + msg['type']) + + +def ignore(arg): + pass +ignore._annspecialcase_ = 'specialize:argtype(0)' + +def addPlayer(player_id): + name = "player no. " + str(player_id) + #name = "player no. %d" % player_id + #File "/Users/eric/projects/pypy-dist/pypy/translator/js/jts.py", line 52, in lltype_to_cts + # raise NotImplementedError("Type %r" % (t,)) + # NotImplementedError: Type + prev_player_id = player.id + if player.id >= 0: + exported_methods.remove_player(player.id, player.sessionid, ignore) + player.id = -1 + if player_id != prev_player_id: + exported_methods.player_name(player_id, name, player.sessionid, ignore) + exported_methods.add_player(player_id, player.sessionid, ignore) + player.id = player_id + + +def keydown(key): + #c = chr(int(key.keyCode)).lower() + #c = int(key.keyCode) + key = key._event + try: + c = key.keyCode + if c > ord('0') and c < ord('9'): + addPlayer(int(chr(c))) + #for i in km.keymappings: + # log(str(i)) + if c in km.keymappings: + km.add_key_down(km.keymappings[c]) + #else: + except Exception, e: + log(str(e)) + +def keyup(key): + key = key._event + c = key.keyCode + if c > ord('0') and c < ord('9'): + pass #don't print warning + elif c in km.keymappings: + km.add_key_up(km.keymappings[c]) + else: + logWarning('unknown keyup: ' + str(c)) + +#def ignore_dispatcher(msgs): +# pass + +def bnb_dispatcher(msgs): + s = ":".join([str(i) for i in km.get_keys()]) + exported_methods.get_message(player.sessionid, player.id, s, + bnb_dispatcher) + render_frame(msgs) + +def render_frame(msgs): + for msg in msgs['messages']: + process_message(msg) + stats.register_frame() + document.title = str(stats.n_sprites) + " sprites " + str(stats.fps) + +def session_dispatcher(sessionid): + player.sessionid = sessionid + connect(document, 'onkeydown', keydown) + connect(document, 'onkeyup', keyup) + exported_methods.get_message(player.sessionid, player.id, "", + bnb_dispatcher) + +def bnb(): + createLoggingPane(True) + log("keys: [0-9] to select player, [wsad] to walk around") + exported_methods.initialize_session(session_dispatcher) + +def run_bnb(): + from pypy.translator.js.examples.bnb.bnb import BnbRoot + from pypy.translator.js.lib import server + addr = ('', 7070) + httpd = server.create_server(handler=BnbRoot, server_address=addr) + httpd.source = rpython2javascript(sys.modules[__name__], ['bnb']) + httpd.serve_forever() + +if __name__ == '__main__': + run_bnb() Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/client.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,154 @@ + +from pypy.translator.js.modules import dom +from pypy.translator.js.modules.mochikit import log, createLoggingPane +from pypy.translator.js.examples.console.console import exported_methods + +class Glob(object): + def __init__(self): + self.console_running = False + self.next_console = "" + self.text_to_show = [] + self.pss = [] + +glob = Glob() + +def add_text_to_dom(txt): + data_elem = dom.document.getElementById("data") + if data_elem.childNodes: + data = data_elem.childNodes[0].nodeValue + txt + else: + data = txt + while data_elem.childNodes: + data_elem.removeChild(data_elem.childNodes[0]) + data_elem.appendChild(dom.document.createTextNode(data)) + + +def add_text(txt, server_flag, fn=add_text_to_dom): + if not server_flag: + if txt.find("\n") != len(txt) - 1: + if txt[-1] == '\n': + txt = txt[:-1] + lst = txt.split("\n") + add_text_to_dom(lst[0] + "\n") + glob.text_to_show += lst[1:] + else: + add_text_to_dom(txt) + else: + for ps in glob.pss: + if glob.text_to_show: + num = txt.find(ps) + if txt.startswith(ps): + txt = txt[len(ps):] + add_text_to_dom(ps + glob.text_to_show.pop(0) + "\n") + add_text(txt, True) + return + if txt.startswith("\n" + ps): + txt = txt[len(ps) + 1:] + add_text_to_dom(ps + glob.text_to_show.pop(0) + "\n") + add_text(txt, True) + return + add_text_to_dom(txt) + +def create_text(txt): + return dom.document.createTextNode(txt) + +def set_text(txt): + data_elem = dom.document.getElementById("data") + while data_elem.childNodes: + data_elem.removeChild(data_elem.childNodes[0]) + data_elem.appendChild(dom.document.createTextNode(txt)) + +def refresh_console(msg): + inp_elem = dom.document.getElementById("inp") + #inp_elem.disabled = False + if msg[0] == "refresh": + data = msg[1] + if data: + inp_elem.scrollIntoView() + inp_elem.focus() + exported_methods.refresh_empty(glob.sess_id, refresh_console) + add_text(data, True) + elif msg[0] == 'disconnected': + inp_elem.disabled = True + name_bar = dom.document.getElementById("namebar") + name_bar.style.color = "red" + text = name_bar.lastChild.nodeValue + name_bar.removeChild(name_bar.lastChild) + name_bar.appendChild(create_text(text + " [DEFUNCT]")) + glob.console_running = False + if glob.next_console: + next = glob.next_console + glob.next_console = "" + load_console(next) + +def set_sessid(data): + sessid = int(data[0]) + help_msg = data[1] + glob.pss = data[2:] + glob.sess_id = sessid + inp_elem = dom.document.getElementById("inp") + inp_elem.disabled = False + name_bar = dom.document.getElementById("namebar") + name_bar.style.color = "black" + name_bar.removeChild(name_bar.lastChild) + name_bar.appendChild(create_text("Python console")) + dom.document.getElementById("helpcontents").innerHTML = help_msg + exported_methods.refresh_empty(sessid, refresh_console) + +def empty_callback(msg): + inp_elem = dom.document.getElementById("inp") + #inp_elem.disabled = False + inp_elem.scrollIntoView() + +def keypressed(key): + kc = key.keyCode + if kc == ord("\r"): + inp_elem = dom.document.getElementById("inp") + cmd = inp_elem.value + inp_elem.value = '' + add_text(cmd + "\n", False) + #if not cmd: + # exported_methods.refresh(glob.sess_id, cmd, empty_callback) + #else: + exported_methods.refresh(glob.sess_id, cmd + "\n", refresh_console) + +def nothing(msg): + pass + +def nothing2(msg): + pass + +def cleanup_console(): + inp_elem = dom.document.getElementById("inp") + inp_elem.disabled = True + set_text("") + glob.text_to_show = [] # better safe than sorry + exported_methods.kill_console(glob.sess_id, nothing2) + +def load_console(python="python"): + if glob.console_running: + cleanup_console() + glob.next_console = python + return + inp_elem = dom.document.getElementById("inp") + main = dom.document.getElementById("main") + main.style.visibility = "visible" + inp_elem.disabled = False + inp_elem.focus() + glob.console_running = True + exported_methods.get_console(python, set_sessid) + +def add_snippet(snippet): + add_text(snippet, False) + exported_methods.refresh(glob.sess_id, snippet, refresh_console) + +def execute_snippet(name='python', number=3): + exported_methods.execute_snippet(name, number, add_snippet) + +def console_onload(): + #createLoggingPane(True) + #inp_elem = dom.document.getElementById("inp") + #inp_elem.focus() + dom.document.onkeypress = keypressed + #exported_methods.get_console("python", set_sessid) + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/console.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/console.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,155 @@ +import py + +from pypy.translator.js.lib import server +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js.lib.support import callback +from pypy.translator.js import commproxy +from pypy.translator.js.examples.console.session import Interpreter, Killed +from pypy.translator.js.examples.console.docloader import DocLoader +from py.__.green.server.httpserver import GreenHTTPServer +from py.__.green.greensock2 import ConnexionClosed + +commproxy.USE_MOCHIKIT = True + +FUNCTION_LIST = ["load_console", "console_onload", "execute_snippet"] + +class Ignore(Exception): + pass + +def js_source(): + import client + return rpython2javascript(client, FUNCTION_LIST) + +def line_split(ret, max_len): + to_ret = [] + for line in ret.split("\n"): + if len(line) > max_len: + to_ret += [line[i*max_len:(i+1)*max_len] for i in + range(len(line)/max_len)] + i = len(line)/max_len + else: + i = 0 + to_ret.append(line[i*max_len:]) + return "\n".join(to_ret) + + +STATIC_DIR = py.path.local(__file__) +for x in range(6): + STATIC_DIR = STATIC_DIR.dirpath() +STATIC_DIR = STATIC_DIR.join("compiled") +DOCDIR = STATIC_DIR.dirpath().join("pypy", "doc", "play1") +CONSOLES = ['python', 'pypy-c', 'pypy-c-thunk', 'pypy-c-taint', 'pypy-cli', 'pyrolog-c', 'pypy-c-jit'] + +class Sessions(object): + def __init__(self): + self.sessions = {} + self.updating = {} + testfile = py.path.local(__file__).dirpath().join("play1_snippets.py") + self.docloader = DocLoader(docdir=DOCDIR, consoles=CONSOLES, + testfile=testfile) + + def new_session(self, python="python"): + if not py.path.local().sysfind(python): + python = str(STATIC_DIR.join(python)) + ip = Interpreter(python) + self.sessions[ip.pid] = ip + self.updating[ip.pid] = False + return ip.pid + + def update_session(self, pid, to_write=None): + ip = self.sessions[pid] + if self.updating[pid]: + ip.write_only(to_write) + raise Ignore() + self.updating[pid] = True + ret = ip.interact(to_write) + self.updating[pid] = False + if not ret: + return "" + MAX_LEN = 80 + return line_split(ret, MAX_LEN) + + def kill_session(self, pid): + ip = self.sessions[pid] + ip.pipe.stdin.close() + del self.sessions[pid] + del self.updating[pid] + + def get_ps(self, python): + if python == 'python': + return ['>>> ', '... '] + if python.startswith('pypy'): + return ['>>>> ', '.... '] + if python == 'pyrolog-c': + return ['>?-'] + return [] + +# We hack here, cause in exposed methods we don't have global 'server' +# state +sessions = Sessions() + +class ExportedMethods(server.ExportedMethods): + @callback(retval=[str]) + def get_console(self, python="python"): + retval = sessions.new_session(python) + return [str(retval), sessions.docloader.get_html(python)] +\ + sessions.get_ps(python) + + def _refresh(self, pid, to_write): + try: + return ["refresh", sessions.update_session(int(pid), to_write)] + except (KeyError, IOError, Killed, ConnexionClosed): + return ["disconnected"] + except Ignore: + return ["ignore"] + + @callback(retval=[str]) + def refresh(self, pid=0, to_write=""): + return self._refresh(pid, to_write) + + @callback(retval=[str]) + def refresh_empty(self, pid=0): + #print "Empty refresh %d" % int(pid) + return self._refresh(pid, None) + + @callback(retval=str) + def execute_snippet(self, name='aaa', number=3): + return sessions.docloader.get_snippet(name, int(number)) + "\n" + + @callback() + def kill_console(self, pid=0): + try: + sessions.kill_session(int(pid)) + except KeyError: + pass + +exported_methods = ExportedMethods() + +static_dir = py.path.local(__file__).dirpath().join("data") + +class Root(server.Collection): + exported_methods = exported_methods + index = server.FsFile(static_dir.join("console.html")) + style_css = server.FsFile(static_dir.dirpath().dirpath().join("data"). + join("style.css")) + MochiKit = server.StaticDir('MochiKit') + + def source_js(self): + if hasattr(self.server, 'source_console'): + source = self.server.source_console + else: + source = js_source() + self.server.source_console = source + return "text/javascript", source + source_js.exposed = True + +class Handler(server.NewHandler): + application = Root() + application.some = Root() + application.other = Root() + +if __name__ == '__main__': + addr = ('', 8007) + httpd = server.create_server(server_address=addr, handler=Handler, + server=GreenHTTPServer) + httpd.serve_forever() Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/data/console.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/data/console.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,43 @@ + + + + + + + Console + + + PyPy + + + + + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/docloader.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/docloader.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,43 @@ + +""" Simple module for loading documentation of various +pypy-cs from doc directory +""" + +import py + +class DocLoader(object): + def __init__(self, consoles, docdir, testfile): + self.consoles = consoles + self.docdir = py.path.local(docdir) + assert self.docdir.check(dir=1) + self.testfile = testfile + assert self.testfile.check() + self.htmls = {} + self.snippets = {} + self.load() + + def get_html(self, console): + return self.htmls[console] + + def get_snippet(self, console, num): + return str(self.snippets[console][num]) + + def load(self): + def mangle_name(name): + return name.replace("-", "_").replace(".", "_") + + def mangle(source): + source = source.strip() + del source.lines[0] + return source.deindent() + + testmod = self.testfile.pyimport() + for console in self.consoles: + html = self.docdir.join(console + '.html').read() + snip_class = getattr(testmod, 'AppTest_' + mangle_name(console)) + snippets = [mangle(py.code.Source(getattr(snip_class, name))) + for name in + dir(snip_class) if name.startswith("test_snippet")] + self.snippets[console] = snippets + self.htmls[console] = html % tuple([str(i) for i in snippets]) + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/ideas.txt ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/ideas.txt Sat Jul 14 12:31:00 2007 @@ -0,0 +1,11 @@ +* Thunk objspace - lazy evaluation in demo/fibonacci2 or sth + +* Taint objspace - examples in the documentation + +* Transparent proxy + +* Stackless - examples in a docs of stackless/WP7 + +* __pypy__.internal_repr - --faassen, ropes + +* changing grammar at the runtime Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/play1_snippets.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/play1_snippets.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,82 @@ +from pypy.conftest import gettestobjspace + +class AppTest_pypy_c(object): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withtproxy": True, + "usemodules":("_stackless",)}) + + def test_snippet_1(self): + from tputil import make_proxy + history = [] + def recorder(operation): + history.append(operation) + return operation.delegate() + + l = make_proxy(recorder, obj=[]) + +class AppTest_pypy_c_thunk(object): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.name": 'thunk'}) + + def test_snippet_1(self): + from __pypy__ import thunk + def f(): + print 'computing...' + return 6*7 + + x = thunk(f) + +class AppTest_pypy_c_taint(object): + def setup_class(cls): + cls.space = gettestobjspace(**{'objspace.name': 'taint'}) + + def test_snippet_1(self): + from __pypy__ import taint + x = taint(6) + +class AppTest_pypy_cli(object): + def setup_class(cls): + cls.space = gettestobjspace(**{'usemodules': 'clr'}) + + def test_snippet_1(self): + import clr + ArrayList = clr.load_cli_class('System.Collections', 'ArrayList') + obj = ArrayList() + obj.Add(1) + +class AppTest_pyrolog_c(object): + pass + +class AppTest_python(object): + pass + +class AppTest_pypy_c_jit(object): + def setup_class(cls): + cls.space = gettestobjspace(**{'usemodules':('pypyjit',)}) + + def test_snippet_1(self): + import time + + def f1(n): + "Arbitrary test function." + i = 0 + x = 1 + while i%s") + code = tmpdir.ensure("test_snippets2.py").write(str(py.code.Source(''' + class AppTest_one(object): + def test_snippet_1(self): + x = 1 + ''')) + '\n') + ld = DocLoader(docdir=tmpdir, consoles=['one'], testfile=tmpdir.join('test_snippets2.py')) + assert ld.get_html('one') == "x = 1" + assert ld.get_snippet('one', 0) == 'x = 1' Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/test/test_session.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/test/test_session.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,71 @@ + +""" Various tests around interpreter in a subprocess +""" + +import py + +from py.__.green.greensock2 import allof, sleep +from py.__.green.pipe.fd import FDInput +from pypy.translator.js.examples.console.session import Interpreter, Killed + +import sys +if sys.platform == 'nt': + py.test.skip("Those tests doesn't run on windows (yet)") + +def test_greensock_reader_timeouter(): + i = Interpreter("python", timeout=3) + while not i.interact().endswith(">>> "): + pass + assert i.interact() is None + assert i.interact("a\n") is not None + +def test_two_interpreters(): + i = Interpreter("python", timeout=3) + i2 = Interpreter("python", timeout=3) + while not i.interact().endswith(">>> "): + pass + while not i2.interact().endswith(">>> "): + pass + + def f(): + return i.interact("import time;time.sleep(1)\n") + def g(): + return i2.interact("a\n") + + one, two = allof(g, f) + assert two.startswith(">>") + assert one.startswith("Traceback") + +def test_multiline_command(): + i = Interpreter("python", timeout=3) + while not i.interact().endswith(">>> "): + pass + val = i.interact("def f(x):\n y = x + 3\n return y\n\nf(8)\n") + while val is not None: + assert 'Traceback' not in val + assert 'Syntax' not in val + print val + val = i.interact() + +def test_kill_timeout(): + py.test.skip("XXX") + i = Interpreter("python", kill_timeout=1, timeout=3) + while not i.interact().endswith(">>> "): + pass + i.interact() + +def test_kill_timeout_outside(): + py.test.skip("XXX") + i = Interpreter("python", kill_timeout=1, timeout=3) + while not i.interact().endswith(">>> "): + pass + sleep(8) + i.interact() + +def test_does_not_die(): + i = Interpreter("python", kill_timeout=1, timeout=3) + while not i.interact().endswith(">>> "): + pass + i.interact("import time\n") + for _ in range(6): + i.interact("time.sleep(.2)\n") Added: pypy/branch/flex-backend/pypy/translator/flex/examples/console/test/test_snippets.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/console/test/test_snippets.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,3 @@ +from pypy.translator.js.examples.console.play1_snippets import * + +del AppTest_pypy_cli Added: pypy/branch/flex-backend/pypy/translator/flex/examples/data/error.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/data/error.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,19 @@ + + + %(message)s + + + + + PyPy + +
+

%(message)s

+

%(explain)s

+
+ Added: pypy/branch/flex-backend/pypy/translator/flex/examples/data/index.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/data/index.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,39 @@ + + + pypy.js various demos + + + + + PyPy + + + +
+

This site presents various demos created with + PyPy's compiler toolchain + to generate JavaScript code out of + RPython.

+ +

It also allows you to play with PyPy's unique features on consoles + presenting a PyPy standard interpreter translated to C or + CLI (.NET, running Mono), forwarded from the server. + On each console there is + a little help box which explains the features. +

+ +

The demos should work under Firefox, Mozilla and Opera. Support for + other browsers is uncertain. The Bub'n'Bros demo is only known to work + with Firefox so far.

+ +

Note: these demos are set up on a replicable Xen instance, but + please do not vandalise the machine, otherwise we will be forced to + take it down.

+
+ + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/data/launcher.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/data/launcher.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,9 @@ + + + Python console + + + +

Console Launching, please wait...

+ + \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/examples/data/py-web1.png ============================================================================== Binary file. No diff available. Added: pypy/branch/flex-backend/pypy/translator/flex/examples/data/style.css ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/data/style.css Sat Jul 14 12:31:00 2007 @@ -0,0 +1,56 @@ +body,body.editor,body.body { + font: 90% "Times New Roman", Arial, Verdana, Helvetica, serif; + background: White; + color: Black; +} + +div#main { + position: absolute; + top: 130px; + left: 170px; + margin-right: 5px; +} + +div#contents { + float: left; + width: 160px; +} + +#inp { + font: 90% fixed; +} + +#help { + background: #EEE; +} + +.menubar { + font-weight: bold; + float: left; + font: 120% Verdana, Helvetica, Arial, sans-serif; + text-decoration: none; + padding-right: 0.3em; + +} + +.title{ + /*border-spacing: 20px;*/ + font: 160% Verdana, Helvetica, Arial, sans-serif; + color: #3ba6ec; + vertical-align: center; + padding-bottom: 0.3em; +} + +div#metaspace { + position: absolute; + top: 20px; + left: 170px; +} + +.main { +} + +.error { + color: red; + font-weight: bold; +} Added: pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/client.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,51 @@ +"""rpython javascript code""" + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.translator.js.lib.support import callback + +from pypy.translator.js.modules import mochikit +from pypy.translator.js.modules import dom + +class PingHandler(BasicExternal): + """Server side code which handles javascript calls""" + _render_xmlhttp = True + + @callback(retval={str:str}) + def ping(self, ping_str="aa"): + """Simply returns the string prefixed with a PONG""" + return dict(response="PONG: %s" % ping_str) + +ping_handler = PingHandler() + +def jsping(): + mochikit.logDebug("pinging") + ping_handler.ping("PING", callback) + +def callback(data): + mochikit.logDebug("Got response: " + data["response"]) + log = dom.document.getElementById("log") + mochikit.logDebug("got log element") + try: + s = "

" + data["response"] + "

" + except KeyError: + mochikit.logDebug("Can't find data") + s = "

" + "Error" + "

" + mochikit.logDebug("Adding: " + s) + log.innerHTML += s + mochikit.logDebug("added message") + +def doping_onclick(event): + mochikit.logDebug("calling pinger") + jsping() + +def ping_init(): + mochikit.createLoggingPane(True) + button = dom.document.getElementById("doping") + button.onclick = doping_onclick + mochikit.logDebug("Ping button setup") + +if __name__ == "__main__": + # circular import + from pypy.translator.js.examples.djangoping import client + from pypy.translator.js.main import rpython2javascript + print rpython2javascript(client, ["ping_init"]) Added: pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/manage.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/manage.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,11 @@ +#!/usr/bin/env python +from django.core.management import execute_manager +try: + import settings # Assumed to be in the same directory. +except ImportError: + import sys + sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__) + sys.exit(1) + +if __name__ == "__main__": + execute_manager(settings) Added: pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/settings.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/settings.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,76 @@ +# Django settings for djangoping project. + +DEBUG = True +TEMPLATE_DEBUG = DEBUG + +ADMINS = ( + # ('Your Name', 'your_email at domain.com'), +) + +MANAGERS = ADMINS + +DATABASE_ENGINE = '' # 'postgresql', 'mysql', 'sqlite3' or 'ado_mssql'. +DATABASE_NAME = '' # Or path to database file if using sqlite3. +DATABASE_USER = '' # Not used with sqlite3. +DATABASE_PASSWORD = '' # Not used with sqlite3. +DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3. +DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3. + +# Local time zone for this installation. All choices can be found here: +# http://www.postgresql.org/docs/current/static/datetime-keywords.html#DATETIME-TIMEZONE-SET-TABLE +TIME_ZONE = 'America/Chicago' + +# Language code for this installation. All choices can be found here: +# http://www.w3.org/TR/REC-html40/struct/dirlang.html#langcodes +# http://blogs.law.harvard.edu/tech/stories/storyReader$15 +LANGUAGE_CODE = 'en-us' + +SITE_ID = 1 + +# Absolute path to the directory that holds media. +# Example: "/home/media/media.lawrence.com/" +MEDIA_ROOT = '' + +# URL that handles the media served from MEDIA_ROOT. +# Example: "http://media.lawrence.com" +MEDIA_URL = '' + +# URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a +# trailing slash. +# Examples: "http://foo.com/media/", "/media/". +ADMIN_MEDIA_PREFIX = '/media/' + +# Make this unique, and don't share it with anybody. +SECRET_KEY = 'z-n46o5v2gd^n2!tqlher5)w5!l at 0em+d_o+-3qhdulg7qc_lk' + +# List of callables that know how to import templates from various sources. +TEMPLATE_LOADERS = ( + 'django.template.loaders.filesystem.load_template_source', + 'django.template.loaders.app_directories.load_template_source', +# 'django.template.loaders.eggs.load_template_source', +) + +MIDDLEWARE_CLASSES = ( + 'django.middleware.common.CommonMiddleware', + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.middleware.doc.XViewMiddleware', +) + +ROOT_URLCONF = 'pypy.translator.js.examples.djangoping.urls' + +import os +# grab a sister package to get its __file__ +from pypy.translator.js.examples.djangoping import client +TEMPLATE_DIRS = ( + # Put strings here, like "/home/html/django_templates". + # Always use forward slashes, even on Windows. + os.path.join(os.path.dirname(client.__file__), "templates"), +) + +INSTALLED_APPS = ( + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.sites', +) Added: pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/templates/index.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/djangoping/templates/index.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,14 @@ + + + + + Python pinger + + + +

Guestbook

+
+ +
+
+ name:
+ message:
+
+ +
+ + + """ + return 'text/html', html + index.exposed = True + + # the (generated) javascript + def guestbook_js(self): + if hasattr(self.server, 'source'): + source = self.server.source + else: + source = guestbook_client() + self.server.source = source + return "text/javascript", source + guestbook_js.exposed = True + +if __name__ == '__main__': + addr = ('', 8008) + httpd = server.create_server(server_address=addr, handler=Handler) + httpd.serve_forever() Added: pypy/branch/flex-backend/pypy/translator/flex/examples/guestbook_client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/guestbook_client.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,35 @@ +""" rpython guestbook client-side code + + this code can be tested in CPython, but will also be converted to + JavaScript to provide the client-side functionality for the guestbook + example +""" + +from pypy.translator.js.modules import dom +from pypy.translator.js.examples.guestbook import exported_methods + +def _get_messages_callback(messages): + for message in messages: + add_html_message(message) + +def init_guestbook(): + exported_methods.get_messages(_get_messages_callback) + +def _add_message_callback(message): + add_html_message(message) + +def add_message(): + doc = dom.window.document + name = doc.getElementById('name').value + message = doc.getElementById('message').value + exported_methods.add_message(name, message, _add_message_callback) + +def add_html_message(text=''): + doc = dom.window.document + div = doc.getElementById('messages') + msgdiv = doc.createElement('div') + msgdiv.style.border = '1px solid black' + msgdiv.style.margin = '1em' + msgdiv.appendChild(doc.createTextNode(text)) + div.appendChild(msgdiv) + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/over_client.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/over_client.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,14 @@ + +""" Client side of overmind.py +""" + +from pypy.translator.js.modules import dom + +def callback(port): + hname = dom.window.location.hostname + dom.window.location.assign("http://%s:%d" % (hname, port)) + +def bnb_redirect(): + loc = dom.window.location + new_loc = loc.protocol + "//" + loc.hostname + ":7070" + loc.assign(new_loc) Added: pypy/branch/flex-backend/pypy/translator/flex/examples/overmind.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/overmind.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,83 @@ +#!/usr/bin/env python +""" This is script which collects all the demos and +run them when needed +""" + +import autopath + +from pypy.translator.js.lib import server +from pypy.translator.js.lib.support import callback +from pypy.rpython.ootypesystem.bltregistry import described +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js.examples.console import console +from py.__.green.server.httpserver import GreenHTTPServer + +import os +import py + +FUNCTION_LIST = ['bnb_redirect'] +TIMEOUT = 300 +pids = [] + +def js_source(function_list): + import over_client + return rpython2javascript(over_client, FUNCTION_LIST) + +static_dir = py.path.local(__file__).dirpath().join("data") + +class Root(server.Collection): + index = server.FsFile(static_dir.join("index.html")) + style_css = server.FsFile(static_dir.join("style.css"), + content_type="text/css") + terminal = server.Static(static_dir.join("terminal.html")) + console = console.Root() + py_web1_png = server.FsFile(static_dir.join("py-web1.png"), + content_type="image/png") + + def source_js(self): + if hasattr(self.server, 'source'): + source = self.server.source + else: + source = js_source(FUNCTION_LIST) + self.server.source = source + return "text/javascript", source + source_js.exposed = True + + def bnb(self): + return ''' + + + + + + + ''' + bnb.exposed = True + + def handle_error(self, exc, e_value, tb): + import traceback + tb_formatted = '\n'.join(traceback.format_tb(tb)) + \ + "%s: %s" % (exc, e_value) + log_file = open("/tmp/play1_error_log", "a") + log_file.write(tb_formatted) + log_file.close() + print tb_formatted + +class Handler(server.NewHandler): + application = Root() + + error_message_format = static_dir.join('error.html').read() + #console = server.Static(os.path.join(static_dir, "launcher.html")) + +if __name__ == '__main__': + try: + addr = ('', 8008) + httpd = server.create_server(server_address=addr, handler=Handler, + server=GreenHTTPServer) + httpd.serve_forever() + except KeyboardInterrupt: + for pid in pids: + # eventually os.kill stuff + os.kill(pid, 15) + os.waitpid(pid, 0) + Added: pypy/branch/flex-backend/pypy/translator/flex/examples/pythonconsole.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/pythonconsole.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,207 @@ +#!/usr/bin/env python +""" A basic Python console from your browser. + +This depends on MochiKit for proper quoting. +You need to provide the files as ..jsdemo/MochiKit/*.js. +(Symlinks are your friends.) + +Try to type: import time; time.sleep(2); print 'hi' +""" + +import autopath + +import sys, os, cStringIO +from cgi import parse_qs +from pypy.translator.js.modules.dom import setTimeout, document, window +from pypy.translator.js.modules.mochikit import connect, disconnect +from pypy.rpython.ootypesystem.bltregistry import MethodDesc, BasicExternal +from pypy.translator.js import commproxy +from pypy.rpython.extfunc import genericcallable + +from pypy.translator.js.lib import support +from pypy.translator.js.lib import server + +commproxy.USE_MOCHIKIT = True + +from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer +from SocketServer import ThreadingMixIn +import time + +HTML_PAGE = """ + + + Example + + + + +

Console

+

Note that a default timeout for the console is 5 minutes, after that time +console just dies and stops responding

+ +
+ + + + + +""" + +httpd = None + +def callback(data): + inp_elem = document.getElementById("inp") + inp_elem.disabled = False + answer = data.get('answer', '') + add_text(answer) + inp_elem.focus() + +def add_text(text): + data_elem = document.getElementById("data") + lines = text.split('\n') + lines.pop() + for line in lines: + pre = document.createElement('pre') + pre.style.margin = '0px' + pre.appendChild(document.createTextNode(line)) + data_elem.appendChild(pre) + +class Storage(object): + def __init__(self): + self.level = 0 + self.cmd = "" + +storage = Storage() + +def keypressed(key): + kc = key._event.keyCode + if kc == ord("\r"): + inp_elem = document.getElementById("inp") + cmd = inp_elem.value + if storage.level == 0: + add_text(">>> %s\n" % (cmd,)) + else: + add_text("... %s\n" % (cmd,)) + inp_elem.value = '' + if cmd: + storage.cmd += cmd + "\n" + if cmd.endswith(':'): + storage.level = 1 + elif storage.level == 0 or cmd == "": + if (not storage.level) or (not cmd): + inp_elem.disabled = True + httpd.some_callback(storage.cmd, callback) + storage.cmd = "" + storage.level = 0 + +def setup_page(): + connect(document, 'onkeypress', keypressed) + document.getElementById("inp").focus() + +class Server(HTTPServer, BasicExternal): + # Methods and signatures how they are rendered for JS + _methods = { + 'some_callback' : MethodDesc([('cmd', str), + ('callback', genericcallable([{str:str}]))], + {str:str}) + } + + _render_xmlhttp = True + + def __init__(self, *args, **kwargs): + HTTPServer.__init__(self, *args, **kwargs) + self.source = None + self.locals = {} + +class RequestHandler(BaseHTTPRequestHandler): + def do_GET(self): + path = self.path + if '?' in path: + i = path.index('?') + else: + i = len(path) + kwds = parse_qs(path[i+1:]) + path = path[:i].split("/") + if not path[0]: + del path[0] + if not path: + path = [''] + cmd = path[0] + args = path[1:] + method_to_call = getattr(self, "run_" + cmd, None) + if method_to_call is None: + self.send_error(404, "File %r not found" % (self.path,)) + else: + method_to_call(*args, **kwds) + + do_POST = do_GET + + def run_(self): + self.run_index() + + def run_index(self): + self.serve_data("text/html", HTML_PAGE) + + def run_MochiKit(self, filename): + assert filename in os.listdir('MochiKit') + pathname = os.path.join('MochiKit', filename) + f = open(pathname, 'r') + data = f.read() + f.close() + self.serve_data("text/javascript", data) + + def run_some_callback(self, cmd=[""]): + cmd = cmd[0] + self.server.last_activity = time.time() + if cmd: + buf = cStringIO.StringIO() + out1 = sys.stdout + err1 = sys.stderr + try: + sys.stdout = sys.stderr = buf + try: + exec compile(cmd, '?', 'single') in self.server.locals + except Exception: + import traceback + traceback.print_exc() + finally: + sys.stdout = out1 + sys.stderr = err1 + answer = buf.getvalue() + else: + answer = "" + self.serve_data("text/json", repr({'answer': answer})) + + def run_jssource(self): + if self.server.source: + source = self.server.source + else: + source = support.js_source([setup_page]) + self.server.source = source + self.serve_data("text/javascript", source) + + def serve_data(self, content_type, data): + self.send_response(200) + self.send_header("Content-type", content_type) + self.send_header("Content-length", len(data)) + self.send_header('Expires', 'Mon, 26 Jul 1997 05:00:00 GMT') + self.send_header('Last-Modified', + time.strftime("%a, %d %b %Y %H:%M:%S GMT")) + self.send_header('Cache-Control', 'no-cache, must-revalidate') + self.send_header('Cache-Control', 'post-check=0, pre-check=0') + self.send_header('Pragma', 'no-cache') + self.end_headers() + self.wfile.write(data) + + +def build_http_server(server_address=('', 8001)): + global httpd + httpd = Server(server_address, RequestHandler) + print 'http://127.0.0.1:%d' % (server_address[1],) + +def _main(address=('', 8001)): + build_http_server(server_address=address) + httpd.serve_forever() + +if __name__ == '__main__': + _main() Added: pypy/branch/flex-backend/pypy/translator/flex/examples/serialise.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/serialise.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,15 @@ + +""" example of (very simple) serialiser +""" + +def serialise(obj): + if isinstance(obj, str): + return "S" + obj + elif isinstance(obj, int): + return "I" + str(obj) + return "?" + +serialise._annspecialcase_ = "specialize:argtype(0)" + +def serialisetest(): + return serialise("aaa") + serialise(3) + serialise(None) Added: pypy/branch/flex-backend/pypy/translator/flex/examples/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/examples/test/test_examples.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/examples/test/test_examples.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,38 @@ + +""" Various build tests +""" +import py +from pypy.translator.js.main import rpython2javascript + +def test_console_build(): + from pypy.translator.js.examples import pythonconsole + httpd = pythonconsole.Server(('', 0), pythonconsole.RequestHandler) + pythonconsole.httpd = httpd + # XXX obscure hack + assert rpython2javascript(pythonconsole, ['setup_page'], use_pdb=False) + +def test_bnb_build(): + from pypy.translator.js.examples.bnb import start_bnb + assert rpython2javascript(start_bnb, ['bnb'], use_pdb=False) + +def test_overmind_build(): + from pypy.translator.js.examples import overmind, over_client + assert rpython2javascript(over_client, overmind.FUNCTION_LIST, + use_pdb=False) + +def test_guestbook_build(): + from pypy.translator.js.examples import guestbook, guestbook_client + assert rpython2javascript(guestbook_client, guestbook.FUNCTION_LIST, + use_pdb=False) + + +def test_console_2_build(): + from pypy.translator.js.examples.console import console, client + assert rpython2javascript(client, console.FUNCTION_LIST, + use_pdb=True) + +def test_ping_play1(): + from urllib import URLopener + u = URLopener() + text = "pypy.js various demos" + assert u.open("http://play1.pypy.org/").read().find(text) != -1 Added: pypy/branch/flex-backend/pypy/translator/flex/function.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/function.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,233 @@ +try: + set +except NameError: + from sets import Set as set + +from pypy.objspace.flow import model as flowmodel +from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong +from pypy.rpython.ootypesystem import ootype +from pypy.translator.oosupport.metavm import Generator,InstructionList +from pypy.translator.oosupport import function + +from pypy.translator.flex.log import log +from types import FunctionType + +import re + +class BaseGenerator(object): + def load(self, v): + if isinstance(v, flowmodel.Variable): + if v.name in self.argset: + selftype, selfname = self.args[0] + if self.is_method and v.name == selfname: + self.ilasm.load_self() + else: + self.ilasm.load_arg(v) + else: + self.ilasm.load_local(v) + elif isinstance(v, flowmodel.Constant): + self.db.load_const(v.concretetype, v.value, self.ilasm) + elif isinstance(v, str): + self.ilasm.load_const("'" + v + "'") + else: + assert False + + def store(self, v): + assert isinstance(v, flowmodel.Variable) + if v.concretetype is not Void: + self.ilasm.store_local(v) + else: + self.ilasm.store_void() + + def change_name(self, name, to_name): + self.ilasm.change_name(name, to_name) + + def add_comment(self, text): + pass + + def function_signature(self, graph): + return self.cts.graph_to_signature(graph, False) + + def class_name(self, ooinstance): + return ooinstance._name + + def emit(self, instr, *args): + self.ilasm.emit(instr, *args) + + def call_graph(self, graph): + self.db.pending_function(graph) + func_sig = self.function_signature(graph) + self.ilasm.call(func_sig) + + def call_external(self, name, args): + self.ilasm.call((name, args)) + + #def call_signature(self, signature): + # self.ilasm.call(signature) + + def cast_to(self, lltype): + cts_type = self.cts.lltype_to_cts(lltype, False) + self.ilasm.castclass(cts_type) + + def new(self, obj): + self.ilasm.new(self.cts.obj_name(obj)) + + def set_field(self, obj, name): + self.ilasm.set_field(obj, name) + #self.ilasm.set_field(self.field_name(obj,name)) + + def get_field(self, useless_stuff, name): + self.ilasm.get_field(name) + + def call_method(self, obj, name): + func_name, signature = self.cts.method_signature(obj, name) + self.ilasm.call_method(obj, name, signature) + + def call_external_method(self, name, arg_len): + self.ilasm.call_method(None, name, [0]*arg_len) + + def instantiate(self): + self.ilasm.runtimenew() + + def downcast(self, TYPE): + pass + + def load_special(self, v): + # special case for loading value + # when setting builtin field we need to load function instead of None + # FIXME: we cheat here + if isinstance(v, flowmodel.Constant) and v.concretetype is ootype.Void and isinstance(v.value, FunctionType): + graph = self.db.translator.annotator.bookkeeper.getdesc(v.value).cachedgraph(None) + self.db.pending_function(graph) + name = graph.name + self.ilasm.load_str(name) + else: + self.load(v) + + def cast_function(self, name, num): + self.ilasm.cast_function(name, num) + + def prefix_op(self, st): + self.ilasm.prefix_op(st) + + def load_str(self, s): + self.ilasm.load_str(s) + + def load_void(self): + self.ilasm.load_void() + + def list_setitem(self, base_obj, item, val): + self.load(base_obj) + self.load(val) + self.load(item) + self.ilasm.list_setitem() + + def list_getitem(self, base_obj, item): + self.load(base_obj) + self.load(item) + self.ilasm.list_getitem() + + def push_primitive_constant(self, TYPE, value): + self.db.load_const(TYPE, value, self.ilasm) + + def branch_unconditionally(self, target_label): + self.ilasm.jump_block(self.block_map[target_label]) + + def branch_conditionally(self, exitcase, target_label): + self.ilasm.branch_if(exitcase) + self.ilasm.jump_block(self.block_map[target_label]) + self.ilasm.close_branch() + +class Function(function.Function, BaseGenerator): + def __init__(self, db, graph, name=None, is_method=False, + is_entrypoint=False, _class=None): + self._class = _class + super(Function, self).__init__(db, graph, name, is_method, is_entrypoint) + self._set_args() + self._set_locals() + self.order = 0 + self.name = name or self.db.get_uniquename(self.graph, self.graph.name) + + def _setup_link(self, link, is_exc_link = False): + target = link.target + for to_load, to_store in zip(link.args, target.inputargs): + if to_load.concretetype is not Void: + if is_exc_link and isinstance(to_load, flowmodel.Variable) and re.match("last_exc_value", to_load.name): + self.ilasm.load_str("exc") + else: + self.load(to_load) + self.store(to_store) + + def _create_generator(self, ilasm): + return self + + def begin_render(self): + block_map = {} + for blocknum, block in enumerate(self.graph.iterblocks()): + block_map[self._get_block_name(block)] = blocknum + self.block_map = block_map + + if self.is_method: + args = self.args[1:] # self is implicit + else: + args = self.args + if self.is_method: + self.ilasm.begin_method(self.name, self._class, [i[1] for i in args]) + else: + self.ilasm.begin_function(self.name, args) + self.ilasm.set_locals(",".join([i[1] for i in self.locals])) + self.ilasm.begin_for() + + def render_return_block(self, block): + return_var = block.inputargs[0] + if return_var.concretetype is not Void: + self.load(return_var) + self.ilasm.ret() + else: + self.ilasm.load_void() + self.ilasm.ret() + + def end_render(self): + self.ilasm.end_for() + self.ilasm.end_function() + + def render_raise_block(self, block): + self.ilasm.throw(block.inputargs[1]) + + def end_try(self, target_label): + self.ilasm.jump_block(self.block_map[target_label]) + self.ilasm.catch() + #self.ilasm.close_branch() + + def record_ll_meta_exc(self, ll_meta_exc): + pass + + def begin_catch(self, llexitcase): + real_name = self.cts.lltype_to_cts(llexitcase._inst.class_._INSTANCE) + s = "isinstanceof(exc, %s)"%real_name + self.ilasm.branch_if_string(s) + + def end_catch(self, target_label): + """ Ends the catch block, and branchs to the given target_label as the + last item in the catch block """ + self.ilasm.close_branch() + + def store_exception_and_link(self, link): + self._setup_link(link, True) + self.ilasm.jump_block(self.block_map[self._get_block_name(link.target)]) + + def after_except_block(self): + #self.ilasm.close_branch() + self.ilasm.throw_real("exc") + self.ilasm.close_branch() + + def set_label(self, label): + self.ilasm.write_case(self.block_map[label]) + #self.ilasm.label(label) + + def begin_try(self): + self.ilasm.begin_try() + + def clean_stack(self): + self.ilasm.clean_stack() Added: pypy/branch/flex-backend/pypy/translator/flex/helper.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/helper.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,41 @@ + +""" Some helpers +""" + +from pypy.translator.flex.modules.dom import document + +def escape(s): + #return s.replace("&", "&").replace("<", "<").replace(">", ">"). \ + # replace("'", "\\'").replace(" ", " ").replace("\n", "
") + return s + +def create_debug_div(): + debug_div = document.createElement("div") + debug_div.setAttribute("id", "debug_div") + # XXX attach it somewhere... + #body = document.getElementsByTagName('body')[0] + document.childNodes[0].childNodes[1].appendChild(debug_div) + return debug_div + +def __show_traceback(tb, exc): + debug_div = document.getElementById("debug_div") + if not debug_div: + # create div here + debug_div = create_debug_div() + + pre_div = document.createElement("pre") + pre_div.style.color = "#FF0000" + debug_div.appendChild(pre_div) + txt = document.createTextNode("") + pre_div.appendChild(txt) + for tb_entry in tb[1:]: + # list of tuples... + fun_name, args, filename, lineno = tb_entry + # some source maybe? or so? + line1 = escape("%s %s" % (fun_name, args)) + line2 = escape(" %s: %s\n" % (filename, lineno)) + txt.nodeValue += line1 + '\n' + line2 + + txt.nodeValue += str(exc) + +__show_traceback.explicit_traceback = True Added: pypy/branch/flex-backend/pypy/translator/flex/js.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/js.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,133 @@ +''' +reference material: + http://webreference.com/javascript/reference/core_ref/ + http://webreference.com/programming/javascript/ + http://mochikit.com/ + http://www.mozilla.org/js/spidermonkey/ + svn co http://codespeak.net/svn/kupu/trunk/ecmaunit +''' + +import py +import os + +from pypy.rpython.rmodel import inputconst +from pypy.rpython.typesystem import getfunctionptr +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.ootypesystem import ootype +from pypy.tool.udir import udir +from pypy.translator.flex.log import log + +from pypy.translator.flex.asmgen import AsmGen +from pypy.translator.flex.jts import JTS +from pypy.translator.flex.opcodes import opcodes +from pypy.translator.flex.function import Function +from pypy.translator.flex.database import LowLevelDatabase + +from pypy.translator.oosupport.genoo import GenOO + +from heapq import heappush, heappop + +from StringIO import StringIO + +def _path_join(root_path, *paths): + path = root_path + for p in paths: + path = os.path.join(path, p) + return path + +class Tee(object): + def __init__(self, *args): + self.outfiles = args + + def write(self, s): + for outfile in self.outfiles: + outfile.write(s) + + def close(self): + for outfile in self.outfiles: + if outfile is not sys.stdout: + outfile.close() + +class JS(GenOO): + TypeSystem = JTS + opcodes = opcodes + Function = Function + Database = LowLevelDatabase + + def __init__(self, translator, functions=[], stackless=False, compress=False, \ + logging=False, use_debug=False): + if not isinstance(functions, list): + functions = [functions] + GenOO.__init__(self, udir, translator, None) + + pending_graphs = [translator.annotator.bookkeeper.getdesc(f).cachedgraph(None) for f in functions ] + for graph in pending_graphs: + self.db.pending_function(graph) + + self.db.translator = translator + self.use_debug = use_debug + self.assembly_name = self.translator.graphs[0].name + self.tmpfile = udir.join(self.assembly_name + '.js') + + def gen_pendings(self): + while self.db._pending_nodes: + node = self.db._pending_nodes.pop() + to_render = [] + nparent = node + while nparent.order != 0: + nparent = nparent.parent + to_render.append(nparent) + to_render.reverse() + for i in to_render: + i.render(self.ilasm) + + node.render(self.ilasm) + + def generate_communication_proxy(self): + """ Render necessary stuff aroundc communication + proxies + """ + for proxy in self.db.proxies: + proxy.render(self.ilasm) + + + def create_assembler(self): + out = self.tmpfile.open('w') + return AsmGen(out, self.assembly_name) + + def generate_source(self): + self.ilasm = self.create_assembler() + self.fix_names() + self.gen_entrypoint() + while self.db._pending_nodes: + self.gen_pendings() + self.db.gen_constants(self.ilasm, self.db._pending_nodes) + + self.ilasm.end_consts() + self.ilasm.close() + assert len(self.ilasm.right_hand) == 0 + return self.tmpfile.strpath + + def write_source(self): + + # write down additional functions + # FIXME: when used with browser option it should probably + # not be used as inlined, rather another script to load + # this is just workaround + self.generate_source() + sio = StringIO() + data = self.tmpfile.open().read() + + src_filename = _path_join(os.path.dirname(__file__), 'jssrc', 'misc.as') + flex_filename = _path_join(os.path.dirname(__file__), 'jssrc', 'flex.mxml') + f = self.tmpfile.open("w") + lib = open(src_filename).read() + flex = open(flex_filename).read() + self.ilasm = AsmGen(sio, self.assembly_name ) + self.generate_communication_proxy() + f.write(flex%(lib, data)) + f.close() + + self.filename = self.tmpfile + + return self.tmpfile Added: pypy/branch/flex-backend/pypy/translator/flex/jsbuiltin.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/jsbuiltin.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,84 @@ +""" JavaScript builtin mappings +""" + +from pypy.translator.oosupport.metavm import InstructionList, PushAllArgs,\ + _PushAllArgs +from pypy.translator.flex.metavm import SetBuiltinField, ListGetitem, ListSetitem, \ + GetBuiltinField, CallBuiltin, Call, SetTimeout, ListContains,\ + NewBuiltin, SetOnEvent, ListRemove, CallBuiltinMethod, _GetPredefinedField,\ + _SetPredefinedField + +from pypy.rpython.ootypesystem import ootype + +class _Builtins(object): + def __init__(self): + list_resize = _SetPredefinedField('length') + + self.builtin_map = { + 'll_js_jseval' : CallBuiltin('eval'), + 'set_on_keydown' : SetOnEvent('onkeydown'), + 'set_on_keyup' : SetOnEvent('onkeyup'), + 'setTimeout' : SetTimeout, + 'll_int_str' : CallBuiltinMethod('toString', [2]), + 'll_strconcat' : InstructionList([_PushAllArgs(slice(1, None)), '+']), + 'll_int' : CallBuiltin('parseInt'), + #'alert' : CallBuiltin('alert'), + 'seval' : CallBuiltin('seval'), + 'date': NewBuiltin('Date'), + 'll_math.ll_math_fmod' : InstructionList([_PushAllArgs(slice(1, None)), '%']), + 'll_time_time' : CallBuiltin('time'), + 'll_time_clock' : CallBuiltin('clock'), + 'll_os_write' : CallBuiltin('print'), + } + self.builtin_obj_map = { + ootype.String.__class__: { + 'll_strconcat' : InstructionList([_PushAllArgs(slice(1, None)), '+']), + 'll_strlen' : _GetPredefinedField('length'), + 'll_stritem_nonneg' : CallBuiltinMethod('charAt', slice(1,None)), + 'll_streq' : InstructionList([_PushAllArgs(slice(1, None)), '==']), + 'll_strcmp' : CallBuiltin('strcmp'), + 'll_startswith' : CallBuiltin('startswith'), + 'll_endswith' : CallBuiltin('endswith'), + 'll_split_chr' : CallBuiltin('splitchr'), + 'll_substring' : CallBuiltin('substring'), + 'll_lower' : CallBuiltinMethod('toLowerCase', slice(1, None)), + 'll_upper' : CallBuiltinMethod('toUpperCase', slice(1, None)), + 'll_find' : CallBuiltin('findIndexOf'), + 'll_find_char' : CallBuiltin('findIndexOf'), + 'll_contains' : CallBuiltin('findIndexOfTrue'), + 'll_replace_chr_chr' : CallBuiltinMethod('replace', slice(1, None), ['g']), + 'll_count_char' : CallBuiltin('countCharOf'), + 'll_count' : CallBuiltin('countOf'), + }, + ootype.List: { + 'll_setitem_fast' : ListSetitem, + 'll_getitem_fast' : ListGetitem, + '_ll_resize' : list_resize, + '_ll_resize_ge' : list_resize, + '_ll_resize_le' : list_resize, + 'll_length' : _GetPredefinedField('length'), + }, + ootype.Dict: { + 'll_get' : ListGetitem, + 'll_set' : ListSetitem, + 'll_contains' : ListContains, + 'll_get_items_iterator' : CallBuiltin('dict_items_iterator'), + 'll_length' : CallBuiltin('get_dict_len'), + 'll_remove' : ListRemove, + 'll_clear': CallBuiltin('clear_dict'), + }, + ootype.Record: { + 'll_get' : ListGetitem, + 'll_set' : ListSetitem, + 'll_contains' : ListContains, + } + } + self.fix_opcodes() + + def fix_opcodes(self): + from pypy.translator.flex.metavm import fix_opcodes + #fix_opcodes(self.builtin_map) + #for value in self.builtin_obj_map.values(): + # fix_opcodes(value) + +Builtins = _Builtins() Added: pypy/branch/flex-backend/pypy/translator/flex/json.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/json.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,310 @@ +import string +import types + +## json.py implements a JSON (http://json.org) reader and writer. +## Copyright (C) 2005 Patrick D. Logan +## Contact mailto:patrickdlogan at stardecisions.com +## +## This library is free software; you can redistribute it and/or +## modify it under the terms of the GNU Lesser General Public +## License as published by the Free Software Foundation; either +## version 2.1 of the License, or (at your option) any later version. +## +## This library is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +## Lesser General Public License for more details. +## +## You should have received a copy of the GNU Lesser General Public +## License along with this library; if not, write to the Free Software +## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +class _StringGenerator(object): + def __init__(self, string): + self.string = string + self.index = -1 + def peek(self): + i = self.index + 1 + if i < len(self.string): + return self.string[i] + else: + return None + def next(self): + self.index += 1 + if self.index < len(self.string): + return self.string[self.index] + else: + raise StopIteration + def all(self): + return self.string + +class WriteException(Exception): + pass + +class ReadException(Exception): + pass + +class JsonReader(object): + hex_digits = {'A': 10,'B': 11,'C': 12,'D': 13,'E': 14,'F':15} + escapes = {'t':'\t','n':'\n','f':'\f','r':'\r','b':'\b'} + + def read(self, s): + self._generator = _StringGenerator(s) + result = self._read() + return result + + def _read(self): + self._eatWhitespace() + peek = self._peek() + if peek is None: + raise ReadException, "Nothing to read: '%s'" % self._generator.all() + if peek == '{': + return self._readObject() + elif peek == '[': + return self._readArray() + elif peek == '"': + return self._readString() + elif peek == '-' or peek.isdigit(): + return self._readNumber() + elif peek == 't': + return self._readTrue() + elif peek == 'f': + return self._readFalse() + elif peek == 'n': + return self._readNull() + elif peek == '/': + self._readComment() + return self._read() + else: + raise ReadException, "Input is not valid JSON: '%s'" % self._generator.all() + + def _readTrue(self): + self._assertNext('t', "true") + self._assertNext('r', "true") + self._assertNext('u', "true") + self._assertNext('e', "true") + return True + + def _readFalse(self): + self._assertNext('f', "false") + self._assertNext('a', "false") + self._assertNext('l', "false") + self._assertNext('s', "false") + self._assertNext('e', "false") + return False + + def _readNull(self): + self._assertNext('n', "null") + self._assertNext('u', "null") + self._assertNext('l', "null") + self._assertNext('l', "null") + return None + + def _assertNext(self, ch, target): + if self._next() != ch: + raise ReadException, "Trying to read %s: '%s'" % (target, self._generator.all()) + + def _readNumber(self): + isfloat = False + result = self._next() + peek = self._peek() + while peek is not None and (peek.isdigit() or peek == "."): + isfloat = isfloat or peek == "." + result = result + self._next() + peek = self._peek() + try: + if isfloat: + return float(result) + else: + return int(result) + except ValueError: + raise ReadException, "Not a valid JSON number: '%s'" % result + + def _readString(self): + result = "" + assert self._next() == '"' + try: + while self._peek() != '"': + ch = self._next() + if ch == "\\": + ch = self._next() + if ch in 'brnft': + ch = self.escapes[ch] + elif ch == "u": + ch4096 = self._next() + ch256 = self._next() + ch16 = self._next() + ch1 = self._next() + n = 4096 * self._hexDigitToInt(ch4096) + n += 256 * self._hexDigitToInt(ch256) + n += 16 * self._hexDigitToInt(ch16) + n += self._hexDigitToInt(ch1) + ch = unichr(n) + elif ch not in '"/\\': + raise ReadException, "Not a valid escaped JSON character: '%s' in %s" % (ch, self._generator.all()) + result = result + ch + except StopIteration: + raise ReadException, "Not a valid JSON string: '%s'" % self._generator.all() + assert self._next() == '"' + return result + + def _hexDigitToInt(self, ch): + try: + result = self.hex_digits[ch.upper()] + except KeyError: + try: + result = int(ch) + except ValueError: + raise ReadException, "The character %s is not a hex digit." % ch + return result + + def _readComment(self): + assert self._next() == "/" + second = self._next() + if second == "/": + self._readDoubleSolidusComment() + elif second == '*': + self._readCStyleComment() + else: + raise ReadException, "Not a valid JSON comment: %s" % self._generator.all() + + def _readCStyleComment(self): + try: + done = False + while not done: + ch = self._next() + done = (ch == "*" and self._peek() == "/") + if not done and ch == "/" and self._peek() == "*": + raise ReadException, "Not a valid JSON comment: %s, '/*' cannot be embedded in the comment." % self._generator.all() + self._next() + except StopIteration: + raise ReadException, "Not a valid JSON comment: %s, expected */" % self._generator.all() + + def _readDoubleSolidusComment(self): + try: + ch = self._next() + while ch != "\r" and ch != "\n": + ch = self._next() + except StopIteration: + pass + + def _readArray(self): + result = [] + assert self._next() == '[' + done = self._peek() == ']' + while not done: + item = self._read() + result.append(item) + self._eatWhitespace() + done = self._peek() == ']' + if not done: + ch = self._next() + if ch != ",": + raise ReadException, "Not a valid JSON array: '%s' due to: '%s'" % (self._generator.all(), ch) + assert ']' == self._next() + return result + + def _readObject(self): + result = {} + assert self._next() == '{' + done = self._peek() == '}' + while not done: + key = self._read() + if type(key) is not types.StringType: + raise ReadException, "Not a valid JSON object key (should be a string): %s" % key + self._eatWhitespace() + ch = self._next() + if ch != ":": + raise ReadException, "Not a valid JSON object: '%s' due to: '%s'" % (self._generator.all(), ch) + self._eatWhitespace() + val = self._read() + result[key] = val + self._eatWhitespace() + done = self._peek() == '}' + if not done: + ch = self._next() + if ch != ",": + raise ReadException, "Not a valid JSON array: '%s' due to: '%s'" % (self._generator.all(), ch) + assert self._next() == "}" + return result + + def _eatWhitespace(self): + p = self._peek() + while p is not None and p in string.whitespace or p == '/': + if p == '/': + self._readComment() + else: + self._next() + p = self._peek() + + def _peek(self): + return self._generator.peek() + + def _next(self): + return self._generator.next() + +class JsonWriter(object): + + def _append(self, s): + self._results.append(s) + + def write(self, obj, escaped_forward_slash=False): + self._escaped_forward_slash = escaped_forward_slash + self._results = [] + self._write(obj) + return "".join(self._results) + + def _write(self, obj): + ty = type(obj) + if ty is types.DictType: + n = len(obj) + self._append("{") + for k, v in obj.items(): + self._write(k) + self._append(":") + self._write(v) + n = n - 1 + if n > 0: + self._append(",") + self._append("}") + elif ty is types.ListType or ty is types.TupleType: + n = len(obj) + self._append("[") + for item in obj: + self._write(item) + n = n - 1 + if n > 0: + self._append(",") + self._append("]") + elif ty is types.StringType or ty is types.UnicodeType: + self._append('"') + obj = obj.replace('\\', r'\\') + if self._escaped_forward_slash: + obj = obj.replace('/', r'\/') + obj = obj.replace('"', r'\"') + obj = obj.replace('\b', r'\b') + obj = obj.replace('\f', r'\f') + obj = obj.replace('\n', r'\n') + obj = obj.replace('\r', r'\r') + obj = obj.replace('\t', r'\t') + self._append(obj) + self._append('"') + elif ty is types.IntType or ty is types.LongType: + self._append(str(obj)) + elif ty is types.FloatType: + self._append("%f" % obj) + elif obj is True: + self._append("true") + elif obj is False: + self._append("false") + elif obj is None: + self._append("null") + else: + raise WriteException, "Cannot write in JSON: %s" % repr(obj) + +def write(obj, escaped_forward_slash=False): + return JsonWriter().write(obj, escaped_forward_slash) + +def read(s): + return JsonReader().read(s) Added: pypy/branch/flex-backend/pypy/translator/flex/jssrc/flex.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/jssrc/flex.mxml Sat Jul 14 12:31:00 2007 @@ -0,0 +1,18 @@ + + + + + + + + Added: pypy/branch/flex-backend/pypy/translator/flex/jssrc/misc.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/jssrc/misc.as Sat Jul 14 12:31:00 2007 @@ -0,0 +1,241 @@ + + +// we should put the imports here. +import mx.controls.Button; + + +import py.*; +//import py.f.DictIter; + + +// starts hand written code +var MALLOC_ZERO_FILLED = 0; + +var print = trace; + + + + +Function.prototype.method = function (name, func) { + this.prototype[name] = func; + return this; +}; + +function inherits(child, parent) { + child.parent = parent; + for (var i in parent.prototype) { + if (!child.prototype[i]) { + child.prototype[i] = parent.prototype[i]; + } + } +} + +function isinstanceof(self, what) { + + //return self instanceof what; + + + if (!self) { + return (false); + } + var t = self.constructor; + while ( t ) { + if (t == what) { + return (true); + } + t = t.parent; + } + return (false); +} + +/*function delitem(fn, l, i) { + for(var j = i; j < l.length-1; ++j) { + l[j] = l[j+1]; + } + l.length--; +}*/ + +function strcmp(s1, s2) { + if ( s1 < s2 ) { + return ( -1 ); + } else if ( s1 == s2 ) { + return ( 0 ); + } + return (1); +} + +function startswith(s1, s2) { + if (s1.length < s2.length) { + return(false); + } + for (var i = 0; i < s2.length; ++i){ + if (s1.charAt(i) != s2.charAt(i)) { + return(false); + } + } + return(true); +} + +function endswith(s1, s2) { + if (s2.length > s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} + + +function __flash_main() { + + + var aaa = new Button(); + aaa.label = "before consts"; + addChild ( aaa ); + + + + __load_consts_flex() + + aaa.label = "after consts"; + flash_main(1) +} + + +public function flexTrace( text:String ):void { + var myUrl:URLRequest = new URLRequest("javascript:console.log('" + text + "');void(0);"); + sendToURL(myUrl); +} + +// wrapper for throw, because flex compiler weirdness. +function throwit(e) { + throw(e); +} + + +// ends hand written code + Added: pypy/branch/flex-backend/pypy/translator/flex/jssrc/misc.js ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/jssrc/misc.js Sat Jul 14 12:31:00 2007 @@ -0,0 +1,229 @@ +// starts hand written code +MALLOC_ZERO_FILLED = 0 + +try { + log; + print = log; +} catch(e) { +} + +Function.prototype.method = function (name, func) { + this.prototype[name] = func; + return this; +}; + +function inherits(child, parent) { + child.parent = parent; + for (i in parent.prototype) { + if (!child.prototype[i]) { + child.prototype[i] = parent.prototype[i]; + } + } +} + +function isinstanceof(self, what) { + if (!self) { + return (false); + } + t = self.constructor; + while ( t ) { + if (t == what) { + return (true); + } + t = t.parent; + } + return (false); +} + +/*function delitem(fn, l, i) { + for(j = i; j < l.length-1; ++j) { + l[j] = l[j+1]; + } + l.length--; +}*/ + +function strcmp(s1, s2) { + if ( s1 < s2 ) { + return ( -1 ); + } else if ( s1 == s2 ) { + return ( 0 ); + } + return (1); +} + +function startswith(s1, s2) { + if (s1.length < s2.length) { + return(false); + } + for (i = 0; i < s2.length; ++i){ + if (s1.charAt(i) != s2.charAt(i)) { + return(false); + } + } + return(true); +} + +function endswith(s1, s2) { + if (s2.length > s1.length) { + return(false); + } + for (i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} +// ends hand written code Added: pypy/branch/flex-backend/pypy/translator/flex/jts.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/jts.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,158 @@ +""" JavaScript type system +""" + +from pypy.rpython.ootypesystem import ootype +from pypy.rpython.lltypesystem import lltype +from pypy.translator.cli import oopspec + +from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong, Primitive +from pypy.rpython.lltypesystem.lltype import Char, UniChar +from pypy.rpython.ootypesystem.ootype import String, _string, List, StaticMethod +from pypy.rlib.objectmodel import Symbolic + +from pypy.translator.flex.log import log + +from types import FunctionType +from pypy.rpython.extfunc import is_external + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc, ExternalType + +try: + set +except NameError: + from sets import Set as set + +class JTS(object): + """ Class implementing JavaScript type system + calls with mapping similiar to cts + """ + def __init__(self, db): + self.db = db + + #def __class(self, name): + # return name.replace(".", "_") + + def escape_name(self, name): + return name.replace('.', '_') + + def llvar_to_cts(self, var): + return 'var ', var.name + + def lltype_to_cts(self, t): + if isinstance(t, ootype.Instance): + self.db.pending_class(t) + return self.escape_name(t._name) + elif isinstance(t, ootype.List): + return "Array" + elif isinstance(t, lltype.Primitive): + return "var" + elif isinstance(t, ootype.Record): + return "Object" + elif isinstance(t, ootype.String.__class__): + return '""' + elif isinstance(t, ootype.Dict): + return "Object" + elif isinstance(t, ootype.DictItemsIterator): + return "Object" + elif t is ootype.StringBuilder: + return "StringBuilder" + #return "var" + raise NotImplementedError("Type %r" % (t,)) + + def graph_to_signature(self, graph, is_method = False, func_name = None): + func_name = func_name or self.db.get_uniquename(graph,graph.name) + + args = [arg for arg in graph.getargs() if + arg.concretetype is not ootype.Void] + if is_method: + args = args[1:] + + return func_name,args + + def method_signature(self, obj, name): + # TODO: use callvirt only when strictly necessary + if isinstance(obj, ootype.Instance): + owner, meth = obj._lookup(name) + METH = meth._TYPE + return obj._name, METH.ARGS + elif isinstance(obj, ootype.BuiltinType): + meth = oopspec.get_method(obj, name) + class_name = self.lltype_to_cts(obj) + return class_name,meth.ARGS + else: + assert False + + def obj_name(self, obj): + #import pdb + #pdb.set_trace() + if isinstance(obj, ExternalType): + basic_external = obj._class_ + if not hasattr(basic_external, "_render_class"): + raise TypeError("This class %s can not be instantiated." % (basic_external,)) + return basic_external._render_class + return self.lltype_to_cts(obj) + + def primitive_repr(self, _type, v): + if _type is Bool: + if v == False: + val = 'false' + else: + val = 'true' + elif _type is Void: + val = 'undefined' + elif isinstance(_type,String.__class__): + val = '%r'%v._str + elif isinstance(_type,List): + # FIXME: It's not ok to use always empty list + val = "[]" + elif isinstance(_type,StaticMethod): + if hasattr(v, 'graph') and not is_external(v): + self.db.pending_function(v.graph) + else: + self.db.pending_abstract_function(v) + val = v._name + val = val.replace('.', '_') + if val == '?': + val = 'undefined' + elif _type is UniChar or _type is Char: + #log("Constant %r"%v) + s = repr(v) + if s.startswith('u'): + s = s[1:] + if s != "'\''": + s.replace("'", '"') + val = s + elif isinstance(v, Symbolic): + val = v.expr + elif isinstance(_type, Primitive): + #log("Type: %r"%_type) + val = str(v) + else: + assert False, "Unknown constant %r"%_type + val = str(v) + return val + + #def lltype_to_cts(self, t, include_class=True): + # return 'var' +## if isinstance(t, ootype.Instance): +## self.db.pending_class(t) +## return self.__class(t._name, include_class) +## elif isinstance(t, ootype.Record): +## name = self.db.pending_record(t) +## return self.__class(name, include_class) +## elif isinstance(t, ootype.StaticMethod): +## return 'void' # TODO: is it correct to ignore StaticMethod? +## elif isinstance(t, ootype.List): +## item_type = self.lltype_to_cts(t._ITEMTYPE) +## return self.__class(PYPY_LIST % item_type, include_class) +## elif isinstance(t, ootype.Dict): +## key_type = self.lltype_to_cts(t._KEYTYPE) +## value_type = self.lltype_to_cts(t._VALUETYPE) +## return self.__class(PYPY_DICT % (key_type, value_type), include_class) +## elif isinstance(t, ootype.DictItemsIterator): +## key_type = self.lltype_to_cts(t._KEYTYPE) +## value_type = self.lltype_to_cts(t._VALUETYPE) +## return self.__class(PYPY_DICT_ITEMS_ITERATOR % (key_type, value_type), include_class) +## +## return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Added: pypy/branch/flex-backend/pypy/translator/flex/lib/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/lib/server.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/server.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,396 @@ + +""" This is example of totally basic server for XMLHttp request +built on top of BaseHTTPServer. + +Construction is like that: + +you take your own implementation of Handler and subclass it +to provide whatever you like. Each request is checked first for +apropriate method in handler (with dots replaced as _) and this method +needs to have set attribute exposed + +If method is not there, we instead try to search exported_methods (attribute +of handler) for apropriate JSON call. We write down a JSON which we get as +a return value (note that right now arguments could be only strings) and +pass them to caller +""" + +import traceback + +HTTP_STATUS_MESSAGES = { + 200: 'OK', + 204: 'No Content', + 301: 'Moved permanently', + 302: 'Found', + 304: 'Not modified', + 401: 'Unauthorized', + 403: 'Forbidden', + 404: 'Not found', + 500: 'Server error', + 501: 'Not implemented', +} + +class HTTPError(Exception): + """ raised on HTTP errors """ + def __init__(self, status, data=None): + self.status = status + self.message = HTTP_STATUS_MESSAGES[status] + self.data = data + + def __str__(self): + data = '' + if self.data: + data = ' (%s)' % (self.data,) + return '' % (self.status, self.message, data) + +from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler + +import re +import time +import random +import os +import sys + +import py +from pypy.translator.js.lib.url import parse_url + +from pypy.translator.js import json + +from pypy.rpython.ootypesystem.bltregistry import MethodDesc, BasicExternal,\ + described +from pypy.translator.js.main import rpython2javascript +from pypy.translator.js import commproxy + +commproxy.USE_MOCHIKIT = False + +class Collection(object): + """ an HTTP collection + + essentially this is a container object that has a path that ends on a + slash, and support for PATH_INFO (so can have (virtual or not) + children) + + children are callable attributes of ourselves that have an 'exposed' + attribute themselves, that accept 3 arguments: 'handler', a reference + to the BaseHTTPHandler that handles the request (XXX should be + abstracted?), 'path', the requested path to the object, and 'query', + the (unparsed!) GET query string (without a preceding ?) + """ + + def traverse(self, path, orgpath): + """ traverse path relative to self + + 'path' is the path requested by the client, split on '/', but + relative from the current object: parent Collection items may have + removed items (they will have, actually, unless 'self' is the root + of the website) from the beginning on traversal to 'self' + + path is split on '/', the first item is removed and used to do + a lookup on self, if that fails a 404 is raised, if successful + the item is used to continue traversal (if the object found is + a Collection type) or to handle the request (if the object found + is a callable with .exposed set to True) + + if path equals '', a lookup for 'index' is done + + can be overridden in subclasses to implement different path + handling (PATH_INFO-like stuff) + """ + name = path.pop(0) + if name == '': + name = 'index' + name = name.replace(".", "_") + resource = getattr(self, name, None) + if (resource is None or (not isinstance(resource, Collection) and + (not callable(resource) or + not getattr(resource, 'exposed', True)))): + raise HTTPError(404) + if path: + if not isinstance(resource, Collection): + raise HTTPError(500) # no PATH_INFO allowed for non-Collection + return resource.traverse(path, orgpath) + else: + if isinstance(resource, Collection): + # targeting a collection directly: redirect to its 'index' + raise HTTPError(301, orgpath + '/') + if not getattr(resource, 'exposed', False): + # don't reveal what is not accessible... + raise HTTPError(404) + return resource + +class ExportedMethods(BasicExternal, Collection): + _render_base_path = "exported_methods" + def traverse(self, path, orgpath): + """ traverse path relative to self + + 'path' is the path requested by the client, split on '/', but + relative from the current object: parent Collection items may have + removed items (they will have, actually, unless 'self' is the root + of the website) from the beginning on traversal to 'self' + + path is split on '/', the first item is removed and used to do + a lookup on self, if that fails a 404 is raised, if successful + the item is used to continue traversal (if the object found is + a Collection type) or to handle the request (if the object found + is a callable with .exposed set to True) + + if path equals '', a lookup for 'index' is done + + can be overridden in subclasses to implement different path + handling (PATH_INFO-like stuff) + """ + name = path.pop(0) + name = name.replace(".", "_") + resource = getattr(self, name, None) + if not resource: + raise HTTPError(404) + return lambda **args : ('text/json', json.write(resource(**args))) + _render_xmlhttp = True + +exported_methods = ExportedMethods() + +def patch_handler(handler_class): + """ This function takes care of adding necessary + attributed to Static objects + """ + for name, value in handler_class.__dict__.iteritems(): + if isinstance(value, Static) and value.path is None: + assert hasattr(handler_class, "static_dir") + value.path = os.path.join(str(handler_class.static_dir), + name + ".html") + +class TestHandler(BaseHTTPRequestHandler): + exported_methods = exported_methods + + def do_GET(self): + path, args = parse_url(self.path) + if not path: + path = ["index"] + name_path = path[0].replace(".", "_") + if len(path) > 1: + rest = os.path.sep.join(path[1:]) + else: + rest = None + method_to_call = getattr(self, name_path, None) + if method_to_call is None or not getattr(method_to_call, 'exposed', None): + exec_meth = getattr(self.exported_methods, name_path, None) + if exec_meth is None: + self.send_error(404, "File %s not found" % path) + else: + self.serve_data('text/json', json.write(exec_meth(**args)), + True) + else: + if rest: + outp = method_to_call(rest, **args) + else: + outp = method_to_call(**args) + if isinstance(outp, (str, unicode)): + self.serve_data('text/html', outp) + elif isinstance(outp, tuple): + self.serve_data(*outp) + else: + raise ValueError("Don't know how to serve %s" % (outp,)) + + def log_message(self, format, *args): + # XXX just discard it + pass + + do_POST = do_GET + + def serve_data(self, content_type, data, nocache=False): + self.send_response(200) + self.send_header("Content-type", content_type) + self.send_header("Content-length", len(data)) + if nocache: + self.send_nocache_headers() + self.end_headers() + self.wfile.write(data) + + def send_nocache_headers(self): + self.send_header('Expires', 'Mon, 26 Jul 1997 05:00:00 GMT') + self.send_header('Last-Modified', + time.strftime("%a, %d %b %Y %H:%M:%S GMT")) + self.send_header('Cache-Control', 'no-cache, must-revalidate') + self.send_header('Cache-Control', 'post-check=0, pre-check=0') + self.send_header('Pragma', 'no-cache') + +class Static(object): + exposed = True + + def __init__(self, path=None): + self.path = path + + def __call__(self): + return open(str(self.path)).read() + +class FsFile(object): + exposed = True + debug = False + def __init__(self, path, content_type="text/html"): + self._path = path + self._content_type = content_type + + _data = None + def __call__(self): + if self._data is None or self.debug: + self._data = self._path.read() + return ({'Content-Type': self._content_type}, self._data) + +class StaticDir(Collection): + exposed = True + + def __init__(self, path, type=None): + self.path = path + self.type = type + + def traverse(self, path, orgpath): + data = open(os.path.join(str(self.path), *path)).read() + if self.type: + return lambda : (self.type, data) + return lambda : data + +def create_server(server_address = ('', 8000), handler=TestHandler, + server=HTTPServer): + """ Parameters: + spawn - create new thread and return (by default it doesn't return) + fork - do a real fork + timeout - kill process after X seconds (actually doesn't work for threads) + port_file - function to be called with port number + """ + patch_handler(handler) + httpd = server(server_address, handler) + httpd.last_activity = time.time() + print "Server started, listening on %s:%s" %\ + (httpd.server_address[0],httpd.server_port) + return httpd + +def start_server_in_new_thread(server): + import thread + thread.start_new_thread(server.serve_forever, ()) + +def start_server_in_new_process(server, timeout=None): + pid = os.fork() + if not pid: + if timeout: + def f(httpd): + while 1: + time.sleep(.3) + if time.time() - httpd.last_activity > timeout: + httpd.server_close() + import os + os.kill(os.getpid(), 15) + import thread + thread.start_new_thread(f, (server,)) + + server.serve_forever() + os._exit(0) + return pid + +Handler = TestHandler +# deprecate TestHandler name + +class NewHandler(BaseHTTPRequestHandler): + """ BaseHTTPRequestHandler that does object publishing + """ + + application = None # attach web root (Collection object) here!! + bufsize = 1024 + + def do_GET(self, send_body=True): + """ perform a request """ + path, query = self.process_path(self.path) + _, args = parse_url("?" + query) + try: + resource = self.find_resource(path) + # XXX strange hack + if hasattr(resource, 'im_self'): + resource.im_self.server = self.server + retval = resource(**args) + if isinstance(retval, str): + headers = {'Content-Type': 'text/html'} + data = retval + else: + headers, data = retval + if isinstance(headers, str): + headers = {'Content-Type': headers} + except HTTPError, e: + status = e.status + headers, data = self.process_http_error(e) + except: + exc, e, tb = sys.exc_info() + tb_formatted = '\n'.join(traceback.format_tb(tb)) + status = 200 + data = 'An error has occurred: %s - %s\n\n%s' % (exc, e, + tb_formatted) + headers = {'Content-Type': 'text/plain'} + if hasattr(self.application, 'handle_error'): + self.application.handle_error(exc, e, tb) + else: + status = 200 + if not 'content-type' in [k.lower() for k in headers]: + headers['Content-Type'] = 'text/html; charset=UTF-8' + self.response(status, headers, data, send_body) + + do_POST = do_GET + + def do_HEAD(self): + return self.do_GET(False) + + def process_path(self, path): + """ split the path in a path and a query part# + + returns a tuple (path, query), where path is a string and + query a dictionary containing the GET vars (URL decoded and such) + """ + path = path.split('?') + if len(path) > 2: + raise ValueError('illegal path %s' % (path,)) + p = path[0] + q = len(path) > 1 and path[1] or '' + return p, q + + def find_resource(self, path): + """ find the resource for a given path + """ + if not path: + raise HTTPError(301, '/') + assert path.startswith('/') + chunks = path.split('/') + chunks.pop(0) # empty item + return self.application.traverse(chunks, path) + + def process_http_error(self, e): + """ create the response body and headers for errors + """ + headers = {'Content-Type': 'text/html'} # XXX need more headers here? + if e.status in [301, 302]: + headers['Location'] = e.data + body = 'Redirecting to %s' % (e.data,) + else: + message, explain = self.responses[e.status] + body = self.error_message_format % {'code': e.status, 'message': message, + 'explain': explain} + return headers, body + + def response(self, status, headers, body, send_body=True): + """ generate the HTTP response and send it to the client + """ + self.send_response(status) + if (isinstance(body, str) and + not 'content-length' in [k.lower() for k in headers]): + headers['Content-Length'] = len(body) + for keyword, value in headers.iteritems(): + self.send_header(keyword, value) + self.end_headers() + if not send_body: + return + if isinstance(body, str): + self.wfile.write(body) + elif hasattr(body, 'read'): + while 1: + data = body.read(self.bufsize) + if data == '': + break + self.wfile.write(data) + else: + raise ValueError('body is not a plain string or file-like object') Added: pypy/branch/flex-backend/pypy/translator/flex/lib/support.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/support.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,48 @@ + +""" Various simple support functions +""" + +from pypy.rpython.ootypesystem.bltregistry import described, load_dict_args,\ + MethodDesc + +from pypy.rpython.extfunc import genericcallable + +def callback(retval=None, args={}): + """ Variant of described decorator, which flows + an additional argument with return value of decorated + function, used specifically for callbacks + """ + def decorator(func): + defs = func.func_defaults + if defs is None: + defs = () + vars = func.func_code.co_varnames[:func.func_code.co_argcount] + if isinstance(args, dict): + arg_list = load_dict_args(vars, defs, args) + else: + arg_list = args + arg_list.append(("callback", genericcallable(args=[retval]))) + func._method = (func.__name__, MethodDesc(arg_list, retval)) + return func + + return decorator + +import sys, new +from pypy.translator.js.main import rpython2javascript + +def js_source(functions, use_pdb=True): + mod = new.module('_js_src') + function_names = [] + for func in functions: + name = func.__name__ + if hasattr(mod, name): + raise ValueError("exported function name %r is duplicated" + % (name,)) + mod.__dict__[name] = func + function_names.append(name) + sys.modules['_js_src'] = mod + try: + return rpython2javascript(mod, function_names, use_pdb=use_pdb) + finally: + del sys.modules['_js_src'] + Added: pypy/branch/flex-backend/pypy/translator/flex/lib/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_server.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_server.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,77 @@ + +""" Server testing +""" + +import py +from pypy.translator.js.lib import server +from urllib import URLopener +import os + +class Handler(server.TestHandler): + def index(self): + return "xxx" + index.exposed = True + +def test_basic_startup(): + import thread + # XXX: how to do this without threads? + httpd = server.HTTPServer(('127.0.0.1', 21210), Handler) + thread.start_new_thread(httpd.serve_forever, ()) + assert URLopener().open("http://127.0.0.1:21210/index").read() == "xxx" + assert URLopener().open("http://127.0.0.1:21210/").read() == "xxx" + +def test_own_startup(): + httpd = server.create_server(server_address=('127.0.0.1', 21211), + handler=Handler) + server.start_server_in_new_thread(httpd) + assert URLopener().open("http://127.0.0.1:21210/index").read() == "xxx" + +def test_static_page(): + import thread + tmpdir = py.test.ensuretemp("server_static_page") + tmpdir.ensure("test.html").write("") + + class StaticHandler(server.TestHandler): + static_dir = str(tmpdir) + index = server.Static(os.path.join(static_dir, "test.html")) + + httpd = server.HTTPServer(('127.0.0.1', 21212), StaticHandler) + thread.start_new_thread(httpd.serve_forever, ()) + assert URLopener().open("http://127.0.0.1:21212/index").read() == \ + "" + +def test_static_page_implicit(): + import thread + tmpdir = py.test.ensuretemp("server_static_page_implicit") + tmpdir.ensure("index.html").write("") + + class StaticHandler(server.TestHandler): + static_dir = str(tmpdir) + index = server.Static() + + server.patch_handler(StaticHandler) + httpd = server.HTTPServer(('127.0.0.1', 21213), StaticHandler) + thread.start_new_thread(httpd.serve_forever, ()) + assert URLopener().open("http://127.0.0.1:21213/index").read() == \ + "" + + +def test_static_directory(): + py.test.skip("Fails") + import thread + tmpdir = py.test.ensuretemp("server_static_dir") + tmpdir.ensure("a", dir=1) + tmpdir.join("a").ensure("a.txt").write("aaa") + tmpdir.join("a").ensure("b.txt").write("bbb") + + class StaticDir(server.Handler): + static_dir = tmpdir + a_dir = server.StaticDir(tmpdir.join("a")) + + httpd = server.HTTPServer(('127.0.0.1', 0), StaticDir) + port = httpd.server_port + thread.start_new_thread(httpd.serve_forever, ()) + addr = "http://127.0.0.1:%d/" % port + assert URLopener().open(addr + "a_dir/a.txt").read() == "aaa" + assert URLopener().open(addr + "a_dir/b.txt").read() == "bbb" + Added: pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_server_g.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_server_g.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,132 @@ +import py +from pypy.translator.js.lib.server import * + +class NonInitHandler(NewHandler): + request_version = '1.0' + + def __init__(self): + pass + + def log_request(self, code='-', size='-'): + pass + +class SomePage(object): + """ test resource """ + exposed = True + def __call__(self, handler, path, query): + return ('text/plain', 'foo') + +def raising_page(handler, path, query): + xxx +raising_page.exposed = True + +def build_app_structure(): + app = Collection() + app.sub = Collection() + app.sub = Collection() + app.sub.index = SomePage() + app.error = raising_page + return app + +class TestCollection(object): + def test_traverse(self): + app = build_app_structure() + assert app.traverse(['sub', 'index'], '/sub/index') is app.sub.index + assert app.traverse(['sub', ''], '/sub/') is app.sub.index + try: + app.traverse(['sub'], '/sub') + except HTTPError, e: + assert e.status == 301 + assert e.data == '/sub/' + else: + py.test.fail('should have redirected') + # 404 errors (first -> no index) + py.test.raises(HTTPError, "app.traverse([''], '/')") + py.test.raises(HTTPError, "app.traverse(['other', ''], '/other/')") + +class TestResource(object): + pass + +class TestHandler(object): + def setup_method(self, method): + self.handler = NonInitHandler() + self.handler.wfile = self.wfile = py.std.StringIO.StringIO() + + def test_process_path(self): + path, query = self.handler.process_path('') + assert path == '' + assert query == '' + + path, query = self.handler.process_path('/foo') + assert path == '/foo' + assert query == '' + + path, query = self.handler.process_path('/foo?bar') + assert path == '/foo' + assert query == 'bar' + + py.test.raises(ValueError, "self.handler.process_path('/foo?bar?baz')") + + def test_find_resource(self): + app = build_app_structure() + self.handler.application = app + assert self.handler.find_resource('/sub/index') is app.sub.index + assert self.handler.find_resource('/sub/') is app.sub.index + try: + self.handler.find_resource('/sub') + except HTTPError, e: + assert e.status == 301 + assert e.data == '/sub/' + else: + py.test.raises('should have raised a redirect') + try: + self.handler.find_resource('') + except HTTPError, e: + assert e.status == 301 + assert e.data == '/' + else: + py.test.raises('should have raised a redirect') + py.test.raises(HTTPError, "self.handler.find_resource('/foo/')") + + def test_response(self): + self.handler.response(200, {'Content-Type': 'text/plain'}, 'foo') + response = self.wfile.getvalue() + assert response.startswith('HTTP/1.0 200 OK') + assert 'Content-Type: text/plain\r\n' in response + assert 'Content-Length: 3\r\n' in response + assert response.endswith('\r\n\r\nfoo') + + def test_get_response_file(self): + rfile = py.std.StringIO.StringIO() + rfile.write('foo\nbar\nbaz') + rfile.seek(0) + self.handler.response(200, {'Content-Type': 'text/plain'}, rfile) + response = self.wfile.getvalue() + assert response.endswith('\r\n\r\nfoo\nbar\nbaz') + + def test_get_response_wrong_body(self): + py.test.raises(ValueError, "self.handler.response(200, {}, u'xxx')") + + def test_handle_error(self): + l = [] + def f(*args): + l.append(args) + app = build_app_structure() + self.handler.application = app + self.handler.path = '/error' + self.handler.do_GET() + self.handler.application.handle_error = f + self.handler.do_GET() + assert len(l) == 1 + +class TestFsFile(object): + def test_call(self): + temp = py.test.ensuretemp('TestStaticResource.test_handle') + foo = temp.ensure('foo.txt') + foo.write('foo') + r = FsFile(foo, 'text/plain') + ret = r()#None, '/bar/foo.txt', '') + assert ret[0] == {'Content-Type': 'text/plain'} + assert ret[1] == 'foo' + + Added: pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_support.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_support.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,15 @@ + +""" Tests for support module +""" + +from pypy.translator.js.lib import support + +def test_callback(): + @support.callback(retval=int) + def f(self, a=8, b=3.2): + pass + + methdesc = f._method[1] + assert len(methdesc.args) == 3 + assert methdesc.args[2].name == 'callback' + Added: pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_url.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/test/test_url.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,11 @@ + +from pypy.translator.js.lib.url import parse_url + +def test_url(): + assert parse_url("path") == (["path"], {}) + assert parse_url("a/b/c/d") == (["a", "b", "c", "d"], {}) + assert parse_url("/a/b") == (["a", "b"], {}) + assert parse_url("/a/b/c/") == (["a", "b", "c"], {}) + assert parse_url("a/b?q=a&c=z") == (["a","b"], {"q":"a", "c":"z"}) + got = parse_url('/get_message?sid=2ed&pid=-1') + assert got == (["get_message"], {'pid':'-1', 'sid':'2ed'}) Added: pypy/branch/flex-backend/pypy/translator/flex/lib/url.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/lib/url.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,45 @@ + +""" Some support files for mapping urls, mostly bindings +for existing cgi stuff +""" + +import cgi +import urllib + +class URL(object): + def __init__(self, path, vars): + self.path = path + self.vars = vars + + def __eq__(self, other): + if isinstance(other, URL): + return self.path == other.path and self.vars == other.vars + if isinstance(other, tuple): + if len(other) != 2: + return False + return self.path, self.vars == other + return False + + def __ne__(self, other): + return not self == other + + def __iter__(self): + return iter((self.path, self.vars)) + +def parse_url(path): + """ Parse a/b/c?q=a into ('a', 'b', 'c') {'q':'a'} + """ + if '?' in path: + path, var_str = path.split("?") + vars_orig = cgi.parse_qs(var_str) + # if vars has a list inside... + vars = {} + for i, v in vars_orig.items(): + if isinstance(v, list): + vars[i] = v[0] + else: + vars[i] = v + else: + vars = {} + parts = [urllib.unquote(i) for i in path.split("/") if i] + return URL(parts, vars) Added: pypy/branch/flex-backend/pypy/translator/flex/log.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/log.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,4 @@ +import py +from pypy.tool.ansi_print import ansi_log +log = py.log.Producer("flex") +py.log.setconsumer("flex", ansi_log) Added: pypy/branch/flex-backend/pypy/translator/flex/main.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/main.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,166 @@ +"""Contains high level javascript compilation function +""" + +import autopath + +#from pypy.translator.flex.test.runtest import compile_function +#from pypy.translator.translator import TranslationContext +from pypy.translator.driver import TranslationDriver +from pypy.translator.flex.js import JS +from pypy.tool.error import AnnotatorError, FlowingError, debug +from pypy.rlib.nonconst import NonConstant +from pypy.annotation.policy import AnnotatorPolicy +from py.compat import optparse +from pypy.config.config import OptionDescription, BoolOption, StrOption +from pypy.config.config import Config, to_optparse +import py +import sys + +js_optiondescr = OptionDescription("jscompile", "", [ + BoolOption("view", "View flow graphs", + default=False, cmdline="--view"), + BoolOption("use_pdb", "Use debugger", + default=False, cmdline="--pdb"), + StrOption("output", "File to save results (default output.mxml)", + default="output.mxml", cmdline="--output")]) + + +class FunctionNotFound(Exception): + pass + +class BadSignature(Exception): + pass + +class JsPolicy(AnnotatorPolicy): + allow_someobjects = False + +def get_args(func_data): + l = [] + for i in xrange(func_data.func_code.co_argcount): + l.append("NonConstant(%s)" % repr(func_data.func_defaults[i])) + return ",".join(l) + +def get_arg_names(func_data): + return ",".join(func_data.func_code.co_varnames\ + [:func_data.func_code.co_argcount]) + +def rpython2javascript_main(argv, jsconfig): + if len(argv) == 0: + print "usage: module " + sys.exit(0) + module_name = argv[0] + if not module_name.endswith('.py'): + module_name += ".py" + mod = py.path.local(module_name).pyimport() + if len(argv) == 1: + function_names = [] + for function_name in dir(mod): + function = getattr(mod, function_name) + if callable(function) and getattr(function, '_client', False): + function_names.append( function_name ) + if not function_names: + print "Cannot find any function with _client=True in %s"\ + % module_name + sys.exit(1) + else: + function_names = argv[1:] + source = rpython2javascript(mod, function_names, jsconfig=jsconfig) + if not source: + print "Exiting, source not generated" + sys.exit(1) + open(jsconfig.output, "w").write(source) + print "Written file %s" % jsconfig.output + +# some strange function source +source_ssf_base = """ + +import %(module_name)s +from pypy.translator.flex.helper import __show_traceback +from pypy.rlib.nonconst import NonConstant as NonConst + +%(function_defs)s + +def some_strange_function_which_will_never_be_called(): + +%(functions)s +""" + +wrapped_function_def_base = """ +def %(fun_name)s(%(arg_names)s): + try: + traceback_handler.enter(NonConst("entrypoint"), NonConst("()"), NonConst(""), NonConst(0)) + %(module_name)s.%(fun_name)s(%(arg_names)s) + traceback_handler.leave(NonConst("entrypoint")) + except Exception, e: + new_tb = traceback_handler.tb[:] + __show_traceback(new_tb, str(e)) + +%(fun_name)s.explicit_traceback = True +""" + +function_base = "%(module_name)s.%(fun_name)s(%(args)s)" +wrapped_function_base = "%(fun_name)s(%(args)s)" + +def get_source_ssf(mod, module_name, function_names): + #source_ssf = "\n".join(["import %s" % module_name, "def some_strange_function_which_will_never_be_called():"] + [" "+\ + # module_name+"."+fun_name+get_args(mod.__dict__[fun_name]) for fun_name in function_names]) + function_list = [] + function_def_list = [] + for fun_name in function_names: + args = get_args(mod.__dict__[fun_name]) + arg_names = get_arg_names(mod.__dict__[fun_name]) + base = function_base + function_list.append(py.code.Source(base % locals())) + function_defs = "\n\n".join([str(i) for i in function_def_list]) + functions = "\n".join([str(i.indent()) for i in function_list]) + retval = source_ssf_base % locals() + print retval + return retval + +def rpython2javascript(mod, function_names, jsconfig=None, use_pdb=False): + if isinstance(function_names, str): + function_names = [function_names] + # avoid confusion + if mod is None: + # this means actual module, which is quite hairy to get in python, + # so we cheat + import sys + mod = sys.modules[sys._getframe(1).f_globals['__name__']] + + if jsconfig is None: + jsconfig = Config(js_optiondescr) + if use_pdb: + jsconfig.use_pdb = True + module_name = mod.__name__ + if not function_names and 'main' in mod.__dict__: + function_names.append('main') + for func_name in function_names: + if func_name not in mod.__dict__: + raise FunctionNotFound("function %r was not found in module %r" % (func_name, module_name)) + func_code = mod.__dict__[func_name] + if func_code.func_defaults: + lgt = len(func_code.func_defaults) + else: + lgt = 0 + if func_code.func_code.co_argcount > 0 and func_code.func_code. \ + co_argcount != lgt: + raise BadSignature("Function %s does not have default arguments" % func_name) + source_ssf = get_source_ssf(mod, module_name, function_names) + exec(source_ssf) in globals() + # now we gonna just cut off not needed function + # XXX: Really do that + #options = optparse.Values(defaults=DEFAULT_OPTIONS) + from pypy.config.pypyoption import get_pypy_config + config = get_pypy_config(translating=True) + driver = TranslationDriver(config=config) + try: + driver.setup(some_strange_function_which_will_never_be_called, [], policy = JsPolicy()) + driver.proceed(["compile_flex"]) + if jsconfig.view: + driver.translator.view() + return driver.gen.tmpfile.open().read() + # XXX: Add some possibility to write down selected file + except Exception, e: + # do something nice with it + raise + debug(driver, use_pdb) Added: pypy/branch/flex-backend/pypy/translator/flex/metavm.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/metavm.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,297 @@ +""" Opcode meaning objects, descendants of MicroInstruction +""" + +#from pypy.translator.js.jsbuiltin import Builtins +from pypy.translator.oosupport.metavm import PushArg, PushAllArgs, StoreResult,\ + InstructionList, New, GetField, MicroInstruction + +from pypy.translator.flex.log import log +from pypy.rpython.ootypesystem import ootype +from types import FunctionType +from pypy.objspace.flow.model import Constant + +class NewBuiltin(MicroInstruction): + def __init__(self, arg): + self.arg = arg + + def render(self, generator, op): + generator.ilasm.new(self.arg) + +class _ListSetitem(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[1]) + generator.load(op.args[3]) + generator.load(op.args[2]) + generator.ilasm.list_setitem() +ListSetitem = _ListSetitem() + +class _ListGetitem(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[1]) + generator.load(op.args[2]) + generator.ilasm.list_getitem() +ListGetitem = _ListGetitem() + +class _ListContains(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[1]) + generator.load(op.args[2]) + generator.ilasm.list_getitem() + generator.ilasm.load_void() + generator.emit('!=') +ListContains = _ListContains() + +class _Call(MicroInstruction): + def render(self, generator, op): + graph = op.args[0].value.graph + self._render_function(generator, graph, op.args) + + def _render_builtin(self, generator, builtin, args): + for func_arg in args[1:]: # push parameters + generator.load(func_arg) + generator.call_external(builtin, args[1:]) + + def _render_builtin_prepared_args(self, generator, builtin, args): + for func_arg in args: + generator.load_str(func_arg) + generator.call_external(builtin, args) + + def _render_builtin_method(self, generator, builtin, args): + for func_arg in args: + generator.load_special(func_arg) + generator.call_external_method(builtin, len(args)-1) + + def _render_function(self, generator, graph, args): + for func_arg in args[1:]: # push parameters + if func_arg.concretetype is not ootype.Void: + generator.load(func_arg) + generator.call_graph(graph) + + def _render_method(self, generator, method_name, args): + this = args[0] + for arg in args: # push parametes + generator.load(arg) + generator.call_method(this.concretetype, method_name) + +Call = _Call() + +class CallBuiltin(_Call): + def __init__(self, builtin): + self.builtin = builtin + + def render(self, generator, op): + self._render_builtin(generator, self.builtin, op.args) + +class CallBuiltinMethod(_Call): + def __init__(self, builtin, slice=None, additional_args=[]): + self.builtin = builtin + self.slice = slice + self.additional_args = additional_args + + def render(self, generator, op): + if self.slice is not None: + args = op.args[self.slice] + else: + args = op.args + args += self.additional_args + self._render_builtin_method(generator, self.builtin, args) + +class _SameAs(MicroInstruction): + def render(self, generator, op): + generator.change_name(op.result, op.args[0]) + +class _CastFun(MicroInstruction): + def __init__(self, name, num): + self.name = name + self.num = num + + def render(self, generator, op): + log("Args: %r"%op.args) + generator.cast_function(self.name, self.num) + +class _Prefix(MicroInstruction): + def __init__(self, st): + self.st = st + + def render(self, generator, op): + generator.prefix_op(self.st) + +class _NotImplemented(MicroInstruction): + def __init__(self, reason): + self.reason = reason + + def render(self, generator, op): + raise NotImplementedError(self.reason) + +class _CastMethod(MicroInstruction): + def __init__(self, method_name, num=0): + self.method_name = method_name + self.num = num + + def render(self, generator, op): + generator.call_external_method(self.method_name, self.num) + +class _LoadConst(MicroInstruction): + def __init__(self, value): + self.value = value + + def render(self, generator, op): + generator.load(Constant(self.value, ootype.typeOf(self.value))) + +class _GetBuiltinField(MicroInstruction): + def render(self, generator, op): + this = op.args[0] + field = op.args[1].value[1:] + generator.load(this) + generator.get_field(None, field) + +class _GetPredefinedField(MicroInstruction): + def __init__(self, field): + self.field = field + + def render(self, generator, op): + this = op.args[1] + generator.load(this) + generator.get_field(None, self.field) + +GetBuiltinField = _GetBuiltinField() + +class _SetBuiltinField(MicroInstruction): + def render(self, generator, op): + this = op.args[0] + field = op.args[1].value + if not field.startswith('o'): + generator.load_void() + else: + value = op.args[2] + field_name = field[1:] + self.run_it(generator, this, field_name, value) + + def run_it(self, generator, this, field_name, value): + generator.load(this) + generator.load_special(value) + generator.set_field(None, field_name) + +class _SetPredefinedField(_SetBuiltinField): + def __init__(self, field): + self.field = field + + def render(self, generator, op): + value = op.args[2] + this = op.args[1] + self.run_it(generator, this, self.field, value) + +class _SetExternalField(_SetBuiltinField): + def render(self, generator, op): + self.run_it(generator, op.args[0], op.args[1].value, op.args[2]) + +SetBuiltinField = _SetBuiltinField() +SetExternalField = _SetExternalField() + +class _CallMethod(_Call): + def render(self, generator, op): + method = op.args[0] + self._render_method(generator, method.value, op.args[1:]) + +class _CallBuiltinObject(_Call): + def render(self, generator, op): + this = op.args[1].concretetype + method = op.args[0] + method_name = this._methods[method.value]._name[1:] + generator.load(op.args[1]) + self._render_builtin_method(generator, method_name, op.args[1:]) + +class _CallExternalObject(_Call): + def render(self, generator, op): + this = op.args[1].concretetype + method = op.args[0] + method_name = method.value + #generator.load(op.args[1]) + self._render_builtin_method(generator, method_name, op.args[1:]) + +CallBuiltinObject = _CallBuiltinObject() +CallExternalObject = _CallExternalObject() + +class _IsInstance(MicroInstruction): + def render(self, generator, op): + # FIXME: just temporary hack + generator.load(op.args[0]) + generator.ilasm.load_const(op.args[1].value._name.replace('.', '_'))#[-1]) + generator.cast_function("isinstanceof", 2) + +class _IndirectCall(MicroInstruction): + def render(self, generator, op): + for func_arg in op.args[1:]: # push parameters + generator.load(func_arg) + generator.call_external(op.args[0].name, op.args[1:]) + +class _SetTimeout(MicroInstruction): + # FIXME: Dirty hack for javascript callback stuff + def render(self, generator, op): + val = op.args[1].value + assert(isinstance(val, ootype._static_meth)) + #if isinstance(val, ootype.StaticMethod): + real_name = val._name + generator.db.pending_function(val.graph) + #generator.db.pending_function(val.graph) + #else: + # concrete = val.concretize() + # real_name = concrete.value._name + # generator.db.pending_function(concrete.value.graph) + generator.load_str("'%s()'" % real_name) + generator.load(op.args[2]) + generator.call_external('setTimeout',[0]*2) + +class _DiscardStack(MicroInstruction): + def render(self, generator, op): + generator.clean_stack() + +class SetOnEvent(MicroInstruction): + def __init__(self, field): + self.field = field + + # FIXME: Dirty hack for javascript callback stuff + def render(self, generator, op): + val = op.args[1].value + val = val.concretize().value + assert(isinstance(val, ootype._static_meth)) + real_name = val._name + generator.db.pending_function(val.graph) + generator.load_str("document") + generator.load_str(real_name) + generator.set_field(None, self.field) + +class _CheckLength(MicroInstruction): + def render(self, generator, op): + assert not generator.ilasm.right_hand + +class _ListRemove(MicroInstruction): + def render(self, generator, op): + generator.list_getitem(op.args[1], op.args[2]) + generator.call_external('delete', [0]) + +ListRemove = _ListRemove() +CheckLength = _CheckLength() +SetTimeout = _SetTimeout() +IndirectCall = _IndirectCall() +IsInstance = _IsInstance() +CallMethod = _CallMethod() +CopyName = [PushAllArgs, _SameAs ()] +CastString = _CastFun("convertToString", 1) +SameAs = CopyName +DiscardStack = _DiscardStack() + +def fix_opcodes(opcodes): + for key, value in opcodes.iteritems(): + if type(value) is str: + value = InstructionList([PushAllArgs, value, StoreResult, CheckLength]) + elif value == []: + value = InstructionList([CheckLength]) + elif value is not None: + if StoreResult not in value: + value.append(StoreResult) + if CheckLength not in value: + value.append(CheckLength) + value = InstructionList(value) + + opcodes[key] = value Added: pypy/branch/flex-backend/pypy/translator/flex/modules/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/__init__.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,3 @@ + +""" Support classes needed for javascript to translate +""" Added: pypy/branch/flex-backend/pypy/translator/flex/modules/dom.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/dom.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,815 @@ + +"""Document Object Model support + + this provides a mock browser API, both the standard DOM level 2 stuff as + the browser-specific additions + + in addition this provides the necessary descriptions that allow rpython + code that calls the browser DOM API to be translated + + note that the API is not and will not be complete: more exotic features + will most probably not behave as expected, or are not implemented at all + + http://www.w3.org/DOM/ - main standard + http://www.w3schools.com/dhtml/dhtml_dom.asp - more informal stuff + http://developer.mozilla.org/en/docs/Gecko_DOM_Reference - Gecko reference +""" + +import time +import re +import urllib +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.rlib.nonconst import NonConstant + +from pypy.rpython.extfunc import genericcallable, register_external +from xml.dom import minidom + +from pypy.annotation.signature import annotation +from pypy.annotation import model as annmodel + +# EventTarget is the base class for Nodes and Window +class EventTarget(BasicExternal): + def addEventListener(self, type, listener, useCapture): + if not hasattr(self._original, '_events'): + self._original._events = [] + # XXX note that useCapture is ignored... + self._original._events.append((type, listener, useCapture)) + + def dispatchEvent(self, event): + if event._cancelled: + return + event.currentTarget = self + if event.target is None: + event.target = self + if event.originalTarget is None: + event.originalTarget = self + if hasattr(self._original, '_events'): + for etype, handler, capture in self._original._events: + if etype == event.type: + handler(event) + if event._cancelled or event.cancelBubble: + return + parent = getattr(self, 'parentNode', None) + if parent is not None: + parent.dispatchEvent(event) + + def removeEventListener(self, type, listener, useCapture): + if not hasattr(self._original, '_events'): + raise ValueError('no registration for listener') + filtered = [] + for data in self._original._events: + if data != (type, listener, useCapture): + filtered.append(data) + if filtered == self._original._events: + raise ValueError('no registration for listener') + self._original._events = filtered + +# XML node (level 2 basically) implementation +# the following classes are mostly wrappers around minidom nodes that try to +# mimic HTML DOM behaviour by implementing browser API and changing the +# behaviour a bit + +class Node(EventTarget): + """base class of all node types""" + _original = None + + def __init__(self, node=None): + self._original = node + + def __getattr__(self, name): + """attribute access gets proxied to the contained minidom node + + all returned minidom nodes are wrapped as Nodes + """ + try: + return super(Node, self).__getattr__(name) + except AttributeError: + pass + if (name not in self._fields and + (not hasattr(self, '_methods') or name not in self._methods)): + raise NameError, name + value = getattr(self._original, name) + return _wrap(value) + + def __setattr__(self, name, value): + """set an attribute on the wrapped node""" + if name in dir(self) or name.startswith('_'): + return super(Node, self).__setattr__(name, value) + if name not in self._fields: + raise NameError, name + setattr(self._original, name, value) + + def __eq__(self, other): + original = getattr(other, '_original', other) + return original is self._original + + def __ne__(self, other): + original = getattr(other, '_original', other) + return original is not self._original + + def getElementsByTagName(self, name): + name = name.lower() + return self.__getattr__('getElementsByTagName')(name) + + def __repr__(self): + return '<%s %s>' % (self.__class__.__name__, self.nodeName) + + def _getClassName(self): + return self.getAttribute('class') + + def _setClassName(self, name): + self.setAttribute('class', name) + + className = property(_getClassName, _setClassName) + + def _getId(self): + return self.getAttribute('id') + + def _setId(self, id): + self.setAttribute('id', id) + + id = property(_getId, _setId) + +class Element(Node): + nodeType = 1 + style = None + + def _style(self): + style = getattr(self._original, '_style', None) + if style is not None: + return style + styles = {} + if self._original.hasAttribute('style'): + for t in self._original.getAttribute('style').split(';'): + name, value = t.split(':') + dashcharpairs = re.findall('-\w', name) + for p in dashcharpairs: + name = name.replace(p, p[1].upper()) + styles[name.strip()] = value.strip() + style = Style(styles) + self._original._style = style + return style + style = property(_style) + + def _nodeName(self): + return self._original.nodeName.upper() + nodeName = property(_nodeName) + + def _get_innerHTML(self): + ret = [] + for child in self.childNodes: + ret.append(_serialize_html(child)) + return ''.join(ret) + + def _set_innerHTML(self, html): + dom = minidom.parseString('%s' % (html,)) + while self.childNodes: + self.removeChild(self.lastChild) + for child in dom.documentElement.childNodes: + child = self.ownerDocument.importNode(child, True) + self._original.appendChild(child) + del dom + + innerHTML = property(_get_innerHTML, _set_innerHTML) + + def scrollIntoView(self): + pass + +class Attribute(Node): + nodeType = 2 + +class Text(Node): + nodeType = 3 + +class Comment(Node): + nodeType = 8 + +class Document(Node): + nodeType = 9 + + def createEvent(self, group=''): + """create an event + + note that the group argument is ignored + """ + if group in ('KeyboardEvent', 'KeyboardEvents'): + return KeyEvent() + elif group in ('MouseEvent', 'MouseEvents'): + return MouseEvent() + return Event() + + def getElementById(self, id): + nodes = self.getElementsByTagName('*') + for node in nodes: + if node.getAttribute('id') == id: + return node + +# the standard DOM stuff that doesn't directly deal with XML +# note that we're mimicking the standard (Mozilla) APIs, so things tested +# against this code may not work in Internet Explorer + +# XXX note that we store the events on the wrapped minidom node to avoid losing +# them on re-wrapping +class Event(BasicExternal): + def initEvent(self, type, bubbles, cancelable): + self.type = type + self.cancelBubble = not bubbles + self.cancelable = cancelable + self.target = None + self.currentTarget = None + self.originalTarget = None + self._cancelled = False + + def preventDefault(self): + if not self.cancelable: + raise TypeError('event can not be canceled') + self._cancelled = True + + def stopPropagation(self): + self.cancelBubble = True + +class KeyEvent(Event): + pass + +class MouseEvent(Event): + pass + +class Style(BasicExternal): + def __init__(self, styles={}): + for name, value in styles.iteritems(): + setattr(self, name, value) + + def __getattr__(self, name): + if name not in self._fields: + raise AttributeError, name + return None + + def _tostring(self): + ret = [] + for name in sorted(self._fields): + value = getattr(self, name, None) + if value is not None: + ret.append(' ') + for char in name: + if char.upper() == char: + char = '-%s' % (char.lower(),) + ret.append(char) + ret.append(': %s;' % (value,)) + return ''.join(ret[1:]) + +# non-DOM ('DOM level 0') stuff + +# Window is the main environment, the root node of the JS object tree + +class Location(BasicExternal): + _fields = { + 'hostname' : str, + 'href' : str, + 'hash' : str, + 'host' : str, + 'pathname' : str, + 'port' : str, + 'protocol' : str, + 'search' : str, + } + _methods = { + 'assign' : MethodDesc([str]), + 'reload' : MethodDesc([bool]), + 'replace' : MethodDesc([str]), + 'toString' : MethodDesc([], str), + } + +class Navigator(BasicExternal): + def __init__(self): + self.appName = 'Netscape' + +class Window(EventTarget): + def __init__(self, html=('Untitled document' + ''), parent=None): + super(Window, self).__init__() + self._html = html + self.document = Document(minidom.parseString(html)) + + # references to windows + self.content = self + self.self = self + self.window = self + self.parent = parent or self + self.top = self.parent + while 1: + if self.top.parent is self.top: + break + self.top = self.top.parent + + # other properties + self.closed = True + self._location = 'about:blank' + + self._original = self # for EventTarget interface (XXX a bit nasty) + + self.navigator = Navigator() + + def __getattr__(self, name): + return globals()[name] + + def _getLocation(self): + return self._location + + def _setLocation(self, newloc): + url = urllib.urlopen(newloc) + html = url.read() + self.document = Document(minidom.parseString(html)) + + location = property(_getLocation, _setLocation) + +scrollX = 0 +scrollMaxX = 0 +scrollY = 0 +scrollMaxY = 0 + +def some_fun(): + pass + +def setTimeout(func, delay): + pass +register_external(setTimeout, args=[genericcallable([]), int], result=None) + +window = Window() +document = window.document +Window._render_name = 'window' +Document._render_name = 'document' + +# rtyper stuff + +EventTarget._fields = { + 'onabort' : genericcallable([Event]), + 'onblur' : genericcallable([Event]), + 'onchange' : genericcallable([Event]), + 'onclick' : genericcallable([MouseEvent]), + 'onclose' : genericcallable([MouseEvent]), + 'ondblclick' : genericcallable([MouseEvent]), + 'ondragdrop' : genericcallable([MouseEvent]), + 'onerror' : genericcallable([MouseEvent]), + 'onfocus' : genericcallable([Event]), + 'onkeydown' : genericcallable([KeyEvent]), + 'onkeypress' : genericcallable([KeyEvent]), + 'onkeyup' : genericcallable([KeyEvent]), + 'onload' : genericcallable([KeyEvent]), + 'onmousedown' : genericcallable([MouseEvent]), + 'onmousemove' : genericcallable([MouseEvent]), + 'onmouseup' : genericcallable([MouseEvent]), + 'onmouseover' : genericcallable([MouseEvent]), + 'onresize' : genericcallable([Event]), + 'onscroll' : genericcallable([MouseEvent]), + 'onselect' : genericcallable([MouseEvent]), + 'onsubmit' : genericcallable([MouseEvent]), + 'onunload' : genericcallable([Event]), +} + +lambda_returning_true = genericcallable([Event]) + +EventTarget._methods = { + 'addEventListener' : MethodDesc([str, lambda_returning_true, bool]), + 'dispatchEvent' : MethodDesc([str], bool), + 'removeEventListener' : MethodDesc([str, lambda_returning_true, bool]), +} + +Node._fields = EventTarget._fields.copy() +Node._fields.update({ + 'childNodes' : [Element], + 'firstChild' : Element, + 'lastChild' : Element, + 'localName' : str, + 'name' : str, + 'namespaceURI' : str, + 'nextSibling' : Element, + 'nodeName' : str, + 'nodeType' : int, + 'nodeValue' : str, + 'ownerDocument' : Document, + 'parentNode' : Element, + 'prefix' : str, + 'previousSibling': Element, + 'tagName' : str, + 'textContent' : str, +}) + +Node._methods = EventTarget._methods.copy() +Node._methods.update({ + 'appendChild' : MethodDesc([Element]), + 'cloneNode' : MethodDesc([int], Element), + 'getElementsByTagName' : MethodDesc([str], [Element]), + 'hasChildNodes' : MethodDesc([], bool), + 'insertBefore' : MethodDesc([Element], Element), + 'normalize' : MethodDesc([]), + 'removeChild' : MethodDesc([Element]), + 'replaceChild' : MethodDesc([Element], Element), +}) + +Element._fields = Node._fields.copy() +Element._fields.update({ + 'attributes' : [Attribute], + 'className' : str, + 'clientHeight' : int, + 'clientWidth' : int, + 'clientLeft' : int, + 'clientTop' : int, + 'dir' : str, + 'innerHTML' : str, + 'id' : str, + 'lang' : str, + 'offsetHeight' : int, + 'offsetLeft' : int, + 'offsetParent' : int, + 'offsetTop' : int, + 'offsetWidth' : int, + 'scrollHeight' : int, + 'scrollLeft' : int, + 'scrollTop' : int, + 'scrollWidth' : int, + 'disabled': bool, + # HTML specific + 'style' : Style, + 'tabIndex' : int, + # XXX: From HTMLInputElement to make pythonconsole work. + 'value': str, + 'checked': bool, + # IMG specific + 'src': str, +}) + +Element._methods = Node._methods.copy() +Element._methods.update({ + 'getAttribute' : MethodDesc([str], str), + 'getAttributeNS' : MethodDesc([str], str), + 'getAttributeNode' : MethodDesc([str], Element), + 'getAttributeNodeNS' : MethodDesc([str], Element), + 'hasAttribute' : MethodDesc([str], bool), + 'hasAttributeNS' : MethodDesc([str], bool), + 'hasAttributes' : MethodDesc([], bool), + 'removeAttribute' : MethodDesc([str]), + 'removeAttributeNS' : MethodDesc([str]), + 'removeAttributeNode' : MethodDesc([Element], str), + 'setAttribute' : MethodDesc([str, str]), + 'setAttributeNS' : MethodDesc([str]), + 'setAttributeNode' : MethodDesc([Element], Element), + 'setAttributeNodeNS' : MethodDesc([str, Element], Element), + # HTML specific + 'blur' : MethodDesc([]), + 'click' : MethodDesc([]), + 'focus' : MethodDesc([]), + 'scrollIntoView' : MethodDesc([]), + 'supports' : MethodDesc([str, float]), +}) + +Document._fields = Node._fields.copy() +Document._fields.update({ + 'characterSet' : str, + # 'contentWindow' : Window(), XXX doesn't exist, only on iframe + 'doctype' : str, + 'documentElement' : Element, + 'styleSheets' : [Style], + 'alinkColor' : str, + 'bgColor' : str, + 'body' : Element, + 'cookie' : str, + 'defaultView' : Window, + 'domain' : str, + 'embeds' : [Element], + 'fgColor' : str, + 'forms' : [Element], + 'height' : int, + 'images' : [Element], + 'lastModified' : str, + 'linkColor' : str, + 'links' : [Element], + 'referrer' : str, + 'title' : str, + 'URL' : str, + 'vlinkColor' : str, + 'width' : int, +}) + +Document._methods = Node._methods.copy() +Document._methods.update({ + 'createAttribute' : MethodDesc([str], Element), + 'createDocumentFragment' : MethodDesc([], Element), + 'createElement' : MethodDesc([str], Element), + 'createElementNS' : MethodDesc([str], Element), + 'createEvent' : MethodDesc([str], Event), + 'createTextNode' : MethodDesc([str], Element), + #'createRange' : MethodDesc(["aa"], Range()) - don't know what to do here + 'getElementById' : MethodDesc([str], Element), + 'getElementsByName' : MethodDesc([str], [Element]), + 'importNode' : MethodDesc([Element, bool], Element), + 'clear' : MethodDesc([]), + 'close' : MethodDesc([]), + 'open' : MethodDesc([]), + 'write' : MethodDesc([str]), + 'writeln' : MethodDesc([str]), +}) + +Window._fields = EventTarget._fields.copy() +Window._fields.update({ + 'content' : Window, + 'closed' : bool, + # 'crypto' : Crypto() - not implemented in Gecko, leave alone + 'defaultStatus' : str, + 'document' : Document, + # 'frameElement' : - leave alone + 'frames' : [Window], + 'history' : [str], + 'innerHeight' : int, + 'innerWidth' : int, + 'length' : int, + 'location' : Location, + 'name' : str, + # 'preference' : # denied in gecko + 'opener' : Window, + 'outerHeight' : int, + 'outerWidth' : int, + 'pageXOffset' : int, + 'pageYOffset' : int, + 'parent' : Window, + # 'personalbar' : - disallowed + # 'screen' : Screen() - not part of the standard, allow it if you want + 'screenX' : int, + 'screenY' : int, + 'scrollMaxX' : int, + 'scrollMaxY' : int, + 'scrollX' : int, + 'scrollY' : int, + 'self' : Window, + 'status' : str, + 'top' : Window, + 'window' : Window, + 'navigator': Navigator, +}) + +Window._methods = Node._methods.copy() +Window._methods.update({ + 'alert' : MethodDesc([str]), + 'atob' : MethodDesc([str], str), + 'back' : MethodDesc([]), + 'blur' : MethodDesc([]), + 'btoa' : MethodDesc([str], str), + 'close' : MethodDesc([]), + 'confirm' : MethodDesc([str], bool), + 'dump' : MethodDesc([str]), + 'escape' : MethodDesc([str], str), + #'find' : MethodDesc(["aa"], - gecko only + 'focus' : MethodDesc([]), + 'forward' : MethodDesc([]), + 'getComputedStyle' : MethodDesc([Element, str], Style), + 'home' : MethodDesc([]), + 'open' : MethodDesc([str]), +}) + +Style._fields = { + 'azimuth' : str, + 'background' : str, + 'backgroundAttachment' : str, + 'backgroundColor' : str, + 'backgroundImage' : str, + 'backgroundPosition' : str, + 'backgroundRepeat' : str, + 'border' : str, + 'borderBottom' : str, + 'borderBottomColor' : str, + 'borderBottomStyle' : str, + 'borderBottomWidth' : str, + 'borderCollapse' : str, + 'borderColor' : str, + 'borderLeft' : str, + 'borderLeftColor' : str, + 'borderLeftStyle' : str, + 'borderLeftWidth' : str, + 'borderRight' : str, + 'borderRightColor' : str, + 'borderRightStyle' : str, + 'borderRightWidth' : str, + 'borderSpacing' : str, + 'borderStyle' : str, + 'borderTop' : str, + 'borderTopColor' : str, + 'borderTopStyle' : str, + 'borderTopWidth' : str, + 'borderWidth' : str, + 'bottom' : str, + 'captionSide' : str, + 'clear' : str, + 'clip' : str, + 'color' : str, + 'content' : str, + 'counterIncrement' : str, + 'counterReset' : str, + 'cssFloat' : str, + 'cssText' : str, + 'cue' : str, + 'cueAfter' : str, + 'onBefore' : str, + 'cursor' : str, + 'direction' : str, + 'displays' : str, + 'elevation' : str, + 'emptyCells' : str, + 'font' : str, + 'fontFamily' : str, + 'fontSize' : str, + 'fontSizeAdjust' : str, + 'fontStretch' : str, + 'fontStyle' : str, + 'fontVariant' : str, + 'fontWeight' : str, + 'height' : str, + 'left' : str, + 'length' : str, + 'letterSpacing' : str, + 'lineHeight' : str, + 'listStyle' : str, + 'listStyleImage' : str, + 'listStylePosition' : str, + 'listStyleType' : str, + 'margin' : str, + 'marginBottom' : str, + 'marginLeft' : str, + 'marginRight' : str, + 'marginTop' : str, + 'markerOffset' : str, + 'marks' : str, + 'maxHeight' : str, + 'maxWidth' : str, + 'minHeight' : str, + 'minWidth' : str, + 'MozBinding' : str, + 'MozOpacity' : str, + 'orphans' : str, + 'outline' : str, + 'outlineColor' : str, + 'outlineStyle' : str, + 'outlineWidth' : str, + 'overflow' : str, + 'padding' : str, + 'paddingBottom' : str, + 'paddingLeft' : str, + 'paddingRight' : str, + 'paddingTop' : str, + 'page' : str, + 'pageBreakAfter' : str, + 'pageBreakBefore' : str, + 'pageBreakInside' : str, + 'parentRule' : str, + 'pause' : str, + 'pauseAfter' : str, + 'pauseBefore' : str, + 'pitch' : str, + 'pitchRange' : str, + 'playDuring' : str, + 'position' : str, + 'quotes' : str, + 'richness' : str, + 'right' : str, + 'size' : str, + 'speak' : str, + 'speakHeader' : str, + 'speakNumeral' : str, + 'speakPunctuation' : str, + 'speechRate' : str, + 'stress' : str, + 'tableLayout' : str, + 'textAlign' : str, + 'textDecoration' : str, + 'textIndent' : str, + 'textShadow' : str, + 'textTransform' : str, + 'top' : str, + 'unicodeBidi' : str, + 'verticalAlign' : str, + 'visibility' : str, + 'voiceFamily' : str, + 'volume' : str, + 'whiteSpace' : str, + 'widows' : str, + 'width' : str, + 'wordSpacing' : str, + 'zIndex' : str, +} + +Event._fields = { + 'bubbles': bool, + 'cancelBubble': bool, + 'cancelable': bool, + 'currentTarget': Element, + 'detail': int, + 'relatedTarget': Element, + 'target': Element, + 'type': str, + 'returnValue': bool, + 'which': int, + 'keyCode' : int, + 'charCode': int, + 'altKey' : bool, + 'ctrlKey' : bool, + 'shiftKey': bool, +} + +Event._methods = { + 'initEvent': MethodDesc([str, bool, bool]), + 'preventDefault': MethodDesc([]), + 'stopPropagation': MethodDesc([]), +} + +KeyEvent._methods = Event._methods.copy() + +KeyEvent._fields = Event._fields.copy() + +Navigator._methods = { +} +Navigator._fields = { + 'appName': str, +} + +class _FunctionWrapper(object): + """makes sure function return values are wrapped if appropriate""" + def __init__(self, callable): + self._original = callable + + def __call__(self, *args, **kwargs): + args = list(args) + for i, arg in enumerate(args): + if isinstance(arg, Node): + args[i] = arg._original + for name, arg in kwargs.iteritems(): + if isinstance(arg, Node): + kwargs[arg] = arg._original + value = self._original(*args, **kwargs) + return _wrap(value) + +_typetoclass = { + 1: Element, + 2: Attribute, + 3: Text, + 8: Comment, + 9: Document, +} +def _wrap(value): + if isinstance(value, minidom.Node): + nodeclass = _typetoclass[value.nodeType] + return nodeclass(value) + elif callable(value): + return _FunctionWrapper(value) + # nothing fancier in minidom, i hope... + # XXX and please don't add anything fancier either ;) + elif isinstance(value, list): + return [_wrap(x) for x in value] + return value + +# some helper functions + +def _quote_html(text): + for char, e in [('&', 'amp'), ('<', 'lt'), ('>', 'gt'), ('"', 'quot'), + ("'", 'apos')]: + text = text.replace(char, '&%s;' % (e,)) + return text + +_singletons = ['link', 'meta'] +def _serialize_html(node): + ret = [] + if node.nodeType in [3, 8]: + return node.nodeValue + elif node.nodeType == 1: + original = getattr(node, '_original', node) + nodeName = original.nodeName + ret += ['<', nodeName] + if len(node.attributes): + for aname in node.attributes.keys(): + if aname == 'style': + continue + attr = node.attributes[aname] + ret.append(' %s="%s"' % (attr.nodeName, + _quote_html(attr.nodeValue))) + styles = getattr(original, '_style', None) + if styles: + ret.append(' style="%s"' % (_quote_html(styles._tostring()),)) + if len(node.childNodes) or nodeName not in _singletons: + ret.append('>') + for child in node.childNodes: + if child.nodeType == 1: + ret.append(_serialize_html(child)) + else: + ret.append(_quote_html(child.nodeValue)) + ret += [''] + else: + ret.append(' />') + else: + raise ValueError('unsupported node type %s' % (node.nodeType,)) + return ''.join(ret) + +def alert(msg): + window.alert(msg) + +# initialization + +# set the global 'window' instance to an empty HTML document, override using +# dom.window = Window(html) (this will also set dom.document) + Added: pypy/branch/flex-backend/pypy/translator/flex/modules/flex.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/flex.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,19 @@ +from pypy.rpython.extfunc import genericcallable, register_external +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc + + +class Button( BasicExternal ): + + _fields = { + 'label' : str, + '_x' : int, + '_y' : int, + } + _render_class = 'Button' + +def addChild(e): + pass + +register_external(addChild, args=[Button]) +#addChild._annspecialcase_ = 'specialize:argtype(0)' + Added: pypy/branch/flex-backend/pypy/translator/flex/modules/mochikit.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/mochikit.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,75 @@ + +""" mochikit wrappers +""" + +from pypy.rpython.extfunc import genericcallable, register_external +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.translator.js.modules import dom + +# MochiKit.LoggingPane + +def createLoggingPane(var): + pass +register_external(createLoggingPane, args=[bool]) + +# MochiKit.Logging + +def log(data): + print data +register_external(log, args=None) + +def logDebug(data): + print "D:", data +register_external(logDebug, args=None) + +def logWarning(data): + print "Warning:", data +register_external(logWarning, args=None) + +def logError(data): + print "ERROR:", data +register_external(logError, args=None) + +def logFatal(data): + print "FATAL:", data +register_external(logFatal, args=None) + +# MochiKit.DOM + +def escapeHTML(data): + return data +register_external(escapeHTML, args=[str], result=str) + +# MochiKit.Base + +def serializeJSON(data): + pass +register_external(serializeJSON, args=None, result=str) + +# MochiKit.Signal + +class Event(BasicExternal): + pass + +Event._fields = { + '_event': dom.Event, +} + +Event._methods = { + 'preventDefault': MethodDesc([]), +} + + +def connect(src, signal, dest): + print 'connecting signal %s' % (signal,) +register_external(connect, args=[dom.EventTarget, str, genericcallable([Event])], + result=int) + +def disconnect(id): + pass +register_external(disconnect, args=[int]) + +def disconnectAll(src, signal): + print 'disconnecting all handlers for signal: %s' % (signal,) +register_external(disconnectAll, args=[dom.EventTarget, str]) + Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/__init__.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,11 @@ +from pygame import * + +import mixer +import sprite +import font +import time +import event +import image + +class error(Exception): + pass \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/event.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/event.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,7 @@ +def get(): + return [Event() for x in range(2) ] + +class Event: + def __init__(self): + self.type = 1 + \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/font.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/font.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,9 @@ +import pygame + +class Font: + def __init__(self, fontfile, size): + self.size = size + + + def render(self, text, alpha, color): + return pygame.Surface( (len(text)*self.size, self.size) ) \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/image.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/image.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,4 @@ +import pygame + +def load(filename): + return pygame.Surface((10,10)) \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/locals.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/locals.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,227 @@ +ACTIVEEVENT = 1 +ANYFORMAT = 268435456 +ASYNCBLIT = 4 +AUDIO_S16 = 32784 +AUDIO_S16LSB = 32784 +AUDIO_S16MSB = 36880 +AUDIO_S16SYS = 36880 +AUDIO_S8 = 32776 +AUDIO_U16 = 16 +AUDIO_U16LSB = 16 +AUDIO_U16MSB = 4112 +AUDIO_U16SYS = 4112 +AUDIO_U8 = 8 +DOUBLEBUF = 1073741824 +FULLSCREEN = -2147483648 +GL_ACCUM_ALPHA_SIZE = 11 +GL_ACCUM_BLUE_SIZE = 10 +GL_ACCUM_GREEN_SIZE = 9 +GL_ACCUM_RED_SIZE = 8 +GL_ALPHA_SIZE = 3 +GL_BLUE_SIZE = 2 +GL_BUFFER_SIZE = 4 +GL_DEPTH_SIZE = 6 +GL_DOUBLEBUFFER = 5 +GL_GREEN_SIZE = 1 +GL_MULTISAMPLEBUFFERS = 13 +GL_MULTISAMPLESAMPLES = 14 +GL_RED_SIZE = 0 +GL_STENCIL_SIZE = 7 +GL_STEREO = 12 +HAT_CENTERED = 0 +HAT_DOWN = 4 +HAT_LEFT = 8 +HAT_LEFTDOWN = 12 +HAT_LEFTUP = 9 +HAT_RIGHT = 2 +HAT_RIGHTDOWN = 6 +HAT_RIGHTUP = 3 +HAT_UP = 1 +HWACCEL = 256 +HWPALETTE = 536870912 +HWSURFACE = 1 +IYUV_OVERLAY = 1448433993 +JOYAXISMOTION = 7 +JOYBALLMOTION = 8 +JOYBUTTONDOWN = 10 +JOYBUTTONUP = 11 +JOYHATMOTION = 9 +KEYDOWN = 2 +KEYUP = 3 +KMOD_ALT = 768 +KMOD_CAPS = 8192 +KMOD_CTRL = 192 +KMOD_LALT = 256 +KMOD_LCTRL = 64 +KMOD_LMETA = 1024 +KMOD_LSHIFT = 1 +KMOD_META = 3072 +KMOD_MODE = 16384 +KMOD_NONE = 0 +KMOD_NUM = 4096 +KMOD_RALT = 512 +KMOD_RCTRL = 128 +KMOD_RMETA = 2048 +KMOD_RSHIFT = 2 +KMOD_SHIFT = 3 +K_0 = 48 +K_1 = 49 +K_2 = 50 +K_3 = 51 +K_4 = 52 +K_5 = 53 +K_6 = 54 +K_7 = 55 +K_8 = 56 +K_9 = 57 +K_AMPERSAND = 38 +K_ASTERISK = 42 +K_AT = 64 +K_BACKQUOTE = 96 +K_BACKSLASH = 92 +K_BACKSPACE = 8 +K_BREAK = 318 +K_CAPSLOCK = 301 +K_CARET = 94 +K_CLEAR = 12 +K_COLON = 58 +K_COMMA = 44 +K_DELETE = 127 +K_DOLLAR = 36 +K_DOWN = 274 +K_END = 279 +K_EQUALS = 61 +K_ESCAPE = 27 +K_EURO = 321 +K_EXCLAIM = 33 +K_F1 = 282 +K_F10 = 291 +K_F11 = 292 +K_F12 = 293 +K_F13 = 294 +K_F14 = 295 +K_F15 = 296 +K_F2 = 283 +K_F3 = 284 +K_F4 = 285 +K_F5 = 286 +K_F6 = 287 +K_F7 = 288 +K_F8 = 289 +K_F9 = 290 +K_FIRST = 0 +K_GREATER = 62 +K_HASH = 35 +K_HELP = 315 +K_HOME = 278 +K_INSERT = 277 +K_KP0 = 256 +K_KP1 = 257 +K_KP2 = 258 +K_KP3 = 259 +K_KP4 = 260 +K_KP5 = 261 +K_KP6 = 262 +K_KP7 = 263 +K_KP8 = 264 +K_KP9 = 265 +K_KP_DIVIDE = 267 +K_KP_ENTER = 271 +K_KP_EQUALS = 272 +K_KP_MINUS = 269 +K_KP_MULTIPLY = 268 +K_KP_PERIOD = 266 +K_KP_PLUS = 270 +K_LALT = 308 +K_LAST = 323 +K_LCTRL = 306 +K_LEFT = 276 +K_LEFTBRACKET = 91 +K_LEFTPAREN = 40 +K_LESS = 60 +K_LMETA = 310 +K_LSHIFT = 304 +K_LSUPER = 311 +K_MENU = 319 +K_MINUS = 45 +K_MODE = 313 +K_NUMLOCK = 300 +K_PAGEDOWN = 281 +K_PAGEUP = 280 +K_PAUSE = 19 +K_PERIOD = 46 +K_PLUS = 43 +K_POWER = 320 +K_PRINT = 316 +K_QUESTION = 63 +K_QUOTE = 39 +K_QUOTEDBL = 34 +K_RALT = 307 +K_RCTRL = 305 +K_RETURN = 13 +K_RIGHT = 275 +K_RIGHTBRACKET = 93 +K_RIGHTPAREN = 41 +K_RMETA = 309 +K_RSHIFT = 303 +K_RSUPER = 312 +K_SCROLLOCK = 302 +K_SEMICOLON = 59 +K_SLASH = 47 +K_SPACE = 32 +K_SYSREQ = 317 +K_TAB = 9 +K_UNDERSCORE = 95 +K_UNKNOWN = 0 +K_UP = 273 +K_a = 97 +K_b = 98 +K_c = 99 +K_d = 100 +K_e = 101 +K_f = 102 +K_g = 103 +K_h = 104 +K_i = 105 +K_j = 106 +K_k = 107 +K_l = 108 +K_m = 109 +K_n = 110 +K_o = 111 +K_p = 112 +K_q = 113 +K_r = 114 +K_s = 115 +K_t = 116 +K_u = 117 +K_v = 118 +K_w = 119 +K_x = 120 +K_y = 121 +K_z = 122 +MOUSEBUTTONDOWN = 5 +MOUSEBUTTONUP = 6 +MOUSEMOTION = 4 +NOEVENT = 0 +NOFRAME = 32 +NUMEVENTS = 32 +OPENGL = 2 +OPENGLBLIT = 10 +PREALLOC = 16777216 +QUIT = 12 +RESIZABLE = 16 +RLEACCEL = 16384 +RLEACCELOK = 8192 +SRCALPHA = 65536 +SRCCOLORKEY = 4096 +SWSURFACE = 0 +SYSWMEVENT = 13 +TIMER_RESOLUTION = 10 +USEREVENT = 24 +UYVY_OVERLAY = 1498831189 +VIDEOEXPOSE = 17 +VIDEORESIZE = 16 +YUY2_OVERLAY = 844715353 +YV12_OVERLAY = 842094169 +YVYU_OVERLAY = 1431918169 Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/mixer.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/mixer.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,6 @@ +def get_init(): + return True + +class Sound: + def __init__(self, filename): + self.filename = filename \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/pygame.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/pygame.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,65 @@ + +def init(): + pass + +class Rect: + pass + +class Surface: + def __init__(self, size): + self.size = size + + def convert(self): + return Surface(self.size) + + def fill(self, color): + pass + + def blit(self, surface, position): + pass + blit._annspecialcase_ = 'specialize:argtype(2)' + + def get_size(self): + return self.size + + def get_rect(self, centerx=1): + return Rect() + + def get_width(self): + return self.size[0] + + def get_at(self, position): + return (100,100,100) + + def set_colorkey(self, colorkey, options): + pass + + + + + + +class Display: + def get_surface(self): + return Surface((10,10)) + + def set_mode(self, (x, y)): + return Surface((x,y)) + + def set_caption(self, caption): + pass + + def flip(self): + pass + + def get_rect(self): + pass + +display = Display() + +class Mouse: + def set_visible(self, bool): + pass + +mouse = Mouse() + Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/sprite.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/sprite.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,14 @@ + +class Sprite: + def __init__(self): + pass + +class RenderPlain: + def __init__(self, sprites): + pass + + def update(self): + pass + + def draw(self, surface): + pass \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/time.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/pygame/time.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,3 @@ +class Clock: + def tick(self, delta): + pass \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/modules/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/modules/test/html/anim.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/test/html/anim.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,36 @@ + + + + + + + +
+ + +
+ + Added: pypy/branch/flex-backend/pypy/translator/flex/modules/test/html/test.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/test/html/test.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,35 @@ + + + + + +

Blah

+
+ + +
+ + Added: pypy/branch/flex-backend/pypy/translator/flex/modules/test/test_dom.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/test/test_dom.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,385 @@ + +# -*- encoding: utf-8 -*- +import py +from pypy.translator.js.modules import dom +from pypy.translator.js.main import rpython2javascript +from xml.dom.minidom import parseString +import sys + +TRANSLATING = False +USE_PDB = False +# XXX: How to get it from config.option??? + +class handler: + called = False + event = dom.Event() + def __call__(self, e): + self.called = True + self.event = e + e.stopPropagation() + +def get_window(): + if TRANSLATING: + return dom.window + else: + return dom.Window() + +def test_quote_html(): + assert dom._quote_html('foo&bar') == 'foo&bar' + assert dom._quote_html('foo"&bar') == 'foo"&bar' + +def test_serialize_html(): + def roundtrip(html): + return dom._serialize_html(parseString(html).documentElement) + html = '
content
' + assert roundtrip(html) == html + html = '' + assert roundtrip(html) == html + html = '' + assert roundtrip(html) == '' + html = '
\n
foo
\n
' + assert roundtrip(html) == html + html = '
foo&bar
' + assert roundtrip(html) == html + +def code_init(): + window = get_window() + nodeType = window.document.nodeType + docel = window.document.documentElement.nodeName + children = len(window.document.documentElement.childNodes) + + return nodeType, docel, children + +def test_init(): + nodeType, docel, children = code_init() + assert nodeType == 9 + assert docel == 'HTML' + assert children == 2 + +def test_init_failing(): + py.test.raises(py.std.xml.parsers.expat.ExpatError, + 'dom.Window(html="")') + +def code_wrap(): + window = get_window() + document = window.document + div = document.createElement('div') + document.documentElement.appendChild(div) + return document, div + +def test_wrap(): + document, div = code_wrap() + assert isinstance(div, dom.Element) # wrapped node + assert div.nodeType == 1 + assert document.documentElement.childNodes[-1]._original is div._original + +def code_nodeValue(): + window = get_window() + document = window.document + td = document.createElement('td') + td.appendChild(document.createTextNode('foo')) + td.childNodes[0].nodeValue = 'bar' + return td.childNodes[0].nodeValue + +def test_nodeValue(): + nodevalue = code_nodeValue() + assert nodevalue == 'bar' + +def code_node_eq(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + firstref = body.parentNode.lastChild + secondref = window.document.createElement('body') + return body, firstref, secondref + +def test_node_eq(): + body, ref1, ref2 = code_node_eq() + assert ref1 == body + assert ref2 != body + +def code_get_element_by_id(): + window = get_window() + document = window.document + div = document.createElement('div') + div.id = 'foo' + window.document.getElementsByTagName('body')[0].appendChild(div) + div = window.document.getElementById('foo') + return div + +def test_get_element_by_id(): + div = code_get_element_by_id() + assert div.nodeName == 'DIV' + +def code_element_style(): + window = get_window() + document = window.document + div = document.createElement('div') + div.style.backgroundColor = 'green' + return div + +def test_element_style(): + div = code_element_style() + assert div.style + assert not div.style.color + assert div.style.backgroundColor == 'green' + py.test.raises(AttributeError, 'div.style.nonExistent') + +def code_get_elements_by_tag_name(): + window = get_window() + document = window.document + div1 = document.createElement('div') + div1.appendChild(document.createTextNode('foo')) + div2 = document.createElement('div') + div2.appendChild(document.createTextNode('bar')) + body = document.getElementsByTagName('body')[0] + body.appendChild(div1) + body.appendChild(div2) + return document + +def test_get_elements_by_tag_name(): + document = code_get_elements_by_tag_name() + divs = document.getElementsByTagName('div') + assert len(divs) == 2 + divs = document.getElementsByTagName('DIV') + assert len(divs) == 2 + +def test_window_references(): + window = dom.Window() + assert window is window.window + assert window is window.self + assert window is window.parent + assert window is window.top + + window2 = dom.Window(parent=window) + assert window2.parent is window + assert window2.top is window + + window3 = dom.Window(parent=window2) + assert window3.parent is window2 + assert window3.top is window + +def code_read_innerHTML(): + window = get_window() + document = window.document + h1 = document.createElement('h1') + h1.appendChild(document.createTextNode('some document')) + p = document.createElement('p') + p.id = 'content' + p.appendChild(document.createTextNode('some content')) + body = document.getElementsByTagName('body')[0] + body.appendChild(h1) + body.appendChild(p) + return document.documentElement.innerHTML + +def test_read_innerHTML(): + html = code_read_innerHTML() + assert html == ('Untitled document' + '

some document

' + '

some content

') + +def code_read_innerHTML_singletons(): + window = get_window() + document = window.document + head = document.getElementsByTagName('head')[0] + meta = document.createElement('meta') + meta.setAttribute('name', 'foo') + meta.setAttribute('content', 'bar') + head.appendChild(meta) + headhtml = window.document.getElementsByTagName('head')[0].innerHTML + return headhtml + +def test_read_innerHTML_singletons(): + headhtml = code_read_innerHTML_singletons() + assert py.std.re.search(']*\/>', headhtml) + +def code_set_innerHTML(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.innerHTML = '
some content
' + return body + +def test_set_innerHTML(): + body = code_set_innerHTML() + assert body.innerHTML == '
some content
' + assert body.childNodes[0].nodeName == 'DIV' + div = body.childNodes[0] + html = div.innerHTML + assert html == 'some content' + +def code_set_innerHTML_empty(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.innerHTML = '' + body.appendChild(window.document.createTextNode('foobar')) + return body + +def test_set_innerHTML_empty(): + body = code_set_innerHTML_empty() + html = body.innerHTML + assert html == 'foobar' + +def code_event_init_1(): + window = get_window() + e = window.document.createEvent() + e.initEvent('click', True, True) + return window, e + +def code_event_init_2(): + window, event = code_event_init_1() + body = window.document.getElementsByTagName('body')[0] + body.dispatchEvent(event) + return body, event + +def test_event_init(): + window, e = code_event_init_1() + assert e.cancelable == True + assert e.target == None + body, e = code_event_init_2() + assert e.target is body + +def code_event_handling(): + h = handler() + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', True, True) + body.dispatchEvent(e) + return h + +def test_event_handling(): + h = code_event_handling() + assert h.called == True + +def code_event_bubbling_1(): + h = handler() + window = get_window() + body = window.document.getElementsByTagName('body')[0] + div = window.document.createElement('div') + body.appendChild(div) + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', False, True) + div.dispatchEvent(e) + return div, h + +def code_event_bubbling_2(): + div, h = code_event_bubbling_1() + e = div.ownerDocument.createEvent() + e.initEvent('click', True, True) + div.dispatchEvent(e) + return h + +def test_event_bubbling(): + div, h = code_event_bubbling_1() + assert not h.called + h = code_event_bubbling_2() + assert h.called == True + +def code_remove_event_listener(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + div = window.document.createElement('div') + body.appendChild(div) + h = handler() + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', True, True) + body.dispatchEvent(e) + return body, h + +def code_remove_event_listener_2(): + body, h = code_remove_event_listener() + h.called = False + body.removeEventListener('click', h, False) + e = body.ownerDocument.createEvent() + e.initEvent('click', True, True) + body.dispatchEvent(e) + return h + +def test_remove_event_listener(): + body, h = code_remove_event_listener() + assert h.called == True + h = code_remove_event_listener_2() + assert h.called == False + +def code_event_vars(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + div = window.document.createElement('div') + body.appendChild(div) + h = handler() + body.addEventListener('click', h, False) + e = window.document.createEvent() + e.initEvent('click', True, True) + div.dispatchEvent(e) + return body, div, h + +def test_event_vars(): + body, div, h = code_event_vars() + assert h.event.target == div + assert h.event.originalTarget == div + assert h.event.currentTarget == body + +def code_class_name(): + window = get_window() + document = window.document + body = document.getElementsByTagName('body')[0] + div = document.createElement('div') + div.appendChild(document.createTextNode('foo')) + div.setAttribute('class', 'foo') + body.appendChild(div) + div = window.document.getElementsByTagName('div')[0] + return body, div + +def test_class_name(): + body, div = code_class_name() + assert div.className == 'foo' + assert not body.className + div.className = 'bar' + assert div.className == 'bar' + assert body.innerHTML == '
foo
' + +def code_read_styles(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.innerHTML = ('
foo' + '
') + return body.childNodes[0].style + +def test_read_styles(): + style = code_read_styles() + assert style.color == 'red' + bgcolor = style.backgroundColor + assert bgcolor == 'green' + +def code_write_styles(): + window = get_window() + body = window.document.getElementsByTagName('body')[0] + body.style.color = 'green' + body.style.backgroundColor = 'red' + return body + +def test_write_styles(): + body = code_write_styles() + assert dom._serialize_html(body) == ('') + +def test_document_location(): + py.test.skip("To be written") + +def test_build(): + py.test.skip('B?rken') + global TRANSLATING + TRANSLATING = True + for var in globals(): + if var.startswith('code_'): + # just build it + #def f(): + assert rpython2javascript(sys.modules[__name__], [var], use_pdb=False) + + TRANSLATING = False + +def test_navigator(): + window = get_window() + assert window.navigator.appName == 'Netscape' Added: pypy/branch/flex-backend/pypy/translator/flex/modules/test/test_mochikit.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/modules/test/test_mochikit.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,14 @@ + +from pypy.translator.js.lib.support import js_source +from pypy.translator.js.modules.mochikit import * + +class TestRender(object): + def test_escape_html(self): + def x(): + escapeHTML("xxx") + "xxx" + assert js_source([x], use_pdb=False).find("escapeHTML (") != -1 + + def test_serialize_json(self): + def x(): + return serializeJSON(3) + serializeJSON("aaa") + serializeJSON([1]) + assert js_source([x]) Added: pypy/branch/flex-backend/pypy/translator/flex/opcodes.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/opcodes.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,158 @@ +""" opcode definitions +""" + +from pypy.translator.oosupport.metavm import PushArg, PushAllArgs, StoreResult,\ + InstructionList, New, GetField, MicroInstruction, RuntimeNew, PushPrimitive + +from pypy.translator.oosupport.metavm import _GetFieldDispatcher, _SetFieldDispatcher, \ + _CallDispatcher, _MethodDispatcher, SetField + +from pypy.translator.flex.metavm import IsInstance, Call, CallMethod,\ + CopyName, CastString, _Prefix, _CastFun, _NotImplemented, CallBuiltin,\ + CallBuiltinObject, GetBuiltinField, SetBuiltinField, IndirectCall,\ + CallExternalObject, SetExternalField, _CastMethod, _LoadConst,\ + DiscardStack, CheckLength, fix_opcodes + +from pypy.translator.flex.jsbuiltin import Builtins +from pypy.rpython.ootypesystem import ootype + +DoNothing = [] + +from pypy.translator.flex.log import log + +class_map = { 'Call' : Call, + 'CallMethod' : CallMethod, + 'CallBuiltinObject' : CallBuiltinObject, + 'CallBuiltin' : CallBuiltin, + 'GetBuiltinField' : GetBuiltinField, + 'GetField' : GetField, + 'SetField' : SetField, + 'SetBuiltinField' : SetBuiltinField, + 'CallExternalObject' : CallExternalObject, + 'SetExternalField' : SetExternalField, +} + +opcodes = {'int_mul': '*', + 'int_add': '+', + 'int_sub': '-', + 'int_sub_ovf': '-', # XXX overflow + 'int_floordiv': '/', + 'int_mod': '%', + 'int_mod_ovf': '%', # XXX: what's that? + 'int_mod_zer': '%', # XXX: fix zero stuff + 'int_and': '&', + 'int_or': '|', + 'int_xor': '^', + 'int_lshift': '<<', + 'int_lshift_ovf': '<<', # XXX overflow + 'int_rshift': '>>', + 'int_rshift_ovf': '>>', # XXX overflow + 'int_lt': '<', + 'int_le': '<=', + 'int_eq': '==', + 'int_ne': '!=', + 'int_ge': '>=', + 'int_gt': '>', + + 'uint_mul': '*', + 'uint_add': '+', + 'uint_sub': '-', + 'uint_floordiv': '/', + 'uint_mod': '%', + 'uint_and': '&', + 'uint_or': '|', + 'uint_xor': '^', + 'uint_lshift': '<<', + 'uint_rshift': '>>', + 'uint_lt': '<', + 'uint_le': '<=', + 'uint_eq': '==', + 'uint_ne': '!=', + 'uint_ge': '>=', + 'uint_gt': '>', + + 'unichar_lt': '<', + 'unichar_le': '<=', + 'unichar_eq': '==', + 'unichar_ne': '!=', + 'unichar_ge': '>=', + 'unichar_gt': '>', + + 'char_lt': '<', + 'char_le': '<=', + 'char_eq': '==', + 'char_ne': '!=', + 'char_ge': '>=', + 'char_gt': '>', + + 'float_mul': '*', + 'float_add': '+', + 'float_sub': '-', + 'float_truediv': '/', + 'float_lt': '<', + 'float_le': '<=', + 'float_eq': '==', + 'float_ne': '!=', + 'float_ge': '>=', + 'float_gt': '>', + + 'ptr_eq': '==', + 'ptr_ne': '!=', + + 'bool_not': [PushAllArgs,_Prefix('!')], + 'int_neg': [PushAllArgs,_Prefix('-')], + 'int_invert': [PushAllArgs,_Prefix('~')], + 'float_neg': [PushAllArgs,_Prefix('-')], + + 'float_pow': [PushAllArgs,_CastFun('Math.pow',2)], + 'int_abs': [PushAllArgs,_CastFun('Math.abs',1)], + 'float_abs': [PushAllArgs,_CastFun('Math.abs',1)], + + 'int_is_true': [PushAllArgs,_Prefix('!!')], + 'uint_is_true': [PushAllArgs,_Prefix('!!')], + 'float_is_true': [PushAllArgs,_Prefix('!!')], + 'is_true': [PushAllArgs,_Prefix('!!')], + + 'direct_call' : [_CallDispatcher(Builtins, class_map)], + 'indirect_call' : [IndirectCall], + 'same_as' : CopyName, + 'new' : [New], + 'runtimenew' : [RuntimeNew], + 'instanceof' : [IsInstance], + #'subclassof' : [IsSubclassOf], + + # objects + + 'oosetfield' : [_SetFieldDispatcher(Builtins, class_map)], + 'oogetfield' : [_GetFieldDispatcher(Builtins, class_map)], + 'oosend' : [_MethodDispatcher(Builtins, class_map)], + 'ooupcast' : CopyName, + 'oodowncast' : CopyName, + 'oononnull' : [PushAllArgs,_Prefix('!!')], + 'oostring' : [PushArg(0),CastString], + 'ooparse_int' : [PushAllArgs,_CastFun("parseInt",2)], + 'ooparse_float' : [PushAllArgs,_CastFun("parseFloat",1)], + 'oois' : '===', + 'cast_bool_to_int': CopyName, + 'cast_bool_to_uint': CopyName, + 'cast_bool_to_float': CopyName, + 'cast_char_to_int': [PushAllArgs,_LoadConst(0),_CastMethod("charCodeAt",1)], + 'cast_unichar_to_int': [PushAllArgs,_LoadConst(0),_CastMethod("charCodeAt",1)], + 'cast_int_to_char': [PushAllArgs,_CastFun("String.fromCharCode",1)], + 'cast_int_to_unichar': [PushAllArgs,_CastFun("String.fromCharCode",1)], + 'cast_int_to_uint': CopyName, + 'cast_int_to_float': CopyName, + 'cast_int_to_longlong': CopyName, + 'cast_uint_to_int': CopyName, + 'cast_uint_to_float': CopyName, + 'cast_float_to_int': [PushAllArgs,_CastFun("Math.floor",1)], + 'cast_float_to_uint': [PushAllArgs,_CastFun("Math.floor",1)], + 'cast_float_to_longlong': [PushAllArgs,_CastFun("Math.floor",1)], + 'truncate_longlong_to_int': CopyName, + + 'debug_assert' : DoNothing, + 'resume_point' : DoNothing, + 'is_early_constant': [PushPrimitive(ootype.Bool, False)], +} + +fix_opcodes(opcodes) Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/MyFirst.html ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/MyFirst.html Sat Jul 14 12:31:00 2007 @@ -0,0 +1,11 @@ + + + + + + + Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/MyFirst.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/MyFirst.mxml Sat Jul 14 12:31:00 2007 @@ -0,0 +1,74 @@ + + + + + + + + + + Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/chimp.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/chimp.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,6 @@ +#/usr/bin/env python """ This simple example is used for the line-by-line tutorial that comes with pygame. It is based on a 'popular' web banner. Note there are comments here, but for the full explanation, follow along in the tutorial. """ #Import Modules import os +import pypy.translator.flex.modules.pygame as pygame from pypy.translator.flex.modules.pygame.locals import * if not pygame.font: print 'Warning, fonts disabled' if not pygame.mixer: print 'Warning, sound disabled' #functions to create our resources def load_image(name, colorkey=None): fullname = os.path.join('data', name) try: image = pygame.image.load(fullname) except pygame.error, message: print 'Cannot load image:', fullname raise SystemExit, message image = image.convert() if colorkey is not None: if colorkey is -1: colorkey = image.get_at((0,0)) image.set_colorkey(colorkey, RLEACCEL) return image, image.get_rect() + +class NoneSound: def play(self): pass + def load_sound(name): if not pygame.mixer or not pygame.mixer.get_init(): return NoneSound() fullname = os.path.join('data', name) try: sound = pygame.mixer.Sound(fullname) except pygame.error, message: print 'Cannot load sound:', fullname raise SystemExit, message return sound #classes for our game objects class Fist(pygame.sprite.Sprite): """moves a clenched fist on the screen, following the mouse""" def __init__(self): pygame.sprite.Sprite.__init__(self) #call Sprite initializer self.image, self.rect = load_image('fist.bmp', -1) self.punching = 0 def update(self): "move the fist based on the mouse position" pos = pygame.mouse.get_pos() self.rect.midtop = pos if self.punching: self.rect.move_ip(5, 10) def punch(self, target): "returns true if the fist collides with the target" if not self.punching: self.punching = 1 hitbox = self.rect.inflate(-5, -5) return hitbox.colliderect(target.rect) def unpunch(self): "called to pull the fist back" self.punching = 0 class Chimp(pygame.sprite.Sprite): """moves a monkey critter across the screen. it can spin the monkey when it is punched.""" def __init__(self): pygame.sprite.Sprite.__init__(self) #call Sprite intializer self.image, self.rect = load_image('chimp.bmp', -1) screen = pygame.display.get_surface() self.area = screen.get_rect() self.rect.topleft = 10, 10 self.move = 9 self.dizzy = 0 def update(self): "walk or spin, depending on the monkeys state" if self.dizzy: self._spin() else: self._walk() def _walk(self): "move the monkey across the screen, and turn at the ends" newpos = self.rect.move((self.move, 0)) if self.rect.left < self.area.left or \ self.rect.right > self.area.right: self.move = -self.move newpos = self.rect.move((self.move, 0)) self.image = pygame.transform.flip(self.image, 1, 0) self.rect = newpos def _spin(self): "spin the monkey image" center = self.rect.center self.dizzy = self.dizzy + 12 if self.dizzy >= 360: self.dizzy = 0 self.image = self.original else: rotate = pygame.transform.rotate self.image = rotate(self.original, self.dizzy) self.rect = self.image.get_rect(center=center) def punched(self): "this will cause the monkey to start spinning" if not self.dizzy: self.dizzy = 1 self.original = self.image def flash_main(): """this function is called when the program starts. it initializes everything it needs, then runs in a loop until the function returns.""" #Initialize Everything pygame.init() screen = pygame.display.set_mode((468, 60)) pygame.display.set_caption('Monkey Fever') pygame.mouse.set_visible(0) #Create The Backgound background = pygame.Surface(screen.get_size()) background = background.convert() background.fill((250, 250, 250)) #Put Text On The Background, Centered if pygame.font: font = pygame.font.Font(None, 36) text = font.render("Pummel The Chimp, And Win $$$", 1, (10, 10, 10)) + textpos = text.get_rect(centerx=background.get_width()/2) background.blit(text, textpos) #Display The Background screen.blit(background, (0, 0)) pygame.display.flip() #Prepare Game Objects clock = pygame.time.Clock() whiff_sound = load_sound('whiff.wav') punch_sound = load_sound('punch.wav') chimp = Chimp() fist = Fist() allsprites = pygame.sprite.RenderPlain((fist, chimp)) #Main Loop while 1: clock.tick(60) #Handle Input Events for event in pygame.event.get(): if event.type == QUIT: return elif event.type == KEYDOWN and event.key == K_ESCAPE: return elif event.type == MOUSEBUTTONDOWN: if fist.punch(chimp): punch_sound.play() #punch chimp.punched() else: whiff_sound.play() #miss elif event.type is MOUSEBUTTONUP: fist.unpunch() allsprites.update() #Draw Everything screen.blit(background, (0, 0)) allsprites.draw(screen) pygame.display.flip() #Game Over #this calls the 'main' function when this script is executed if __name__ == '__main__': flash_main() \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/fun.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/fun.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,17 @@ +from pypy.translator.flex.modules.flex import * + + +def flash_main(a=1): + #abutton = Button() + #abutton.label = "I'm a button!" + #addChild(abutton) + + for x in range(20): + abutton = Button() + abutton.label = "I'm button :" + str(x) + addChild(abutton) + + return a+1 + + + Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/misc.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/misc.as Sat Jul 14 12:31:00 2007 @@ -0,0 +1,236 @@ + + import py.*; + //import py.f.DictIter; + + +// starts hand written code +var MALLOC_ZERO_FILLED = 0; + +var print = trace; + + + + +Function.prototype.method = function (name, func) { + this.prototype[name] = func; + return this; +}; + +function inherits(child, parent) { + child.parent = parent; + for (var i in parent.prototype) { + if (!child.prototype[i]) { + child.prototype[i] = parent.prototype[i]; + } + } +} + +function isinstanceof(self, what) { + if (!self) { + return (false); + } + var t = self.constructor; + while ( t ) { + if (t == what) { + return (true); + } + t = t.parent; + } + return (false); +} + +/*function delitem(fn, l, i) { + for(var j = i; j < l.length-1; ++j) { + l[j] = l[j+1]; + } + l.length--; +}*/ + +function strcmp(s1, s2) { + if ( s1 < s2 ) { + return ( -1 ); + } else if ( s1 == s2 ) { + return ( 0 ); + } + return (1); +} + +function startswith(s1, s2) { + if (s1.length < s2.length) { + return(false); + } + for (var i = 0; i < s2.length; ++i){ + if (s1.charAt(i) != s2.charAt(i)) { + return(false); + } + } + return(true); +} + +function endswith(s1, s2) { + if (s2.length > s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} +// ends hand written code +function some_strange_function_which_will_never_be_called () { + var v1,v2; + var block = 0; + for(;;){ + switch(block){ + case 0: + v1 = 2; + v2 = xx ( v1 ); + block = 1; + break; + case 1: + return ( undefined ); + } + } +} + +function xx (y_0) { + var v3,v4; + var block = 0; + for(;;){ + switch(block){ + case 0: + v4 = (y_0+1); + v3 = v4; + block = 1; + break; + case 1: + return ( v3 ); + } + } +} + + +var __consts_0 = {}; Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/output.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/output.mxml Sat Jul 14 12:31:00 2007 @@ -0,0 +1,462 @@ + + + + + + s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} + + +function __flash_main() { + + + var aaa = new Button(); + aaa.label = "before consts"; + addChild ( aaa ); + + + + __load_consts_flex() + + aaa.label = "after consts"; + flash_main(1) +} + + +public function flexTrace( text:String ):void { + var myUrl:URLRequest = new URLRequest("javascript:console.log('" + text + "');void(0);"); + sendToURL(myUrl); +} + +// wrapper for throw, because flex compiler weirdness. +function throwit(e) { + throw(e); +} + + +// ends hand written code + + + +function some_strange_function_which_will_never_be_called () { + var v1,v2; + var block = 0; + for(;;){ + switch(block){ + case 0: + v1 = 1; + v2 = flash_main ( v1 ); + block = 1; + break; + case 1: + return ( undefined ); + } + } +} + +function flash_main (a_0) { + var v3,v4,v5,a_1,v6,v7,last_exc_value_0,a_2,x_0,v8,v9,v10,v11,v14,v15; + var block = 0; + for(;;){ + switch(block){ + case 0: + v4 = ll_newrange__Record_start__Signed__stop__Si_Signed_Signed ( 0,20 ); + v5 = ll_rangeiter__Record_stop__Signed__next__Sig_Record_start__Signed__stop__Si ( v4 ); + a_1 = a_0; + v6 = v5; + block = 1; + break; + case 1: + try { + v7 = ll_rangenext_up__Record_stop__Signed__next__Sig_Signed ( v6,1 ); + a_2 = a_1; + x_0 = v7; + v8 = v6; + block = 2; + break; + } + catch (exc){ + if (isinstanceof(exc, exceptions_StopIteration)) + { + v14 = a_1; + block = 3; + break; + } + throwit(exc); + } + case 2: + v9 = new Button(); + v10 = ll_int_str__IntegerR_SignedConst_Signed ( x_0 ); + v11 = ll_strconcat__String_String ( __consts_0.const_str,v10 ); + v9.label = v11; + addChild ( v9 ); + a_1 = a_2; + v6 = v8; + block = 1; + break; + case 3: + v15 = (v14+1); + v3 = v15; + block = 4; + break; + case 4: + return ( v3 ); + } + } +} + +function ll_strconcat__String_String (obj_0,arg0_0) { + var v39,v40,v41; + var block = 0; + for(;;){ + switch(block){ + case 0: + v41 = (obj_0+arg0_0); + v39 = v41; + block = 1; + break; + case 1: + return ( v39 ); + } + } +} + +function ll_newrange__Record_start__Signed__stop__Si_Signed_Signed (start_0,stop_0) { + var v16,v17; + var block = 0; + for(;;){ + switch(block){ + case 0: + v17 = new Object(); + v17.start = start_0; + v17.stop = stop_0; + v16 = v17; + block = 1; + break; + case 1: + return ( v16 ); + } + } +} + +function ll_rangeiter__Record_stop__Signed__next__Sig_Record_start__Signed__stop__Si (rng_0) { + var v20,v21,v22,v24; + var block = 0; + for(;;){ + switch(block){ + case 0: + v21 = new Object(); + v22 = rng_0.start; + v21.next = v22; + v24 = rng_0.stop; + v21.stop = v24; + v20 = v21; + block = 1; + break; + case 1: + return ( v20 ); + } + } +} + +function ll_int_str__IntegerR_SignedConst_Signed (i_0) { + var v37,v38; + var block = 0; + for(;;){ + switch(block){ + case 0: + v38 = ll_int2dec__Signed ( i_0 ); + v37 = v38; + block = 1; + break; + case 1: + return ( v37 ); + } + } +} + +function ll_rangenext_up__Record_stop__Signed__next__Sig_Signed (iter_0,step_0) { + var v26,v27,v28,v29,v30,iter_1,next_0,v31,v32,v34,v35,v36,etype_0,evalue_0; + var block = 0; + for(;;){ + switch(block){ + case 0: + v27 = iter_0.next; + v28 = iter_0.stop; + v29 = (v27>=v28); + iter_1 = iter_0; + next_0 = v27; + v31 = step_0; + if (v29 == false) + { + block = 1; + break; + } + block = 3; + break; + case 1: + v32 = (next_0+v31); + iter_1.next = v32; + v26 = next_0; + block = 2; + break; + case 3: + v34 = __consts_0.exceptions_StopIteration; + v35 = v34.meta; + etype_0 = v35; + evalue_0 = v34; + block = 4; + break; + case 4: + throwit(evalue_0); + case 2: + return ( v26 ); + } + } +} + +function ll_int2dec__Signed (i_1) { + var v42,v43; + var block = 0; + for(;;){ + switch(block){ + case 0: + v43 = convertToString ( i_1 ); + v42 = v43; + block = 1; + break; + case 1: + return ( v42 ); + } + } +} + +var __consts_0 = {} +function __load_consts_flex() +{ + __consts_0.exceptions_StopIteration__3 = exceptions_StopIteration; + __consts_0.exceptions_StopIteration_meta = new exceptions_StopIteration_meta(); + __consts_0.exceptions_StopIteration = new exceptions_StopIteration(); + __consts_0.const_str = "I'm button :"; + __consts_0.exceptions_StopIteration_meta.class_ = __consts_0.exceptions_StopIteration__3; + __consts_0.exceptions_StopIteration.meta = __consts_0.exceptions_StopIteration_meta; +} + + ]]> + + Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/output2.mxml ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/output2.mxml Sat Jul 14 12:31:00 2007 @@ -0,0 +1,260 @@ + + + + + + s1.length) { + return(false); + } + for (var i = s1.length-s2.length; i < s1.length; ++i) { + if (s1.charAt(i) != s2.charAt(i - s1.length + s2.length)) { + return(false); + } + } + return(true); +} + +function splitchr(s, ch) { + var i, lst, next; + lst = []; + next = ""; + for (var i = 0; i end || start > s1.length) { + return -1; + } + s1 = s1.substr(start, end-start); + var res = s1.indexOf(s2); + if (res == -1) { + return -1; + } + return res + start; +} + +function findIndexOfTrue(s1, s2) { + return findIndexOf(s1, s2, 0, s1.length) != -1; +} + +function countCharOf(s, c, start, end) { + s = s.substring(start, end); + var i = 0; + for (var c1 in s) { + if (s[c1] == c) { + i++; + } + } + return(i); +} + +function countOf(s, s1, start, end) { + var ret = findIndexOf(s, s1, start, end); + var i = 0; + var lgt = 1; + if (s1.length > 0) { + lgt = s1.length; + } + while (ret != -1) { + i++; + ret = findIndexOf(s, s1, ret + lgt, end); + } + return (i); +} + +function convertToString(stuff) { + if (stuff === undefined) { + return ("undefined"); + } + return (stuff.toString()); +} +// ends hand written code + + +function some_strange_function_which_will_never_be_called () { + var v1,v2; + var block = 0; + for(;;){ + switch(block){ + case 0: + v1 = 1; + v2 = flash_main ( v1 ); + block = 1; + break; + case 1: + return ( undefined ); + } + } +} + +function flash_main (a_0) { + var v3,v4,v6; + var block = 0; + for(;;){ + switch(block){ + case 0: + var v4 = new Button(); + v4.label = __consts_0.const_str; + addChild (v4); + v6 = (a_0+1); + v3 = v6; + block = 1; + break; + case 1: + return ( v3 ); + } + } +} + + +var __consts_0 = new Constants(); +//__consts_0.const_str = "I'm a button!"; +//__consts_0['const_str'] = "I'm a button!"; + + ]]> + + Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/py/DictIter.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/py/DictIter.as Sat Jul 14 12:31:00 2007 @@ -0,0 +1,31 @@ + +package py { + +public class DictIter { + + var current_key; + var l; + + public function DictIter() { + this.current_key = null; + } + + function ll_go_next() { + var ret = this.l.length != 0; + this.current_key = this.l.pop(); + return ret; + } + + function ll_current_key() { + if (this.current_key) { + return this.current_key; + } else { + return null; + } + } + +} + + + +} Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/py/StringBuilder.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/py/StringBuilder.as Sat Jul 14 12:31:00 2007 @@ -0,0 +1,33 @@ +package py { + + +public class StringBuilder { + + public function StringBuilder() { + this.l = []; + } + + function ll_append_char(s) { + this.l.length += 1; + this.l[this.l.length - 1] = s; + } + + function ll_append(s) { + this.l.push(s); + } + + function ll_allocate(t) { + } + + function ll_build() { + var s; + s = ""; + for (var i in this.l) { + s += this.l[i]; + } + return (s); + } + +} + +} Added: pypy/branch/flex-backend/pypy/translator/flex/sandbox/py/pygame/Rect.as ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/sandbox/py/pygame/Rect.as Sat Jul 14 12:31:00 2007 @@ -0,0 +1,45 @@ +class Rect { + public var x : Float; + public var y : Float; + public var w : Float; + public var h : Float; + + public var centerx(get_centerx,set_centerx) : Float; + public var centery(get_centery,set_centery) : Float; + public var top(get_top,set_top): Float; + public var right(get_right,set_right): Float; + public var bottom(get_bottom,set_bottom): Float; + public var left(get_left,set_left): Float; + + public function new(x_:Float,y_:Float,w_:Float,h_:Float) { + x = x_; + y = y_; + w = w_; + h = h_; + } + + public function colliderect(b:Rect) { + var a = this; + if (a.bottom < b.top) { return false; } + if (a.left > b.right) { return false; } + if (a.top > b.bottom) { return false; } + if (a.right < b.left) { return false; } + return true; + } + + private function get_centerx() { return x+w/2; } + private function set_centerx(v:Float) { x = v-w/2; return x+w/2; } + private function get_centery() { return y+h/2; } + private function set_centery(v:Float) { y = v-h/2; return y+h/2; } + private function get_top() { return y; } + private function set_top(v:Float) { y = v; return y; } + private function get_right() { return x+w; } + private function set_right(v:Float) { x = v-w; return x+w; } + private function get_bottom() { return y+h; } + private function set_bottom(v:Float) { y = v-h; return y+h; } + private function get_left() { return x; } + private function set_left(v:Float) { x = v; return x; } +} + + + \ No newline at end of file Added: pypy/branch/flex-backend/pypy/translator/flex/support.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/support.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,64 @@ +from pypy.translator.gensupp import NameManager +#from pypy.translator.js.optimize import is_optimized_function + +class JavascriptNameManager(NameManager): + def __init__(self, db): + NameManager.__init__(self) + self.db = db + self.reserved = {} + + #http://javascript.about.com/library/blreserved.htm + reserved_words = ''' + abstract as boolean break byte case catch + char class continue const debugger default delete + do double else enum export extends false + final finally float for function goto if implements + import in instanceof int interface is long + namespace native new null package private protected + public return short static super switch synchronized + this throw throws transient true try typeof + use var void volatile while with alert + ''' + for name in reserved_words.split(): + self.reserved[name] = True + + #http://javascript.about.com/library/blclassobj.htm + # XXX WAAAHHH!!! IE alert :( there are a lot of objects here that are + # _not_ in standard JS, see + # http://devedge-temp.mozilla.org/library/manuals/2000/javascript/1.5/reference/ + predefined_classes_and_objects = ''' + Anchor anchors Applet applets Area Array Body + Button Checkbox Date document Error EvalError FileUpload + Form forms frame frames Function Hidden History + history Image images Link links location Math + MimeType mimetypes navigator Number Object Option options + Password Plugin plugins Radio RangeError ReferenceError RegExp + Reset screen Script Select String Style StyleSheet + Submit SyntaxError Text Textarea TypeError URIError window + ''' + for name in predefined_classes_and_objects.split(): + self.reserved[name] = True + + #http://javascript.about.com/library/blglobal.htm + global_properties_and_methods = ''' + _content closed Components controllers crypto defaultstatus directories + document frames history innerHeight innerWidth length location + locationbar menubar name navigator opener outerHeight outerWidth + pageXOffset pageYOffset parent personalbar pkcs11 prompter screen + screenX screenY scrollbars scrollX scrollY self statusbar + toolbar top window + ''' + for name in global_properties_and_methods.split(): + self.reserved[name] = True + + self.make_reserved_names(' '.join(self.reserved)) + + self.predefined = set(predefined_classes_and_objects) + + #def uniquename(self, name, lenmax=0): + # return NameManager.uniquename(self, , lenmax) + + def ensure_non_reserved(self, name): + while name in self.reserved: + name += '_' + return name Added: pypy/branch/flex-backend/pypy/translator/flex/test/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/test/browsertest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/browsertest.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,177 @@ +from BaseHTTPServer import HTTPServer as BaseHTTPServer, BaseHTTPRequestHandler +import py +from os import system +from cgi import parse_qs +from sys import platform +from time import sleep +import webbrowser +from pypy.translator.js.log import log +log = log.browsertest + +class HTTPServer(BaseHTTPServer): + allow_reuse_address = True + +class config: + http_port = 10001 + + html_page = """ + + + + + %(jsfilename)s +
+ +
+
+ +""" + + refresh_page = """ + + + + +
+// testcase: %(jstestcase)s
+%(jscode)s
+
+ +""" + + +class TestCase(object): + def __init__(self, jsfilename, jstestcase): + self.jsfilename = jsfilename + self.jscode = open(jsfilename).read() + self.jstestcase = jstestcase + self.result = None + + +class TestHandler(BaseHTTPRequestHandler): + """The HTTP handler class that provides the tests and handles results""" + + def do_GET(self): + global do_status + if self.path != "/test.html": + self.send_error(404, "File /test.html not found") + return + jsfilename = jstest.jsfilename + jstestcase = jstest.jstestcase + jscode = jstest.jscode + if self.server.html_page: + if self.server.is_interactive: + isinteractive = '' + else: + isinteractive = 'resultform.submit();' + try: + html_page = open(self.server.html_page).read() % locals() + except IOError: + log("HTML FILE WAS NOT FOUND!!!!") + self.send_error(404, "File %s not found" % self.server.html_page) + return + else: + html_page = config.html_page % locals() + + open("html_page.html", "w").write(html_page) + self.serve_data('text/html', html_page) + do_status = 'do_GET' + + def do_POST(self): + global do_status + if self.path != "/test.html": + self.send_error(404, "File /test.html not found") + return + form = parse_qs(self.rfile.read(int(self.headers['content-length']))) + if self.server.is_interactive: + if not form.has_key('ok'): + jstest.result = 'Not clicked OK' + else: + jstest.result = 'OK' + #assert False, "Clicked not ok" + else: + jstest.result = form['result'][0] + + #we force a page refresh here because of two reason: + # 1. we don't have the next testcase ready yet + # 2. browser should ask again when we do have a test + jsfilename = jstest.jsfilename + jstestcase = jstest.jstestcase + jscode = jstest.jscode + refresh_page = config.refresh_page % locals() + self.serve_data('text/html', refresh_page) + do_status = 'do_POST' + + def serve_data(self, content_type, data): + self.send_response(200) + self.send_header("Content-type", content_type) + self.send_header("Content-length", len(data)) + self.end_headers() + self.wfile.write(data) + + +class BrowserTest(object): + """The browser driver""" + + def start_server(self, port, html_page, is_interactive): + server_address = ('', port) + self.httpd = HTTPServer(server_address, TestHandler) + self.httpd.is_interactive = is_interactive + self.httpd.html_page = html_page + + def get_result(self): + global do_status + do_status = None + while do_status != 'do_GET': + self.httpd.handle_request() + while do_status != 'do_POST': + self.httpd.handle_request() + return jstest.result + + +def jstest(jsfilename, jstestcase, browser_to_use, html_page = None, is_interactive = False): + global driver, jstest + jstest = TestCase(str(jsfilename), str(jstestcase)) + + try: + driver.httpd.html_page = html_page + driver.httpd.is_interactive = is_interactive + except: + driver = BrowserTest() + driver.start_server(config.http_port, html_page, is_interactive) + if browser_to_use == 'default': + browser_to_use = None + if browser_to_use != 'none': + webbrowser.get(browser_to_use).open('http://localhost:%d/test.html' % config.http_port) + + result = driver.get_result() + return result Added: pypy/branch/flex-backend/pypy/translator/flex/test/runtest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/runtest.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,215 @@ + +import py, os, re, subprocess +from pypy.translator.translator import TranslationContext +from pypy.translator.backendopt.all import backend_optimizations +from pypy.translator.js.js import JS +from pypy.translator.js.test.browsertest import jstest +from pypy.translator.js import conftest +from pypy.translator.js.log import log +from pypy.conftest import option +from pypy.rpython.test.tool import BaseRtypingTest, OORtypeMixin +from pypy.rlib.nonconst import NonConstant +from pypy.rpython.ootypesystem import ootype + +from pypy.rpython.llinterp import LLException + +log = log.runtest +use_browsertest = conftest.option.browser +use_tg = conftest.option.tg + +port = 8080 + +class JSException(LLException): + pass + +def _CLI_is_on_path(): + if py.path.local.sysfind('js') is None: #we recommend Spidermonkey + return False + return True + +class compile_function(object): + def __init__(self, function, annotations, stackless=False, view=False, html=None, is_interactive=False, root = None, run_browser = True, policy = None): + if not use_browsertest and not _CLI_is_on_path(): + py.test.skip('Javascript CLI (js) not found') + + self.html = html + self.is_interactive = is_interactive + t = TranslationContext() + + if policy is None: + from pypy.annotation.policy import AnnotatorPolicy + policy = AnnotatorPolicy() + policy.allow_someobjects = False + + ann = t.buildannotator(policy=policy) + ann.build_types(function, annotations) + if view or option.view: + t.view() + t.buildrtyper(type_system="ootype").specialize() + + if view or option.view: + t.view() + #self.js = JS(t, [function, callback_function], stackless) + self.js = JS(t, function, stackless) + self.js.write_source() + if root is None and use_tg: + from pypy.translator.js.demo.jsdemo.controllers import Root + self.root = Root + else: + self.root = root + self.run_browser = run_browser + self.function_calls = [] + + def source(self): + return self.js.tmpfile.open().read() + + def _conv(self, v): + if isinstance(v, str): + return repr(v) + return str(v).lower() + + def __call__(self, *kwds): + return self.call(None, kwds) + + def call(self, entry_function, kwds): + args = ', '.join([self._conv(kw) for kw in kwds]) #lowerstr for (py)False->(js)false, etc. + + if entry_function is None: + entry_function = self.js.translator.graphs[0].name + else: + entry_function = self.js.translator.annotator.bookkeeper.getdesc(entry_function).cached_graph(None) + function_call = "%s(%s)" % (entry_function, args) + self.function_calls.append(function_call) + #if self.js.stackless: + # function_call = "slp_entry_point('%s')" % function_call + + if use_browsertest: + if not use_tg: + log("Used html: %r" % self.html) + output = jstest(self.js.filename, function_call, use_browsertest, self.html, self.is_interactive) + else: + global port + from pypy.translator.js.test.tgtest import run_tgtest + out = run_tgtest(self, tg_root = self.root, port=port, run_browser=self.run_browser).results + assert out[1] == 'undefined' or out[1] == "" + output = out[0] + port += 1 + else: +# cmd = 'echo "load(\'%s\'); print(%s)" | js 2>&1' % (self.js.filename, function_call) +# log(cmd) +# output = os.popen(cmd).read().strip() + js = subprocess.Popen(["js"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + input = "load(%r);\n" % self.js.filename.strpath + for call in self.function_calls[:-1]: + input += "%s;\n" % call + input += "print(\"'\" + %s + \"'\");\n" % self.function_calls[-1] + js.stdin.write(input) + stdout, stderr = js.communicate() + output = (stderr + stdout).strip() + for s in output.split('\n'): + log(s) + + m = re.match("'(.*)'", output, re.DOTALL) + if not m: + log("Error: %s" % output) + raise JSException(output) + return self.reinterpret(m.group(1)) + + def reinterpret(cls, s): + #while s.startswith(" "): + # s = s[1:] # :-) quite inneficient, but who cares + if s == 'false': + res = False + elif s == 'true': + res = True + elif s == 'undefined': + res = None + elif s == 'inf': + res = 1e300 * 1e300 + elif s == 'NaN': + res = (1e300 * 1e300) / (1e300 * 1e300) + elif s.startswith('[') or s.startswith('('): + l = s[1:-1].split(',') + res = [cls.reinterpret(i) for i in l] + else: + try: + res = float(s) + if float(int(res)) == res: + return int(res) + except ValueError: + res = str(s) + return res + reinterpret = classmethod(reinterpret) + +class JsTest(BaseRtypingTest, OORtypeMixin): + def _compile(self, _fn, args, policy=None): + argnames = _fn.func_code.co_varnames[:_fn.func_code.co_argcount] + func_name = _fn.func_name + if func_name == '': + func_name = 'func' + source = py.code.Source(""" + def %s(): + from pypy.rlib.nonconst import NonConstant + res = _fn(%s) + if isinstance(res, type(None)): + return None + else: + return str(res)""" + % (func_name, ",".join(["%s=NonConstant(%r)" % (name, i) for + name, i in zip(argnames, args)]))) + exec source.compile() in locals() + return compile_function(locals()[func_name], [], policy=policy) + + def string_to_ll(self, s): + return s + + def interpret(self, fn, args, policy=None): + f = self._compile(fn, args, policy) + res = f(*args) + return res + + def interpret_raises(self, exception, fn, args): + #import exceptions # needed by eval + #try: + #import pdb; pdb.set_trace() + try: + res = self.interpret(fn, args) + except JSException, e: + s = e.args[0] + assert s.startswith('uncaught exception:') + assert re.search(exception.__name__, s) + else: + raise AssertionError("Did not raise, returned %s" % res) + #except ExceptionWrapper, ex: + # assert issubclass(eval(ex.class_name), exception) + #else: + # assert False, 'function did raise no exception at all' + + def ll_to_string(self, s): + return str(s) + + def ll_to_list(self, l): + return l + + def ll_unpack_tuple(self, t, length): + assert len(t) == length + return tuple(t) + + def class_name(self, value): + return value[:-8].split('.')[-1] + + def is_of_instance_type(self, val): + m = re.match("^<.* object>$", val) + return bool(m) + + def read_attr(self, obj, name): + py.test.skip('read_attr not supported on genjs tests') + +def check_source_contains(compiled_function, pattern): + import re + + source = compiled_function.js.tmpfile.open().read() + return re.search(pattern, source) Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_basicexternal.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_basicexternal.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,129 @@ + +""" BasicExternal testers +""" + +import py + +from pypy.rpython.ootypesystem.bltregistry import MethodDesc, BasicExternal, described +from pypy.translator.js.test.runtest import compile_function, check_source_contains +from pypy.translator.js.tester import schedule_callbacks + +class A(BasicExternal): + @described(retval=int) + def some_code(self, var="aa"): + pass + +a = A() +a._render_name = 'a' + +class B(object): + pass + +def test_decorator(): + def dec_fun(): + a.some_code("aa") + + fun = compile_function(dec_fun, []) + assert check_source_contains(fun, "\.some_code") + +def test_basicexternal_element(): + def be_fun(): + b = B() + b.a = a + b.a.some_code("aa") + + fun = compile_function(be_fun, []) + assert check_source_contains(fun, "\.some_code") + +##def test_basicexternal_raise(): +## #py.test.skip("Constant BasicExternals not implemented") +## def raising_fun(): +## try: +## b = B() +## except: +## pass +## else: +## return 3 +## +## fun = compile_function(raising_fun, []) +## assert fun() == 3 + +class EE(BasicExternal): + @described(retval=int) + def bb(self): + pass + +ee = EE() +ee._render_name = 'ee' + +def test_prebuild_basicexternal(): + def tt_fun(): + ee.bb() + + fun = compile_function(tt_fun, []) + assert check_source_contains(fun, "EE = ee") + +class C(BasicExternal): + @described(retval=int) + def f(self): + pass + +c = C() +c._render_name = 'c' + +def test_basicexternal_raise_method_call(): + def raising_method_call(): + try: + c.f() + except: + pass + + fun = compile_function(raising_method_call, []) + assert len(C._methods) == 1 + assert 'f' in C._methods + +class D(BasicExternal): + _fields = { + 'a': {str:str}, + 'b': [str], + } + +D._fields['c'] = [D] + +d = D() +d._render_name = 'd' + +def test_basicexternal_list(): + def getaa(item): + return d.c[item] + + def return_list(i): + one = getaa(i) + if one: + two = getaa(i + 3) + return two + return one + + fun2 = compile_function(return_list, [int]) + +def test_basicextenal_dict(): + def return_dict(): + return d.a + + fun1 = compile_function(return_dict, []) + +def test_method_call(): + class Meth(BasicExternal): + @described(retval=int) + def meth(self): + return 8 + + l = [] + + def callback(i): + l.append(i) + + meth = Meth() + meth.meth(callback) + schedule_callbacks(meth) + assert l[0] == 8 Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_bltn.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_bltn.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,79 @@ +""" blttest - some tests of builtin stuff +""" + +import py + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, MethodDesc +from pypy.translator.js.test.runtest import compile_function, check_source_contains +from pypy.rpython.extfunc import genericcallable +from pypy.translator.js import commproxy + +# check rendering dom.document +def test_simple_builtin(): + from pypy.translator.js.modules.dom import document + def test_document_call(): + return document.getElementById("some_id") + + fn = compile_function(test_document_call, []) + assert check_source_contains(fn, "= document") + assert check_source_contains(fn, "\.getElementById") + +# check rendering transparent proxy +class SomeProxy(BasicExternal): + _render_xmlhttp = True + + _methods = { + 'some_method' : MethodDesc([], int), + } + +class SomeNode(BasicExternal): + _fields = { + 'some_callback' : genericcallable([int], int), + } + +SomeProxyInstance = SomeProxy() + +def test_simple_proxy(): + try: + oldproxy = commproxy.USE_MOCHIKIT + commproxy.USE_MOCHIKIT = True + def simple_proxy(): + retval = SomeProxyInstance.some_method() + + fn = compile_function(simple_proxy, []) + assert check_source_contains(fn, "loadJSONDoc\('some_method'") + finally: + commproxy.USE_MOCHIKIT = oldproxy + +SomeNodeInstance = SomeNode() +SomeNodeInstance._render_name = 's' + +# next will try out the callback +def test_callback(): + def callback(a): + return a + + def callback_stuff(): + SomeNodeInstance.some_callback = callback + + fn = compile_function(callback_stuff, []) + assert check_source_contains(fn, "\.some_callback = callback") + +def test_get_elements(): + from pypy.translator.js.modules import dom + + def getaa(tname): + return dom.document.getElementsByTagName(tname)[0].nodeValue + + def some_stuff(): + one = getaa("some") + if one: + two = getaa("other") + return two + # if two: + # return one + two + # else: + # return one + return one + + fn = compile_function(some_stuff, []) Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_class.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_class.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,146 @@ +from __future__ import division +import py + +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.llvm.test import llvmsnippet + +class TestClass(object): + def test_classsimple(self): + f = compile_function(llvmsnippet.class_simple, []) + assert f() == 14 + + def test_classsimple1(self): + f = compile_function(llvmsnippet.class_simple1, [int]) + assert f(2) == 10 + + def test_id_int(self): + #py.test.skip("Full list support not implemented") + f = compile_function(llvmsnippet.id_int, [int]) + for i in range(1, 20): + assert f(i) == i + + def test_classsimple2(self): + f = compile_function(llvmsnippet.class_simple2, [int]) + assert f(2) == 10 + + def test_inherit1(self): + f = compile_function(llvmsnippet.class_inherit1, []) + assert f() == 11 + + def test_inherit2(self): + #py.test.skip("Inheritance not implemented") + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + f = compile_function(llvmsnippet.class_inherit2, []) + assert f() == 1 + + def test_method_of_base_class(self): + #py.test.skip("Inheritance not implemented") + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + f = compile_function(llvmsnippet.method_of_base_class, []) + assert f() == 14 + + def test_attribute_from_base_class(self): + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.attribute_from_base_class, []) + assert f() == 4 + + def test_direct_call_of_virtual_method(self): + #py.test.skip("Inheritance not implemented") + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + f = compile_function(llvmsnippet.direct_call_of_virtual_method, []) + assert f() == 14 + + def test_flow_type(self): + #py.test.skip("isinstanceof not implemented") + f = compile_function(llvmsnippet.flow_type, []) + assert f() == 16 + + def test_merge_class(self): + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.merge_classes, [bool]) + assert f(True) == 1 + assert f(False) == 2 + + def test_attribute_instance(self): + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.attribute_instance, [bool]) + assert f(True) == 1 + assert f(False) == 2 + + def test_global_instance(self): #issue we restart every test with a fresh set of globals + f = compile_function(llvmsnippet.global_instance, [int]) + assert f(-1) == llvmsnippet.global_instance(-1) + for i in range(20): + x = f(i) + y = llvmsnippet.global_instance(i) + assert x == y + + def test_getset(self): + #py.test.skip("Basic arguments of const objects not implemented") + f = compile_function(llvmsnippet.testgetset, [int]) + assert f(15) == 25 + + def test_call_degrading_func(self): + #py.test.skip("isinstanceof not implemented") + f = compile_function(llvmsnippet.call_degrading_func, [bool]) + assert f(True) == llvmsnippet.call_degrading_func(True) + assert f(False) == llvmsnippet.call_degrading_func(False) + + def test_circular_classdef(self): + #py.test.skip("Problems with constant names") + #py.test.skip("Inheritance not implemented") + f = compile_function(llvmsnippet.circular_classdef, []) + assert f() == 10 + +class A(object): + def __init__(self, a): + self.a = a + if a is None: + self.b = 0 + else: + self.b = a.b + 3 + +a = A(None) + +def test_circular_class(): + def circular_class(): + b = A(a) + return b.b + + fn = compile_function(circular_class, []) + assert fn() == 3 + +INIT_VAL = 0 + +class B(object): + def __init__(self): + self.a = [INIT_VAL] * 30 + +b = B() + +def test_init_list(): + def init_list(i): + if i == 8: + b.a = [1,1,1] + return b.a[2] + + fn = compile_function(init_list, [int]) + assert fn(3) == INIT_VAL + assert fn(8) == 1 + +class C(object): + pass + +def test_instance_str(): + def instance_str(): + return str(C()) + + fn = compile_function(instance_str, []) + assert fn() == '' + +def test_instance_ret(): + def instance_ret(): + return str(C()) + + fn = compile_function(instance_ret, []) + assert fn() == '' Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_exc_operation.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_exc_operation.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,195 @@ +import py +import sys +from pypy.translator.js.test.runtest import compile_function +from pypy.rlib.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift +from pypy.translator.test import snippet + +py.test.skip("Exception work in progress") + +def test_zerodiv_int(): + def zerodiv_int(n): + try: + r=100/n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodiv_int, [int]) + for i in (-50,0,50): + assert f(i) == zerodiv_int(i) + +def test_zerodiv_uint(): + def zerodiv_uint(n): + try: + r=100/n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodiv_uint, [r_uint]) + for i in (0,50,100): + assert f(i) == zerodiv_uint(i) + +def test_zerodivrem_int(): + def zerodivrem_int(n): + try: + r=100%n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodivrem_int, [int]) + for i in (-50,0,50): + assert f(i) == zerodivrem_int(i) + +def test_zerodivrem_uint(): + def zerodivrem_uint(n): + try: + r=100%n + except ZeroDivisionError: + return n+7 + return r + f = compile_function(zerodivrem_uint, [r_uint]) + for i in (0,50,100): + assert f(i) == zerodivrem_uint(i) + +def DONTtest_neg_int_ovf(): #issue with Javascript Number() having a larger range + def neg_int_ovf(n): + try: + r=ovfcheck(-n) + except OverflowError: + return 123 + return r + f = compile_function(neg_int_ovf, [int]) + for i in (-sys.maxint-1, -sys.maxint, 0, sys.maxint-1, sys.maxint): + assert f(i) == neg_int_ovf(i) + +def DONTtest_abs_int_ovf(): #issue with Javascript Number() having a larger range + def abs_int_ovf(n): + try: + r=ovfcheck(abs(n)) + except OverflowError: + return 123 + return r + f = compile_function(abs_int_ovf, [int]) + for i in (-sys.maxint-1, -sys.maxint, 0, sys.maxint-1, sys.maxint): + assert f(i) == abs_int_ovf(i) + +############################ + +#raises(...) fails because we do'nt reraise javascript exceptions on the python level + +def DONTtest_int_ovf(): #issue with Javascript Number() having a larger range + def int_ovf_fn(i): + try: + return snippet.add_func(i) + except OverflowError: + return 123 + except: + return 1234 + fn = compile_function(int_ovf_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == int_ovf_fn(i) + +def test_int_div_ovf_zer(): + def int_div_ovf_zer_fn(i): + try: + return snippet.div_func(i) + except OverflowError: + return 123 + except ZeroDivisionError: + return 1234 + except: + return 12345 + fn = compile_function(int_div_ovf_zer_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == int_div_ovf_zer_fn(i) + +def DONTtest_int_mod_ovf_zer(): #issue with Javascript Number() having a larger range + def int_mod_ovf_zer_fn(i): + try: + return snippet.mod_func(i) + except OverflowError: + return 123 + except ZeroDivisionError: + return 1234 + except: + return 12345 + fn = compile_function(int_mod_ovf_zer_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == int_mod_ovf_zer_fn(i) + +def DONTtest_int_rshift_val(): #issue with Javascript Number() having a larger range + def rshift_fn(i): + try: + return snippet.rshift_func(i) + except ValueError: + return 123 + except: + return 1234 + fn = compile_function(rshift_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == rshift_fn(i) + +def test_int_lshift_ovf_val(): + def lshift_fn(i): + try: + return snippet.lshift_func(i) + except ValueError: + return 123 + except OverflowError: + return 1234 + except: + return 12345 + fn = compile_function(lshift_fn, [int]) + for i in (-sys.maxint-1, -1, 0, 1, sys.maxint): + assert fn(i) == lshift_fn(i) + +def test_uint_arith(): + py.test.skip("unsigned number becomes negative because we should cast to unsigned when necessary") + def fn(i): + try: + return ~(i*(i+1))/(i-1) + except ZeroDivisionError: + return r_uint(91872331) + f = compile_function(fn, [r_uint]) + for value in range(15): + i = r_uint(value) + assert f(i) == fn(i) + +def test_int_add_ovf(): + def add_func(i): + try: + return ovfcheck(i + 1) + except OverflowError: + return 123 + f = compile_function(add_func, [int]) + assert f(0) == add_func(0) + assert f(0) == 1 + assert f(sys.maxint) == add_func(sys.maxint) + assert f(sys.maxint) == 123 + +def test_int_sub_ovf(): + def sub_func(i): + try: + return ovfcheck(i - 1) + except OverflowError: + return 123 + n_ovf = 0 + f = compile_function(sub_func, [int]) + for i in (-1, 0, 1, sys.maxint, -sys.maxint, -sys.maxint-1): + result = f(i) + assert result == sub_func(i) + n_ovf += result == 123 + assert n_ovf == 1 + +#As JavaScript uses floating-point numbers the accuracy is only assured +#for integers between: -9007199254740992 (-2^53) and 9007199254740992 (2^53) +def DONTtest_shift_with_overflow(): #issue with Javascript Number() having a larger range + def shiftleft(x, y): + return x << y + def shiftright(x, y): + return x >> y + shl = compile_function(shiftleft , [int, int]) + shr = compile_function(shiftright, [int, int]) + for i in [1, 2, 3, 100000, 2000000, sys.maxint - 1]: + for j in [1, 2, 3, 100000, 2000000, sys.maxint - 1]: + assert shl(i, j) == i << j + assert shr(i, j) == i >> j Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_exception.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_exception.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,287 @@ +import py + +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.test.snippet import try_raise_choose +from pypy.rlib.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift + +import sys + +#py.test.skip("Exception work in progress") + +class TestException(Exception): + pass + +class MyException(Exception): + def __init__(self, n): + self.n = n + +def getitem(l, i): #LookupError, KeyError + if not isinstance(i, int): + raise TypeError + if i < 0: + i = len(l) - i + if i>= len(l): + raise IndexError + return l[i] + +def test_simple1(): + def raise_(i): + if i: + raise TestException() + else: + return 3 + def fn(i): + try: + a = raise_(i) + 11 + b = raise_(i) + 12 + c = raise_(i) + 13 + return a+b+c + except TestException: + return 7 + else: + return 3 + f = compile_function(fn, [int]) + assert f(0) == fn(0) + assert f(1) == fn(1) + +def test_simple2(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except: + return 2 + return 4 + f = compile_function(fn, [int]) + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == fn(10) + +def test_simple3(): + def raise_(i): + if i == 0: + raise TestException() + elif i == 1: + raise MyException(42) + else: + return 3 + def fn(i): + try: + a = raise_(i) + 11 + b = raise_(i) + 12 + c = raise_(i) + 13 + return a+b+c + except TestException: + return 7 + except MyException: + return 123 + except: + return 22 + return 66 + f = compile_function(fn, [int]) + assert f(0) == fn(0) + assert f(1) == fn(1) + assert f(2) == fn(2) + +def test_pass_exc(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except: + pass + return 4 + f = compile_function(fn, [int]) + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == fn(10) + +def test_reraise1(): + def fn2(n): + lst = range(10) + try: + getitem(lst,n) + except: + raise + return 4 + def fn(n): + try: + return fn2(n) + except: + return 123 + f = compile_function(fn, [int]) + assert f(-1) == 123 + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == 123 + assert f(10) == fn(10) + +def test_reraise2(): + def fn2(n): + lst = range(10) + try: + getitem(lst,n) + except Exception, e: + raise e + return 4 + def fn(n): + try: + return fn2(n) + except: + return 123 + f = compile_function(fn, [int]) + assert f(-1) == 123 + assert f(-1) == fn(-1) + assert f( 0) == fn( 0) + assert f(10) == 123 + assert f(10) == fn(10) + +def test_simple_exception(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except IndexError: + return 2 + return 4 + f = compile_function(fn, [int]) + for i in range(10): + assert f(i) == fn(i) + for i in range(10, 20): + assert f(i) == fn(i) + +def test_two_exceptionsA(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except IndexError: + return 2 + except KeyError: + return 3 + return 4 + f = compile_function(fn, [int]) + for i in range(10): + assert f(i) == fn(i) + for i in range(10, 20): + assert f(i) == fn(i) + +def test_catch_base_exception(): + def fn(n): + lst = range(10) + try: + getitem(lst,n) + except LookupError: + return 2 + return 4 + f = compile_function(fn, [int]) + for i in range(10): + assert f(i) == fn(i) + for i in range(10, 20): + assert f(i) == fn(i) + +def test_catches(): + def raises(i): + if i == 3: + raise MyException, 12 + if i == 4: + raise IndexError + if i > 5: + raise MyException(i) + return 1 + def fn(i): + try: + return raises(i) + except MyException, e: + return e.n + except: + return 123 + f = compile_function(fn, [int]) + assert f(1) == fn(1) + assert f(2) == fn(2) + assert f(3) == fn(3) + #py.test.raises(RuntimeError, "f(4)") #XXX would like to test: py.test.not_raises(....) + assert f(5) == fn(5) + assert f(6) == fn(6) + assert f(13) == fn(13) + +def test_try_raise_choose(): + f = compile_function(try_raise_choose, [int]) + for i in [-1, 0, 1, 2]: + assert f(i) == i + +def test_two_exceptionsB(): + def fn1(): + raise Exception + def fn2(): + return 10 + def two_exceptionsB(): + r = 50 + try: + fn1() + r += 1 + except: + r += 100 + try: + r += fn2() + except: + r += 300 + r += fn2() + return r + f = compile_function(two_exceptionsB, []) + assert f() == two_exceptionsB() + +def test_raise_outside_testfn(): + def raiser(n): + if n < 0: + raise ValueError("hello") + else: + raise MyException("world") + + def intermediate(n): + raiser(n) + + def testfn(n): + try: + intermediate(n) + except ValueError: + return 1 + except Exception: + return 2 + return 0 + + saved = no_magic() + try: + f = compile_function(testfn, [int]) + assert f(1) == testfn(1) + assert f(-1) == testfn(-1) + finally: + restore_magic(saved) + +def no_magic(): + import __builtin__ + try: + py.magic.revert(__builtin__, 'AssertionError') + return True + except ValueError: + return False + +def restore_magic(saved): + if saved: + py.magic.invoke(assertion=True) + +def test_always_raise(): + def function_raise2(i): + if i == 3: + raise IndexError() + else: + pass + + def function_raise1(i): + try: + function_raise2(i) + except Exception, e: + return str(e) + + fn = compile_function(function_raise1, [int]) + fn(3) Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_extfunc.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_extfunc.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,17 @@ + +""" Some external functions tests +""" + +from pypy.translator.js.test.runtest import compile_function, check_source_contains + +def test_set_timeout(): + from pypy.translator.js.modules.dom import setTimeout + + def to_timeout(): + pass + + def s_timeout_call(): + setTimeout(to_timeout, 300) + + c = compile_function(s_timeout_call, []) + assert check_source_contains(c, "setTimeout \( 'to_timeout\(\)',300 \)") Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_loops.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_loops.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,66 @@ +import py + +from pypy.translator.js.test.runtest import compile_function + +import sys + +def test_if_simple(): + def f(x): + if x == 1: + return 1 + else: + return 2 + fn = compile_function(f,[int]) + assert fn(1) == f(1) + assert fn(2) == f(2) + +def test_if_call(): + def check(x): + if x == 1: + return True + return False + + def f(x): + if check(x): + return 1 + else: + return 2 + + fn = compile_function(f,[int]) + assert fn(1) == f(1) + assert fn(2) == f(2) + +def test_while(): + def f(s): + while s > 1: + s -= 1 + return s + + fn = compile_function(f,[int]) + assert fn(38) == f(38) + assert fn(5) == f(5) + +def test_while_break(): + def f(s): + while s > 1: + if s == 8: + break + s -= 1 + return s + + fn = compile_function(f,[int]) + assert fn(38) == f(38) + assert fn(5) == f(5) + +def test_while_if(): + def f(x): + while x > 1: + if x%2 == 0: + x -= 7 + else: + x -= 3 + return x + + fn = compile_function(f,[int]) + assert fn(38) == f(38) + assert fn(5) == f(5) Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_main.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_main.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,69 @@ + +""" tests of rpython2javascript function +""" + +from pypy.translator.js.main import rpython2javascript, rpython2javascript_main,\ + js_optiondescr +from pypy.translator.js import conftest +from pypy.rpython.ootypesystem.bltregistry import BasicExternal, described +import py +import sys +from pypy.tool.udir import udir +from pypy.config.config import Config, to_optparse + + +#if not conftest.option.browser: +# py.test.skip("Works only in browser (right now?)") + +def setup_module(mod): + mod.jsconfig = Config(js_optiondescr) + +class A(BasicExternal): + def method(self, a={'a':'a'}): + pass + method = described(retval={str:str})(method) + +a = A() +a._render_name = 'a' + +def fun(x='3'): + return a.method({'a':x})['a'] + +def fff(): + pass + +def test_bookkeeper_cleanup(): + assert rpython2javascript(sys.modules[__name__], ["fun"]) + assert rpython2javascript(sys.modules[__name__], ["fun"]) + +def test_module_none(): + assert rpython2javascript(None, "fff") + +class TestJsMain(object): + def _test_not_raises(self, mod_file, args_rest=[]): + try: + rpython2javascript_main([str(mod_file)] + args_rest, + jsconfig) + except SystemExit: + py.test.fail("Exited") + + def _test_raises(self, mod_file, args_rest): + py.test.raises(SystemExit, rpython2javascript_main, + [str(mod_file)] + args_rest, jsconfig) + + def test_main_one(self): + udir.ensure("js_one.py").write(py.code.Source(""" + def f(): + pass + f._client = True + """)) + self._test_not_raises(udir.join("js_one.py")) + + def test_main_two(self): + udir.ensure("js_two.py").write(py.code.Source(""" + def f(): + pass + """)) + self._test_not_raises(udir.join("js_two.py"), ["f"]) + self._test_raises(udir.join("js_two.py"), []) + Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rclass.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rclass.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,77 @@ +import py +from pypy.translator.js.test.runtest import JsTest +from pypy.rpython.test.test_exception import BaseTestException +from pypy.rpython.test.test_rclass import BaseTestRclass +from pypy.rpython.test.test_rtuple import BaseTestRtuple +from pypy.rpython.test.test_rstr import BaseTestRstr + +class TestJsException(JsTest, BaseTestException): + pass + +# ====> ../../../rpython/test/test_rclass.py + +class TestJsClass(JsTest, BaseTestRclass): + def test___class___attribute(self): + class Base(object): pass + class A(Base): pass + class B(Base): pass + class C(A): pass + def seelater(): + C() + def f(n): + if n == 1: + x = A() + else: + x = B() + y = B() + result = x.__class__, y.__class__ + seelater() + return result + def g(): + cls1, cls2 = f(1) + return cls1 is A, cls2 is B + + res = self.interpret(g, []) + assert res[0] + assert res[1] + + def test_mixin(self): + class Mixin(object): + _mixin_ = True + + def m(self, v): + return v + + class Base(object): + pass + + class A(Base, Mixin): + pass + + class B(Base, Mixin): + pass + + class C(B): + pass + + def f(): + a = A() + v0 = a.m(2) + b = B() + v1 = b.m('x') + c = C() + v2 = c.m('y') + return v0, v1, v2 + + res = self.interpret(f, []) + assert isinstance(res[0], int) + + def test_hash_preservation(self): + py.test.skip("WIP") + + def test_issubclass_type(self): + py.test.skip("WIP") + + #def test_isinstance(self): + # py.test.skip("WIP") + Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rdict.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rdict.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,45 @@ + +import py +from pypy.rpython.test.test_rdict import BaseTestRdict +from pypy.translator.js.test.runtest import JsTest + +# ====> ../../../rpython/test/test_rdict.py + +class TestJsDict(JsTest, BaseTestRdict): + def test_tuple_dict(self): + py.test.skip("rdict not implemented") + + def test_dict_itermethods(self): + py.test.skip("itermethods not implemented") + + def test_dict_copy(self): + py.test.skip("itermethods not implemented") + + def test_dict_update(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_iterkeys(self): + py.test.skip("itermethods not implemented") + + def test_dict_values(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_values(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_itervalues(self): + py.test.skip("itermethods not implemented") + + def test_dict_items(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_items(self): + py.test.skip("itermethods not implemented") + + def test_dict_inst_iteritems(self): + py.test.skip("itermethods not implemented") + + + def test_specific_obscure_bug(self): + py.test.skip("rdict not implemented") + Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rfloat.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rfloat.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,24 @@ + +import py +from pypy.translator.js.test.runtest import JsTest +from pypy.rpython.test.test_rfloat import BaseTestRfloat +from pypy.rlib.rarithmetic import r_uint, r_longlong + +class TestJsFloat(JsTest, BaseTestRfloat): + def test_from_r_uint(self): + py.test.skip("Not implemented") + + def test_longlong_conversion(self): + def fn(f): + return r_longlong(f) + + res = self.interpret(fn, [1.0]) + assert res == 1 + #assert self.is_of_type(res, r_longlong) + res = self.interpret(fn, [2.34]) + assert res == fn(2.34) + big = float(0x7fffffffffffffff) + x = big - 1.e10 + assert x != big + y = fn(x) + assert fn(x) == 9223372026854775808 Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rlist.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rlist.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,85 @@ + +import py +from pypy.rpython.test.test_rlist import BaseTestRlist +from pypy.translator.js.test.runtest import JsTest + +class Foo: + pass + +class Bar(Foo): + pass + +# ====> ../../../rpython/test/test_rlist.py + +class TestJsList(JsTest, BaseTestRlist): + def test_append(self): + def dummyfn(): + l = [] + l.append(50) + l.append(60) + l.append(70) + l.append(80) + l.append(90) + return len(l), l[0], l[-1] + res = self.interpret(dummyfn, []) + assert res == [5, 50, 90] + + def test_slice(self): + py.test.skip("Imperfect testing machinery") + def dummyfn(): + l = [5, 6, 7, 8, 9] + return l[:2], l[1:4], l[3:] + res = self.interpret(dummyfn, []) + + def dummyfn(): + l = [5, 6, 7, 8] + l.append(9) + return l[:2], l[1:4], l[3:] + res = self.interpret(dummyfn, []) + assert res == ([5, 6], [6, 7, 8], [8, 9]) + + def test_setslice(self): + def dummyfn(): + l = [10, 9, 8, 7] + l[:2] = [6, 5] + return l[0], l[1], l[2], l[3] + res = self.interpret(dummyfn, ()) + assert res == [6, 5, 8, 7] + + def test_insert_bug(self): + def dummyfn(n): + l = [1] + l = l[:] + l.pop(0) + if n < 0: + l.insert(0, 42) + else: + l.insert(n, 42) + return l + res = self.interpret(dummyfn, [0]) + assert len(res) == 1 + assert res[0] == 42 + res = self.interpret(dummyfn, [-1]) + assert len(res) == 1 + assert res[0] == 42 + + def test_list_str(self): + pass + + def test_inst_list(self): + def fn(): + l = [None] + l[0] = Foo() + l.append(Bar()) + l2 = [l[1], l[0], l[0]] + l.extend(l2) + for x in l2: + l.append(x) + x = l.pop() + x = l.pop() + x = l.pop() + x = l2.pop() + return str(x)+";"+str(l) + res = self.ll_to_string(self.interpret(fn, [])) + res = res.replace('pypy.translator.js.test.test_rlist.', '') + assert res == ';[, , , , ]' Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rpbc.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rpbc.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,36 @@ + +import py +from pypy.translator.js.test.runtest import JsTest +from pypy.rpython.test.test_rpbc import BaseTestRPBC + +# ====> ../../../rpython/test/test_rpbc.py + +class TestJsPBC(JsTest, BaseTestRPBC): + def test_single_pbc_getattr(self): + class C: + def __init__(self, v1, v2): + self.v1 = v1 + self.v2 = v2 + def _freeze_(self): + return True + c1 = C(11, lambda: "hello") + c2 = C(22, lambda: 623) + def f1(l, c): + l.append(c.v1) + def f2(c): + return c.v2 + def f3(c): + return c.v2 + def g(): + l = [] + f1(l, c1) + f1(l, c2) + return f2(c1)(), f3(c2)() + + res = self.interpret(g, []) + assert res[0] == "hello" + assert res[1] == 623 + + def test_call_memoized_function_with_bools(self): + py.test.skip("WIP") + Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rsnippet.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rsnippet.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,453 @@ +from __future__ import division + +import sys +import py + +from pypy.rlib.rarithmetic import r_uint +from pypy.translator.js.test.runtest import compile_function + +def test_simple_function_pointer(): + #py.test.skip("ootypesystem problems with lists") + def f1(x): + return x + 1 + def f2(x): + return x + 2 + + l = [f1, f2] + + def pointersimple(i): + return l[i](i) + + f = compile_function(pointersimple, [int]) + assert f(0) == pointersimple(0) + assert f(1) == pointersimple(1) + +def test_return1(): + def simple1(): + return 1 + f = compile_function(simple1, []) + assert f() == 1 + +def test_invoke_function_pointer(): + #py.test.skip("ootypesystem problems with lists") + def f1(x): + return x + 1 + def f2(x): + return x + 2 + + l = [f1, f2] + + def invokepointer(i): + try: + return l[i](i) + except: + return 123 + + f = compile_function(invokepointer, [int]) + assert f(0) == invokepointer(0) + assert f(1) == invokepointer(1) + +def test_simple_branching(): + def simple5(b): + if b: + x = 12 + else: + x = 13 + return x + f = compile_function(simple5, [bool]) + assert f(True) == 12 + assert f(False) == 13 + +def test_int_ops(): + def ops(i): + x = 0 + x += i < i + x += i <= i + x += i == i + x += i != i + x += i >= i + x += i > i + x += x % i + x += i + 1 * i // i - 1 + return int(x+0.0001) + f = compile_function(ops, [int]) + assert f(1) == ops(1) + assert f(2) == ops(2) + +def test_uint_ops(): + def ops(i): + x = 0 + x += i < i + x += i <= i + x += i == i + x += i != i + x += i >= i + x += i > i + x += x % i + x += i + 1 * i // i - 1 + return int(x+0.0001) + f = compile_function(ops, [r_uint]) + assert f(1) == ops(1) + assert f(2) == ops(2) + +def test_float_ops(): + def ops(flt): + x = 0 + x += flt < flt + x += flt <= flt + x += flt == flt + x += flt != flt + x += flt >= flt + x += flt > flt + x += int(flt + 1 * flt / flt - 1) + return x + f = compile_function(ops, [float]) + assert f(1) == ops(1) + assert f(2) == ops(2) + +def test_while_loop(): + def factorial(i): + r = 1 + while i>1: + r *= i + i -= 1 + return r + f = compile_function(factorial, [int]) + assert factorial(4) == 24 + assert factorial(5) == 120 + f = compile_function(factorial, [float]) + assert factorial(4.) == 24. + assert factorial(5.) == 120. + +def test_return_void(): + def return_void(i): + return None + def call_return_void(i): + return_void(i) + return 1 + f = compile_function(call_return_void, [int]) + assert f(10) == 1 + +def test_break_while_loop(): + def factorial(i): + r = 1 + while 1: + if i<=1: + break + r *= i + i -= 1 + return r + f = compile_function(factorial, [int]) + assert factorial(4) == 24 + assert factorial(5) == 120 + +def test_primitive_is_true(): + def var_is_true(v): + return bool(v) + f = compile_function(var_is_true, [int]) + assert f(256) + assert not f(0) + f = compile_function(var_is_true, [r_uint]) + assert f(r_uint(256)) + assert not f(r_uint(0)) + f = compile_function(var_is_true, [float]) + assert f(256.0) + assert not f(0.0) + +def test_function_call(): + def callee(): + return 1 + def caller(): + return 3 + callee() + f = compile_function(caller, []) + assert f() == 4 + +def test_recursive_call(): + def call_ackermann(n, m): + return ackermann(n, m) + def ackermann(n, m): + if n == 0: + return m + 1 + if m == 0: + return ackermann(n - 1, 1) + return ackermann(n - 1, ackermann(n, m - 1)) + f = compile_function(call_ackermann, [int, int]) + assert f(0, 2) == 3 + +def test_tuple_getitem(): + def tuple_getitem(i): + l = (4,5,i) + return l[1] + f = compile_function(tuple_getitem, [int]) + assert f(1) == tuple_getitem(1) + +def test_nested_tuple(): + def nested_tuple(i): + l = (1,(1,2,i),i) + return l[1][2] + f = compile_function(nested_tuple, [int]) + assert f(4) == 4 + +def test_prebuilt_tuples(): + t1 = (1,2,3,4) + t2 = (5,6,7,8) + def callee_tuple(t): + return t[0] + def caller_tuple(i): + if i: + return callee_tuple(t1) + i + else: + return callee_tuple(t2) + i + f = compile_function(caller_tuple, [int]) + assert f(0) == 5 + assert f(1) == 2 + +def test_pbc_fns(): + #py.test.skip("Indirect call not implemented") + def f2(x): + return x+1 + def f3(x): + return x+2 + def g(y): + if y < 0: + f = f2 + else: + f = f3 + return f(y+3) + f = compile_function(g, [int]) + assert f(-1) == 3 + assert f(0) == 5 + +def test_simple_chars(): #XXX test this also without optimize_call(...) + #py.test.skip("Method mapping not implemented") + def char_constant2(s): + s = s + s + s + return len(s + '.') + def char_constant(): + return char_constant2("kk") + f = compile_function(char_constant, []) + assert f() == 7 + +def test_list_getitem(): + #py.test.skip("List suuport") + def list_getitem(i): + l = [1,2,i+1] + return l[i] + f = compile_function(list_getitem, [int]) + assert f(0) == 1 + assert f(1) == 2 + assert f(2) == 3 + +def test_list_list_getitem(): + #py.test.skip("List support") + def list_list_getitem(): + l = [[1]] + return l[0][0] + f = compile_function(list_list_getitem, []) + assert f() == 1 + +def test_list_getitem_pbc(): + #py.test.skip("pbc support") + l = [1,2] + def list_getitem_pbc(i): + return l[i] + f = compile_function(list_getitem_pbc, [int]) + assert f(0) == 1 + assert f(1) == 2 + +def test_list_list_getitem_pbc(): + #py.test.skip("pbc support") + l = [[0, 1], [0, 1]] + def list_list_getitem_pbc(i): + return l[i][i] + f = compile_function(list_list_getitem_pbc, [int]) + assert f(0) == 0 + assert f(1) == 1 + +def test_list_basic_ops(): + #py.test.skip("List support") + def list_basic_ops(i, j): + l = [1,2,3] + l.insert(0, 42) + del l[1] + l.append(i) + listlen = len(l) + l.extend(l) + del l[listlen:] + l += [5,6] + l[1] = i + return l[j] + f = compile_function(list_basic_ops, [int, int]) + for i in range(6): + for j in range(6): + assert f(i,j) == list_basic_ops(i,j) + +def test_string_simple(): + #py.test.skip("ord semantics") + def string_simple(i): + return ord(str(i)) + f = compile_function(string_simple, [int]) + assert f(3) == string_simple(3) + +def test_string_simple_ops(): + #py.test.skip("ord semantics") + def string_simple_ops(i): + res = 0 + s = str(i) + s2 = s + s + s + s + s3 = s + s + s + s + res += s != s2 + res += s2 == s3 + res += ord(s) + return res + f = compile_function(string_simple_ops, [int]) + assert f(5) == ord('5') + 2 + +def test_string_getitem1(): + l = "Hello, World" + def string_getitem1(i): + return l[i] + f = compile_function(string_getitem1, [int]) + assert f(0) == "H" + +def test_string_getitem2(): + def string_test(i): + l = "Hello, World" + return l[i] + f = compile_function(string_test, [int]) + assert f(0) == "H" + +def test_list_of_string(): + a = ["hello", "world"] + def string_simple(i, j, k, l): + s = a[i][j] + a[k][l] + return s[0] + s[1] + f = compile_function(string_simple, [int, int, int, int]) + assert f(0, 0, 1, 4) == 'hd' + +def test_attrs_class(): + class MyBase: + pass + def attrs_class(a): + obj = MyBase() + obj.z = a + return obj.z * 4 + f = compile_function(attrs_class, [int]) + assert f(4) == 16 + +def test_attrs_class_pbc(): +# py.test.skip("pbc support") + class MyBase: + pass + obj = MyBase() + obj.z = 4 + def attrs_class_pbc(): + return obj.z * 4 + f = compile_function(attrs_class_pbc, []) + assert f() == 16 + +def test_method_call(): + class MyBase: + def m(self): return self.z + obj = MyBase() + obj.z = 4 + def method_call(): + return obj.m() + f = compile_function(method_call, []) + assert f() == 4 + +def test_dict_creation(): + d = {'hello' : 23, + 'world' : 21} + l = ["hello", "world"] + def createdict(i, j): + return d[l[i]] + d[l[j]] + assert createdict(0,1) == 44 + f = compile_function(createdict, [int, int]) + assert f(0,1) == createdict(0,1) + +def test_closure(): + class A: + def set(self, x): + self.x = x + def get(self): + return self.x + class B(A): + def get(self): + return A.get(self) * 2 + a = A() + b = B() + def createfn(y): + z = 42 + def fn(x): + return x + y + z + a.get() + b.get() + f2 = lambda: A.get(b) + def getfn(x): + if x % 2: + f = A.get + else: + f = B.get + return f + def proxy(s): + f1 = s.get + t = 0 + for ii in range(5): + f3 = getfn(ii) + t += f1() + t += f2() + t += f3(s) + return t + setattr(B, "f2", proxy) + return fn + fn = createfn(10) + def testf(x): + a.set(10) + b.set(25) + return fn(x) + b.get() + b.f2() + f = compile_function(testf, [int]) + assert f(1) == testf(1) + assert f(2) == testf(2) + +def test_cast_str(): + def cast_str(x): + return str(x)+str(x)+'px' + + f = compile_function(cast_str, [int]) + assert f(1) == cast_str(1) + assert f(10) == cast_str(10) + +class A(object): + pass + +def some_fun(): + return 3 + +def test_me(): + def some_test(): + a = A() + a.some_fun = some_fun + return a.some_fun() + + fn = compile_function(some_test, []) + assert fn() == some_test() + +def test_symbolic(): + from pypy.rlib.objectmodel import malloc_zero_filled + + def symbolic1(): + if malloc_zero_filled: + return 3 + return 2 + + fn = compile_function(symbolic1, []) + assert fn() == 2 + +def test_is_early_constant(): + from pypy.rlib import jit + def f(x): + if jit._is_early_constant(x): + return 42 + return 0 + fn = compile_function(f, [int]) + res = fn(5) + assert res == 0 Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_rsnippet1.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_rsnippet1.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,91 @@ +from __future__ import division +import sys + +import py + +from pypy.objspace.flow.model import Constant, Variable +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.llvm.test import llvmsnippet + +class TestGenLLVM(object): + def test_simple1(self): + f = compile_function(llvmsnippet.simple1, []) + assert f() == 1 + + def test_simple2(self): + f = compile_function(llvmsnippet.simple2, []) + assert f() == False + + def test_simple4(self): + f = compile_function(llvmsnippet.simple4, []) + assert f() == 4 + + def test_simple5(self): + f = compile_function(llvmsnippet.simple5, [int]) + assert f(1) == 12 + assert f(0) == 13 + + def test_ackermann(self): + f = compile_function(llvmsnippet.ackermann, [int, int]) + for i in range(4): # (otherwise too much recursion) max 4 in Safari, max 7 in Firefox, IE allows more recursion + assert f(0, i) == i + 1 + assert f(1, i) == i + 2 + assert f(2, i) == 2 * i + 3 + assert f(3, i) == 2 ** (i + 3) - 3 + + def test_calling(self): + f = compile_function(llvmsnippet.calling1, [int]) + assert f(10) == 1 + + def test_call_default_arguments(self): + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.call_default_arguments, [int, int]) + for i in range(3): + assert f(i + 3, i) == llvmsnippet.call_default_arguments(i + 3, i) + + def DONTtest_call_list_default_argument(self): #issue we restart every test with a fresh set of globals + f = compile_function(llvmsnippet.call_list_default_argument, [int]) + for i in range(20): + assert f(i) == llvmsnippet.call_list_default_argument(i) + + def test_shift(self): + shl = compile_function(llvmsnippet.shiftleft, [int, int]) + shr = compile_function(llvmsnippet.shiftright, [int, int]) + assert shl(42,2) == llvmsnippet.shiftleft(42, 2) + assert shr(42,2) == llvmsnippet.shiftright(42,2) + +class TestFloat(object): + def test_float_f1(self): + f = compile_function(llvmsnippet.float_f1, [float]) + assert f(1.0) == 2.2 + + def test_float_int_bool(self): + f = compile_function(llvmsnippet.float_int_bool, [float]) + assert f(3.0) == 9.0 + + +class TestString(object): + def test_f2(self): + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.string_f2, [int, int]) + assert f(1, 0) == "a" + + +class TestPBC(object): + #py.test.skip("pbc not implemented") + def test_pbc_function1(self): + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.pbc_function1, [int]) + assert f(0) == 2 + assert f(1) == 4 + assert f(2) == 6 + assert f(3) == 8 + + def test_pbc_function2(self): + #py.test.skip("issue 'null' for Ptr's? or recurse into Ptr.TO?) see: opwriter.py") + #py.test.skip("Method mapping not implemented") + f = compile_function(llvmsnippet.pbc_function2, [int]) + assert f(0) == 13 + assert f(1) == 15 + assert f(2) == 17 + assert f(3) == 19 Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_runtest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_runtest.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,75 @@ +import py +from pypy.translator.js.test.runtest import compile_function + +#test returntypes +def test_bool_return(): + def bool_return_False(): + return False + def bool_return_True(): + return True + f_false = compile_function(bool_return_False, []) + assert f_false() == bool_return_False() + f_true = compile_function(bool_return_True , []) + assert f_true() == bool_return_True() + +def test_int_return(): + def int_return(): + return 42 + f = compile_function(int_return, []) + assert f() == int_return() + +def test_float_return(): + def float_return(): + return 42.5 + f = compile_function(float_return, []) + assert f() == float_return() + +#test paramtypes +def test_bool_param(): + def bool_param(b): + return b + f = compile_function(bool_param, [bool]) + assert f(False) == bool_param(False) + assert f(True ) == bool_param(True ) + +def test_int_param(): + def int_param(n): + return n * 2 + f = compile_function(int_param, [int]) + assert f(42) == int_param(42) + +def test_float_param(): + def float_param(f): + return f * 3.0 + f = compile_function(float_param, [float]) + assert f(12.5) == float_param(12.5) + +def test_combined_params(): + def combined_params(b, n, f): + return int(int(b) * 5 + n * 2 + f * 3.0) + f = compile_function(combined_params, [bool,int,float]) + assert f(False, 13, 12.5) == combined_params(False, 13, 12.5) + assert f(True , 13, 12.5) == combined_params(True , 13, 12.5) + +def test_return_function(): + rp = compile_function.reinterpret + assert rp('[a,b]') == ["a", "b"] + #assert rp('(true,[a,b])') == [True, ["a", "b"]] + +def test_return_newline(): + def fun_newline(): + return "\n" + fun = compile_function(fun_newline, []) + assert fun() == "\n" + +##def test_multiple_function(): +## def one(): +## return 1 +## +## def two(x): +## return x +## +## f = compile_function([one, two], [[], [int]]) +## assert f.call(one) == 1 +## assert f.call(two, 3) == 3 +## Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_seq.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_seq.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,138 @@ +from __future__ import division +import py + +from pypy.objspace.flow.model import Constant, Variable +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.llvm.test import llvmsnippet + +class TestList(object): + def test_normal_list(self): + def normal_list(): + l = [1,2,3] + return l[1] + + fn = compile_function(normal_list, []) + assert fn() == 2 + + def test_var_list(self): + def var_list(): + l = [] + for i in xrange(10): + l.append(i) + return l[8] + + fn = compile_function(var_list, []) + assert fn() == 8 + + def test_slice(self): + def l_test(): + l = [] + for i in xrange(10): + l.append(i) + return len(l[3:8]) + + fn = compile_function(l_test, []) + assert fn() == l_test() + + def test_init_0(self): + def l_init(): + l = [0] * 100 + return l[38] + + fn = compile_function(l_init, []) + assert fn() == 0 + +class TestDict(object): + def test_dict_iter(self): + def dict_iter(): + sum = 0 + d = {'a':3, 'b':4, 'c':8} + for k in d: + sum += d[k] + return sum + + fn = compile_function(dict_iter, []) + assert fn() == dict_iter() + + def test_const_dict(self): + c = {'aa':'asd', 'ab':'ds', 'ac':'asddsa', 'ad':'sadsad', 'ae':'sadsa'} + + def const_dict(ind): + return c[ind] + + fn = compile_function(const_dict, [str]) + for i in c.keys(): + assert fn(i) == const_dict(i) + + def test_sophisticated_const(self): + c = {'aa':1, 'bb':2, 'cc':3, 'dd':4} + c1 = {'fff' : 'aa', 'zzz' : 'bb', 'xxx' : 'cc', 'bbb' : 'dd'} + c2 = {'xxx' : 'aa', 'kkk' : 'bb', 'aaaa' : 'cc', 'www' : 'dd'} + def soph_const(i1, i2): + return c1[i1] + c2[i2] + + fn = compile_function(soph_const, [str, str]) + for x in c1.keys(): + for y in c2.keys(): + assert fn(x, y) == soph_const(x, y) + + def test_dict_iterator(self): + c = {'aa':1, 'bb':2, 'cc':3, 'dd':4} + def dict_iter(): + sum = 0 + for i in c: + sum += c[i] + return sum + + fn = compile_function(dict_iter, []) + assert fn() == dict_iter() + +class TestTuple(object): + def test_f1(self): + f = compile_function(llvmsnippet.tuple_f1, [int]) + assert f(10) == 10 + assert f(15) == 15 + + def test_f3(self): + f = compile_function(llvmsnippet.tuple_f3, [int]) + assert f(10) == 10 + assert f(15) == 15 + assert f(30) == 30 + + def test_constant_tuple(self): + f = compile_function(llvmsnippet.constant_tuple, [int]) + for i in range(3, 7): + assert f(i) == i + 3 + +class TestString(object): + def test_upperlower(self): + def upperlower(): + s = "aBaF" + return s.upper() + s.lower() + + fn = compile_function(upperlower, []) + assert fn() == "ABAFabaf" + + def test_slice(self): + def one_slice(s): + return s[1:] + + def two_slice(s): + return s[2:] + + fn = compile_function(one_slice, [str]) + assert fn("dupa") == "upa" + fn = compile_function(two_slice, [str]) + assert fn("kupa") == "pa" + +def test_simple_seq(): + def fun(i): + if i: + a = [("ab", "cd"), ("ef", "xy")] + else: + a = [("xz", "pr"), ("as", "fg")] + return ",".join(["%s : %s" % (i, j) for i,j in a]) + + fn = compile_function(fun, [int]) + assert fn(0) == fun(0) + assert fn(1) == fun(1) Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_snippet.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_snippet.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,94 @@ +from __future__ import division + +from pypy.translator.js.test.runtest import compile_function +from pypy.translator.test import snippet as test + +import py + +class TestSnippet(object): + def test_if_then_else(self): + f = compile_function(test.if_then_else, [int, int, int]) + assert f(0, 12, 13) == 13 + assert f(13, 12, 13) == 12 + + def test_my_gcd(self): + f = compile_function(test.my_gcd, [int, int]) + assert f(15, 5) == 5 + assert f(18, 42) == 6 + + def test_is_perfect_number(self): + f = compile_function(test.is_perfect_number, [int]) + assert f(28) == 1 + assert f(123) == 0 + assert f(496) == 1 + + def test_my_bool(self): + f = compile_function(test.my_bool, [int]) + assert f(10) == 1 + assert f(1) == 1 + assert f(0) == 0 + + def test_two_plus_two(self): + f = compile_function(test.two_plus_two, []) + assert f() == 4 + + def test_sieve_of_eratosthenes(self): + #py.test.skip("Loop error in loop detection software") + f = compile_function(test.sieve_of_eratosthenes, []) + assert f() == 1028 + + def test_nested_whiles(self): + #py.test.skip("Loop error in loop detection software") + f = compile_function(test.nested_whiles, [int, int]) + assert test.nested_whiles(3,2) == f(3,2) + + def test_simple_func(self): + f = compile_function(test.simple_func, [int]) + assert f(1027) == 1028 + + def test_while_func(self): + while_func = compile_function(test.while_func, [int]) + assert while_func(10) == 55 + + def test_time_waster(self): + #py.test.skip("Loop error in loop detection software") + f = compile_function(test.time_waster, [int]) + assert f(1) == 1 + assert f(2) == 2 + assert f(3) == 6 + assert f(4) == 12 + + def test_int_id(self): + f = compile_function(test.int_id, [int]) + assert f(1027) == 1027 + + def test_factorial2(self): + #py.test.skip("unknown error") + factorial2 = compile_function(test.factorial2, [int]) + assert factorial2(5) == 120 + + def test_factorial(self): + #py.test.skip("unknown error") + factorial = compile_function(test.factorial, [int]) + assert factorial(5) == 120 + + def test_set_attr(self): + set_attr = compile_function(test.set_attr, []) + assert set_attr() == 2 + + def test_merge_setattr(self): + merge_setattr = compile_function(test.merge_setattr, [bool]) + assert merge_setattr(1) == 1 + + def test_simple_method(self): + simple_method = compile_function(test.simple_method, [int]) + assert simple_method(65) == 65 + + def test_with_init(self): + with_init = compile_function(test.with_init, [int]) + assert with_init(42) == 42 + + def test_with_more_init(self): + with_more_init = compile_function(test.with_more_init, [int, bool]) + assert with_more_init(42, True) == 42 + assert with_more_init(42, False) == -42 Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_str.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_str.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,72 @@ + +import py +from pypy.translator.js.test.runtest import JsTest +import pypy.translator.oosupport.test_template.string as oostring + + +# ====> ../../../rpython/test/test_rstr.py + +class TestJsString(JsTest, oostring.BaseTestString): + def test_char_unichar_eq(self): + py.test.skip("Cannot test it yet") + + def test_char_unichar_eq_2(self): + py.test.skip("Cannot test it yet") + + def test_unichar_const(self): + py.test.skip("Cannot test it yet") + + def test_unichar_eq(self): + py.test.skip("Cannot test it yet") + + def test_unichar_ord(self): + py.test.skip("Cannot test it yet") + + def test_unichar_hash(self): + py.test.skip("Cannot test it yet") + + def test_rfind(self): + py.test.skip("Not implemented") + + def test_rfind_empty_string(self): + py.test.skip("Not implemented") + + def test_find_char(self): + py.test.skip("Not implemented") + + def test_strip(self): + py.test.skip("Not implemented") + + def test_upper(self): + #XXX Testing machinery is quite confused by js print + strings = ['', ' ', 'upper', 'UpPeR', ',uppEr,'] + #for i in range(256): + # if chr(i) != "\x00" and chr(i) != '(' and chr(i) != '[': + # strings.append(chr(i)) + def fn(i): + return strings[i].upper() + for i in range(len(strings)): + res = self.interpret(fn, [i]) + assert self.ll_to_string(res) == fn(i) + + def test_lower(self): + #XXX Testing machinery is quite confused by js print + strings = ['', ' ', 'lower', 'LoWeR', ',lowEr,'] + #for i in range(256): strings.append(chr(i)) + def fn(i): + return strings[i].lower() + for i in range(len(strings)): + res = self.interpret(fn, [i]) + assert self.ll_to_string(res) == fn(i) + + def test_strformat(self): + py.test.skip("string formatting not implemented for base different than 10") + + def test_float(self): + py.test.skip("returning NaN instead of raising ValueError") + + def test_hash(self): + py.test.skip("Not implemented") + + def test_hash_value(self): + py.test.skip("Not implemented") Added: pypy/branch/flex-backend/pypy/translator/flex/test/test_typed.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/test_typed.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,331 @@ +import sys +import py +from py.test import raises +from pypy.translator.test import snippet +from pypy.rlib.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift + +from pypy.translator.js.test.runtest import compile_function + +def test_call_five(): + def wrapper(): + lst = snippet.call_five() + res = list((len(lst), lst[0])) + expected = [1, 5] + return res == expected + fn = compile_function(wrapper, []) + result = fn() + assert result == wrapper() + +def test_get_set_del_slice(): + def get_set_del_nonneg_slice(): # no neg slices for now! + l = [ord('a'), ord('b'), ord('c'), ord('d'), ord('e'), ord('f'), ord('g'), ord('h'), ord('i'), ord('j')] + del l[:1] + bound = len(l)-1 + if bound >= 0: + del l[bound:] + del l[2:4] + #l[:1] = [3] + #bound = len(l)-1 + #assert bound >= 0 + #l[bound:] = [9] no setting slice into lists for now + #l[2:4] = [8,11] + l[0], l[-1], l[2], l[3] = 3, 9, 8, 11 + + list_3_c = l[:2] + list_9 = l[5:] + list_11_h = l[3:5] + return list((len(l), l[0], l[1], l[2], l[3], l[4], l[5], + len(list_3_c), list_3_c[0], list_3_c[1], + len(list_9), list_9[0], + len(list_11_h), list_11_h[0], list_11_h[1])) + + def wrapper(): + res = get_set_del_nonneg_slice() + expected = [6, 3, ord('c'), 8, 11, ord('h'), 9, + 2, 3, ord('c'), + 1, 9, + 2, 11, ord('h')] + + return res == expected + + fn = compile_function(wrapper, []) + result = fn() + assert result + +def test_is(): + def testfn(): + l1 = [] + return l1 is l1 + fn = compile_function(testfn, []) + result = fn() + assert result == True + def testfn(): + l1 = [] + return l1 is None + fn = compile_function(testfn, []) + result = fn() + assert result == False + +def test_nones(): + a = [None] * 4 + def nones(): + a.append(None) + return len(a) + fn = compile_function(nones, []) + result = fn() + assert result == 4 + +def test_str_compare(): + def testfn_eq(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] == s2[j] + fn = compile_function(testfn_eq, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_eq(i, j) + + def testfn_ne(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] != s2[j] + fn = compile_function(testfn_ne, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_ne(i, j) + + def testfn_lt(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] < s2[j] + fn = compile_function(testfn_lt, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_lt(i, j) + + def testfn_le(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] <= s2[j] + fn = compile_function(testfn_le, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_le(i, j) + + def testfn_gt(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] > s2[j] + fn = compile_function(testfn_gt, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_gt(i, j) + + def testfn_ge(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] + return s1[i] >= s2[j] + fn = compile_function(testfn_ge, [int, int]) + for i in range(2): + for j in range(6): + res = fn(i, j) + assert res == testfn_ge(i, j) + +def test_str_methods(): + def testfn_startswith(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'ne', 'e', 'twos', 'foobar', 'fortytwo'] + return s1[i].startswith(s2[j]) + fn = compile_function(testfn_startswith, [int, int]) + for i in range(2): + for j in range(9): + res = fn(i, j) + assert res == testfn_startswith(i, j) + + def testfn_endswith(i, j): + s1 = ['one', 'two'] + s2 = ['one', 'two', 'o', 'on', 'ne', 'e', 'twos', 'foobar', 'fortytwo'] + return s1[i].endswith(s2[j]) + fn = compile_function(testfn_endswith, [int, int]) + for i in range(2): + for j in range(9): + res = fn(i, j) + assert res == testfn_endswith(i, j) + +def test_str_join(): + #py.test.skip("issue with malloc_varsize of varsized struct (rpystring here)") + #py.test.skip("stringBuilder support") + def testfn(i, j): + s1 = [ '', ',', ' and '] + s2 = [ [], ['foo'], ['bar', 'baz', 'bazz']] + return len(s1[i].join(s2[j])) + + fn = compile_function(testfn, [int, int]) + for i in range(3): + for j in range(3): + res = fn(i, j) + assert res == testfn(i, j) + +def test_unichr_eq(): + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i, j): + return l[i] == l[j] + fn = compile_function(f, [int, int]) + for i in range(len(l)): + for j in range(len(l)): + res = fn(i, j) + assert res == f(i,j) + +def test_unichr_ne(): + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i, j): + return l[i] != l[j] + fn = compile_function(f, [int, int]) + for i in range(len(l)): + for j in range(len(l)): + res = fn(i, j) + assert res == f(i, j) + +def test_unichr_ord(): + #py.test.skip("ord semantics") + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i): + return ord(l[i]) + fn = compile_function(f, [int]) + for i in range(len(l)): + res = fn(i) + assert res == f(i) + +def test_unichr_unichr(): + #py.test.skip("Unicode support") + l = list(u'Hello world') + def f(i, j): + return l[i] == unichr(j) + fn = compile_function(f, [int, int]) + for i in range(len(l)): + for j in range(len(l)): + res = fn(i, ord(l[j])) + assert res == f(i, ord(l[j])) + +# floats +def test_float_operations(): + #py.test.skip("issue with ll_math_fmod calling itself") + import math + def func(x, y): + z = x + y / 2.1 * x + z = math.fmod(z, 60.0) + z = pow(z, 2) + z = -z + return int(z) + + fn = compile_function(func, [float, float]) + r1 = fn(5.0, 6.0) + r2 = func(5.0, 6.0) + assert r1 == r2-1 #-1 for stupid spidermonkey rounding error + +def test_rpbc_bound_method_static_call(): + class R: + def meth(self): + return 0 + r = R() + m = r.meth + def fn(): + return m() + res = compile_function(fn, [])() + assert res == 0 + +def test_constant_return_disagreement(): + class R: + def meth(self): + return 0 + r = R() + def fn(): + return r.meth() + res = compile_function(fn, [])() + assert res == 0 + +def test_stringformatting(): + #py.test.skip("StringBuilder not implemented") + def fn(i): + return "you said %d, you did" % i + def wrapper(i): + res = fn(i) + return res == "you said 42, you did" + + f = compile_function(wrapper, [int]) + assert f(42) + +def test_int2str(): + def fn(i): + return str(i) + def wrapper(i): + res = fn(i) + return res == "42" + + f = compile_function(wrapper, [int]) + assert f(42) + +def test_int_invert(): + def fn(i): + return ~i + f = compile_function(fn, [int]) + for i in range(-15, 15): + assert f(i) == fn(i) + +def DONTtest_uint_invert(): #issue with Javascript Number() having a larger range + def fn(i): + inverted = ~i + inverted -= sys.maxint + return inverted + f = compile_function(fn, [r_uint]) + for value in range(1, 15): + i = r_uint(value) + assert str(f(i)) == str(fn(i)) + s = 0xfffffff + for value in range(s, s+1024, 64): + i = r_uint(value) + assert str(f(i)) == str(fn(i)) + +def test_int_abs(): + def int_abs_(n): + return abs(n) + f = compile_function(int_abs_, [int]) + for i in (-25, 0, 75): + assert f(i) == int_abs_(i) + +def test_float_abs(): + def float_abs_(n): + return abs(n) + f = compile_function(float_abs_, [float]) + for i in (-100.1 -50.2, -0.0, 0.0, 25.3, 50.4): + assert f(i) == float_abs_(i) + +def test_cast_to_int(): + def casting(v): + return int(ord(chr(v))) + f = compile_function(casting, [int]) + for ii in range(255): + assert f(ii) == ii + +def test_char_comparisons(): + #py.test.skip("chr/ord semantics") + def comps(v): + x = chr(v) + res = 0 + res += x < chr(0) + res += x > chr(1) + res += x >= chr(127) + res += x < chr(128) + res += x < chr(250) + return res + f = compile_function(comps, [int]) + for ii in range(255): + assert f(ii) == comps(ii) Added: pypy/branch/flex-backend/pypy/translator/flex/test/tgtest.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/test/tgtest.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,42 @@ + +""" TurboGears browser testing utility +""" + +import thread + +import pkg_resources +pkg_resources.require("TurboGears") + +import cherrypy +import os +import sys +import webbrowser + +from pypy.translator.js.demo.jsdemo import controllers + +conf_file = os.path.join(os.path.dirname(controllers.__file__), "..", "dev.cfg") + +class run_tgtest(object): + def __init__(self, compiled_fun, tg_root = None, port = 8080, run_browser=True): + def cont(): + cherrypy.server.wait() + if run_browser: + webbrowser.open("http://localhost:%d/" % port) + cherrypy.root.wait_for_results() + self.results = cherrypy.root.results + cherrypy.server.stop() + cherrypy.server.interrupt = SystemExit() + + cherrypy.config.update(file=conf_file) + cherrypy.config.update({'global':{'server.socketPort':port}}) + + if tg_root is None: + cherrypy.root = controllers.Root() + else: + cherrypy.root = tg_root() + cherrypy.root.jssource = compiled_fun.js.tmpfile.open().read() + cherrypy.root.jsname = compiled_fun.js.translator.graphs[0].name + + thread.start_new_thread(cont, ()) + sys.path.insert(1, os.path.join(os.path.dirname(controllers.__file__), "..")) + cherrypy.server.start() Added: pypy/branch/flex-backend/pypy/translator/flex/tester.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/tester.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,34 @@ + +""" tester - support module for testing js code inside python + +Needs to be imported in case one wants tests involving calling BasicExternal +methods +""" + +from pypy.rpython.ootypesystem.bltregistry import BasicExternal + +def __getattr__(self, attr): + val = super(BasicExternal, self).__getattribute__(attr) + if not callable(val) or attr not in self._methods: + return val # we don't do anything special + # otherwise.... + def wrapper(*args, **kwargs): + args = list(args) + # do this only if last arg is callable + if not (len(args) > 0 and callable(args[-1])): + return val(*args, **kwargs) + callback = args.pop() + res = val(*args, **kwargs) + if not hasattr(self, '__callbacks'): + self.__callbacks = [] + self.__callbacks.append((callback, res)) + wrapper.func_name = attr + return wrapper + +BasicExternal.__getattribute__ = __getattr__ + +def schedule_callbacks(*args): + for arg in args: + if hasattr(arg, '__callbacks'): + for callback, res in arg.__callbacks: + callback(res) Added: pypy/branch/flex-backend/pypy/translator/flex/tutorial/__init__.py ============================================================================== Added: pypy/branch/flex-backend/pypy/translator/flex/tutorial/step1.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/tutorial/step1.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,27 @@ +#!/usr/bin/env python +""" + +This is simple all-in-one self-containing server, +which just shows almost-empty HTML page + +""" + +# here we import server, which is derivative of +# BaseHTTPServer from python standard library +from pypy.translator.js.lib import server + +# We create handler, which will handle all our requests +class Handler(server.TestHandler): + + def index(self): + # provide some html contents + return "

Something

" + # this line is necessary to make server show something, + # otherwise method is considered private-only + index.exposed = True + +if __name__ == '__main__': + # let's start our server, + # this will create running server instance on port 8000 by default, + # which will run until we press Control-C + server.create_server(handler=Handler).serve_forever() Added: pypy/branch/flex-backend/pypy/translator/flex/tutorial/step2.py ============================================================================== --- (empty file) +++ pypy/branch/flex-backend/pypy/translator/flex/tutorial/step2.py Sat Jul 14 12:31:00 2007 @@ -0,0 +1,52 @@ +#!/usr/bin/env python +""" +In this example, we'll show how to add javascript to our simple +server from previous example +""" + +from pypy.translator.js.lib import server +import sys + +# here we have virtual script "source.js" which we generate +# on-the-fly when requested +HTML = """ + + +