From arigo at codespeak.net Sat Apr 1 09:54:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 09:54:47 +0200 (CEST) Subject: [pypy-svn] r25199 - in pypy/dist/pypy: doc doc/discussion doc/weekly interpreter/pyparser/test/samples jit/codegen jit/codegen/llvm jit/codegen/llvm/test lib/logic/computation_space lib/logic/gecode_wrapper objspace objspace/std objspace/test rpython/ootypesystem rpython/rctypes/test tool translator/backendopt translator/backendopt/test translator/goal translator/llvm/pyllvm translator/llvm/pyllvm/llvmcapi translator/llvm/pyllvm/test translator/squeak translator/squeak/test Message-ID: <20060401075447.A635C100C3@code0.codespeak.net> Author: arigo Date: Sat Apr 1 09:54:38 2006 New Revision: 25199 Modified: pypy/dist/pypy/doc/discussion/use_case_of_logic.txt (props changed) pypy/dist/pypy/doc/jit.txt (props changed) pypy/dist/pypy/doc/weekly/summary-2006-03-15.txt (props changed) pypy/dist/pypy/interpreter/pyparser/test/samples/snippet_with_1.py (props changed) pypy/dist/pypy/interpreter/pyparser/test/samples/snippet_with_2.py (props changed) pypy/dist/pypy/jit/codegen/ (props changed) pypy/dist/pypy/jit/codegen/__init__.py (props changed) pypy/dist/pypy/jit/codegen/llvm/ (props changed) pypy/dist/pypy/jit/codegen/llvm/__init__.py (props changed) pypy/dist/pypy/jit/codegen/llvm/jitcode.py (props changed) pypy/dist/pypy/jit/codegen/llvm/rgenop.py (props changed) pypy/dist/pypy/jit/codegen/llvm/test/ (props changed) pypy/dist/pypy/jit/codegen/llvm/test/test_rgenop.py (props changed) pypy/dist/pypy/lib/logic/computation_space/parsesudoku.py (props changed) pypy/dist/pypy/lib/logic/gecode_wrapper/ (props changed) pypy/dist/pypy/lib/logic/gecode_wrapper/reine.py (props changed) pypy/dist/pypy/objspace/logic.py (props changed) pypy/dist/pypy/objspace/std/setobject.py (props changed) pypy/dist/pypy/objspace/std/settype.py (props changed) pypy/dist/pypy/objspace/test/test_logicobjspace.py (props changed) pypy/dist/pypy/rpython/ootypesystem/rtuple.py (props changed) pypy/dist/pypy/rpython/rctypes/test/__init__.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (props changed) pypy/dist/pypy/tool/__future__.py (props changed) pypy/dist/pypy/translator/backendopt/canraise.py (props changed) pypy/dist/pypy/translator/backendopt/stat.py (props changed) pypy/dist/pypy/translator/backendopt/test/test_canraise.py (props changed) pypy/dist/pypy/translator/goal/gcbench.py (props changed) pypy/dist/pypy/translator/goal/targetgcbench.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/build.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi/ (props changed) pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi/cc.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi/create_pyllvm.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi/pyllvm.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi/setup.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/test/__init__.py (props changed) pypy/dist/pypy/translator/llvm/pyllvm/test/ll_snippet.py (props changed) pypy/dist/pypy/translator/squeak/codeformatter.py (props changed) pypy/dist/pypy/translator/squeak/conftest.py (props changed) pypy/dist/pypy/translator/squeak/node.py (props changed) pypy/dist/pypy/translator/squeak/opformatter.py (props changed) pypy/dist/pypy/translator/squeak/test/runtest.py (props changed) pypy/dist/pypy/translator/squeak/test/support.py (props changed) pypy/dist/pypy/translator/squeak/test/test_codeformatter.py (props changed) pypy/dist/pypy/translator/squeak/test/test_datastructure.py (props changed) pypy/dist/pypy/translator/squeak/test/test_llops.py (props changed) Log: fixeol From arigo at codespeak.net Sat Apr 1 10:02:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 10:02:16 +0200 (CEST) Subject: [pypy-svn] r25200 - in pypy/dist/pypy/rpython: . test Message-ID: <20060401080216.4E490100C8@code0.codespeak.net> Author: arigo Date: Sat Apr 1 10:02:15 2006 New Revision: 25200 Modified: pypy/dist/pypy/rpython/rpbc.py pypy/dist/pypy/rpython/test/test_rpbc.py Log: (pedronis, arre, arigo) Yet Another Case of conversion missing in rpbc.py. Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Sat Apr 1 10:02:15 2006 @@ -1,6 +1,6 @@ import types import sys -from pypy.annotation.pairtype import pairtype +from pypy.annotation.pairtype import pair, pairtype from pypy.annotation import model as annmodel from pypy.annotation import description from pypy.objspace.flow.model import Constant @@ -552,6 +552,11 @@ # now hop2 looks like simple_call(function, self, args...) return hop2.dispatch() +class __extend__(pairtype(MethodOfFrozenPBCRepr, MethodOfFrozenPBCRepr)): + + def convert_from_to((r_from, r_to), v, llops): + return pair(r_from.r_im_self, r_to.r_im_self).convert_from_to(v, llops) + # __ None ____________________________________________________ class NoneFrozenPBCRepr(SingleFrozenPBCRepr): Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Sat Apr 1 10:02:15 2006 @@ -1166,7 +1166,6 @@ assert res == 65*66 def test_convert_multiple_to_single_method_of_frozen_pbc(self): - py.test.skip("in-progress") class A: def meth(self, frmeth): return frmeth(100) From arigo at codespeak.net Sat Apr 1 10:19:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 10:19:18 +0200 (CEST) Subject: [pypy-svn] r25201 - in pypy/dist/pypy/jit: . test Message-ID: <20060401081918.0AAA4100C3@code0.codespeak.net> Author: arigo Date: Sat Apr 1 10:19:17 2006 New Revision: 25201 Modified: pypy/dist/pypy/jit/rtimeshift.py pypy/dist/pypy/jit/test/test_hint_timeshift.py Log: (arre, pedronis, arigo) Substructures start to work. Modified: pypy/dist/pypy/jit/rtimeshift.py ============================================================================== --- pypy/dist/pypy/jit/rtimeshift.py (original) +++ pypy/dist/pypy/jit/rtimeshift.py Sat Apr 1 10:19:17 2006 @@ -122,9 +122,7 @@ op_args[0] = typedesc.gv_type self.genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, typedesc.gv_ptrtype) - for i in range(len(boxes)): - RedBox.op_setfield(self, jitstate, typedesc.fielddescs[i], - boxes[i]) + typedesc.materialize_content(jitstate, self.genvar, boxes) return self.genvar def is_forced(self): @@ -145,7 +143,6 @@ bigbox = VirtualRedBox(self.typedesc) memo[self] = bigbox for i in range(len(bigbox.content_boxes)): - # XXX need a memo for sharing gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, gv_fldtype, @@ -203,7 +200,6 @@ bigbox.parentbox = parentcopybox typedesc = self.fielddesc.inlined_typedesc for i in range(len(bigbox.content_boxes)): - # XXX need a memo for sharing gv_fldtype = typedesc.fielddescs[i].gv_resulttype bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, gv_fldtype, @@ -383,6 +379,30 @@ clist.append(box) return clist + def materialize_content(self, jitstate, gv, boxes): + for i in range(len(boxes)): + smallbox = boxes[i] + fielddesc = self.fielddescs[i] + if fielddesc.inlined_typedesc: + op_args = lltype.malloc(VARLIST.TO, 2) + op_args[0] = gv + op_args[1] = fielddesc.gv_fieldname + gv_sub = rgenop.genop(jitstate.curblock, 'getsubstruct', + op_args, fielddesc.gv_resulttype) + assert isinstance(smallbox, SubVirtualRedBox) + subboxes = smallbox.content_boxes + smallbox.content_boxes = None + fielddesc.inlined_typedesc.materialize_content(jitstate, + gv_sub, + subboxes) + else: + op_args = lltype.malloc(VARLIST.TO, 3) + op_args[0] = gv + op_args[1] = fielddesc.gv_fieldname + op_args[2] = smallbox.getgenvar(jitstate) + rgenop.genop(jitstate.curblock, 'setfield', op_args, + gv_Void) + def make(T): try: return StructTypeDesc._type_cache[T] Modified: pypy/dist/pypy/jit/test/test_hint_timeshift.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_timeshift.py (original) +++ pypy/dist/pypy/jit/test/test_hint_timeshift.py Sat Apr 1 10:19:17 2006 @@ -7,7 +7,7 @@ from pypy.jit import rtimeshift, hintrtyper from pypy.jit.test.test_llabstractinterp import annotation, summary from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rpython.objectmodel import hint +from pypy.rpython.objectmodel import hint, keepalive_until_here from pypy.rpython import rgenop, rstr from pypy.annotation import model as annmodel from pypy.rpython.llinterp import LLInterpreter @@ -386,7 +386,6 @@ assert insns == {'int_lt': 1, 'int_mul': 1} def test_red_subcontainer(): - py.test.skip("in-progress") S = lltype.Struct('S', ('n', lltype.Signed)) T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) def ll_function(k): @@ -395,7 +394,9 @@ s.n = k if k < 0: return -123 - return s.n * (k-1) + result = s.n * (k-1) + keepalive_until_here(t) + return result insns, res = timeshift(ll_function, [7], [], policy=P_NOVIRTUAL) assert res == 42 #assert insns == ... in-progress From arigo at codespeak.net Sat Apr 1 10:40:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 10:40:59 +0200 (CEST) Subject: [pypy-svn] r25202 - in pypy/dist/pypy/jit: . test Message-ID: <20060401084059.00B2C100C3@code0.codespeak.net> Author: arigo Date: Sat Apr 1 10:40:59 2006 New Revision: 25202 Modified: pypy/dist/pypy/jit/rtimeshift.py pypy/dist/pypy/jit/test/test_hint_timeshift.py Log: (pedronis, arre, arigo) Now really what this test was supposed to test: substructures that stay virtual. Modified: pypy/dist/pypy/jit/rtimeshift.py ============================================================================== --- pypy/dist/pypy/jit/rtimeshift.py (original) +++ pypy/dist/pypy/jit/rtimeshift.py Sat Apr 1 10:40:59 2006 @@ -32,7 +32,7 @@ def same_constant(self, other): return False - def getallvariables(self, jitstate, result_gv): + def getallvariables(self, jitstate, result_gv, memo): pass def copybox(self, newblock, gv_type, memo): @@ -58,6 +58,14 @@ rgenop.genop(jitstate.curblock, 'setfield', op_args, gv_Void) + def op_getsubstruct(self, jitstate, fielddesc): + op_args = lltype.malloc(VARLIST.TO, 2) + op_args[0] = self.getgenvar(jitstate) + op_args[1] = fielddesc.gv_fieldname + genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, + fielddesc.gv_resulttype) + return VarRedBox(genvar) + class VarRedBox(RedBox): "A red box that contains a run-time variable." @@ -68,8 +76,10 @@ def getgenvar(self, jitstate): return self.genvar - def getallvariables(self, jitstate, result_gv): - result_gv.append(self.genvar) + def getallvariables(self, jitstate, result_gv, memo): + if self not in memo: + result_gv.append(self.genvar) + memo[self] = None def _copybox(self, newblock, gv_type, memo): newgenvar = rgenop.geninputarg(newblock, gv_type) @@ -105,7 +115,32 @@ return box.content_addr -class VirtualRedBox(RedBox): +class BigRedBox(RedBox): + "A (big) red box that contains (small) red boxes inside." + + #def __init__(self, content_boxes): + # self.content_boxes = content_boxes + + def op_getfield(self, jitstate, fielddesc): + if self.content_boxes is None: + return RedBox.op_getfield(self, jitstate, fielddesc) + else: + return self.content_boxes[fielddesc.fieldindex] + + def op_setfield(self, jitstate, fielddesc, valuebox): + if self.content_boxes is None: + RedBox.op_setfield(self, jitstate, fielddesc, valuebox) + else: + self.content_boxes[fielddesc.fieldindex] = valuebox + + def op_getsubstruct(self, jitstate, fielddesc): + if self.content_boxes is None: + return RedBox.op_getsubstruct(self, jitstate, fielddesc) + else: + return self.content_boxes[fielddesc.fieldindex] + + +class VirtualRedBox(BigRedBox): "A red box that contains (for now) a virtual Struct." def __init__(self, typedesc): @@ -128,12 +163,14 @@ def is_forced(self): return bool(self.genvar) - def getallvariables(self, jitstate, result_gv): + def getallvariables(self, jitstate, result_gv, memo): if self.genvar: - result_gv.append(self.genvar) + if self not in memo: + result_gv.append(self.genvar) + memo[self] = None else: for smallbox in self.content_boxes: - smallbox.getallvariables(jitstate, result_gv) + smallbox.getallvariables(jitstate, result_gv, memo) def _copybox(self, newblock, gv_type, memo): if self.genvar: @@ -149,19 +186,7 @@ memo) return bigbox - def op_getfield(self, jitstate, fielddesc): - if self.content_boxes is None: - return RedBox.op_getfield(self, jitstate, fielddesc) - else: - return self.content_boxes[fielddesc.fieldindex] - - def op_setfield(self, jitstate, fielddesc, valuebox): - if self.content_boxes is None: - RedBox.op_setfield(self, jitstate, fielddesc, valuebox) - else: - self.content_boxes[fielddesc.fieldindex] = valuebox - -class SubVirtualRedBox(RedBox): +class SubVirtualRedBox(BigRedBox): def __init__(self, parentbox, fielddesc): self.parentbox = parentbox @@ -181,12 +206,14 @@ def is_forced(self): return self.parentbox.is_forced() - def getallvariables(self, jitstate, result_gv): + def getallvariables(self, jitstate, result_gv, memo): if self.is_forced(): - result_gv.append(self.getgenvar(jitstate)) + if self not in memo: + result_gv.append(self.getgenvar(jitstate)) + memo[self] = None else: for smallbox in self.content_boxes: - smallbox.getallvariables(jitstate, result_gv) + smallbox.getallvariables(jitstate, result_gv, memo) def _copybox(self, newblock, gv_type, memo): if self.is_forced(): @@ -492,12 +519,7 @@ if isinstance(argbox, ConstRedBox): res = getattr(argbox.ll_getvalue(fielddesc.PTRTYPE), fielddesc.fieldname) return ConstRedBox.ll_fromvalue(res) - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = argbox.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, - fielddesc.gv_resulttype) - return VarRedBox(genvar) + return argbox.op_getsubstruct(jitstate, fielddesc) def ll_generate_getarrayitem(jitstate, fielddesc, argbox, indexbox): @@ -566,11 +588,12 @@ def enter_block(jitstate, redboxes, types_gv): newblock = rgenop.newblock() incoming = [] - memo = {} + memo1 = {} + memo2 = {} for i in range(len(redboxes)): redbox = redboxes[i] - redbox.getallvariables(jitstate, incoming) - redboxes[i] = redbox.copybox(newblock, types_gv[i], memo) + redbox.getallvariables(jitstate, incoming, memo1) + redboxes[i] = redbox.copybox(newblock, types_gv[i], memo2) rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) jitstate.curblock = newblock jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) Modified: pypy/dist/pypy/jit/test/test_hint_timeshift.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_timeshift.py (original) +++ pypy/dist/pypy/jit/test/test_hint_timeshift.py Sat Apr 1 10:40:59 2006 @@ -399,4 +399,4 @@ return result insns, res = timeshift(ll_function, [7], [], policy=P_NOVIRTUAL) assert res == 42 - #assert insns == ... in-progress + assert insns == {'int_lt': 1, 'int_mul': 1, 'int_sub': 1} From arigo at codespeak.net Sat Apr 1 15:57:42 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 15:57:42 +0200 (CEST) Subject: [pypy-svn] r25205 - in pypy/dist/pypy: jit jit/test rpython Message-ID: <20060401135742.AFCFF100B2@code0.codespeak.net> Author: arigo Date: Sat Apr 1 15:57:41 2006 New Revision: 25205 Modified: pypy/dist/pypy/jit/hinttimeshift.py pypy/dist/pypy/jit/rtimeshift.py pypy/dist/pypy/jit/test/test_hint_timeshift.py pypy/dist/pypy/rpython/rgenop.py Log: (pedronis, arre, arigo) Merging of virtual red boxes, plus a lot of small bugs found along this path. Modified: pypy/dist/pypy/jit/hinttimeshift.py ============================================================================== --- pypy/dist/pypy/jit/hinttimeshift.py (original) +++ pypy/dist/pypy/jit/hinttimeshift.py Sat Apr 1 15:57:41 2006 @@ -1,3 +1,4 @@ +import py from pypy.rpython.lltypesystem import lltype from pypy.objspace.flow import model as flowmodel from pypy.annotation import model as annmodel @@ -90,6 +91,9 @@ reenter_vars = [v_jitstate] for var in link.args[1:]: + if isinstance(var, flowmodel.Constant): + reenter_vars.append(var) + continue i = inputargs.index(var) r = args_r[i] v_box = self.read_out_box(llops, v_boxes, i) @@ -126,6 +130,7 @@ def timeshift_graph(self, graph): self.graph = graph self.dispatch_to = [] + self.statecaches = [] entering_links = flowmodel.mkentrymap(graph) originalblocks = list(graph.iterblocks()) @@ -160,6 +165,20 @@ # fix its concretetypes self.insert_dispatch_logic(returnblock) + # hack to allow the state caches to be cleared + miniglobals = {} + source = ["def clearcaches():"] + if self.statecaches: + for i, cache in enumerate(self.statecaches): + source.append(" c%d.clear()" % i) + miniglobals["c%d" % i] = cache + else: + source.append(" pass") + exec py.code.Source('\n'.join(source)).compile() in miniglobals + clearcaches = miniglobals['clearcaches'] + self.ll_clearcaches = self.annhelper.getgraph(clearcaches, [], + annmodel.s_None) + def insert_jitstate_arg(self, block): # pass 'jitstate' as an extra argument around the whole graph if block.operations != (): @@ -315,6 +334,8 @@ v_oldjitstate = newinputargs[0] cache = {} + self.statecaches.append(cache) + def merge_point(jitstate, key, boxes): return rtimeshift.retrieve_jitstate_for_merge(cache, jitstate, key, boxes, TYPES_gv) Modified: pypy/dist/pypy/jit/rtimeshift.py ============================================================================== --- pypy/dist/pypy/jit/rtimeshift.py (original) +++ pypy/dist/pypy/jit/rtimeshift.py Sat Apr 1 15:57:41 2006 @@ -41,6 +41,23 @@ except KeyError: return self._copybox(newblock, gv_type, memo) + def match(self, jitstate, newbox, incoming, memo): + if self in memo: + return memo[self] is newbox + if newbox in memo: + return memo[newbox] is self + memo[self] = newbox + memo[newbox] = self + return self._match(jitstate, newbox, incoming, memo) + + def union_for_new_block(self, jitstate, newbox, newblock, + gv_type, incoming, memo): + try: + return memo[newbox] + except KeyError: + return self._union_for_new_block(jitstate, newbox, newblock, + gv_type, incoming, memo) + # generic implementation of some operations def op_getfield(self, jitstate, fielddesc): op_args = lltype.malloc(VARLIST.TO, 2) @@ -86,6 +103,20 @@ memo[self] = newbox = VarRedBox(newgenvar) return newbox + incoming.append(newbox.getgenvar(jitstate)) + return True + + def _match(self, jitstate, newbox, incoming, memo): + incoming.append(newbox.getgenvar(jitstate)) + return True + + def _union_for_new_block(self, jitstate, newbox, newblock, + gv_type, incoming, memo): + incoming.append(newbox.getgenvar(jitstate)) + newgenvar = rgenop.geninputarg(newblock, gv_type) + memo[newbox] = newnewbox = VarRedBox(newgenvar) + return newnewbox + VCONTAINER = lltype.GcStruct("vcontainer") @@ -186,6 +217,47 @@ memo) return bigbox + def _match(self, jitstate, newbox, incoming, memo): + if self.genvar: + incoming.append(newbox.getgenvar(jitstate)) + return True + if not isinstance(newbox, VirtualRedBox): + return False + for i in range(len(self.content_boxes)): + mysmallbox = self.content_boxes[i] + newsmallbox = newbox.content_boxes[i] + if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): + return False + else: + return True + + def inlined_structs_are_compatible(self, newbox): + return (isinstance(newbox, VirtualRedBox) and + self.typedesc.compare_content_boxes(self.content_boxes, + newbox.content_boxes)) + + def _union_for_new_block(self, jitstate, newbox, newblock, + gv_type, incoming, memo): + if self.genvar or not self.inlined_structs_are_compatible(newbox): + incoming.append(newbox.getgenvar(jitstate)) + newgenvar = rgenop.geninputarg(newblock, gv_type) + memo[newbox] = newnewbox = VarRedBox(newgenvar) + return newnewbox + bigbox = VirtualRedBox(self.typedesc) + memo[newbox] = bigbox + for i in range(len(bigbox.content_boxes)): + gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype + box = self.content_boxes[i] + bigbox.content_boxes[i] = box.union_for_new_block( + jitstate, + newbox.content_boxes[i], + newblock, + gv_fldtype, + incoming, + memo) + return bigbox + + class SubVirtualRedBox(BigRedBox): def __init__(self, parentbox, fielddesc): @@ -233,6 +305,62 @@ memo) return bigbox + def _match(self, jitstate, newbox, incoming, memo): + if self.is_forced(): + incoming.append(newbox.getgenvar(jitstate)) + return True + if not (isinstance(newbox, SubVirtualRedBox) and + self.fielddesc is newbox.fielddesc and + self.parentbox.match(jitstate, newbox.parentbox, + incoming, memo)): + return False + for i in range(len(self.content_boxes)): + mysmallbox = self.content_boxes[i] + newsmallbox = newbox.content_boxes[i] + if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): + return False + else: + return True + + def inlined_structs_are_compatible(self, newbox): + if (isinstance(newbox, SubVirtualRedBox) and + self.fielddesc is newbox.fielddesc): + return self.parentbox.inlined_structs_are_compatible( + newbox.parentbox) + else: + return False + + def _union_for_new_block(self, jitstate, newbox, newblock, + gv_type, incoming, memo): + if self.is_forced() or not self.inlined_structs_are_compatible(newbox): + incoming.append(newbox.getgenvar(jitstate)) + newgenvar = rgenop.geninputarg(newblock, gv_type) + memo[newbox] = newnewbox = VarRedBox(newgenvar) + return newnewbox + assert isinstance(newbox, SubVirtualRedBox) + bigbox = SubVirtualRedBox(None, self.fielddesc) + memo[newbox] = bigbox + gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype + parentcopybox = self.parentbox.union_for_new_block(jitstate, + newbox.parentbox, + newblock, + gv_parenttype, + incoming, + memo) + bigbox.parentbox = parentcopybox + typedesc = self.fielddesc.inlined_typedesc + for i in range(len(bigbox.content_boxes)): + gv_fldtype = typedesc.fielddescs[i].gv_resulttype + box = self.content_boxes[i] + bigbox.content_boxes[i] = box.union_for_new_block( + jitstate, + newbox.content_boxes[i], + newblock, + gv_fldtype, + incoming, + memo) + return bigbox + class ConstRedBox(RedBox): "A red box that contains a run-time constant." @@ -246,6 +374,20 @@ def copybox(self, newblock, gv_type, memo): return self + def match(self, jitstate, newbox, incoming, memo): + return self.same_constant(newbox) + + def _union_for_new_block(self, jitstate, newbox, newblock, + gv_type, incoming, memo): + if self.same_constant(newbox): + newnewbox = newbox + else: + incoming.append(newbox.getgenvar(jitstate)) + newgenvar = rgenop.geninputarg(newblock, gv_type) + newnewbox = VarRedBox(newgenvar) + memo[newbox] = newnewbox + return newnewbox + def ll_fromvalue(value): T = lltype.typeOf(value) gv = rgenop.genconst(value) @@ -406,6 +548,20 @@ clist.append(box) return clist + def compare_content_boxes(self, content_boxes_1, content_boxes_2): + for i in range(len(self.fielddescs)): + fielddesc = self.fielddescs[i] + if fielddesc.inlined_typedesc: + box1 = content_boxes_1[i] + box2 = content_boxes_2[i] + assert isinstance(box1, BigRedBox) + assert isinstance(box2, BigRedBox) + if not fielddesc.inlined_typedesc.compare_content_boxes( + box1.content_boxes, box2.content_boxes): + return False + else: + return True + def materialize_content(self, jitstate, gv, boxes): for i in range(len(boxes)): smallbox = boxes[i] @@ -545,48 +701,56 @@ # other jitstate/graph level operations -def retrieve_jitstate_for_merge(states_dic, jitstate, key, redboxes, TYPES): +def retrieve_jitstate_for_merge(states_dic, jitstate, key, redboxes, types_gv): if key not in states_dic: - jitstate = enter_block(jitstate, redboxes, TYPES) - states_dic[key] = redboxes[:], jitstate.curblock + jitstate = enter_block(jitstate, redboxes, types_gv) + states_dic[key] = redboxes, jitstate.curblock return jitstate oldboxes, oldblock = states_dic[key] + memo = {} incoming = [] for i in range(len(redboxes)): oldbox = oldboxes[i] newbox = redboxes[i] - if isinstance(oldbox, VarRedBox): # Always a match - incoming.append(newbox.getgenvar(jitstate)) - continue - if oldbox.same_constant(newbox): - continue - # Mismatch. Generalize to a var - break + if not oldbox.match(jitstate, newbox, incoming, memo): + break # mismatch +## if isinstance(oldbox, VarRedBox): # Always a match +## incoming.append(newbox.getgenvar(jitstate)) +## continue +## if oldbox.same_constant(newbox): +## continue +## # Mismatch. Generalize to a var +## break else: rgenop.closelink(jitstate.curoutgoinglink, incoming, oldblock) return None # Make a more general block + jitstate = enter_block(jitstate, redboxes, types_gv) newblock = rgenop.newblock() incoming = [] + memo = {} for i in range(len(redboxes)): oldbox = oldboxes[i] newbox = redboxes[i] - if not oldbox.same_constant(newbox): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, TYPES[i]) - redboxes[i] = VarRedBox(newgenvar) - - rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) + redboxes[i] = oldbox.union_for_new_block(jitstate, newbox, newblock, + types_gv[i], incoming, memo) +## if not oldbox.same_constant(newbox): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, TYPES[i]) +## redboxes[i] = VarRedBox(newgenvar) + link = rgenop.closeblock1(jitstate.curblock) + rgenop.closelink(link, incoming, newblock) jitstate.curblock = newblock - jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) - states_dic[key] = redboxes[:], newblock + #jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) + states_dic[key] = redboxes, newblock return jitstate retrieve_jitstate_for_merge._annspecialcase_ = "specialize:arglltype(2)" def enter_block(jitstate, redboxes, types_gv): newblock = rgenop.newblock() + jitstate.curblock = newblock incoming = [] memo1 = {} memo2 = {} @@ -595,7 +759,6 @@ redbox.getallvariables(jitstate, incoming, memo1) redboxes[i] = redbox.copybox(newblock, types_gv[i], memo2) rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) - jitstate.curblock = newblock jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) return jitstate @@ -622,7 +785,7 @@ novars = lltype.malloc(VARLIST.TO, 0) -def dispatch_next(jitstate, outredboxes, RETURN_TYPE): +def dispatch_next(jitstate, outredboxes, gv_return_type): split_queue = jitstate.split_queue if split_queue: exitindex, later_jitstate, redboxes = split_queue.pop() @@ -635,26 +798,35 @@ return_queue = jitstate.return_queue first_redbox = return_queue[0][1] finalblock = rgenop.newblock() - jitstate.curblock = finalblock - if isinstance(first_redbox, ConstRedBox): - for link, redbox in return_queue: - if not redbox.same_constant(first_redbox): - break - else: - for link, _ in return_queue: - rgenop.closelink(link, novars, finalblock) - finallink = rgenop.closeblock1(finalblock) - jitstate.curoutgoinglink = finallink - jitstate.curvalue = first_redbox - return -1 +## jitstate.curblock = finalblock +## if isinstance(first_redbox, ConstRedBox): +## for link, redbox in return_queue: +## if not redbox.match(first_redbox): +## break +## else: +## for link, _ in return_queue: +## rgenop.closelink(link, novars, finalblock) +## finallink = rgenop.closeblock1(finalblock) +## jitstate.curoutgoinglink = finallink +## jitstate.curvalue = first_redbox +## return -1 - finalvar = rgenop.geninputarg(finalblock, RETURN_TYPE) + finalvar = rgenop.geninputarg(finalblock, gv_return_type) for link, redbox in return_queue: - genvar = redbox.getgenvar(jitstate) - rgenop.closelink(link, [genvar], finalblock) + newblock = rgenop.newblock() + jitstate.curblock = newblock + incoming = [] + memo1 = {} + memo2 = {} + redbox.getallvariables(jitstate, incoming, memo1) + newbox = redbox.copybox(newblock, gv_return_type, memo2) + gv_retval = newbox.getgenvar(jitstate) + rgenop.closelink(link, incoming, newblock) + newlink = rgenop.closeblock1(newblock) + rgenop.closelink(newlink, [gv_retval], finalblock) finallink = rgenop.closeblock1(finalblock) jitstate.curoutgoinglink = finallink - jitstate.curvalue = VarRedBox(finalvar) + jitstate.curvalue = finalvar return -1 def ll_gvar_from_redbox(jitstate, redbox): @@ -672,7 +844,7 @@ self.return_queue = [] self.split_queue = [] self.curblock = rgenop.newblock() - self.curvalue = None + self.curvalue = rgenop.nullvar def end_setup(self): self.curoutgoinglink = rgenop.closeblock1(self.curblock) @@ -712,5 +884,5 @@ return jitstate.curblock def ll_close_jitstate(jitstate): - result_genvar = jitstate.curvalue.getgenvar(jitstate) + result_genvar = jitstate.curvalue jitstate.close(result_genvar) Modified: pypy/dist/pypy/jit/test/test_hint_timeshift.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_timeshift.py (original) +++ pypy/dist/pypy/jit/test/test_hint_timeshift.py Sat Apr 1 15:57:41 2006 @@ -19,11 +19,14 @@ P_NOVIRTUAL = AnnotatorPolicy() P_NOVIRTUAL.novirtualcontainer = True +def getargtypes(annotator, values): + return [annotation(annotator, x) for x in values] + def hannotate(func, values, policy=None, inline=None): # build the normal ll graphs for ll_function t = TranslationContext() a = t.buildannotator() - argtypes = [annotation(a, x) for x in values] + argtypes = getargtypes(a, values) a.build_types(func, argtypes) rtyper = t.buildrtyper() rtyper.specialize() @@ -40,20 +43,40 @@ hannotator.translator.view() return hs, hannotator, rtyper +_cache = {} +_cache_order = [] +def timeshift_cached(ll_function, values, inline, policy): + key = ll_function, inline, policy + try: + result, argtypes = _cache[key] + except KeyError: + if len(_cache_order) >= 3: + del _cache[_cache_order.pop(0)] + hs, ha, rtyper = hannotate(ll_function, values, + inline=inline, policy=policy) + htshift = HintTimeshift(ha, rtyper) + htshift.timeshift() + t = rtyper.annotator.translator + for graph in ha.translator.graphs: + checkgraph(graph) + t.graphs.append(graph) + if conftest.option.view: + t.view() + result = hs, ha, rtyper, htshift + _cache[key] = result, getargtypes(rtyper.annotator, values) + _cache_order.append(key) + else: + hs, ha, rtyper, htshift = result + assert argtypes == getargtypes(rtyper.annotator, values) + return result + def timeshift(ll_function, values, opt_consts=[], inline=None, policy=None): - hs, ha, rtyper = hannotate(ll_function, values, - inline=inline, policy=policy) - htshift = HintTimeshift(ha, rtyper) - htshift.timeshift() - t = rtyper.annotator.translator - for graph in ha.translator.graphs: - checkgraph(graph) - t.graphs.append(graph) - if conftest.option.view: - t.view() + hs, ha, rtyper, htshift = timeshift_cached(ll_function, values, + inline, policy) # run the time-shifted graph-producing graphs graph1 = ha.translator.graphs[0] llinterp = LLInterpreter(rtyper) + llinterp.eval_graph(htshift.ll_clearcaches, []) jitstate = llinterp.eval_graph(htshift.ll_build_jitstate_graph, []) graph1args = [jitstate] residual_graph_args = [] @@ -238,7 +261,30 @@ insns, res = timeshift(ll_function, [70, 4], [0]) assert res == 66 assert insns['int_add'] == 1 - assert insns['int_add'] == 1 + assert insns['int_sub'] == 1 + +def test_merge_const_before_return(): + def ll_function(x): + if x > 0: + y = 17 + else: + y = 22 + x -= 1 + y += 1 + return y+x + insns, res = timeshift(ll_function, [-70], []) + assert res == 23-71 + assert insns == {'int_gt': 1, 'int_add': 2, 'int_sub': 2} + +def test_merge_const_at_return(): + def ll_function(x): + if x > 0: + return 17 + else: + return 22 + insns, res = timeshift(ll_function, [-70], []) + assert res == 22 + assert insns == {'int_gt': 1} def test_arith_plus_minus(): def ll_plus_minus(encoded_insn, nb_insn, x, y): @@ -324,13 +370,31 @@ assert res == 7 assert insns == {} -def test_degenerated_merge_substructure(): - py.test.skip("re in-progress") +def test_degenerated_before_return(): + S = lltype.GcStruct('S', ('n', lltype.Signed)) + T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) + + def ll_function(flag): + t = lltype.malloc(T) + t.s.n = 3 + s = lltype.malloc(S) + s.n = 4 + if flag: + s = t.s + s.n += 1 + return s.n * t.s.n + insns, res = timeshift(ll_function, [0], []) + assert res == 5 * 3 + insns, res = timeshift(ll_function, [1], []) + assert res == 4 * 4 + +def test_degenerated_at_return(): S = lltype.GcStruct('S', ('n', lltype.Signed)) T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) def ll_function(flag): t = lltype.malloc(T) + t.n = 3.25 t.s.n = 3 s = lltype.malloc(S) s.n = 4 @@ -339,7 +403,12 @@ return s insns, res = timeshift(ll_function, [0], []) assert res.n == 4 - assert insns == {'getsubstruct': 2, 'int_is_true': 1, 'malloc': 2, 'setfield': 2} + assert lltype.parentlink(res._obj) == (None, None) + insns, res = timeshift(ll_function, [1], []) + assert res.n == 3 + parent, parentindex = lltype.parentlink(res._obj) + assert parentindex == 's' + assert parent.n == 3.25 def test_plus_minus_all_inlined(): def ll_plus_minus(s, x, y): Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Sat Apr 1 15:57:41 2006 @@ -25,6 +25,8 @@ def geninputarg(blockcontainer, gv_CONCRETE_TYPE): block = from_opaque_object(blockcontainer.obj) + assert not block.operations, "block already contains operations" + assert block.exits == [], "block already closed" CONCRETE_TYPE = from_opaque_object(gv_CONCRETE_TYPE).value v = flowmodel.Variable() v.concretetype = CONCRETE_TYPE @@ -49,6 +51,7 @@ if not isinstance(opname, str): opname = from_rstr(opname) block = from_opaque_object(blockcontainer.obj) + assert block.exits == [], "block already closed" RESULT_TYPE = from_opaque_object(gv_RESULT_TYPE).value opvars = _inputvars(vars) v = flowmodel.Variable() From arigo at codespeak.net Sat Apr 1 16:17:24 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 16:17:24 +0200 (CEST) Subject: [pypy-svn] r25206 - pypy/dist/pypy/jit/test Message-ID: <20060401141724.18F9110088@code0.codespeak.net> Author: arigo Date: Sat Apr 1 16:17:23 2006 New Revision: 25206 Modified: pypy/dist/pypy/jit/test/test_hint_timeshift.py Log: Slightly more useful "t.view()". Modified: pypy/dist/pypy/jit/test/test_hint_timeshift.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_timeshift.py (original) +++ pypy/dist/pypy/jit/test/test_hint_timeshift.py Sat Apr 1 16:17:23 2006 @@ -61,7 +61,8 @@ checkgraph(graph) t.graphs.append(graph) if conftest.option.view: - t.view() + from pypy.translator.tool.graphpage import FlowGraphPage + FlowGraphPage(t, ha.translator.graphs).display() result = hs, ha, rtyper, htshift _cache[key] = result, getargtypes(rtyper.annotator, values) _cache_order.append(key) From arigo at codespeak.net Sat Apr 1 16:44:34 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 16:44:34 +0200 (CEST) Subject: [pypy-svn] r25207 - in pypy/dist/pypy/jit: . test Message-ID: <20060401144434.B500B10061@code0.codespeak.net> Author: arigo Date: Sat Apr 1 16:44:33 2006 New Revision: 25207 Modified: pypy/dist/pypy/jit/rtimeshift.py pypy/dist/pypy/jit/test/test_hint_timeshift.py Log: (pedronis, arre, arigo) Fixed a bug found by a test whose purpose was to show a completely different bug, which it didn't. Modified: pypy/dist/pypy/jit/rtimeshift.py ============================================================================== --- pypy/dist/pypy/jit/rtimeshift.py (original) +++ pypy/dist/pypy/jit/rtimeshift.py Sat Apr 1 16:44:33 2006 @@ -232,7 +232,7 @@ return True def inlined_structs_are_compatible(self, newbox): - return (isinstance(newbox, VirtualRedBox) and + return (isinstance(newbox, VirtualRedBox) and not newbox.genvar and self.typedesc.compare_content_boxes(self.content_boxes, newbox.content_boxes)) @@ -323,7 +323,7 @@ return True def inlined_structs_are_compatible(self, newbox): - if (isinstance(newbox, SubVirtualRedBox) and + if (isinstance(newbox, SubVirtualRedBox) and not newbox.is_forced() and self.fielddesc is newbox.fielddesc): return self.parentbox.inlined_structs_are_compatible( newbox.parentbox) Modified: pypy/dist/pypy/jit/test/test_hint_timeshift.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_timeshift.py (original) +++ pypy/dist/pypy/jit/test/test_hint_timeshift.py Sat Apr 1 16:44:33 2006 @@ -389,6 +389,26 @@ insns, res = timeshift(ll_function, [1], []) assert res == 4 * 4 +def test_degenerated_before_return_2(): + S = lltype.GcStruct('S', ('n', lltype.Signed)) + T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) + + def ll_function(flag): + t = lltype.malloc(T) + t.s.n = 3 + s = lltype.malloc(S) + s.n = 4 + if flag: + pass + else: + s = t.s + s.n += 1 + return s.n * t.s.n + insns, res = timeshift(ll_function, [1], []) + assert res == 5 * 3 + insns, res = timeshift(ll_function, [0], []) + assert res == 4 * 4 + def test_degenerated_at_return(): S = lltype.GcStruct('S', ('n', lltype.Signed)) T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) From arigo at codespeak.net Sat Apr 1 17:30:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 17:30:18 +0200 (CEST) Subject: [pypy-svn] r25210 - in pypy/dist/pypy/jit: . hintannotator hintannotator/test llabstractinterp llabstractinterp/test test timeshifter timeshifter/test tl tl/test Message-ID: <20060401153018.BABD51007F@code0.codespeak.net> Author: arigo Date: Sat Apr 1 17:30:08 2006 New Revision: 25210 Added: pypy/dist/pypy/jit/hintannotator/ (props changed) pypy/dist/pypy/jit/hintannotator/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/hintannotator/annotator.py - copied, changed from r25203, pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintannotator/bookkeeper.py - copied, changed from r25203, pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintannotator/container.py - copied, changed from r25203, pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintannotator/model.py - copied, changed from r25203, pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/hintannotator/test/ (props changed) pypy/dist/pypy/jit/hintannotator/test/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/hintannotator/test/test_annotator.py - copied, changed from r25203, pypy/dist/pypy/jit/test/test_hint_annotation.py pypy/dist/pypy/jit/hintannotator/vlist.py - copied, changed from r25203, pypy/dist/pypy/jit/hintvlist.py pypy/dist/pypy/jit/llabstractinterp/ (props changed) pypy/dist/pypy/jit/llabstractinterp/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/llabstractinterp/llabstractinterp.py - copied, changed from r25203, pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llabstractinterp/llcontainer.py - copied, changed from r25203, pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/llabstractinterp/llvalue.py - copied unchanged from r25203, pypy/dist/pypy/jit/llvalue.py pypy/dist/pypy/jit/llabstractinterp/test/ (props changed) pypy/dist/pypy/jit/llabstractinterp/test/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tl.py - copied, changed from r25203, pypy/dist/pypy/jit/test/test_jit_tl.py pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tlr.py - copied, changed from r25203, pypy/dist/pypy/jit/test/test_jit_tlr.py pypy/dist/pypy/jit/llabstractinterp/test/test_llabstractinterp.py - copied, changed from r25203, pypy/dist/pypy/jit/test/test_llabstractinterp.py pypy/dist/pypy/jit/llabstractinterp/test/test_vlist.py - copied, changed from r25203, pypy/dist/pypy/jit/test/test_vlist.py pypy/dist/pypy/jit/llabstractinterp/vlist.py - copied, changed from r25203, pypy/dist/pypy/jit/vlist.py pypy/dist/pypy/jit/timeshifter/ (props changed) pypy/dist/pypy/jit/timeshifter/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/timeshifter/rtimeshift.py - copied unchanged from r25207, pypy/dist/pypy/jit/rtimeshift.py pypy/dist/pypy/jit/timeshifter/rtyper.py - copied, changed from r25203, pypy/dist/pypy/jit/hintrtyper.py pypy/dist/pypy/jit/timeshifter/test/ (props changed) pypy/dist/pypy/jit/timeshifter/test/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/timeshifter/test/test_timeshift.py - copied, changed from r25207, pypy/dist/pypy/jit/test/test_hint_timeshift.py pypy/dist/pypy/jit/timeshifter/timeshift.py - copied, changed from r25205, pypy/dist/pypy/jit/hinttimeshift.py pypy/dist/pypy/jit/tl/ (props changed) pypy/dist/pypy/jit/tl/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/tl/opcode.py - copied unchanged from r25203, pypy/dist/pypy/jit/tlopcode.py pypy/dist/pypy/jit/tl/test/ (props changed) pypy/dist/pypy/jit/tl/test/__init__.py - copied unchanged from r25203, pypy/dist/pypy/jit/__init__.py pypy/dist/pypy/jit/tl/test/test_tl.py - copied, changed from r25203, pypy/dist/pypy/jit/test/test_tl.py pypy/dist/pypy/jit/tl/test/test_tlr.py (contents, props changed) pypy/dist/pypy/jit/tl/tl.py - copied, changed from r25203, pypy/dist/pypy/jit/tl.py pypy/dist/pypy/jit/tl/tlr.py (contents, props changed) Removed: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/hintrtyper.py pypy/dist/pypy/jit/hinttimeshift.py pypy/dist/pypy/jit/hintvlist.py pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/llvalue.py pypy/dist/pypy/jit/rtimeshift.py pypy/dist/pypy/jit/test/ pypy/dist/pypy/jit/tl.py pypy/dist/pypy/jit/tlopcode.py pypy/dist/pypy/jit/vlist.py Log: (arre, pedronis, arigo) Split the content of the jit directory in subdirectories. Added: pypy/dist/pypy/jit/tl/test/test_tlr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/tl/test/test_tlr.py Sat Apr 1 17:30:08 2006 @@ -0,0 +1,7 @@ +from pypy.jit.tl import tlr + + +def test_square(): + assert tlr.interpret(tlr.SQUARE, 1) == 1 + assert tlr.interpret(tlr.SQUARE, 7) == 49 + assert tlr.interpret(tlr.SQUARE, 9) == 81 Added: pypy/dist/pypy/jit/tl/tlr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/tl/tlr.py Sat Apr 1 17:30:08 2006 @@ -0,0 +1,73 @@ +from pypy.rpython.objectmodel import hint + + +MOV_A_R = 1 +MOV_R_A = 2 +JUMP_IF_A = 3 +SET_A = 4 +ADD_R_TO_A = 5 +RETURN_A = 6 +ALLOCATE = 7 +NEG_A = 8 + + +def interpret(bytecode, a): + """Another Toy Language interpreter, this one register-based.""" + regs = [] + pc = 0 + while True: + opcode = hint(ord(bytecode[pc]), concrete=True) + pc += 1 + if opcode == MOV_A_R: + n = ord(bytecode[pc]) + pc += 1 + regs[n] = a + elif opcode == MOV_R_A: + n = ord(bytecode[pc]) + pc += 1 + a = regs[n] + elif opcode == JUMP_IF_A: + target = ord(bytecode[pc]) + pc += 1 + if a: + pc = target + elif opcode == SET_A: + a = ord(bytecode[pc]) + pc += 1 + elif opcode == ADD_R_TO_A: + n = ord(bytecode[pc]) + pc += 1 + a += regs[n] + elif opcode == RETURN_A: + return a + elif opcode == ALLOCATE: + n = ord(bytecode[pc]) + pc += 1 + regs = [0] * n + elif opcode == NEG_A: + a = -a + +# ____________________________________________________________ +# example bytecode: compute the square of 'a' >= 1 + +SQUARE_LIST = [ + ALLOCATE, 3, + MOV_A_R, 0, # counter + MOV_A_R, 1, # copy of 'a' + SET_A, 0, + MOV_A_R, 2, # accumulator for the result + # 10: + SET_A, 1, + NEG_A, + ADD_R_TO_A, 0, + MOV_A_R, 0, + MOV_R_A, 2, + ADD_R_TO_A, 1, + MOV_A_R, 2, + MOV_R_A, 0, + JUMP_IF_A, 10, + + MOV_R_A, 2, + RETURN_A ] + +SQUARE = ''.join([chr(n) for n in SQUARE_LIST]) From arigo at codespeak.net Sat Apr 1 19:32:25 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 19:32:25 +0200 (CEST) Subject: [pypy-svn] r25211 - in pypy/branch/jit-timeshifter-refactoring: . hintannotator llabstractinterp timeshifter tl Message-ID: <20060401173225.B2ACE10095@code0.codespeak.net> Author: arigo Date: Sat Apr 1 19:32:24 2006 New Revision: 25211 Added: pypy/branch/jit-timeshifter-refactoring/ - copied from r25209, pypy/dist/pypy/jit/ pypy/branch/jit-timeshifter-refactoring/hintannotator/ - copied from r25210, pypy/dist/pypy/jit/hintannotator/ pypy/branch/jit-timeshifter-refactoring/llabstractinterp/ - copied from r25210, pypy/dist/pypy/jit/llabstractinterp/ pypy/branch/jit-timeshifter-refactoring/timeshifter/ - copied from r25210, pypy/dist/pypy/jit/timeshifter/ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py pypy/branch/jit-timeshifter-refactoring/tl/ - copied from r25210, pypy/dist/pypy/jit/tl/ Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: (pedronis, arre, arigo) Start of a refactoring to look a bit more like llabstractinterp.llvalue and, uncoincidentally, a bit more like Psyco. A bit. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sat Apr 1 19:32:24 2006 @@ -27,411 +27,411 @@ return l -class RedBox(object): +##class RedBox(object): - def same_constant(self, other): - return False +## def same_constant(self, other): +## return False - def getallvariables(self, jitstate, result_gv, memo): - pass - - def copybox(self, newblock, gv_type, memo): - try: - return memo[self] - except KeyError: - return self._copybox(newblock, gv_type, memo) - - def match(self, jitstate, newbox, incoming, memo): - if self in memo: - return memo[self] is newbox - if newbox in memo: - return memo[newbox] is self - memo[self] = newbox - memo[newbox] = self - return self._match(jitstate, newbox, incoming, memo) - - def union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - try: - return memo[newbox] - except KeyError: - return self._union_for_new_block(jitstate, newbox, newblock, - gv_type, incoming, memo) - - # generic implementation of some operations - def op_getfield(self, jitstate, fielddesc): - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = self.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, - fielddesc.gv_resulttype) - return VarRedBox(genvar) - - def op_setfield(self, jitstate, fielddesc, valuebox): - op_args = lltype.malloc(VARLIST.TO, 3) - op_args[0] = self.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - op_args[2] = valuebox.getgenvar(jitstate) - rgenop.genop(jitstate.curblock, 'setfield', op_args, - gv_Void) - - def op_getsubstruct(self, jitstate, fielddesc): - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = self.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, - fielddesc.gv_resulttype) - return VarRedBox(genvar) - - -class VarRedBox(RedBox): - "A red box that contains a run-time variable." - - def __init__(self, genvar): - self.genvar = genvar - - def getgenvar(self, jitstate): - return self.genvar - - def getallvariables(self, jitstate, result_gv, memo): - if self not in memo: - result_gv.append(self.genvar) - memo[self] = None - - def _copybox(self, newblock, gv_type, memo): - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[self] = newbox = VarRedBox(newgenvar) - return newbox - - incoming.append(newbox.getgenvar(jitstate)) - return True - - def _match(self, jitstate, newbox, incoming, memo): - incoming.append(newbox.getgenvar(jitstate)) - return True - - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[newbox] = newnewbox = VarRedBox(newgenvar) - return newnewbox +## def getallvariables(self, jitstate, result_gv, memo): +## pass +## def copybox(self, newblock, gv_type, memo): +## try: +## return memo[self] +## except KeyError: +## return self._copybox(newblock, gv_type, memo) + +## def match(self, jitstate, newbox, incoming, memo): +## if self in memo: +## return memo[self] is newbox +## if newbox in memo: +## return memo[newbox] is self +## memo[self] = newbox +## memo[newbox] = self +## return self._match(jitstate, newbox, incoming, memo) + +## def union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## try: +## return memo[newbox] +## except KeyError: +## return self._union_for_new_block(jitstate, newbox, newblock, +## gv_type, incoming, memo) + +## # generic implementation of some operations +## def op_getfield(self, jitstate, fielddesc): +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = self.getgenvar(jitstate) +## op_args[1] = fielddesc.gv_fieldname +## genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, +## fielddesc.gv_resulttype) +## return VarRedBox(genvar) + +## def op_setfield(self, jitstate, fielddesc, valuebox): +## op_args = lltype.malloc(VARLIST.TO, 3) +## op_args[0] = self.getgenvar(jitstate) +## op_args[1] = fielddesc.gv_fieldname +## op_args[2] = valuebox.getgenvar(jitstate) +## rgenop.genop(jitstate.curblock, 'setfield', op_args, +## gv_Void) + +## def op_getsubstruct(self, jitstate, fielddesc): +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = self.getgenvar(jitstate) +## op_args[1] = fielddesc.gv_fieldname +## genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, +## fielddesc.gv_resulttype) +## return VarRedBox(genvar) + + +##class VarRedBox(RedBox): +## "A red box that contains a run-time variable." + +## def __init__(self, genvar): +## self.genvar = genvar + +## def getgenvar(self, jitstate): +## return self.genvar + +## def getallvariables(self, jitstate, result_gv, memo): +## if self not in memo: +## result_gv.append(self.genvar) +## memo[self] = None + +## def _copybox(self, newblock, gv_type, memo): +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[self] = newbox = VarRedBox(newgenvar) +## return newbox + +## incoming.append(newbox.getgenvar(jitstate)) +## return True + +## def _match(self, jitstate, newbox, incoming, memo): +## incoming.append(newbox.getgenvar(jitstate)) +## return True + +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[newbox] = newnewbox = VarRedBox(newgenvar) +## return newnewbox + + +##VCONTAINER = lltype.GcStruct("vcontainer") + +##class ContainerRedBox(RedBox): +## def __init__(self, envelope, content_addr): +## self.envelope = envelope +## self.content_addr = content_addr + +## def getgenvar(self, jitstate): # no support at the moment +## raise RuntimeError("cannot force virtual containers") + +## def ll_make_container_box(envelope, content_addr): +## return ContainerRedBox(envelope, content_addr) +## ll_make_container_box = staticmethod(ll_make_container_box) + +## def ll_make_subcontainer_box(box, content_addr): +## return ContainerRedBox(box.envelope, content_addr) +## ll_make_subcontainer_box = staticmethod(ll_make_subcontainer_box) + + +##def ll_getenvelope(box): +## assert isinstance(box, ContainerRedBox) +## return box.envelope + +##def ll_getcontent(box): +## assert isinstance(box, ContainerRedBox) +## return box.content_addr + + +##class BigRedBox(RedBox): +## "A (big) red box that contains (small) red boxes inside." + +## #def __init__(self, content_boxes): +## # self.content_boxes = content_boxes + +## def op_getfield(self, jitstate, fielddesc): +## if self.content_boxes is None: +## return RedBox.op_getfield(self, jitstate, fielddesc) +## else: +## return self.content_boxes[fielddesc.fieldindex] -VCONTAINER = lltype.GcStruct("vcontainer") - -class ContainerRedBox(RedBox): - def __init__(self, envelope, content_addr): - self.envelope = envelope - self.content_addr = content_addr - - def getgenvar(self, jitstate): # no support at the moment - raise RuntimeError("cannot force virtual containers") - - def ll_make_container_box(envelope, content_addr): - return ContainerRedBox(envelope, content_addr) - ll_make_container_box = staticmethod(ll_make_container_box) - - def ll_make_subcontainer_box(box, content_addr): - return ContainerRedBox(box.envelope, content_addr) - ll_make_subcontainer_box = staticmethod(ll_make_subcontainer_box) - - -def ll_getenvelope(box): - assert isinstance(box, ContainerRedBox) - return box.envelope - -def ll_getcontent(box): - assert isinstance(box, ContainerRedBox) - return box.content_addr - - -class BigRedBox(RedBox): - "A (big) red box that contains (small) red boxes inside." - - #def __init__(self, content_boxes): - # self.content_boxes = content_boxes +## def op_setfield(self, jitstate, fielddesc, valuebox): +## if self.content_boxes is None: +## RedBox.op_setfield(self, jitstate, fielddesc, valuebox) +## else: +## self.content_boxes[fielddesc.fieldindex] = valuebox - def op_getfield(self, jitstate, fielddesc): - if self.content_boxes is None: - return RedBox.op_getfield(self, jitstate, fielddesc) - else: - return self.content_boxes[fielddesc.fieldindex] +## def op_getsubstruct(self, jitstate, fielddesc): +## if self.content_boxes is None: +## return RedBox.op_getsubstruct(self, jitstate, fielddesc) +## else: +## return self.content_boxes[fielddesc.fieldindex] - def op_setfield(self, jitstate, fielddesc, valuebox): - if self.content_boxes is None: - RedBox.op_setfield(self, jitstate, fielddesc, valuebox) - else: - self.content_boxes[fielddesc.fieldindex] = valuebox - def op_getsubstruct(self, jitstate, fielddesc): - if self.content_boxes is None: - return RedBox.op_getsubstruct(self, jitstate, fielddesc) - else: - return self.content_boxes[fielddesc.fieldindex] +##class VirtualRedBox(BigRedBox): +## "A red box that contains (for now) a virtual Struct." +## def __init__(self, typedesc): +## self.content_boxes = typedesc.build_content_boxes(self) +## self.typedesc = typedesc +## self.genvar = rgenop.nullvar + +## def getgenvar(self, jitstate): +## if not self.genvar: +## typedesc = self.typedesc +## boxes = self.content_boxes +## self.content_boxes = None +## op_args = lltype.malloc(VARLIST.TO, 1) +## op_args[0] = typedesc.gv_type +## self.genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, +## typedesc.gv_ptrtype) +## typedesc.materialize_content(jitstate, self.genvar, boxes) +## return self.genvar + +## def is_forced(self): +## return bool(self.genvar) + +## def getallvariables(self, jitstate, result_gv, memo): +## if self.genvar: +## if self not in memo: +## result_gv.append(self.genvar) +## memo[self] = None +## else: +## for smallbox in self.content_boxes: +## smallbox.getallvariables(jitstate, result_gv, memo) -class VirtualRedBox(BigRedBox): - "A red box that contains (for now) a virtual Struct." +## def _copybox(self, newblock, gv_type, memo): +## if self.genvar: +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[self] = newbox = VarRedBox(newgenvar) +## return newbox +## bigbox = VirtualRedBox(self.typedesc) +## memo[self] = bigbox +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype +## bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, +## gv_fldtype, +## memo) +## return bigbox - def __init__(self, typedesc): - self.content_boxes = typedesc.build_content_boxes(self) - self.typedesc = typedesc - self.genvar = rgenop.nullvar - - def getgenvar(self, jitstate): - if not self.genvar: - typedesc = self.typedesc - boxes = self.content_boxes - self.content_boxes = None - op_args = lltype.malloc(VARLIST.TO, 1) - op_args[0] = typedesc.gv_type - self.genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, - typedesc.gv_ptrtype) - typedesc.materialize_content(jitstate, self.genvar, boxes) - return self.genvar - - def is_forced(self): - return bool(self.genvar) - - def getallvariables(self, jitstate, result_gv, memo): - if self.genvar: - if self not in memo: - result_gv.append(self.genvar) - memo[self] = None - else: - for smallbox in self.content_boxes: - smallbox.getallvariables(jitstate, result_gv, memo) +## def _match(self, jitstate, newbox, incoming, memo): +## if self.genvar: +## incoming.append(newbox.getgenvar(jitstate)) +## return True +## if not isinstance(newbox, VirtualRedBox): +## return False +## for i in range(len(self.content_boxes)): +## mysmallbox = self.content_boxes[i] +## newsmallbox = newbox.content_boxes[i] +## if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): +## return False +## else: +## return True - def _copybox(self, newblock, gv_type, memo): - if self.genvar: - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[self] = newbox = VarRedBox(newgenvar) - return newbox - bigbox = VirtualRedBox(self.typedesc) - memo[self] = bigbox - for i in range(len(bigbox.content_boxes)): - gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype - bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, - gv_fldtype, - memo) - return bigbox - - def _match(self, jitstate, newbox, incoming, memo): - if self.genvar: - incoming.append(newbox.getgenvar(jitstate)) - return True - if not isinstance(newbox, VirtualRedBox): - return False - for i in range(len(self.content_boxes)): - mysmallbox = self.content_boxes[i] - newsmallbox = newbox.content_boxes[i] - if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): - return False - else: - return True +## def inlined_structs_are_compatible(self, newbox): +## return (isinstance(newbox, VirtualRedBox) and not newbox.genvar and +## self.typedesc.compare_content_boxes(self.content_boxes, +## newbox.content_boxes)) + +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## if self.genvar or not self.inlined_structs_are_compatible(newbox): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[newbox] = newnewbox = VarRedBox(newgenvar) +## return newnewbox +## bigbox = VirtualRedBox(self.typedesc) +## memo[newbox] = bigbox +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype +## box = self.content_boxes[i] +## bigbox.content_boxes[i] = box.union_for_new_block( +## jitstate, +## newbox.content_boxes[i], +## newblock, +## gv_fldtype, +## incoming, +## memo) +## return bigbox + + +##class SubVirtualRedBox(BigRedBox): + +## def __init__(self, parentbox, fielddesc): +## self.parentbox = parentbox +## self.fielddesc = fielddesc +## typedesc = fielddesc.inlined_typedesc +## self.content_boxes = typedesc.build_content_boxes(self) + +## def getgenvar(self, jitstate): +## gv = self.parentbox.getgenvar(jitstate) +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = gv +## op_args[1] = self.fielddesc.gv_fieldname +## genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, +## self.fielddesc.gv_resulttype) +## return genvar + +## def is_forced(self): +## return self.parentbox.is_forced() + +## def getallvariables(self, jitstate, result_gv, memo): +## if self.is_forced(): +## if self not in memo: +## result_gv.append(self.getgenvar(jitstate)) +## memo[self] = None +## else: +## for smallbox in self.content_boxes: +## smallbox.getallvariables(jitstate, result_gv, memo) - def inlined_structs_are_compatible(self, newbox): - return (isinstance(newbox, VirtualRedBox) and not newbox.genvar and - self.typedesc.compare_content_boxes(self.content_boxes, - newbox.content_boxes)) - - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - if self.genvar or not self.inlined_structs_are_compatible(newbox): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[newbox] = newnewbox = VarRedBox(newgenvar) - return newnewbox - bigbox = VirtualRedBox(self.typedesc) - memo[newbox] = bigbox - for i in range(len(bigbox.content_boxes)): - gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype - box = self.content_boxes[i] - bigbox.content_boxes[i] = box.union_for_new_block( - jitstate, - newbox.content_boxes[i], - newblock, - gv_fldtype, - incoming, - memo) - return bigbox - - -class SubVirtualRedBox(BigRedBox): - - def __init__(self, parentbox, fielddesc): - self.parentbox = parentbox - self.fielddesc = fielddesc - typedesc = fielddesc.inlined_typedesc - self.content_boxes = typedesc.build_content_boxes(self) - - def getgenvar(self, jitstate): - gv = self.parentbox.getgenvar(jitstate) - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = gv - op_args[1] = self.fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, - self.fielddesc.gv_resulttype) - return genvar - - def is_forced(self): - return self.parentbox.is_forced() - - def getallvariables(self, jitstate, result_gv, memo): - if self.is_forced(): - if self not in memo: - result_gv.append(self.getgenvar(jitstate)) - memo[self] = None - else: - for smallbox in self.content_boxes: - smallbox.getallvariables(jitstate, result_gv, memo) +## def _copybox(self, newblock, gv_type, memo): +## if self.is_forced(): +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[self] = newbox = VarRedBox(newgenvar) +## return newbox +## bigbox = SubVirtualRedBox(None, self.fielddesc) +## memo[self] = bigbox +## gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype +## parentcopybox = self.parentbox.copybox(newblock, gv_parenttype, memo) +## bigbox.parentbox = parentcopybox +## typedesc = self.fielddesc.inlined_typedesc +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = typedesc.fielddescs[i].gv_resulttype +## bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, +## gv_fldtype, +## memo) +## return bigbox - def _copybox(self, newblock, gv_type, memo): - if self.is_forced(): - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[self] = newbox = VarRedBox(newgenvar) - return newbox - bigbox = SubVirtualRedBox(None, self.fielddesc) - memo[self] = bigbox - gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype - parentcopybox = self.parentbox.copybox(newblock, gv_parenttype, memo) - bigbox.parentbox = parentcopybox - typedesc = self.fielddesc.inlined_typedesc - for i in range(len(bigbox.content_boxes)): - gv_fldtype = typedesc.fielddescs[i].gv_resulttype - bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, - gv_fldtype, - memo) - return bigbox - - def _match(self, jitstate, newbox, incoming, memo): - if self.is_forced(): - incoming.append(newbox.getgenvar(jitstate)) - return True - if not (isinstance(newbox, SubVirtualRedBox) and - self.fielddesc is newbox.fielddesc and - self.parentbox.match(jitstate, newbox.parentbox, - incoming, memo)): - return False - for i in range(len(self.content_boxes)): - mysmallbox = self.content_boxes[i] - newsmallbox = newbox.content_boxes[i] - if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): - return False - else: - return True +## def _match(self, jitstate, newbox, incoming, memo): +## if self.is_forced(): +## incoming.append(newbox.getgenvar(jitstate)) +## return True +## if not (isinstance(newbox, SubVirtualRedBox) and +## self.fielddesc is newbox.fielddesc and +## self.parentbox.match(jitstate, newbox.parentbox, +## incoming, memo)): +## return False +## for i in range(len(self.content_boxes)): +## mysmallbox = self.content_boxes[i] +## newsmallbox = newbox.content_boxes[i] +## if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): +## return False +## else: +## return True - def inlined_structs_are_compatible(self, newbox): - if (isinstance(newbox, SubVirtualRedBox) and not newbox.is_forced() and - self.fielddesc is newbox.fielddesc): - return self.parentbox.inlined_structs_are_compatible( - newbox.parentbox) - else: - return False +## def inlined_structs_are_compatible(self, newbox): +## if (isinstance(newbox, SubVirtualRedBox) and not newbox.is_forced() and +## self.fielddesc is newbox.fielddesc): +## return self.parentbox.inlined_structs_are_compatible( +## newbox.parentbox) +## else: +## return False - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - if self.is_forced() or not self.inlined_structs_are_compatible(newbox): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[newbox] = newnewbox = VarRedBox(newgenvar) - return newnewbox - assert isinstance(newbox, SubVirtualRedBox) - bigbox = SubVirtualRedBox(None, self.fielddesc) - memo[newbox] = bigbox - gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype - parentcopybox = self.parentbox.union_for_new_block(jitstate, - newbox.parentbox, - newblock, - gv_parenttype, - incoming, - memo) - bigbox.parentbox = parentcopybox - typedesc = self.fielddesc.inlined_typedesc - for i in range(len(bigbox.content_boxes)): - gv_fldtype = typedesc.fielddescs[i].gv_resulttype - box = self.content_boxes[i] - bigbox.content_boxes[i] = box.union_for_new_block( - jitstate, - newbox.content_boxes[i], - newblock, - gv_fldtype, - incoming, - memo) - return bigbox - - -class ConstRedBox(RedBox): - "A red box that contains a run-time constant." - - def __init__(self, genvar): - self.genvar = genvar - - def getgenvar(self, jitstate): - return self.genvar - - def copybox(self, newblock, gv_type, memo): - return self - - def match(self, jitstate, newbox, incoming, memo): - return self.same_constant(newbox) - - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - if self.same_constant(newbox): - newnewbox = newbox - else: - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - newnewbox = VarRedBox(newgenvar) - memo[newbox] = newnewbox - return newnewbox - - def ll_fromvalue(value): - T = lltype.typeOf(value) - gv = rgenop.genconst(value) - if isinstance(T, lltype.Ptr): - return AddrRedBox(gv) - elif T is lltype.Float: - return DoubleRedBox(gv) - else: - assert isinstance(T, lltype.Primitive) - assert T is not lltype.Void, "cannot make red boxes of voids" - # XXX what about long longs? - return IntRedBox(gv) - ll_fromvalue = staticmethod(ll_fromvalue) - - def ll_getvalue(self, T): - # note: this is specialized by low-level type T, as a low-level helper - return rgenop.revealconst(T, self.genvar) +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## if self.is_forced() or not self.inlined_structs_are_compatible(newbox): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[newbox] = newnewbox = VarRedBox(newgenvar) +## return newnewbox +## assert isinstance(newbox, SubVirtualRedBox) +## bigbox = SubVirtualRedBox(None, self.fielddesc) +## memo[newbox] = bigbox +## gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype +## parentcopybox = self.parentbox.union_for_new_block(jitstate, +## newbox.parentbox, +## newblock, +## gv_parenttype, +## incoming, +## memo) +## bigbox.parentbox = parentcopybox +## typedesc = self.fielddesc.inlined_typedesc +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = typedesc.fielddescs[i].gv_resulttype +## box = self.content_boxes[i] +## bigbox.content_boxes[i] = box.union_for_new_block( +## jitstate, +## newbox.content_boxes[i], +## newblock, +## gv_fldtype, +## incoming, +## memo) +## return bigbox + + +##class ConstRedBox(RedBox): +## "A red box that contains a run-time constant." + +## def __init__(self, genvar): +## self.genvar = genvar + +## def getgenvar(self, jitstate): +## return self.genvar + +## def copybox(self, newblock, gv_type, memo): +## return self + +## def match(self, jitstate, newbox, incoming, memo): +## return self.same_constant(newbox) + +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## if self.same_constant(newbox): +## newnewbox = newbox +## else: +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## newnewbox = VarRedBox(newgenvar) +## memo[newbox] = newnewbox +## return newnewbox + +## def ll_fromvalue(value): +## T = lltype.typeOf(value) +## gv = rgenop.genconst(value) +## if isinstance(T, lltype.Ptr): +## return AddrRedBox(gv) +## elif T is lltype.Float: +## return DoubleRedBox(gv) +## else: +## assert isinstance(T, lltype.Primitive) +## assert T is not lltype.Void, "cannot make red boxes of voids" +## # XXX what about long longs? +## return IntRedBox(gv) +## ll_fromvalue = staticmethod(ll_fromvalue) + +## def ll_getvalue(self, T): +## # note: this is specialized by low-level type T, as a low-level helper +## return rgenop.revealconst(T, self.genvar) -def ll_getvalue(box, T): - return box.ll_getvalue(T) +##def ll_getvalue(box, T): +## return box.ll_getvalue(T) -class IntRedBox(ConstRedBox): - "A red box that contains a constant integer-like value." +##class IntRedBox(ConstRedBox): +## "A red box that contains a constant integer-like value." - def same_constant(self, other): - return (isinstance(other, IntRedBox) and - self.ll_getvalue(lltype.Signed) == other.ll_getvalue(lltype.Signed)) +## def same_constant(self, other): +## return (isinstance(other, IntRedBox) and +## self.ll_getvalue(lltype.Signed) == other.ll_getvalue(lltype.Signed)) -class DoubleRedBox(ConstRedBox): - "A red box that contains a constant double-precision floating point value." +##class DoubleRedBox(ConstRedBox): +## "A red box that contains a constant double-precision floating point value." - def same_constant(self, other): - return (isinstance(other, DoubleRedBox) and - self.ll_getvalue(lltype.Float) == other.ll_getvalue(lltype.Float)) +## def same_constant(self, other): +## return (isinstance(other, DoubleRedBox) and +## self.ll_getvalue(lltype.Float) == other.ll_getvalue(lltype.Float)) -class AddrRedBox(ConstRedBox): - "A red box that contains a constant address." +##class AddrRedBox(ConstRedBox): +## "A red box that contains a constant address." - def same_constant(self, other): - return (isinstance(other, AddrRedBox) and - self.ll_getvalue(llmemory.Address) == other.ll_getvalue(llmemory.Address)) +## def same_constant(self, other): +## return (isinstance(other, AddrRedBox) and +## self.ll_getvalue(llmemory.Address) == other.ll_getvalue(llmemory.Address)) # ____________________________________________________________ Added: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- (empty file) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Sat Apr 1 19:32:24 2006 @@ -0,0 +1,264 @@ +from pypy.rpython import rgenop +from pypy.rpython.lltypesystem import lltype + + +class AbstractRedBox(object): + + def __init__(self, genvar=rgenop.nullvar): + self.genvar = genvar # nullvar or a genvar + + def __repr__(self): + if not self.genvar: + return '' + else: + return '<%r>' % (rgenop.reveal(self.genvar),) + + def is_constant(self): + return bool(self.genvar) and rgenop.isconst(self.genvar) + + def getgenvar(self, jitstate): + return self.genvar + + def ll_fromvalue(value): + "Make a constant RedBox from a low-level value." + T = lltype.typeOf(value) + gv = rgenop.genconst(value) + if isinstance(T, lltype.Ptr): + return PtrRedBox(gv) + elif T is lltype.Float: + return DoubleRedBox(gv) + else: + assert isinstance(T, lltype.Primitive) + assert T is not lltype.Void, "cannot make red boxes of voids" + # XXX what about long longs? + return IntRedBox(gv) + ll_fromvalue = staticmethod(ll_fromvalue) + + def ll_getvalue(self, T): + "Return the content of a known-to-be-constant RedBox." + # note: this is specialized by low-level type T, as a low-level helper + return rgenop.revealconst(T, self.genvar) + +def ll_getvalue(box, T): + return box.ll_getvalue(T) + + +class IntRedBox(AbstractRedBox): + "A red box that contains a constant integer-like value." + + def freeze(self, memo): + try: + return memo[self] + except KeyError: + if self.is_constant(): + result = FrozenIntConst(self.genvar) + else: + result = FrozenIntVar() + memo[self] = result + return result + + +class DoubleRedBox(AbstractRedBox): + "A red box that contains a constant double-precision floating point value." + + def freeze(self, memo): + try: + return memo[self] + except KeyError: + if self.is_constant(): + result = FrozenDoubleConst(self.genvar) + else: + result = FrozenDoubleVar() + memo[self] = result + return result + + +class PtrRedBox(AbstractRedBox): + + def __init__(self, genvar=rgenop.nullvar, content=None): + AbstractRedBox.__init__(self, genvar) + self.content = content # None or an AbstractContainer + + def __repr__(self): + if not self.genvar and self.content is not None: + return '' % (self.content,) + else: + return AbstractRedBox.__repr__(self) + + def freeze(self, memo): + try: + return memo[self] + except KeyError: + if self.genvar: + if self.is_constant(): + result = FrozenPtrConst(self.genvar) + else: + result = FrozenPtrVar() + memo[self] = result + else: + result = FrozenPtrVirtual() + memo[self] = result + result.fz_content = self.content.freeze(memo) + return result + + def getgenvar(self, jitstate): + if not self.genvar: + self.genvar = self.content.build_runtime_container(jitstate) + return self.genvar + +# ____________________________________________________________ + +class FrozenValue(object): + """An abstract value frozen in a saved state. + """ + + +class FrozenIntConst(FrozenValue): + + def __init__(self, gv_const): + self.gv_const = gv_const + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + box = memo[self] = IntRedBox(self.gv_const) + return box + + def exactmatch(self, box, outgoingvars, memo): + if (box.is_constant() and + rgenop.revealconst(lltype.Signed, self.gv_const) == + rgenop.revealconst(lltype.Signed, box.genvar)): + return True + else: + outgoingvars.append(box) + return False + + +class FrozenIntVar(FrozenValue): + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + gv_value = rgenop.geninputarg(block, gv_type) + box = memo[self] = IntRedBox(gv_value) + return box + + def exactmatch(self, box, outgoingvars, memo): + if self not in memo: + memo[self] = box + outgoingvars.append(box) + return True + elif memo[self] is box: + return True + else: + outgoingvars.append(box) + return False + + +class FrozenDoubleConst(FrozenValue): + + def __init__(self, gv_const): + self.gv_const = gv_const + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + box = memo[self] = DoubleRedBox(self.gv_const) + return box + + def exactmatch(self, box, outgoingvars, memo): + if (box.is_constant() and + rgenop.revealconst(lltype.Double, self.gv_const) == + rgenop.revealconst(lltype.Double, box.genvar)): + return True + else: + outgoingvars.append(box) + return False + + +class FrozenDoubleVar(FrozenValue): + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + gv_value = rgenop.geninputarg(block, gv_type) + box = memo[self] = DoubleRedBox(gv_value) + return box + + def exactmatch(self, box, outgoingvars, memo): + if self not in memo: + memo[self] = box + outgoingvars.append(box) + return True + elif memo[self] is box: + return True + else: + outgoingvars.append(box) + return False + + +class FrozenPtrConst(FrozenValue): + + def __init__(self, gv_const): + self.gv_const = gv_const + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + box = memo[self] = PtrRedBox(self.gv_const) + return box + + def exactmatch(self, box, outgoingvars, memo): + if (box.is_constant() and + rgenop.revealconst(lltype.Address, self.gv_const) == + rgenop.revealconst(lltype.Address, box.genvar)): + return True + else: + outgoingvars.append(box) + return False + + +class FrozenPtrVar(FrozenValue): + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + gv_value = rgenop.geninputarg(block, gv_type) + box = memo[self] = PtrRedBox(gv_value) + return box + + def exactmatch(self, box, outgoingvars, memo): + if self not in memo: + memo[self] = box + outgoingvars.append(box) + return True + elif memo[self] is box: + return True + else: + outgoingvars.append(box) + return False + + +class FrozenPtrVirtual(FrozenValue): + + def unfreeze(self, memo, block, gv_type): + try: + return memo[self] + except KeyError: + box = memo[self] = PtrRedBox() + box.content = self.fz_content.unfreeze(memo, block) + return box + + def exactmatch(self, box, outgoingvars, memo): + assert isinstance(box, PtrRedBox) + if box.content is None: + outgoingvars.append(box) + return False + else: + return self.fz_content.exactmatch(box.content, outgoingvars, memo) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Sat Apr 1 19:32:24 2006 @@ -3,8 +3,7 @@ from pypy.objspace.flow import model as flowmodel from pypy.annotation import model as annmodel from pypy.annotation import listdef, dictdef -from pypy.jit.timeshifter.rtimeshift import VARLIST, RedBox, VarRedBox -from pypy.jit.timeshifter.rtimeshift import ConstRedBox, JITState +from pypy.jit.timeshifter.rtimeshift import VARLIST, JITState from pypy.jit.timeshifter.rtimeshift import make_types_const from pypy.rpython import rmodel, rlist, rdict, rgenop, annlowlevel from pypy.rpython.lltypesystem import rtuple From arigo at codespeak.net Sat Apr 1 19:37:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 19:37:20 +0200 (CEST) Subject: [pypy-svn] r25212 - pypy/branch/jit-timeshifter-refactoring Message-ID: <20060401173720.256F91008B@code0.codespeak.net> Author: arigo Date: Sat Apr 1 19:37:19 2006 New Revision: 25212 Removed: pypy/branch/jit-timeshifter-refactoring/ Log: Uh? Broken branch. It seems that it resurrected files that have been deleted because of some 'svn up' missing after the 'svn ci' deleting them. Bug. From arigo at codespeak.net Sat Apr 1 19:37:40 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 19:37:40 +0200 (CEST) Subject: [pypy-svn] r25213 - in pypy/branch/jit-timeshifter-refactoring: . timeshifter Message-ID: <20060401173740.D2B1510098@code0.codespeak.net> Author: arigo Date: Sat Apr 1 19:37:39 2006 New Revision: 25213 Added: pypy/branch/jit-timeshifter-refactoring/ - copied from r25211, pypy/dist/pypy/jit/ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py - copied unchanged from r25211, pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: Second try. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sat Apr 1 19:37:39 2006 @@ -27,411 +27,411 @@ return l -class RedBox(object): +##class RedBox(object): - def same_constant(self, other): - return False +## def same_constant(self, other): +## return False - def getallvariables(self, jitstate, result_gv, memo): - pass - - def copybox(self, newblock, gv_type, memo): - try: - return memo[self] - except KeyError: - return self._copybox(newblock, gv_type, memo) - - def match(self, jitstate, newbox, incoming, memo): - if self in memo: - return memo[self] is newbox - if newbox in memo: - return memo[newbox] is self - memo[self] = newbox - memo[newbox] = self - return self._match(jitstate, newbox, incoming, memo) - - def union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - try: - return memo[newbox] - except KeyError: - return self._union_for_new_block(jitstate, newbox, newblock, - gv_type, incoming, memo) - - # generic implementation of some operations - def op_getfield(self, jitstate, fielddesc): - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = self.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, - fielddesc.gv_resulttype) - return VarRedBox(genvar) - - def op_setfield(self, jitstate, fielddesc, valuebox): - op_args = lltype.malloc(VARLIST.TO, 3) - op_args[0] = self.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - op_args[2] = valuebox.getgenvar(jitstate) - rgenop.genop(jitstate.curblock, 'setfield', op_args, - gv_Void) - - def op_getsubstruct(self, jitstate, fielddesc): - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = self.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, - fielddesc.gv_resulttype) - return VarRedBox(genvar) - - -class VarRedBox(RedBox): - "A red box that contains a run-time variable." - - def __init__(self, genvar): - self.genvar = genvar - - def getgenvar(self, jitstate): - return self.genvar - - def getallvariables(self, jitstate, result_gv, memo): - if self not in memo: - result_gv.append(self.genvar) - memo[self] = None - - def _copybox(self, newblock, gv_type, memo): - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[self] = newbox = VarRedBox(newgenvar) - return newbox - - incoming.append(newbox.getgenvar(jitstate)) - return True - - def _match(self, jitstate, newbox, incoming, memo): - incoming.append(newbox.getgenvar(jitstate)) - return True - - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[newbox] = newnewbox = VarRedBox(newgenvar) - return newnewbox +## def getallvariables(self, jitstate, result_gv, memo): +## pass +## def copybox(self, newblock, gv_type, memo): +## try: +## return memo[self] +## except KeyError: +## return self._copybox(newblock, gv_type, memo) + +## def match(self, jitstate, newbox, incoming, memo): +## if self in memo: +## return memo[self] is newbox +## if newbox in memo: +## return memo[newbox] is self +## memo[self] = newbox +## memo[newbox] = self +## return self._match(jitstate, newbox, incoming, memo) + +## def union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## try: +## return memo[newbox] +## except KeyError: +## return self._union_for_new_block(jitstate, newbox, newblock, +## gv_type, incoming, memo) + +## # generic implementation of some operations +## def op_getfield(self, jitstate, fielddesc): +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = self.getgenvar(jitstate) +## op_args[1] = fielddesc.gv_fieldname +## genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, +## fielddesc.gv_resulttype) +## return VarRedBox(genvar) + +## def op_setfield(self, jitstate, fielddesc, valuebox): +## op_args = lltype.malloc(VARLIST.TO, 3) +## op_args[0] = self.getgenvar(jitstate) +## op_args[1] = fielddesc.gv_fieldname +## op_args[2] = valuebox.getgenvar(jitstate) +## rgenop.genop(jitstate.curblock, 'setfield', op_args, +## gv_Void) + +## def op_getsubstruct(self, jitstate, fielddesc): +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = self.getgenvar(jitstate) +## op_args[1] = fielddesc.gv_fieldname +## genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, +## fielddesc.gv_resulttype) +## return VarRedBox(genvar) + + +##class VarRedBox(RedBox): +## "A red box that contains a run-time variable." + +## def __init__(self, genvar): +## self.genvar = genvar + +## def getgenvar(self, jitstate): +## return self.genvar + +## def getallvariables(self, jitstate, result_gv, memo): +## if self not in memo: +## result_gv.append(self.genvar) +## memo[self] = None + +## def _copybox(self, newblock, gv_type, memo): +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[self] = newbox = VarRedBox(newgenvar) +## return newbox + +## incoming.append(newbox.getgenvar(jitstate)) +## return True + +## def _match(self, jitstate, newbox, incoming, memo): +## incoming.append(newbox.getgenvar(jitstate)) +## return True + +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[newbox] = newnewbox = VarRedBox(newgenvar) +## return newnewbox + + +##VCONTAINER = lltype.GcStruct("vcontainer") + +##class ContainerRedBox(RedBox): +## def __init__(self, envelope, content_addr): +## self.envelope = envelope +## self.content_addr = content_addr + +## def getgenvar(self, jitstate): # no support at the moment +## raise RuntimeError("cannot force virtual containers") + +## def ll_make_container_box(envelope, content_addr): +## return ContainerRedBox(envelope, content_addr) +## ll_make_container_box = staticmethod(ll_make_container_box) + +## def ll_make_subcontainer_box(box, content_addr): +## return ContainerRedBox(box.envelope, content_addr) +## ll_make_subcontainer_box = staticmethod(ll_make_subcontainer_box) + + +##def ll_getenvelope(box): +## assert isinstance(box, ContainerRedBox) +## return box.envelope + +##def ll_getcontent(box): +## assert isinstance(box, ContainerRedBox) +## return box.content_addr + + +##class BigRedBox(RedBox): +## "A (big) red box that contains (small) red boxes inside." + +## #def __init__(self, content_boxes): +## # self.content_boxes = content_boxes + +## def op_getfield(self, jitstate, fielddesc): +## if self.content_boxes is None: +## return RedBox.op_getfield(self, jitstate, fielddesc) +## else: +## return self.content_boxes[fielddesc.fieldindex] -VCONTAINER = lltype.GcStruct("vcontainer") - -class ContainerRedBox(RedBox): - def __init__(self, envelope, content_addr): - self.envelope = envelope - self.content_addr = content_addr - - def getgenvar(self, jitstate): # no support at the moment - raise RuntimeError("cannot force virtual containers") - - def ll_make_container_box(envelope, content_addr): - return ContainerRedBox(envelope, content_addr) - ll_make_container_box = staticmethod(ll_make_container_box) - - def ll_make_subcontainer_box(box, content_addr): - return ContainerRedBox(box.envelope, content_addr) - ll_make_subcontainer_box = staticmethod(ll_make_subcontainer_box) - - -def ll_getenvelope(box): - assert isinstance(box, ContainerRedBox) - return box.envelope - -def ll_getcontent(box): - assert isinstance(box, ContainerRedBox) - return box.content_addr - - -class BigRedBox(RedBox): - "A (big) red box that contains (small) red boxes inside." - - #def __init__(self, content_boxes): - # self.content_boxes = content_boxes +## def op_setfield(self, jitstate, fielddesc, valuebox): +## if self.content_boxes is None: +## RedBox.op_setfield(self, jitstate, fielddesc, valuebox) +## else: +## self.content_boxes[fielddesc.fieldindex] = valuebox - def op_getfield(self, jitstate, fielddesc): - if self.content_boxes is None: - return RedBox.op_getfield(self, jitstate, fielddesc) - else: - return self.content_boxes[fielddesc.fieldindex] +## def op_getsubstruct(self, jitstate, fielddesc): +## if self.content_boxes is None: +## return RedBox.op_getsubstruct(self, jitstate, fielddesc) +## else: +## return self.content_boxes[fielddesc.fieldindex] - def op_setfield(self, jitstate, fielddesc, valuebox): - if self.content_boxes is None: - RedBox.op_setfield(self, jitstate, fielddesc, valuebox) - else: - self.content_boxes[fielddesc.fieldindex] = valuebox - def op_getsubstruct(self, jitstate, fielddesc): - if self.content_boxes is None: - return RedBox.op_getsubstruct(self, jitstate, fielddesc) - else: - return self.content_boxes[fielddesc.fieldindex] +##class VirtualRedBox(BigRedBox): +## "A red box that contains (for now) a virtual Struct." +## def __init__(self, typedesc): +## self.content_boxes = typedesc.build_content_boxes(self) +## self.typedesc = typedesc +## self.genvar = rgenop.nullvar + +## def getgenvar(self, jitstate): +## if not self.genvar: +## typedesc = self.typedesc +## boxes = self.content_boxes +## self.content_boxes = None +## op_args = lltype.malloc(VARLIST.TO, 1) +## op_args[0] = typedesc.gv_type +## self.genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, +## typedesc.gv_ptrtype) +## typedesc.materialize_content(jitstate, self.genvar, boxes) +## return self.genvar + +## def is_forced(self): +## return bool(self.genvar) + +## def getallvariables(self, jitstate, result_gv, memo): +## if self.genvar: +## if self not in memo: +## result_gv.append(self.genvar) +## memo[self] = None +## else: +## for smallbox in self.content_boxes: +## smallbox.getallvariables(jitstate, result_gv, memo) -class VirtualRedBox(BigRedBox): - "A red box that contains (for now) a virtual Struct." +## def _copybox(self, newblock, gv_type, memo): +## if self.genvar: +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[self] = newbox = VarRedBox(newgenvar) +## return newbox +## bigbox = VirtualRedBox(self.typedesc) +## memo[self] = bigbox +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype +## bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, +## gv_fldtype, +## memo) +## return bigbox - def __init__(self, typedesc): - self.content_boxes = typedesc.build_content_boxes(self) - self.typedesc = typedesc - self.genvar = rgenop.nullvar - - def getgenvar(self, jitstate): - if not self.genvar: - typedesc = self.typedesc - boxes = self.content_boxes - self.content_boxes = None - op_args = lltype.malloc(VARLIST.TO, 1) - op_args[0] = typedesc.gv_type - self.genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, - typedesc.gv_ptrtype) - typedesc.materialize_content(jitstate, self.genvar, boxes) - return self.genvar - - def is_forced(self): - return bool(self.genvar) - - def getallvariables(self, jitstate, result_gv, memo): - if self.genvar: - if self not in memo: - result_gv.append(self.genvar) - memo[self] = None - else: - for smallbox in self.content_boxes: - smallbox.getallvariables(jitstate, result_gv, memo) +## def _match(self, jitstate, newbox, incoming, memo): +## if self.genvar: +## incoming.append(newbox.getgenvar(jitstate)) +## return True +## if not isinstance(newbox, VirtualRedBox): +## return False +## for i in range(len(self.content_boxes)): +## mysmallbox = self.content_boxes[i] +## newsmallbox = newbox.content_boxes[i] +## if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): +## return False +## else: +## return True - def _copybox(self, newblock, gv_type, memo): - if self.genvar: - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[self] = newbox = VarRedBox(newgenvar) - return newbox - bigbox = VirtualRedBox(self.typedesc) - memo[self] = bigbox - for i in range(len(bigbox.content_boxes)): - gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype - bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, - gv_fldtype, - memo) - return bigbox - - def _match(self, jitstate, newbox, incoming, memo): - if self.genvar: - incoming.append(newbox.getgenvar(jitstate)) - return True - if not isinstance(newbox, VirtualRedBox): - return False - for i in range(len(self.content_boxes)): - mysmallbox = self.content_boxes[i] - newsmallbox = newbox.content_boxes[i] - if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): - return False - else: - return True +## def inlined_structs_are_compatible(self, newbox): +## return (isinstance(newbox, VirtualRedBox) and not newbox.genvar and +## self.typedesc.compare_content_boxes(self.content_boxes, +## newbox.content_boxes)) + +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## if self.genvar or not self.inlined_structs_are_compatible(newbox): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[newbox] = newnewbox = VarRedBox(newgenvar) +## return newnewbox +## bigbox = VirtualRedBox(self.typedesc) +## memo[newbox] = bigbox +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype +## box = self.content_boxes[i] +## bigbox.content_boxes[i] = box.union_for_new_block( +## jitstate, +## newbox.content_boxes[i], +## newblock, +## gv_fldtype, +## incoming, +## memo) +## return bigbox + + +##class SubVirtualRedBox(BigRedBox): + +## def __init__(self, parentbox, fielddesc): +## self.parentbox = parentbox +## self.fielddesc = fielddesc +## typedesc = fielddesc.inlined_typedesc +## self.content_boxes = typedesc.build_content_boxes(self) + +## def getgenvar(self, jitstate): +## gv = self.parentbox.getgenvar(jitstate) +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = gv +## op_args[1] = self.fielddesc.gv_fieldname +## genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, +## self.fielddesc.gv_resulttype) +## return genvar + +## def is_forced(self): +## return self.parentbox.is_forced() + +## def getallvariables(self, jitstate, result_gv, memo): +## if self.is_forced(): +## if self not in memo: +## result_gv.append(self.getgenvar(jitstate)) +## memo[self] = None +## else: +## for smallbox in self.content_boxes: +## smallbox.getallvariables(jitstate, result_gv, memo) - def inlined_structs_are_compatible(self, newbox): - return (isinstance(newbox, VirtualRedBox) and not newbox.genvar and - self.typedesc.compare_content_boxes(self.content_boxes, - newbox.content_boxes)) - - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - if self.genvar or not self.inlined_structs_are_compatible(newbox): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[newbox] = newnewbox = VarRedBox(newgenvar) - return newnewbox - bigbox = VirtualRedBox(self.typedesc) - memo[newbox] = bigbox - for i in range(len(bigbox.content_boxes)): - gv_fldtype = self.typedesc.fielddescs[i].gv_resulttype - box = self.content_boxes[i] - bigbox.content_boxes[i] = box.union_for_new_block( - jitstate, - newbox.content_boxes[i], - newblock, - gv_fldtype, - incoming, - memo) - return bigbox - - -class SubVirtualRedBox(BigRedBox): - - def __init__(self, parentbox, fielddesc): - self.parentbox = parentbox - self.fielddesc = fielddesc - typedesc = fielddesc.inlined_typedesc - self.content_boxes = typedesc.build_content_boxes(self) - - def getgenvar(self, jitstate): - gv = self.parentbox.getgenvar(jitstate) - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = gv - op_args[1] = self.fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, - self.fielddesc.gv_resulttype) - return genvar - - def is_forced(self): - return self.parentbox.is_forced() - - def getallvariables(self, jitstate, result_gv, memo): - if self.is_forced(): - if self not in memo: - result_gv.append(self.getgenvar(jitstate)) - memo[self] = None - else: - for smallbox in self.content_boxes: - smallbox.getallvariables(jitstate, result_gv, memo) +## def _copybox(self, newblock, gv_type, memo): +## if self.is_forced(): +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[self] = newbox = VarRedBox(newgenvar) +## return newbox +## bigbox = SubVirtualRedBox(None, self.fielddesc) +## memo[self] = bigbox +## gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype +## parentcopybox = self.parentbox.copybox(newblock, gv_parenttype, memo) +## bigbox.parentbox = parentcopybox +## typedesc = self.fielddesc.inlined_typedesc +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = typedesc.fielddescs[i].gv_resulttype +## bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, +## gv_fldtype, +## memo) +## return bigbox - def _copybox(self, newblock, gv_type, memo): - if self.is_forced(): - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[self] = newbox = VarRedBox(newgenvar) - return newbox - bigbox = SubVirtualRedBox(None, self.fielddesc) - memo[self] = bigbox - gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype - parentcopybox = self.parentbox.copybox(newblock, gv_parenttype, memo) - bigbox.parentbox = parentcopybox - typedesc = self.fielddesc.inlined_typedesc - for i in range(len(bigbox.content_boxes)): - gv_fldtype = typedesc.fielddescs[i].gv_resulttype - bigbox.content_boxes[i] = self.content_boxes[i].copybox(newblock, - gv_fldtype, - memo) - return bigbox - - def _match(self, jitstate, newbox, incoming, memo): - if self.is_forced(): - incoming.append(newbox.getgenvar(jitstate)) - return True - if not (isinstance(newbox, SubVirtualRedBox) and - self.fielddesc is newbox.fielddesc and - self.parentbox.match(jitstate, newbox.parentbox, - incoming, memo)): - return False - for i in range(len(self.content_boxes)): - mysmallbox = self.content_boxes[i] - newsmallbox = newbox.content_boxes[i] - if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): - return False - else: - return True +## def _match(self, jitstate, newbox, incoming, memo): +## if self.is_forced(): +## incoming.append(newbox.getgenvar(jitstate)) +## return True +## if not (isinstance(newbox, SubVirtualRedBox) and +## self.fielddesc is newbox.fielddesc and +## self.parentbox.match(jitstate, newbox.parentbox, +## incoming, memo)): +## return False +## for i in range(len(self.content_boxes)): +## mysmallbox = self.content_boxes[i] +## newsmallbox = newbox.content_boxes[i] +## if not mysmallbox.match(jitstate, newsmallbox, incoming, memo): +## return False +## else: +## return True - def inlined_structs_are_compatible(self, newbox): - if (isinstance(newbox, SubVirtualRedBox) and not newbox.is_forced() and - self.fielddesc is newbox.fielddesc): - return self.parentbox.inlined_structs_are_compatible( - newbox.parentbox) - else: - return False +## def inlined_structs_are_compatible(self, newbox): +## if (isinstance(newbox, SubVirtualRedBox) and not newbox.is_forced() and +## self.fielddesc is newbox.fielddesc): +## return self.parentbox.inlined_structs_are_compatible( +## newbox.parentbox) +## else: +## return False - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - if self.is_forced() or not self.inlined_structs_are_compatible(newbox): - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - memo[newbox] = newnewbox = VarRedBox(newgenvar) - return newnewbox - assert isinstance(newbox, SubVirtualRedBox) - bigbox = SubVirtualRedBox(None, self.fielddesc) - memo[newbox] = bigbox - gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype - parentcopybox = self.parentbox.union_for_new_block(jitstate, - newbox.parentbox, - newblock, - gv_parenttype, - incoming, - memo) - bigbox.parentbox = parentcopybox - typedesc = self.fielddesc.inlined_typedesc - for i in range(len(bigbox.content_boxes)): - gv_fldtype = typedesc.fielddescs[i].gv_resulttype - box = self.content_boxes[i] - bigbox.content_boxes[i] = box.union_for_new_block( - jitstate, - newbox.content_boxes[i], - newblock, - gv_fldtype, - incoming, - memo) - return bigbox - - -class ConstRedBox(RedBox): - "A red box that contains a run-time constant." - - def __init__(self, genvar): - self.genvar = genvar - - def getgenvar(self, jitstate): - return self.genvar - - def copybox(self, newblock, gv_type, memo): - return self - - def match(self, jitstate, newbox, incoming, memo): - return self.same_constant(newbox) - - def _union_for_new_block(self, jitstate, newbox, newblock, - gv_type, incoming, memo): - if self.same_constant(newbox): - newnewbox = newbox - else: - incoming.append(newbox.getgenvar(jitstate)) - newgenvar = rgenop.geninputarg(newblock, gv_type) - newnewbox = VarRedBox(newgenvar) - memo[newbox] = newnewbox - return newnewbox - - def ll_fromvalue(value): - T = lltype.typeOf(value) - gv = rgenop.genconst(value) - if isinstance(T, lltype.Ptr): - return AddrRedBox(gv) - elif T is lltype.Float: - return DoubleRedBox(gv) - else: - assert isinstance(T, lltype.Primitive) - assert T is not lltype.Void, "cannot make red boxes of voids" - # XXX what about long longs? - return IntRedBox(gv) - ll_fromvalue = staticmethod(ll_fromvalue) - - def ll_getvalue(self, T): - # note: this is specialized by low-level type T, as a low-level helper - return rgenop.revealconst(T, self.genvar) +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## if self.is_forced() or not self.inlined_structs_are_compatible(newbox): +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## memo[newbox] = newnewbox = VarRedBox(newgenvar) +## return newnewbox +## assert isinstance(newbox, SubVirtualRedBox) +## bigbox = SubVirtualRedBox(None, self.fielddesc) +## memo[newbox] = bigbox +## gv_parenttype = self.fielddesc.parenttypedesc.gv_ptrtype +## parentcopybox = self.parentbox.union_for_new_block(jitstate, +## newbox.parentbox, +## newblock, +## gv_parenttype, +## incoming, +## memo) +## bigbox.parentbox = parentcopybox +## typedesc = self.fielddesc.inlined_typedesc +## for i in range(len(bigbox.content_boxes)): +## gv_fldtype = typedesc.fielddescs[i].gv_resulttype +## box = self.content_boxes[i] +## bigbox.content_boxes[i] = box.union_for_new_block( +## jitstate, +## newbox.content_boxes[i], +## newblock, +## gv_fldtype, +## incoming, +## memo) +## return bigbox + + +##class ConstRedBox(RedBox): +## "A red box that contains a run-time constant." + +## def __init__(self, genvar): +## self.genvar = genvar + +## def getgenvar(self, jitstate): +## return self.genvar + +## def copybox(self, newblock, gv_type, memo): +## return self + +## def match(self, jitstate, newbox, incoming, memo): +## return self.same_constant(newbox) + +## def _union_for_new_block(self, jitstate, newbox, newblock, +## gv_type, incoming, memo): +## if self.same_constant(newbox): +## newnewbox = newbox +## else: +## incoming.append(newbox.getgenvar(jitstate)) +## newgenvar = rgenop.geninputarg(newblock, gv_type) +## newnewbox = VarRedBox(newgenvar) +## memo[newbox] = newnewbox +## return newnewbox + +## def ll_fromvalue(value): +## T = lltype.typeOf(value) +## gv = rgenop.genconst(value) +## if isinstance(T, lltype.Ptr): +## return AddrRedBox(gv) +## elif T is lltype.Float: +## return DoubleRedBox(gv) +## else: +## assert isinstance(T, lltype.Primitive) +## assert T is not lltype.Void, "cannot make red boxes of voids" +## # XXX what about long longs? +## return IntRedBox(gv) +## ll_fromvalue = staticmethod(ll_fromvalue) + +## def ll_getvalue(self, T): +## # note: this is specialized by low-level type T, as a low-level helper +## return rgenop.revealconst(T, self.genvar) -def ll_getvalue(box, T): - return box.ll_getvalue(T) +##def ll_getvalue(box, T): +## return box.ll_getvalue(T) -class IntRedBox(ConstRedBox): - "A red box that contains a constant integer-like value." +##class IntRedBox(ConstRedBox): +## "A red box that contains a constant integer-like value." - def same_constant(self, other): - return (isinstance(other, IntRedBox) and - self.ll_getvalue(lltype.Signed) == other.ll_getvalue(lltype.Signed)) +## def same_constant(self, other): +## return (isinstance(other, IntRedBox) and +## self.ll_getvalue(lltype.Signed) == other.ll_getvalue(lltype.Signed)) -class DoubleRedBox(ConstRedBox): - "A red box that contains a constant double-precision floating point value." +##class DoubleRedBox(ConstRedBox): +## "A red box that contains a constant double-precision floating point value." - def same_constant(self, other): - return (isinstance(other, DoubleRedBox) and - self.ll_getvalue(lltype.Float) == other.ll_getvalue(lltype.Float)) +## def same_constant(self, other): +## return (isinstance(other, DoubleRedBox) and +## self.ll_getvalue(lltype.Float) == other.ll_getvalue(lltype.Float)) -class AddrRedBox(ConstRedBox): - "A red box that contains a constant address." +##class AddrRedBox(ConstRedBox): +## "A red box that contains a constant address." - def same_constant(self, other): - return (isinstance(other, AddrRedBox) and - self.ll_getvalue(llmemory.Address) == other.ll_getvalue(llmemory.Address)) +## def same_constant(self, other): +## return (isinstance(other, AddrRedBox) and +## self.ll_getvalue(llmemory.Address) == other.ll_getvalue(llmemory.Address)) # ____________________________________________________________ Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Sat Apr 1 19:37:39 2006 @@ -3,8 +3,7 @@ from pypy.objspace.flow import model as flowmodel from pypy.annotation import model as annmodel from pypy.annotation import listdef, dictdef -from pypy.jit.timeshifter.rtimeshift import VARLIST, RedBox, VarRedBox -from pypy.jit.timeshifter.rtimeshift import ConstRedBox, JITState +from pypy.jit.timeshifter.rtimeshift import VARLIST, JITState from pypy.jit.timeshifter.rtimeshift import make_types_const from pypy.rpython import rmodel, rlist, rdict, rgenop, annlowlevel from pypy.rpython.lltypesystem import rtuple From arigo at codespeak.net Sat Apr 1 19:39:19 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 19:39:19 +0200 (CEST) Subject: [pypy-svn] r25214 - pypy/dist/pypy/rpython Message-ID: <20060401173919.53C0F1008D@code0.codespeak.net> Author: arigo Date: Sat Apr 1 19:39:18 2006 New Revision: 25214 Modified: pypy/dist/pypy/rpython/rgenop.py Log: (arre, pedronis, arigo) rgenop.isconst() helper. Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Sat Apr 1 19:39:18 2006 @@ -84,7 +84,11 @@ return llmemory.cast_ptr_to_adr(c.value) else: return lltype.cast_primitive(T, c.value) - + +def isconst(gv_value): + c = from_opaque_object(gv_value) + return isinstance(c, flowmodel.Constant) + # XXX # temporary interface; it's unclera if genop itself should change to ease dinstinguishing # Void special args from the rest. Or there should be variation for the ops involving them @@ -254,6 +258,7 @@ setannotation(genop, s_ConstOrVar) setannotation(genconst, s_ConstOrVar) revealconst.compute_result_annotation = lambda s_T, s_gv: annmodel.lltype_to_annotation(s_T.const) +setannotation(isconst, annmodel.SomeBool()) setannotation(closeblock1, s_Link) setannotation(closeblock2, s_LinkPair) setannotation(closelink, None) @@ -265,6 +270,7 @@ setspecialize(genop) setspecialize(genconst) setspecialize(revealconst) +setspecialize(isconst) setspecialize(closeblock1) setspecialize(closeblock2) setspecialize(closelink) From arigo at codespeak.net Sat Apr 1 23:18:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 1 Apr 2006 23:18:36 +0200 (CEST) Subject: [pypy-svn] r25215 - in pypy/branch/jit-timeshifter-refactoring/timeshifter: . test Message-ID: <20060401211836.C7EC31009B@code0.codespeak.net> Author: arigo Date: Sat Apr 1 23:18:35 2006 New Revision: 25215 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: (arre, pedronis, arigo) Intermediate check-in. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sat Apr 1 23:18:35 2006 @@ -1,6 +1,7 @@ import operator, weakref from pypy.rpython.lltypesystem import lltype, lloperation, llmemory from pypy.rpython import rgenop +from pypy.jit.timeshifter import rvalue FOLDABLE_OPS = dict.fromkeys(lloperation.enum_foldable_ops()) @@ -19,12 +20,12 @@ "ll_length": ll_fixed_length })) -def make_types_const(TYPES): - n = len(TYPES) - l = lltype.malloc(VARLIST.TO, n) - for i in range(n): - l[i] = rgenop.constTYPE(TYPES[i]) - return l +##def make_types_const(TYPES): +## n = len(TYPES) +## l = lltype.malloc(VARLIST.TO, n) +## for i in range(n): +## l[i] = rgenop.constTYPE(TYPES[i]) +## return l ##class RedBox(object): @@ -454,6 +455,8 @@ self.nb_args = len(ARGS) self.ARGS = ARGS self.RESULT = RESULT + self.gv_RESULT = rgenop.constTYPE(RESULT) + self.redboxcls = rvalue.ll_redboxcls(RESULT) self.canfold = opname in FOLDABLE_OPS def __getattr__(self, name): # .ARGx -> .ARGS[x] @@ -484,34 +487,33 @@ ARG0 = opdesc.ARG0 RESULT = opdesc.RESULT opname = opdesc.name - if opdesc.canfold and isinstance(argbox, ConstRedBox): - arg = argbox.ll_getvalue(ARG0) + if opdesc.canfold and argbox.is_constant(): + arg = rvalue.ll_getvalue(argbox, ARG0) res = opdesc.llop(RESULT, arg) - return ConstRedBox.ll_fromvalue(res) + return rvalue.ll_fromvalue(res) op_args = lltype.malloc(VARLIST.TO, 1) op_args[0] = argbox.getgenvar(jitstate) genvar = rgenop.genop(jitstate.curblock, opdesc.opname, op_args, - rgenop.constTYPE(RESULT)) - return VarRedBox(genvar) + opdesc.gv_RESULT) + return opdesc.redboxcls(opdesc.gv_RESULT, genvar) def ll_generate_operation2(opdesc, jitstate, argbox0, argbox1): ARG0 = opdesc.ARG0 ARG1 = opdesc.ARG1 RESULT = opdesc.RESULT opname = opdesc.name - if opdesc.canfold and (isinstance(argbox0, ConstRedBox) and - isinstance(argbox1, ConstRedBox)): + if opdesc.canfold and argbox0.is_constant() and argbox1.is_constant(): # const propagate - arg0 = argbox0.ll_getvalue(ARG0) - arg1 = argbox1.ll_getvalue(ARG1) + arg0 = rvalue.ll_getvalue(argbox0, ARG0) + arg1 = rvalue.ll_getvalue(argbox1, ARG1) res = opdesc.llop(RESULT, arg0, arg1) - return ConstRedBox.ll_fromvalue(res) + return rvalue.ll_fromvalue(res) op_args = lltype.malloc(VARLIST.TO, 2) op_args[0] = argbox0.getgenvar(jitstate) op_args[1] = argbox1.getgenvar(jitstate) genvar = rgenop.genop(jitstate.curblock, opdesc.opname, op_args, - rgenop.constTYPE(RESULT)) - return VarRedBox(genvar) + opdesc.gv_RESULT) + return opdesc.redboxcls(opdesc.gv_RESULT, genvar) class StructTypeDesc(object): _type_cache = weakref.WeakKeyDictionary() @@ -531,7 +533,7 @@ defaultbox = None else: defaultvalue = desc.RESTYPE._defl() - defaultbox = ConstRedBox.ll_fromvalue(defaultvalue) + defaultbox = rvalue.ll_fromvalue(defaultvalue) defls.append(defaultbox) self.default_boxes = defls @@ -613,6 +615,7 @@ RESTYPE = lltype.Ptr(RESTYPE) self.RESTYPE = RESTYPE self.gv_resulttype = rgenop.constTYPE(RESTYPE) + self.redboxcls = rvalue.ll_redboxcls(RESTYPE) self.immutable = PTRTYPE.TO._hints.get('immutable', False) def _freeze_(self): @@ -660,9 +663,10 @@ pass def ll_generate_getfield(jitstate, fielddesc, argbox): - if fielddesc.immutable and isinstance(argbox, ConstRedBox): - res = getattr(argbox.ll_getvalue(fielddesc.PTRTYPE), fielddesc.fieldname) - return ConstRedBox.ll_fromvalue(res) + if fielddesc.immutable and argbox.is_constant(): + res = getattr(rvalue.ll_getvalue(argbox, fielddesc.PTRTYPE), + fielddesc.fieldname) + return rvalue.ll_fromvalue(res) return argbox.op_getfield(jitstate, fielddesc) gv_Void = rgenop.constTYPE(lltype.Void) @@ -672,23 +676,24 @@ def ll_generate_getsubstruct(jitstate, fielddesc, argbox): - if isinstance(argbox, ConstRedBox): - res = getattr(argbox.ll_getvalue(fielddesc.PTRTYPE), fielddesc.fieldname) - return ConstRedBox.ll_fromvalue(res) + if argbox.is_constant(): + res = getattr(rvalue.ll_getvalue(argbox, fielddesc.PTRTYPE), + fielddesc.fieldname) + return rvalue.ll_fromvalue(res) return argbox.op_getsubstruct(jitstate, fielddesc) def ll_generate_getarrayitem(jitstate, fielddesc, argbox, indexbox): - if (fielddesc.immutable and - isinstance(argbox, ConstRedBox) and isinstance(indexbox, ConstRedBox)): - res = argbox.ll_getvalue(fielddesc.PTRTYPE)[indexbox.ll_getvalue(lltype.Signed)] - return ConstRedBox.ll_fromvalue(res) + if fielddesc.immutable and argbox.is_constant() and indexbox.is_constant(): + array = rvalue.ll_getvalue(argbox, fielddesc.PTRTYPE) + res = array[rvalue.ll_getvalue(indexbox, lltype.Signed)] + return rvalue.ll_fromvalue(res) op_args = lltype.malloc(VARLIST.TO, 2) op_args[0] = argbox.getgenvar(jitstate) op_args[1] = indexbox.getgenvar(jitstate) genvar = rgenop.genop(jitstate.curblock, 'getarrayitem', op_args, fielddesc.gv_resulttype) - return VarRedBox(genvar) + return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) ##def ll_generate_malloc(jitstate, gv_type, gv_resulttype): ## op_args = lltype.malloc(VARLIST.TO, 1) @@ -701,33 +706,46 @@ # other jitstate/graph level operations -def retrieve_jitstate_for_merge(states_dic, jitstate, key, redboxes, types_gv): +def retrieve_jitstate_for_merge(states_dic, jitstate, key, redboxes): if key not in states_dic: - jitstate = enter_block(jitstate, redboxes, types_gv) - states_dic[key] = redboxes, jitstate.curblock + memo = {} + frozens = [redbox.freeze(memo) for redbox in redboxes] + memo = {} + outgoingvarboxes = [] + for i in range(len(redboxes)): + res = frozens[i].exactmatch(redboxes[i], outgoingvarboxes, memo) + assert res, "exactmatch() failed" + newblock = rgenop.newblock() + linkargs = [] + for box in outgoingvarboxes: + linkargs.append(box.getgenvar(None)) + box.genvar = rgenop.geninputarg(newblock, box.gv_type) + rgenop.closelink(jitstate.curoutgoinglink, linkargs, newblock) + jitstate.curblock = newblock + jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) + states_dic[key] = frozens, newblock return jitstate - oldboxes, oldblock = states_dic[key] + frozens, oldblock = states_dic[key] memo = {} - incoming = [] + outgoingvarboxes = [] + exactmatch = True for i in range(len(redboxes)): - oldbox = oldboxes[i] - newbox = redboxes[i] - if not oldbox.match(jitstate, newbox, incoming, memo): - break # mismatch -## if isinstance(oldbox, VarRedBox): # Always a match -## incoming.append(newbox.getgenvar(jitstate)) -## continue -## if oldbox.same_constant(newbox): -## continue -## # Mismatch. Generalize to a var -## break - else: - rgenop.closelink(jitstate.curoutgoinglink, incoming, oldblock) + frozen = frozens[i] + if not frozen.exactmatch(redboxes[i], outgoingvarboxes, memo): + exactmatch = False + + if exactmatch: + jitstate = enter_block(jitstate, outgoingvarboxes) + linkargs = [] + for box in outgoingvarboxes: + linkargs.append(box.getgenvar(jitstate)) + link = rgenop.closeblock1(jitstate.curblock) + rgenop.closelink(link, linkargs, oldblock) return None # Make a more general block - jitstate = enter_block(jitstate, redboxes, types_gv) + jitstate = enter_block(jitstate, redboxes) newblock = rgenop.newblock() incoming = [] memo = {} @@ -748,17 +766,14 @@ return jitstate retrieve_jitstate_for_merge._annspecialcase_ = "specialize:arglltype(2)" -def enter_block(jitstate, redboxes, types_gv): +def enter_block(jitstate, redboxes): newblock = rgenop.newblock() - jitstate.curblock = newblock incoming = [] - memo1 = {} - memo2 = {} + memo = {} for i in range(len(redboxes)): - redbox = redboxes[i] - redbox.getallvariables(jitstate, incoming, memo1) - redboxes[i] = redbox.copybox(newblock, types_gv[i], memo2) + redboxes[i].enter_block(newblock, incoming, memo) rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) + jitstate.curblock = newblock jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) return jitstate @@ -814,13 +829,11 @@ finalvar = rgenop.geninputarg(finalblock, gv_return_type) for link, redbox in return_queue: newblock = rgenop.newblock() - jitstate.curblock = newblock incoming = [] - memo1 = {} - memo2 = {} - redbox.getallvariables(jitstate, incoming, memo1) - newbox = redbox.copybox(newblock, gv_return_type, memo2) - gv_retval = newbox.getgenvar(jitstate) + memo = {} + redbox.enter_block(newblock, incoming, memo) + jitstate.curblock = newblock + gv_retval = redbox.getgenvar(jitstate) rgenop.closelink(link, incoming, newblock) newlink = rgenop.closeblock1(newblock) rgenop.closelink(newlink, [gv_retval], finalblock) @@ -866,19 +879,18 @@ jitstate.setup() return jitstate -def ll_int_box(gv): - return IntRedBox(gv) +def ll_int_box(gv_type, gv): + return rvalue.IntRedBox(gv_type, gv) -def ll_double_box(gv): - return DoubleRedBox(gv) +def ll_double_box(gv_type, gv): + return rvalue.DoubleRedBox(gv_type, gv) -def ll_addr_box(gv): - return AddrRedBox(gv) +def ll_addr_box(gv_type, gv): + return rvalue.PtrRedBox(gv_type, gv) + +def ll_geninputarg(jitstate, gv_TYPE): + return rgenop.geninputarg(jitstate.curblock, gv_TYPE) -def ll_var_box(jitstate, gv_TYPE): - genvar = rgenop.geninputarg(jitstate.curblock, gv_TYPE) - return VarRedBox(genvar) - def ll_end_setup_jitstate(jitstate): jitstate.end_setup() return jitstate.curblock Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py Sat Apr 1 23:18:35 2006 @@ -10,7 +10,7 @@ from pypy.rpython import rgenop from pypy.jit.hintannotator import model as hintmodel from pypy.jit.hintannotator import container as hintcontainer -from pypy.jit.timeshifter import rtimeshift +from pypy.jit.timeshifter import rtimeshift, rvalue class HintTypeSystem(TypeSystem): name = "hinttypesystem" @@ -322,7 +322,7 @@ annmodel.SomePtr(rgenop.CONSTORVAR)) def convert_const(self, ll_value): - redbox = rtimeshift.ConstRedBox.ll_fromvalue(ll_value) + redbox = rvalue.ll_fromvalue(ll_value) timeshifter = self.timeshifter return timeshifter.annhelper.delayedconst(timeshifter.r_RedBox, redbox) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Sat Apr 1 23:18:35 2006 @@ -2,9 +2,11 @@ from pypy.rpython.lltypesystem import lltype -class AbstractRedBox(object): +class RedBox(object): - def __init__(self, genvar=rgenop.nullvar): + def __init__(self, gv_type, genvar=rgenop.nullvar): + assert rgenop.isconst(gv_type) # temporary? + self.gv_type = gv_type self.genvar = genvar # nullvar or a genvar def __repr__(self): @@ -19,31 +21,38 @@ def getgenvar(self, jitstate): return self.genvar - def ll_fromvalue(value): - "Make a constant RedBox from a low-level value." - T = lltype.typeOf(value) - gv = rgenop.genconst(value) - if isinstance(T, lltype.Ptr): - return PtrRedBox(gv) - elif T is lltype.Float: - return DoubleRedBox(gv) - else: - assert isinstance(T, lltype.Primitive) - assert T is not lltype.Void, "cannot make red boxes of voids" - # XXX what about long longs? - return IntRedBox(gv) - ll_fromvalue = staticmethod(ll_fromvalue) - - def ll_getvalue(self, T): - "Return the content of a known-to-be-constant RedBox." - # note: this is specialized by low-level type T, as a low-level helper - return rgenop.revealconst(T, self.genvar) + def enter_block(self, newblock, incoming, memo): + if not self.is_constant() and self not in memo: + incoming.append(self.genvar) + memo[self] = None + self.genvar = rgenop.geninputarg(newblock, self.gv_type) + + +def ll_redboxcls(TYPE): + if isinstance(TYPE, lltype.Ptr): + return PtrRedBox + elif TYPE is lltype.Float: + return DoubleRedBox + else: + assert isinstance(TYPE, lltype.Primitive) + assert TYPE is not lltype.Void, "cannot make red boxes of voids" + # XXX what about long longs? + return IntRedBox + +def ll_fromvalue(value): + "Make a constant RedBox from a low-level value." + T = lltype.typeOf(value) + gv_type = rgenop.constTYPE(T) + gv = rgenop.genconst(value) + cls = ll_redboxcls(T) + return cls(gv_type, gv) def ll_getvalue(box, T): - return box.ll_getvalue(T) + "Return the content of a known-to-be-constant RedBox." + return rgenop.revealconst(T, box.genvar) -class IntRedBox(AbstractRedBox): +class IntRedBox(RedBox): "A red box that contains a constant integer-like value." def freeze(self, memo): @@ -58,7 +67,7 @@ return result -class DoubleRedBox(AbstractRedBox): +class DoubleRedBox(RedBox): "A red box that contains a constant double-precision floating point value." def freeze(self, memo): @@ -73,39 +82,46 @@ return result -class PtrRedBox(AbstractRedBox): +class PtrRedBox(RedBox): - def __init__(self, genvar=rgenop.nullvar, content=None): - AbstractRedBox.__init__(self, genvar) + def __init__(self, gv_type, genvar=rgenop.nullvar, content=None): + RedBox.__init__(self, gv_type, genvar) self.content = content # None or an AbstractContainer def __repr__(self): if not self.genvar and self.content is not None: return '' % (self.content,) else: - return AbstractRedBox.__repr__(self) + return RedBox.__repr__(self) def freeze(self, memo): try: return memo[self] except KeyError: - if self.genvar: + if self.content: + result = FrozenPtrVirtual() + memo[self] = result + result.fz_content = self.content.freeze(memo) + else: if self.is_constant(): result = FrozenPtrConst(self.genvar) else: result = FrozenPtrVar() memo[self] = result - else: - result = FrozenPtrVirtual() - memo[self] = result - result.fz_content = self.content.freeze(memo) return result def getgenvar(self, jitstate): if not self.genvar: + assert self.content self.genvar = self.content.build_runtime_container(jitstate) return self.genvar + def enter_block(self, newblock, incoming, memo): + if self.content: + self.content.enter_block(newblock, incoming, memo) + else: + RedBox.enter_block(self, newblock, incoming, memo) + # ____________________________________________________________ class FrozenValue(object): @@ -122,16 +138,16 @@ try: return memo[self] except KeyError: - box = memo[self] = IntRedBox(self.gv_const) + box = memo[self] = IntRedBox(gv_type, self.gv_const) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): if (box.is_constant() and rgenop.revealconst(lltype.Signed, self.gv_const) == rgenop.revealconst(lltype.Signed, box.genvar)): return True else: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False @@ -142,18 +158,18 @@ return memo[self] except KeyError: gv_value = rgenop.geninputarg(block, gv_type) - box = memo[self] = IntRedBox(gv_value) + box = memo[self] = IntRedBox(gv_type, gv_value) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): if self not in memo: memo[self] = box - outgoingvars.append(box) + outgoingvarboxes.append(box) return True elif memo[self] is box: return True else: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False @@ -166,16 +182,16 @@ try: return memo[self] except KeyError: - box = memo[self] = DoubleRedBox(self.gv_const) + box = memo[self] = DoubleRedBox(gv_type, self.gv_const) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): if (box.is_constant() and rgenop.revealconst(lltype.Double, self.gv_const) == rgenop.revealconst(lltype.Double, box.genvar)): return True else: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False @@ -186,18 +202,18 @@ return memo[self] except KeyError: gv_value = rgenop.geninputarg(block, gv_type) - box = memo[self] = DoubleRedBox(gv_value) + box = memo[self] = DoubleRedBox(gv_type, gv_value) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): if self not in memo: memo[self] = box - outgoingvars.append(box) + outgoingvarboxes.append(box) return True elif memo[self] is box: return True else: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False @@ -210,16 +226,16 @@ try: return memo[self] except KeyError: - box = memo[self] = PtrRedBox(self.gv_const) + box = memo[self] = PtrRedBox(gv_type, self.gv_const) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): if (box.is_constant() and rgenop.revealconst(lltype.Address, self.gv_const) == rgenop.revealconst(lltype.Address, box.genvar)): return True else: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False @@ -230,18 +246,18 @@ return memo[self] except KeyError: gv_value = rgenop.geninputarg(block, gv_type) - box = memo[self] = PtrRedBox(gv_value) + box = memo[self] = PtrRedBox(gv_type, gv_value) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): if self not in memo: memo[self] = box - outgoingvars.append(box) + outgoingvarboxes.append(box) return True elif memo[self] is box: return True else: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False @@ -251,14 +267,15 @@ try: return memo[self] except KeyError: - box = memo[self] = PtrRedBox() + box = memo[self] = PtrRedBox(gv_type) box.content = self.fz_content.unfreeze(memo, block) return box - def exactmatch(self, box, outgoingvars, memo): + def exactmatch(self, box, outgoingvarboxes, memo): assert isinstance(box, PtrRedBox) if box.content is None: - outgoingvars.append(box) + outgoingvarboxes.append(box) return False else: - return self.fz_content.exactmatch(box.content, outgoingvars, memo) + return self.fz_content.exactmatch(box.content, outgoingvarboxes, + memo) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py Sat Apr 1 23:18:35 2006 @@ -93,16 +93,18 @@ # red assert residual_v == [llvalue], "XXX for now" TYPE = htshift.originalconcretetype(v) - box = llinterp.eval_graph(htshift.ll_var_box_graph, [jitstate, - rgenop.constTYPE(TYPE)]) + gv_type = rgenop.constTYPE(TYPE) + gvar = llinterp.eval_graph(htshift.ll_geninputarg_graph, [jitstate, + gv_type]) if i in opt_consts: # XXX what should happen here interface wise is unclear - if isinstance(lltype.typeOf(llvalue), lltype.Ptr): - ll_box_graph = htshift.ll_addr_box_graph - elif isinstance(llvalue, float): - ll_box_graph = htshift.ll_double_box_graph - else: - ll_box_graph = htshift.ll_int_box_graph - box = llinterp.eval_graph(ll_box_graph, [rgenop.genconst(llvalue)]) + gvar = rgenop.genconst(llvalue) + if isinstance(lltype.typeOf(llvalue), lltype.Ptr): + ll_box_graph = htshift.ll_addr_box_graph + elif isinstance(llvalue, float): + ll_box_graph = htshift.ll_double_box_graph + else: + ll_box_graph = htshift.ll_int_box_graph + box = llinterp.eval_graph(ll_box_graph, [gv_type, gvar]) graph1args.append(box) residual_graph_args.append(llvalue) startblock = llinterp.eval_graph(htshift.ll_end_setup_jitstate_graph, [jitstate]) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Sat Apr 1 23:18:35 2006 @@ -3,8 +3,8 @@ from pypy.objspace.flow import model as flowmodel from pypy.annotation import model as annmodel from pypy.annotation import listdef, dictdef +from pypy.jit.timeshifter import rvalue from pypy.jit.timeshifter.rtimeshift import VARLIST, JITState -from pypy.jit.timeshifter.rtimeshift import make_types_const from pypy.rpython import rmodel, rlist, rdict, rgenop, annlowlevel from pypy.rpython.lltypesystem import rtuple from pypy.jit.timeshifter import rtimeshift @@ -25,7 +25,7 @@ self.annhelper = annlowlevel.MixLevelHelperAnnotator(rtyper) self.s_JITState, self.r_JITState = self.s_r_instanceof(JITState) - self.s_RedBox, self.r_RedBox = self.s_r_instanceof(RedBox) + self.s_RedBox, self.r_RedBox = self.s_r_instanceof(rvalue.RedBox) getrepr = self.rtyper.getrepr @@ -47,20 +47,20 @@ [], self.s_JITState) self.ll_int_box_graph = self.annhelper.getgraph( rtimeshift.ll_int_box, - [rgenop.s_ConstOrVar], + [rgenop.s_ConstOrVar, rgenop.s_ConstOrVar], self.s_RedBox) self.ll_addr_box_graph = self.annhelper.getgraph( rtimeshift.ll_addr_box, - [rgenop.s_ConstOrVar], + [rgenop.s_ConstOrVar, rgenop.s_ConstOrVar], self.s_RedBox) self.ll_double_box_graph = self.annhelper.getgraph( rtimeshift.ll_int_box, - [rgenop.s_ConstOrVar], + [rgenop.s_ConstOrVar, rgenop.s_ConstOrVar], self.s_RedBox) - self.ll_var_box_graph = self.annhelper.getgraph( - rtimeshift.ll_var_box, + self.ll_geninputarg_graph = self.annhelper.getgraph( + rtimeshift.ll_geninputarg, [self.s_JITState, annmodel.SomePtr(rgenop.CONSTORVAR)], - self.s_RedBox) + rgenop.s_ConstOrVar) self.ll_end_setup_jitstate_graph = self.annhelper.getgraph( rtimeshift.ll_end_setup_jitstate, [self.s_JITState], @@ -216,7 +216,7 @@ def make_const_box(self, llops, r_green, v_value): v_box = llops.genmixlevelhelpercall( - rtimeshift.ConstRedBox.ll_fromvalue, + rvalue.ll_fromvalue, [r_green.annotation()], [v_value], self.s_RedBox) return v_box @@ -265,7 +265,6 @@ getrepr = self.rtyper.getrepr v_boxes = self.build_box_list(llops, boxes_v) - TYPES_gv = make_types_const(TYPES) if nentrylinks > 1: enter_block_logic = self.bookkeeping_enter_for_join @@ -277,8 +276,7 @@ enter_block_logic(args_r, newinputargs, before_block, llops, - v_boxes, - TYPES_gv) + v_boxes) def build_box_list(self, llops, boxes_v): type_erased_v = [llops.convertvar(v_box, self.r_RedBox, @@ -287,13 +285,11 @@ v_boxes = rlist.newlist(llops, self.r_box_list, type_erased_v) return v_boxes - def bookkeeping_enter_simple(self, args_r, newinputargs, before_block, llops, v_boxes, - TYPES_gv): - c_TYPES = rmodel.inputconst(VARLIST, TYPES_gv) + def bookkeeping_enter_simple(self, args_r, newinputargs, before_block, + llops, v_boxes): v_newjitstate = llops.genmixlevelhelpercall(rtimeshift.enter_block, - [self.s_JITState, self.s_box_list, - annmodel.SomePtr(VARLIST)], - [newinputargs[0], v_boxes, c_TYPES], + [self.s_JITState, self.s_box_list], + [newinputargs[0], v_boxes], self.s_JITState) bridge = before_block.exits[0] @@ -310,8 +306,8 @@ # ll_retrieve_jitstate_for_merge is supposed to use the "constant" dict as cache # mapping green values combinations to frozen states for red boxes values # and generated blocks - def bookkeeping_enter_for_join(self, args_r, newinputargs, before_block, llops, v_boxes, - TYPES_gv): + def bookkeeping_enter_for_join(self, args_r, newinputargs, before_block, + llops, v_boxes): getrepr = self.rtyper.getrepr items_s = [] key_v = [] @@ -338,7 +334,7 @@ def merge_point(jitstate, key, boxes): return rtimeshift.retrieve_jitstate_for_merge(cache, jitstate, - key, boxes, TYPES_gv) + key, boxes) v_newjitstate = llops.genmixlevelhelpercall(merge_point, [self.s_JITState, s_key_tuple, self.s_box_list], From pedronis at codespeak.net Sun Apr 2 11:14:02 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 11:14:02 +0200 (CEST) Subject: [pypy-svn] r25217 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402091402.508A6100AD@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 11:14:01 2006 New Revision: 25217 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: (arre, arigo, pedronis) for first at fixing merge point and split point logic. Introduced a copy method on redboxes. Some other fixes. More tests pass again. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sun Apr 2 11:14:01 2006 @@ -736,7 +736,7 @@ exactmatch = False if exactmatch: - jitstate = enter_block(jitstate, outgoingvarboxes) + jitstate = dyn_enter_block(jitstate, outgoingvarboxes) linkargs = [] for box in outgoingvarboxes: linkargs.append(box.getgenvar(jitstate)) @@ -745,25 +745,39 @@ return None # Make a more general block - jitstate = enter_block(jitstate, redboxes) + jitstate = dyn_enter_block(jitstate, outgoingvarboxes) newblock = rgenop.newblock() - incoming = [] - memo = {} - for i in range(len(redboxes)): - oldbox = oldboxes[i] - newbox = redboxes[i] - redboxes[i] = oldbox.union_for_new_block(jitstate, newbox, newblock, - types_gv[i], incoming, memo) -## if not oldbox.same_constant(newbox): -## incoming.append(newbox.getgenvar(jitstate)) -## newgenvar = rgenop.geninputarg(newblock, TYPES[i]) -## redboxes[i] = VarRedBox(newgenvar) + linkargs = [] + for box in outgoingvarboxes: + linkargs.append(box.getgenvar(jitstate)) + box.genvar = rgenop.geninputarg(newblock, box.gv_type) link = rgenop.closeblock1(jitstate.curblock) - rgenop.closelink(link, incoming, newblock) + rgenop.closelink(link, linkargs, newblock) jitstate.curblock = newblock #jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) - states_dic[key] = redboxes, newblock + memo = {} + frozens = [redbox.freeze(memo) for redbox in redboxes] + states_dic[key] = frozens, newblock return jitstate + +## newblock = rgenop.newblock() +## incoming = [] +## memo = {} +## for i in range(len(redboxes)): +## oldbox = oldboxes[i] +## newbox = redboxes[i] +## redboxes[i] = oldbox.union_for_new_block(jitstate, newbox, newblock, +## types_gv[i], incoming, memo) +## ## if not oldbox.same_constant(newbox): +## ## incoming.append(newbox.getgenvar(jitstate)) +## ## newgenvar = rgenop.geninputarg(newblock, TYPES[i]) +## ## redboxes[i] = VarRedBox(newgenvar) +## link = rgenop.closeblock1(jitstate.curblock) +## rgenop.closelink(link, incoming, newblock) +## jitstate.curblock = newblock +## #jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) +## states_dic[key] = redboxes, newblock +## return jitstate retrieve_jitstate_for_merge._annspecialcase_ = "specialize:arglltype(2)" def enter_block(jitstate, redboxes): @@ -777,14 +791,25 @@ jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) return jitstate +def dyn_enter_block(jitstate, redboxes): + newblock = rgenop.newblock() + incoming = [] + memo = {} + for i in range(len(redboxes)): + redboxes[i].enter_block(newblock, incoming, memo) + rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) + jitstate.curblock = newblock + jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) + return jitstate + def leave_block(jitstate): jitstate.curoutgoinglink = rgenop.closeblock1(jitstate.curblock) return jitstate def leave_block_split(jitstate, switchredbox, exitindex, redboxes): - if isinstance(switchredbox, IntRedBox): + if switchredbox.is_constant(): jitstate.curoutgoinglink = rgenop.closeblock1(jitstate.curblock) - return switchredbox.ll_getvalue(lltype.Bool) + return rvalue.ll_getvalue(switchredbox, lltype.Bool) else: exitgvar = switchredbox.getgenvar(jitstate) linkpair = rgenop.closeblock2(jitstate.curblock, exitgvar) @@ -792,7 +817,9 @@ later_jitstate = jitstate.copystate() jitstate.curoutgoinglink = true_link later_jitstate.curoutgoinglink = false_link - jitstate.split_queue.append((exitindex, later_jitstate, redboxes)) + memo = {} + redboxcopies = [redbox.copy(memo) for redbox in redboxes] + jitstate.split_queue.append((exitindex, later_jitstate, redboxcopies)) return True def schedule_return(jitstate, redbox): Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Sun Apr 2 11:14:01 2006 @@ -1,5 +1,5 @@ from pypy.rpython import rgenop -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory class RedBox(object): @@ -55,6 +55,13 @@ class IntRedBox(RedBox): "A red box that contains a constant integer-like value." + def copy(self, memo): + try: + return memo[self] + except KeyError: + result = memo[self] = IntRedBox(self.gv_type, self.genvar) + return result + def freeze(self, memo): try: return memo[self] @@ -70,6 +77,13 @@ class DoubleRedBox(RedBox): "A red box that contains a constant double-precision floating point value." + def copy(self, memo): + try: + return memo[self] + except KeyError: + result = memo[self] = DoubleRedBox(self.gv_type, self.genvar) + return result + def freeze(self, memo): try: return memo[self] @@ -94,6 +108,16 @@ else: return RedBox.__repr__(self) + def copy(self, memo): + try: + return memo[self] + except KeyError: + result = PtrRedBox(self.gv_type, self.genvar) + memo[self] = result + if self.content: + result.content = self.content.copy(memo) + return result + def freeze(self, memo): try: return memo[self] @@ -231,8 +255,8 @@ def exactmatch(self, box, outgoingvarboxes, memo): if (box.is_constant() and - rgenop.revealconst(lltype.Address, self.gv_const) == - rgenop.revealconst(lltype.Address, box.genvar)): + rgenop.revealconst(llmemory.Address, self.gv_const) == + rgenop.revealconst(llmemory.Address, box.genvar)): return True else: outgoingvarboxes.append(box) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Sun Apr 2 11:14:01 2006 @@ -29,6 +29,7 @@ getrepr = self.rtyper.getrepr + bk = rtyper.annotator.bookkeeper box_list_def = listdef.ListDef(None, self.s_RedBox) box_list_def.mutate() self.s_box_list = annmodel.SomeList(box_list_def) From pedronis at codespeak.net Sun Apr 2 11:31:38 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 11:31:38 +0200 (CEST) Subject: [pypy-svn] r25218 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402093138.2F469100A3@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 11:31:36 2006 New Revision: 25218 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py Log: (arre, pedronis) make test_convert_greenvar_redvar Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py Sun Apr 2 11:31:36 2006 @@ -519,7 +519,7 @@ def convert_from_to((r_from, r_to), v, llops): assert r_from.lowleveltype == r_to.original_concretetype - return llops.genmixlevelhelpercall(rtimeshift.ConstRedBox.ll_fromvalue, + return llops.genmixlevelhelpercall(rvalue.ll_fromvalue, [r_from.annotation()], [v], llops.timeshifter.s_RedBox) From pedronis at codespeak.net Sun Apr 2 11:38:22 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 11:38:22 +0200 (CEST) Subject: [pypy-svn] r25219 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402093822.9C31F100A7@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 11:38:21 2006 New Revision: 25219 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: another small fix making a test pass again. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Sun Apr 2 11:38:21 2006 @@ -104,7 +104,7 @@ c_TYPE = rmodel.inputconst(lltype.Void, r.lowleveltype) s_TYPE = self.rtyper.annotator.bookkeeper.immutablevalue(r.lowleveltype) - v_value = llops.genmixlevelhelpercall(rtimeshift.ll_getvalue, + v_value = llops.genmixlevelhelpercall(rvalue.ll_getvalue, [self.s_RedBox, s_TYPE], [v_box, c_TYPE], r.annotation()) From pedronis at codespeak.net Sun Apr 2 12:25:30 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 12:25:30 +0200 (CEST) Subject: [pypy-svn] r25220 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402102530.A4337100B5@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 12:25:29 2006 New Revision: 25220 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Log: (arigo, arre, pedronis) fix ll_generate_getfield. another test pass again. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sun Apr 2 12:25:29 2006 @@ -667,7 +667,16 @@ res = getattr(rvalue.ll_getvalue(argbox, fielddesc.PTRTYPE), fielddesc.fieldname) return rvalue.ll_fromvalue(res) - return argbox.op_getfield(jitstate, fielddesc) + assert isinstance(argbox, rvalue.PtrRedBox) + if argbox.content is None: + op_args = lltype.malloc(VARLIST.TO, 2) + op_args[0] = argbox.getgenvar(jitstate) + op_args[1] = fielddesc.gv_fieldname + genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, + fielddesc.gv_resulttype) + return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) + else: + return argbox.content.op_getfield(jitstate, fielddesc) gv_Void = rgenop.constTYPE(lltype.Void) From ericvrp at codespeak.net Sun Apr 2 13:08:33 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sun, 2 Apr 2006 13:08:33 +0200 (CEST) Subject: [pypy-svn] r25221 - in pypy/dist/pypy/translator/llvm: llvmcapi llvmcapi/include llvmcapi/include/ExecutionEngine llvmcapi/include/VMCore llvmcapi/lib llvmcapi/lib/ExecutionEngine llvmcapi/lib/VMCore pythonllvm pythonllvm/doc pythonllvm/test Message-ID: <20060402110833.98C83100BF@code0.codespeak.net> Author: ericvrp Date: Sun Apr 2 13:08:28 2006 New Revision: 25221 Added: pypy/dist/pypy/translator/llvm/llvmcapi/ pypy/dist/pypy/translator/llvm/llvmcapi/README.txt (contents, props changed) pypy/dist/pypy/translator/llvm/llvmcapi/include/ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/ModuleProvider.h pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h pypy/dist/pypy/translator/llvm/llvmcapi/lib/ pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp (contents, props changed) pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp (contents, props changed) pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp (contents, props changed) pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/ModuleProvider.cpp (contents, props changed) pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp pypy/dist/pypy/translator/llvm/llvmcapi/setup.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/ pypy/dist/pypy/translator/llvm/pythonllvm/__init__.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/cc.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/doc/ pypy/dist/pypy/translator/llvm/pythonllvm/doc/pyllvm.cpp.org pypy/dist/pypy/translator/llvm/pythonllvm/helper.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/test/ pypy/dist/pypy/translator/llvm/pythonllvm/test/.gdb_history pypy/dist/pypy/translator/llvm/pythonllvm/test/__init__.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/test/addnumbers.c pypy/dist/pypy/translator/llvm/pythonllvm/test/addnumbers.s pypy/dist/pypy/translator/llvm/pythonllvm/test/hello.c pypy/dist/pypy/translator/llvm/pythonllvm/test/hello.s pypy/dist/pypy/translator/llvm/pythonllvm/test/ll_snippet.py (contents, props changed) pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py (contents, props changed) Log: WIP llvm c api library and ctypes wrapper for that. Checkin because I'm moving machines Added: pypy/dist/pypy/translator/llvm/llvmcapi/README.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/README.txt Sun Apr 2 13:08:28 2006 @@ -0,0 +1,4 @@ +Create llvm c api library by running "python setup.py build_ext -i" here + +note: This library building should maybe not be done with distutils at all! + It should probably be done with the regular: ./configure;make;make install Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h Sun Apr 2 13:08:28 2006 @@ -0,0 +1,21 @@ +#ifndef __EXECUIONENGINE_H__ +#define __EXECUIONENGINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/* +void* ExecutionEngine__init__M(void* M); +void* ExecutionEngine__init__MP(void* MP); +*/ +void* ExecutionEngine__create__(void* MP, int ForceInterpreter); +void* ExecutionEngine_getModule(void* EE); +int ExecutionEngine_n_functions(void* EE); +int ExecutionEngine_function_exists(void* EE, const char* funcname); + +#ifdef __cplusplus +}; +#endif + +#endif Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h Sun Apr 2 13:08:28 2006 @@ -0,0 +1,14 @@ +#ifndef __FUNCTION_H__ +#define __FUNCTION_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +// + +#ifdef __cplusplus +}; +#endif + +#endif Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h Sun Apr 2 13:08:28 2006 @@ -0,0 +1,26 @@ +#ifndef __MODULE_H__ +#define __MODULE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +void* Module__init__(const char* ModuleID); +const char* Module_getModuleIdentifier(void* M); +void Module_setModuleIdentifier(void* M, const char* ID); +const char* Module_getTargetTriple(void* M); +void Module_setTargetTriple(void* M, const char* T); +//const char* Module_getEndianness(void* M); +//void Module_setEndianness(void* M, Endianness E); +//const char* Module_getPointerSize(void* M); +//void Module_setPointerSize(void* M, PointerSize PS); +const char* Module_getModuleInlineAsm(void* M); +void Module_setModuleInlineAsm(void* M, const char* Asm); +void Module_ParseAssemblyString(void* M, const char* AsmString); +int Module_verifyModule(void* M); + +#ifdef __cplusplus +}; +#endif + +#endif Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/ModuleProvider.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/ModuleProvider.h Sun Apr 2 13:08:28 2006 @@ -0,0 +1,14 @@ +#ifndef __MODULEPROVIDER_H__ +#define __MODULEPROVIDER_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +void* ExistingModuleProvider__init__(void* M); + +#ifdef __cplusplus +}; +#endif + +#endif Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h Sun Apr 2 13:08:28 2006 @@ -0,0 +1,19 @@ +#ifndef __LLVMCAPI_H__ +#define __LLVMCAPI_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +#include "VMCore/Module.h" +#include "VMCore/ModuleProvider.h" +#include "VMCore/Function.h" +#include "ExecutionEngine/ExecutionEngine.h" + +void toggle_print_machineinstrs(); + +#ifdef __cplusplus +}; +#endif + +#endif Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp Sun Apr 2 13:08:28 2006 @@ -0,0 +1,46 @@ +#include "llvm/ExecutionEngine/ExecutionEngine.h" + +using namespace llvm; + +/* +void* ExecutionEngine__init__M(void* M) { + return new ExecutionEngine((Module*)M); +} + +void* ExecutionEngine__init__MP(void* MP) { + return new ExecutionEngine((ModuleProvider*)MP); +} +*/ + +void* ExecutionEngine__create__(void* MP, int ForceInterpreter) { + return ExecutionEngine::create((ModuleProvider*)MP, (bool)ForceInterpreter); +} + +void* ExecutionEngine_getModule(void* EE) { + return &((ExecutionEngine*)EE)->getModule(); +} + +int ExecutionEngine_n_functions(void* EE) { //move to Module? + Module* mod = &((ExecutionEngine*)EE)->getModule(); + int funccount = 0; + Module::FunctionListType &fns = mod->getFunctionList(); + for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + funccount += 1; + } + } + return funccount; +} + +int Module_function_exists(void* EE, const char* funcname) { //move to Module? + Module* mod = &((ExecutionEngine*)EE)->getModule(); + Module::FunctionListType &fns = mod->getFunctionList(); + for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + if (ii->getName() == funcname) { + return 1; + } + } + } + return 0; +} Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp Sun Apr 2 13:08:28 2006 @@ -0,0 +1,7 @@ +#include "llvm/Function.h" + +using namespace llvm; + +//void* Function__init__(void* M) { +// return new Function((Module*)M); +//} Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp Sun Apr 2 13:08:28 2006 @@ -0,0 +1,58 @@ +#include "llvm/Module.h" +#include "llvm/Analysis/Verifier.h" + +using namespace llvm; + +void* Module__init__(const char* ModuleID) { + return new Module(ModuleID); +} + +const char* Module_getModuleIdentifier(void* M) { + return ((Module*)M)->getModuleIdentifier().c_str(); +} + +void Module_setModuleIdentifier(void* M, const char* ID) { + return ((Module*)M)->setModuleIdentifier(ID); +} + +const char* Module_getTargetTriple(void* M) { + return ((Module*)M)->getTargetTriple().c_str(); +} + +void Module_setTargetTriple(void* M, const char* T) { + return ((Module*)M)->setTargetTriple(T); +} + +/* +const char* Module_getEndianness(void* M) { + return ((Module*)M)->getEndianness().c_str(); +} + +void Module_setEndianness(void* M, Endianness E) { + return ((Module*)M)->setEndianness(E); +} + +const char* Module_getPointerSize(void* M) { + return ((Module*)M)->getPointerSize().c_str(); +} + +void Module_setPointerSize(void* M, PointerSize PS) { + return ((Module*)M)->setPointerSize(PS); +} +*/ + +const char* Module_getModuleInlineAsm(void* M) { + return ((Module*)M)->getModuleInlineAsm().c_str(); +} + +void Module_setModuleInlineAsm(void* M, const char* Asm) { + ((Module*)M)->setModuleInlineAsm(Asm); +} + +void Module_ParseAssemblyString(void* M, const char* AsmString) { //from Assembly/Parser.h + ParseAssemblyString(AsmString, (Module*)M); +} + +int Module_verifyModule(void* M) { //from Analysis/Verifier.h + return verifyModule(*(Module*)M, ThrowExceptionAction); +} Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/ModuleProvider.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/ModuleProvider.cpp Sun Apr 2 13:08:28 2006 @@ -0,0 +1,7 @@ +#include "llvm/ModuleProvider.h" + +using namespace llvm; + +void* ExistingModuleProvider__init__(void* M) { + return new ExistingModuleProvider((Module*)M); +} Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp Sun Apr 2 13:08:28 2006 @@ -0,0 +1,30 @@ +// llvm includes +#include "llvm/Type.h" +#include "llvm/Module.h" +#include "llvm/ModuleProvider.h" +#include "llvm/Assembly/Parser.h" +#include "llvm/Bytecode/Reader.h" +#include "llvm/ExecutionEngine/GenericValue.h" +#include "llvm/ExecutionEngine/ExecutionEngine.h" +#include "llvm/Analysis/Verifier.h" +#include "llvm/DerivedTypes.h" +#include "llvm/Support/CommandLine.h" +#include "llvm/Target/TargetOptions.h" + +// llvm c api includes +#include "llvmcapi.h" + + +using namespace llvm; + + +// llvm c acpi code +#include "VMCore/Module.cpp" +#include "VMCore/ModuleProvider.cpp" +#include "VMCore/Function.cpp" +#include "ExecutionEngine/ExecutionEngine.cpp" + + +void toggle_print_machineinstrs() { + PrintMachineCode = !PrintMachineCode; +} Added: pypy/dist/pypy/translator/llvm/llvmcapi/setup.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/setup.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,19 @@ +from distutils.core import setup +from distutils.extension import Extension +from os import popen + +cxxflags = popen('llvm-config --cxxflags').readline().split() +ldflags = popen('llvm-config --ldflags').readline().split() +libs = popen('llvm-config --libs all').readline().split() + +opts = dict(name='llvmcapi', + sources=['lib/llvmcapi.cpp'], + libraries=[], + include_dirs =["include"] + [f[2:] for f in cxxflags if f.startswith('-I')], + library_dirs =[f[2:] for f in ldflags if f.startswith('-L')], + define_macros=[(f[2:], None) for f in cxxflags if f.startswith('-D')], + extra_objects=libs) + +ext_modules = Extension(**opts) + +setup(name=opts['name'], ext_modules=[ext_modules]) Added: pypy/dist/pypy/translator/llvm/pythonllvm/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,120 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories don't have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.realpath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.realpath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + break + else: + raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) + + checkpaths = sys.path[:] + pypy_root = os.path.join(head, '') + + while checkpaths: + orig = checkpaths.pop() + fullorig = os.path.join(os.path.realpath(orig), '') + if fullorig.startswith(pypy_root): + if os.path.exists(os.path.join(fullorig, '__init__.py')): + sys.path.remove(orig) + try: + sys.path.remove(head) + except ValueError: + pass + sys.path.insert(0, head) + + munged = {} + for name, mod in sys.modules.items(): + fn = getattr(mod, '__file__', None) + if '.' in name or not isinstance(fn, str): + continue + newname = os.path.splitext(os.path.basename(fn))[0] + if not newname.startswith(part + '.'): + continue + path = os.path.join(os.path.dirname(os.path.realpath(fn)), '') + if path.startswith(pypy_root) and newname != part: + modpaths = os.path.normpath(path[len(pypy_root):]).split(os.sep) + if newname != '__init__': + modpaths.append(newname) + modpath = '.'.join(modpaths) + if modpath not in sys.modules: + munged[modpath] = mod + + for name, mod in munged.iteritems(): + if name not in sys.modules: + sys.modules[name] = mod + if '.' in name: + prename = name[:name.rfind('.')] + postname = name[len(prename)+1:] + if prename not in sys.modules: + __import__(prename) + if not hasattr(sys.modules[prename], postname): + setattr(sys.modules[prename], postname, mod) + + return partdir, this_dir + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Added: pypy/dist/pypy/translator/llvm/pythonllvm/cc.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/cc.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,17 @@ +''' +Added this because ctypes on my computer was missing cdecl. +''' +from ctypes import * + +class cdecl(object): + def __init__(self, restype, libname, argtypes): + #XXX does this load once or every time? + self.library = cdll.load("../llvmcapi/" + libname + ".so") + self.restype = restype + self.argtypes = argtypes + + def __call__(self, func): + func._api_ = getattr(self.library, func.__name__) + func._api_.restype = self.restype + func._api_.argtypes = self.argtypes + return func Added: pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,14 @@ +#!/usr/bin/env python +import os + +WRAPDIR = "/Users/eric/projects/ctypes/ctypes/wrap" #XXX get rid of this hardcoded path + +if os.path.exists(WRAPDIR + "/h2xml.py"): + print "creating llvmcapi.py" + os.system("python " + WRAPDIR + "/h2xml.py ../llvmcapi/include/llvmcapi.h -q -I . -o llvmcapi.xml") + os.system("python " + WRAPDIR + "/xml2py.py llvmcapi.xml -l ../llvmcapi/llvmcapi.so -o llvmcapi.tmp") + os.system("sed -e s/from\ ctypes\ import/from\ cc\ import/ llvmcapi.tmp > llvmcapi.py") + os.system("rm -f llvmcapi.tmp llvmcapi.xml") + print "done" +else: + print "skipping llvmcapi.py creation" Added: pypy/dist/pypy/translator/llvm/pythonllvm/doc/pyllvm.cpp.org ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/doc/pyllvm.cpp.org Sun Apr 2 13:08:28 2006 @@ -0,0 +1,473 @@ + +// python includes +#include +#include + +// llvm includes +#include "llvm/Type.h" +#include "llvm/Module.h" +#include "llvm/ModuleProvider.h" +#include "llvm/Assembly/Parser.h" +#include "llvm/Bytecode/Reader.h" +#include "llvm/ExecutionEngine/GenericValue.h" +#include "llvm/ExecutionEngine/ExecutionEngine.h" +#include "llvm/Analysis/Verifier.h" +#include "llvm/DerivedTypes.h" + +#include "llvm/Support/CommandLine.h" +#include "llvm/Target/TargetOptions.h" + + +// c++ includes +#include +#include + +using namespace llvm; + +typedef struct { + PyObject_HEAD + ExecutionEngine *exec; + +} PyExecutionEngine; + + +/* +XXX we should probably support parsing one function at a time too. +This would enable use to use self->exec->recompileAndRelinkFunction(Function* F) +to support rebinding to a new (more blocks?) function. (or self modifying code) +*/ +static PyObject *ee_parse(PyExecutionEngine *self, PyObject *args) { + + // check number of parameters + if (PyTuple_Size(args) < 1 or PyTuple_Size(args) > 2) { + PyErr_SetString(PyExc_TypeError, "expected one or two parameters"); + return NULL; + } + + // get first parameter (llcode) + PyObject *pyllcode = PyTuple_GetItem(args, 0); + if (!PyString_Check(pyllcode)) { + PyErr_SetString(PyExc_TypeError, "first arg expected as string with llcode"); + return NULL; + } + const char* llcode = PyString_AsString(pyllcode); + + // get optional second parameter (fnname) + const char* fnname = NULL; + if (PyTuple_Size(args) > 1) { + PyObject *pyfnname = PyTuple_GetItem(args, 1); + if (!PyString_Check(pyllcode)) { + PyErr_SetString(PyExc_TypeError, "second arg expected as string with functionname"); + return NULL; + } + fnname = PyString_AsString(pyfnname); + } + + // parse and verify llcode + try { + Module *M = &self->exec->getModule(); + + if (fnname) { + Function *F = M->getNamedFunction(fnname); + if (F == NULL) { + PyErr_SetString(PyExc_Exception, "Failed to resolve function to be replaced"); + return NULL; + } + F->setName(""); // now it won't conflicht + ParseAssemblyString(llcode, M); + Function *Fnew = M->getNamedFunction(fnname); + F->replaceAllUsesWith(Fnew); // Everything using the old one uses the new one + self->exec->freeMachineCodeForFunction(F); // still no-op on march 27th 2006 + F->eraseFromParent(); // also, take it out of the JIT (LLVM IR) if it was in it + + /* + XXX this is a really dump implementation!!! Should really be somehthing like... + codegend_uses = list of function that have codegen'd calls to F + for func in codegend_uses: + self->exec->recompileAndRelinkFunction(func); + // what about (codegen-ed) pointers to F? + */ + Module::FunctionListType &fns = M->getFunctionList(); + for (Module::FunctionListType::iterator ii = fns.begin(); + ii != fns.end(); + ++ii) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + self->exec->recompileAndRelinkFunction(ii); + } + } + } else { + ParseAssemblyString(llcode, M); + } + + verifyModule(*M, ThrowExceptionAction); // XXX make this optional for performance? + Py_INCREF(Py_None); + return Py_None; + + } catch (const ParseException &ref) { + PyErr_SetString(PyExc_Exception, ref.getMessage().c_str()); + + } catch (...) { + PyErr_SetString(PyExc_Exception, "Unexpected unknown exception occurred"); + } + + return NULL; +} + +static int from_python_to_args(std::vector &llvmargs, + PyObject *value, + const Type *type) { + // XXX Flesh out + GenericValue res; + if (type->getTypeID() == Type::IntTyID) { + if (!PyInt_Check(value)) { + PyErr_SetString(PyExc_TypeError, "expected an integer type"); + return -1; + } + res.IntVal = PyInt_AsLong(value); + llvmargs.push_back(res); + return 0; + } + + std::string err = "unsupported type: " + type->getDescription(); + PyErr_SetString(PyExc_TypeError, err.c_str()); + return -1; +} + +static PyObject *to_python_value(const GenericValue &value, + const Type *type) { + + // special case for strings - it is your own fault if not NULL terminated + if (type->getTypeID() == Type::PointerTyID && + type->getContainedType(0)->getTypeID() == Type::SByteTyID) { + return PyString_FromString((const char *) value.PointerVal); + } + + PyObject *res; + + switch (type->getTypeID()) { + + case Type::VoidTyID: + Py_INCREF(Py_None); + res = Py_None; + break; + + case Type::BoolTyID: + res = PyBool_FromLong((long) value.BoolVal); + break; + + case Type::UByteTyID: + res = PyInt_FromLong((long) value.UByteVal); + break; + + case Type::SByteTyID: + res = PyInt_FromLong((long) value.SByteVal); + break; + + case Type::UShortTyID: + res = PyInt_FromLong((long) value.UShortVal); + break; + + case Type::ShortTyID: + res = PyInt_FromLong((long) value.ShortVal); + break; + + case Type::UIntTyID: + res = PyLong_FromUnsignedLong(value.UIntVal); + break; + + case Type::IntTyID: + res = PyInt_FromLong(value.IntVal); + break; + + case Type::ULongTyID: + res = PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value.ULongVal); + break; + + case Type::LongTyID: + res = PyLong_FromLongLong((PY_LONG_LONG) value.ULongVal); + break; + + // XXX the rest + default: + std::string err = "unsupported type: " + type->getDescription(); + PyErr_SetString(PyExc_TypeError, err.c_str()); + res = NULL; + } + + return res; +} + +static PyObject *ee_call(PyExecutionEngine *self, PyObject *args) { + + if (PyTuple_Size(args) == 0) { + PyErr_SetString(PyExc_TypeError, "missing functionname"); + return NULL; + } + + PyObject *pyfnname = PyTuple_GetItem(args, 0); + if (!PyString_Check(pyfnname)) { + PyErr_SetString(PyExc_TypeError, "first arg expected as string"); + return NULL; + } + char *fnname = PyString_AsString(pyfnname); + + try { + Function *fn = self->exec->getModule().getNamedFunction(fnname); + if (fn == NULL) { + PyErr_SetString(PyExc_Exception, "Failed to resolve function to call"); + return NULL; + } + + unsigned argcount = fn->getFunctionType()->getNumParams(); + if ((unsigned) PyTuple_Size(args) != argcount + 1) { + PyErr_SetString(PyExc_TypeError, "args not much count"); + return NULL; + } + + std::vector llvmargs; + for (unsigned ii=0; iigetFunctionType()->getParamType(ii)) == -1) { + return NULL; + } + } + + GenericValue ret = self->exec->runFunction(fn, llvmargs); + return to_python_value(ret, fn->getFunctionType()->getReturnType()); + + } catch (...) { + PyErr_SetString(PyExc_Exception, "Unexpected unknown exception occurred"); + return NULL; + } +} + +static PyObject *ee_delete(PyExecutionEngine *self, PyObject *args) { + + if (PyTuple_Size(args) != 1) { + PyErr_SetString(PyExc_TypeError, "missing functionname"); + return NULL; + } + + PyObject *pyfnname = PyTuple_GetItem(args, 0); + if (!PyString_Check(pyfnname)) { + PyErr_SetString(PyExc_TypeError, "functionname expected as string"); + return NULL; + } + char *fnname = PyString_AsString(pyfnname); + + try { + Function *fn = self->exec->getModule().getNamedFunction(fnname); + if (fn == NULL) { + PyErr_SetString(PyExc_Exception, "Failed to resolve function to delete"); + return NULL; + } + + // XXX fn should not be refered to from anywhere! Check for this? + self->exec->freeMachineCodeForFunction(fn); // still no-op on march 27th 2006 + fn->eraseFromParent(); + } catch (...) { + PyErr_SetString(PyExc_Exception, "Unexpected unknown exception occurred"); + return NULL; + } + + Py_INCREF(Py_None); + return Py_None; +} + +static PyObject *ee_functions(PyExecutionEngine *self) { + int funccount = 0; + Module::FunctionListType &fns = self->exec->getModule().getFunctionList(); + // spin thrru and get function count + for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + funccount += 1; + } + } + + PyObject *functions = PyTuple_New(funccount); + + int count = 0; + for (Module::FunctionListType::const_iterator ii = fns.begin(); + ii != fns.end(); + ++ii, funccount++) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + + unsigned argcount = ii->getFunctionType()->getNumParams(); + + PyObject *entry = PyTuple_New(3); + + PyObject *returnId = PyInt_FromLong(ii->getFunctionType()->getReturnType()->getTypeID()); + PyObject *name = PyString_FromString(ii->getName().c_str()); + PyObject *args = PyTuple_New(argcount); + + for (unsigned jj=0; jjgetFunctionType()->getParamType(jj)->getTypeID(); + PyTuple_SetItem(args, (long) jj, PyInt_FromLong(argtype)); + } + + PyTuple_SetItem(entry, 0, returnId); + PyTuple_SetItem(entry, 1, name); + PyTuple_SetItem(entry, 2, args); + + PyTuple_SetItem(functions, count, entry); + count += 1; + } + } + + return functions; +} + +static PyMethodDef ee_methodlist[] = { + {"parse" , (PyCFunction) ee_parse , METH_VARARGS, NULL}, + {"call" , (PyCFunction) ee_call , METH_VARARGS, NULL}, + {"delete" , (PyCFunction) ee_delete , METH_VARARGS, NULL}, + {"functions", (PyCFunction) ee_functions, METH_NOARGS , NULL}, + + {NULL, NULL} +}; + +void ee_dealloc(PyExecutionEngine *self); +PyObject *ee_new(PyTypeObject *type, PyObject *args, PyObject *kwds); + +// PyTypeObject - pythons type structure +PyTypeObject ExecutionEngine_Type = { + PyObject_HEAD_INIT(NULL) + 0, + "ExecutionEngine", + sizeof(PyExecutionEngine), + 0, + (destructor)ee_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + ee_methodlist, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + 0, /* tp_new */ +}; + +PyObject *pyllvm_execution_engine; + +static PyObject *ee_factory() { + + if (pyllvm_execution_engine != NULL) { + PyErr_SetString(PyExc_Exception, "This should not happen"); + return NULL; + } + + ExecutionEngine *exec; + + try { + Module *mod = new Module((const char *) "my module"); + ModuleProvider *mp = new ExistingModuleProvider(mod); + exec = ExecutionEngine::create(mp, false); + assert(exec && "Couldn't create an ExecutionEngine, not even an interpreter?"); + } catch (...) { + PyErr_SetString(PyExc_Exception, "Unexpected unknown exception occurred"); + return NULL; + } + + PyTypeObject *type = &ExecutionEngine_Type; + + PyExecutionEngine *self = (PyExecutionEngine *) type->tp_alloc(type, 0); + self->exec = exec; + + return (PyObject *) self; +} + +void ee_dealloc(PyExecutionEngine *self) { + self->ob_type->tp_free((PyObject*) self); +} + +static PyObject *pyllvm_get_ee(PyObject *self, PyObject *args) { + if (pyllvm_execution_engine != NULL) { + Py_INCREF(pyllvm_execution_engine); + return pyllvm_execution_engine; + } + + pyllvm_execution_engine = ee_factory(); + return pyllvm_execution_engine; +} + +static PyObject *pyllvm_delete_ee(PyObject *self, PyObject *args) { + PyExecutionEngine *ee = (PyExecutionEngine *) pyllvm_execution_engine; + if (ee != NULL) { + + // bye + if (ee->exec != NULL) { + delete ee->exec; + } + + Py_DECREF(pyllvm_execution_engine); + pyllvm_execution_engine = NULL; + } + + Py_INCREF(Py_None); + return Py_None; +} + +static PyObject *pyllvm_toggle_print_machineinstrs(PyObject *self, PyObject *args) { + PrintMachineCode = !PrintMachineCode; + + Py_INCREF(Py_None); + return Py_None; +} + +PyMethodDef pyllvm_functions[] = { + {"get_ee" , pyllvm_get_ee , METH_NOARGS, NULL}, + {"delete_ee" , pyllvm_delete_ee , METH_NOARGS, NULL}, + {"toggle_print_machineinstrs", pyllvm_toggle_print_machineinstrs, METH_NOARGS, NULL}, + {NULL, NULL} +}; + + +#ifdef __cplusplus +extern "C" { +#endif + +void initpyllvm(void) { + PyObject *module = Py_InitModule("pyllvm", pyllvm_functions); + + if(PyType_Ready(&ExecutionEngine_Type) < 0) { + return; + } + + Py_INCREF(&ExecutionEngine_Type); + PyModule_AddObject(module, "ExecutionEngine", + (PyObject*) &ExecutionEngine_Type); + + if (0) { // pass in `lli` type parameters + static char * Args[] = { "", "-print-machineinstrs", 0 }; + int n_args = 2; + cl::ParseCommandLineOptions(n_args, Args); + } +} + +} Added: pypy/dist/pypy/translator/llvm/pythonllvm/helper.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/helper.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,24 @@ +''' +Added this because ctypes on my computer was missing cdecl. +''' +from llvmcapi import * + +class Method(object): + def __init__(self, instance, method): + self.instance = instance + self.method = method + + def __call__(self, *args): + a = [self.instance] + #for arg in args: + # if isinstance(arg, Wrapper): #pass on value to actual C (not Python) object + # a.append(arg.instance) + # else: + # a.append(arg) + a += args + return apply(self.method, a) + +class Wrapper(object): + def __getattr__(self, name): + global_funcname = self.__class__.__name__ + "_" + name + return Method(self.instance, globals()[global_funcname]) Added: pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,90 @@ +# generated by 'xml2py' +# flags 'llvmcapi.xml -l ../llvmcapi/llvmcapi.so -o llvmcapi.tmp' +from cc import * +STRING = c_char_p + + + +def ExecutionEngine__create__(MP, ForceInterpreter): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 12 + return ExecutionEngine__create__._api_(MP, ForceInterpreter) +ExecutionEngine__create__ = cdecl(c_void_p, 'llvmcapi', [c_void_p, c_int]) (ExecutionEngine__create__) + + +def ExecutionEngine_getModule(EE): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 13 + return ExecutionEngine_getModule._api_(EE) +ExecutionEngine_getModule = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (ExecutionEngine_getModule) + + +def ExecutionEngine_n_functions(EE): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 14 + return ExecutionEngine_n_functions._api_(EE) +ExecutionEngine_n_functions = cdecl(c_int, 'llvmcapi', [c_void_p]) (ExecutionEngine_n_functions) + + +def Module__init__(ModuleID): + # ../llvmcapi/include/VMCore/Module.h 8 + return Module__init__._api_(ModuleID) +Module__init__ = cdecl(c_void_p, 'llvmcapi', [STRING]) (Module__init__) + + +def Module_getModuleIdentifier(M): + # ../llvmcapi/include/VMCore/Module.h 9 + return Module_getModuleIdentifier._api_(M) +Module_getModuleIdentifier = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getModuleIdentifier) + + +def Module_setModuleIdentifier(M, ID): + # ../llvmcapi/include/VMCore/Module.h 10 + return Module_setModuleIdentifier._api_(M, ID) +Module_setModuleIdentifier = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setModuleIdentifier) + + +def Module_getTargetTriple(M): + # ../llvmcapi/include/VMCore/Module.h 11 + return Module_getTargetTriple._api_(M) +Module_getTargetTriple = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getTargetTriple) + + +def Module_setTargetTriple(M, T): + # ../llvmcapi/include/VMCore/Module.h 12 + return Module_setTargetTriple._api_(M, T) +Module_setTargetTriple = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setTargetTriple) + + +def Module_getModuleInlineAsm(M): + # ../llvmcapi/include/VMCore/Module.h 17 + return Module_getModuleInlineAsm._api_(M) +Module_getModuleInlineAsm = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getModuleInlineAsm) + + +def Module_setModuleInlineAsm(M, Asm): + # ../llvmcapi/include/VMCore/Module.h 18 + return Module_setModuleInlineAsm._api_(M, Asm) +Module_setModuleInlineAsm = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setModuleInlineAsm) + + +def Module_ParseAssemblyString(M, AsmString): + # ../llvmcapi/include/VMCore/Module.h 19 + return Module_ParseAssemblyString._api_(M, AsmString) +Module_ParseAssemblyString = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_ParseAssemblyString) + + +def Module_verifyModule(M): + # ../llvmcapi/include/VMCore/Module.h 20 + return Module_verifyModule._api_(M) +Module_verifyModule = cdecl(c_int, 'llvmcapi', [c_void_p]) (Module_verifyModule) + + +def ExistingModuleProvider__init__(M): + # ../llvmcapi/include/VMCore/ModuleProvider.h 8 + return ExistingModuleProvider__init__._api_(M) +ExistingModuleProvider__init__ = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (ExistingModuleProvider__init__) + + +def toggle_print_machineinstrs(): + # ../llvmcapi/include/llvmcapi.h 13 + return toggle_print_machineinstrs._api_() +toggle_print_machineinstrs = cdecl(None, 'llvmcapi', []) (toggle_print_machineinstrs) + Added: pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,29 @@ +from helper import * + +class Module(Wrapper): + def __init__(self, ModuleID='mymodule'): + self.instance = Module__init__(ModuleID) + +class ModuleProvider(Wrapper): + pass + +class ExistingModuleProvider(ModuleProvider): + def __init__(self, M=None): + if not M: + M = Module() + self.instance = ExistingModuleProvider__init__(M.instance) + +class ExecutionEngine(Wrapper): + def __init__(self, MP=None, ForceInterpreter=False): + if not MP: + MP = ExistingModuleProvider(); + self.instance = ExecutionEngine__create__(MP.instance, ForceInterpreter) + + def getModule(self): + m = object.__new__(Module) + m.instance = ExecutionEngine_getModule(self.instance) + return m + + def parse(self, llcode): + self.getModule().ParseAssemblyString(llcode) + self.getModule().verifyModule() Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/.gdb_history ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/.gdb_history Sun Apr 2 13:08:28 2006 @@ -0,0 +1 @@ +run Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/addnumbers.c ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/addnumbers.c Sun Apr 2 13:08:28 2006 @@ -0,0 +1,4 @@ +int add(int n, int y) { + return n + y; +} + Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/addnumbers.s ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/addnumbers.s Sun Apr 2 13:08:28 2006 @@ -0,0 +1,441 @@ +; GNU C version 3.4-llvm 20051104 (LLVM 1.6) (i686-pc-linux-gnu) +; compiled by GNU C version 3.4.0. +; GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096 +; options passed: -iprefix -mtune=pentiumpro -auxbase +; options enabled: -feliminate-unused-debug-types -fpeephole +; -ffunction-cse -fkeep-static-consts -fpcc-struct-return -fgcse-lm +; -fgcse-sm -fsched-interblock -fsched-spec -fbranch-count-reg -fcommon +; -fgnu-linker -fargument-alias -fzero-initialized-in-bss -fident +; -fmath-errno -ftrapping-math -m80387 -mhard-float -mno-soft-float +; -mieee-fp -mfp-ret-in-387 -maccumulate-outgoing-args -mno-red-zone +; -mtls-direct-seg-refs -mtune=pentiumpro -march=i386 + +target triple = "i686-pc-linux-gnu" +target pointersize = 32 +target endian = little +deplibs = ["c", "crtend"] + +"complex double" = type { double, double } +"complex float" = type { float, float } +"complex long double" = type { double, double } + +implementation +declare double %acos(double) ;; __builtin_acos +declare float %acosf(float) ;; __builtin_acosf +declare double %acosh(double) ;; __builtin_acosh +declare float %acoshf(float) ;; __builtin_acoshf +declare double %acoshl(double) ;; __builtin_acoshl +declare double %acosl(double) ;; __builtin_acosl +declare double %asin(double) ;; __builtin_asin +declare float %asinf(float) ;; __builtin_asinf +declare double %asinh(double) ;; __builtin_asinh +declare float %asinhf(float) ;; __builtin_asinhf +declare double %asinhl(double) ;; __builtin_asinhl +declare double %asinl(double) ;; __builtin_asinl +declare double %atan(double) ;; __builtin_atan +declare double %atan2(double,double) ;; __builtin_atan2 +declare float %atan2f(float,float) ;; __builtin_atan2f +declare double %atan2l(double,double) ;; __builtin_atan2l +declare float %atanf(float) ;; __builtin_atanf +declare double %atanh(double) ;; __builtin_atanh +declare float %atanhf(float) ;; __builtin_atanhf +declare double %atanhl(double) ;; __builtin_atanhl +declare double %atanl(double) ;; __builtin_atanl +declare double %cbrt(double) ;; __builtin_cbrt +declare float %cbrtf(float) ;; __builtin_cbrtf +declare double %cbrtl(double) ;; __builtin_cbrtl +declare double %ceil(double) ;; __builtin_ceil +declare float %ceilf(float) ;; __builtin_ceilf +declare double %ceill(double) ;; __builtin_ceill +declare double %copysign(double,double) ;; __builtin_copysign +declare float %copysignf(float,float) ;; __builtin_copysignf +declare double %copysignl(double,double) ;; __builtin_copysignl +declare double %cos(double) ;; __builtin_cos +declare float %cosf(float) ;; __builtin_cosf +declare double %cosh(double) ;; __builtin_cosh +declare float %coshf(float) ;; __builtin_coshf +declare double %coshl(double) ;; __builtin_coshl +declare double %cosl(double) ;; __builtin_cosl +declare double %drem(double,double) ;; __builtin_drem +declare float %dremf(float,float) ;; __builtin_dremf +declare double %dreml(double,double) ;; __builtin_dreml +declare double %erf(double) ;; __builtin_erf +declare double %erfc(double) ;; __builtin_erfc +declare float %erfcf(float) ;; __builtin_erfcf +declare double %erfcl(double) ;; __builtin_erfcl +declare float %erff(float) ;; __builtin_erff +declare double %erfl(double) ;; __builtin_erfl +declare double %exp(double) ;; __builtin_exp +declare double %exp10(double) ;; __builtin_exp10 +declare float %exp10f(float) ;; __builtin_exp10f +declare double %exp10l(double) ;; __builtin_exp10l +declare double %exp2(double) ;; __builtin_exp2 +declare float %exp2f(float) ;; __builtin_exp2f +declare double %exp2l(double) ;; __builtin_exp2l +declare float %expf(float) ;; __builtin_expf +declare double %expl(double) ;; __builtin_expl +declare double %expm1(double) ;; __builtin_expm1 +declare float %expm1f(float) ;; __builtin_expm1f +declare double %expm1l(double) ;; __builtin_expm1l +declare double %fabs(double) ;; __builtin_fabs +declare float %fabsf(float) ;; __builtin_fabsf +declare double %fabsl(double) ;; __builtin_fabsl +declare double %fdim(double,double) ;; __builtin_fdim +declare float %fdimf(float,float) ;; __builtin_fdimf +declare double %fdiml(double,double) ;; __builtin_fdiml +declare double %floor(double) ;; __builtin_floor +declare float %floorf(float) ;; __builtin_floorf +declare double %floorl(double) ;; __builtin_floorl +declare double %fma(double,double,double) ;; __builtin_fma +declare float %fmaf(float,float,float) ;; __builtin_fmaf +declare double %fmal(double,double,double) ;; __builtin_fmal +declare double %fmax(double,double) ;; __builtin_fmax +declare float %fmaxf(float,float) ;; __builtin_fmaxf +declare double %fmaxl(double,double) ;; __builtin_fmaxl +declare double %fmin(double,double) ;; __builtin_fmin +declare float %fminf(float,float) ;; __builtin_fminf +declare double %fminl(double,double) ;; __builtin_fminl +declare double %fmod(double,double) ;; __builtin_fmod +declare float %fmodf(float,float) ;; __builtin_fmodf +declare double %fmodl(double,double) ;; __builtin_fmodl +declare double %frexp(double,int*) ;; __builtin_frexp +declare float %frexpf(float,int*) ;; __builtin_frexpf +declare double %frexpl(double,int*) ;; __builtin_frexpl +declare double %gamma(double) ;; __builtin_gamma +declare float %gammaf(float) ;; __builtin_gammaf +declare double %gammal(double) ;; __builtin_gammal +declare double %__builtin_huge_val() +declare float %__builtin_huge_valf() +declare double %__builtin_huge_vall() +declare double %hypot(double,double) ;; __builtin_hypot +declare float %hypotf(float,float) ;; __builtin_hypotf +declare double %hypotl(double,double) ;; __builtin_hypotl +declare int %ilogb(double) ;; __builtin_ilogb +declare int %ilogbf(float) ;; __builtin_ilogbf +declare int %ilogbl(double) ;; __builtin_ilogbl +declare double %__builtin_inf() +declare float %__builtin_inff() +declare double %__builtin_infl() +declare double %j0(double) ;; __builtin_j0 +declare float %j0f(float) ;; __builtin_j0f +declare double %j0l(double) ;; __builtin_j0l +declare double %j1(double) ;; __builtin_j1 +declare float %j1f(float) ;; __builtin_j1f +declare double %j1l(double) ;; __builtin_j1l +declare double %jn(int,double) ;; __builtin_jn +declare float %jnf(int,float) ;; __builtin_jnf +declare double %jnl(int,double) ;; __builtin_jnl +declare double %ldexp(double,int) ;; __builtin_ldexp +declare float %ldexpf(float,int) ;; __builtin_ldexpf +declare double %ldexpl(double,int) ;; __builtin_ldexpl +declare double %lgamma(double) ;; __builtin_lgamma +declare float %lgammaf(float) ;; __builtin_lgammaf +declare double %lgammal(double) ;; __builtin_lgammal +declare long %llrint(double) ;; __builtin_llrint +declare long %llrintf(float) ;; __builtin_llrintf +declare long %llrintl(double) ;; __builtin_llrintl +declare long %llround(double) ;; __builtin_llround +declare long %llroundf(float) ;; __builtin_llroundf +declare long %llroundl(double) ;; __builtin_llroundl +declare double %log(double) ;; __builtin_log +declare double %log10(double) ;; __builtin_log10 +declare float %log10f(float) ;; __builtin_log10f +declare double %log10l(double) ;; __builtin_log10l +declare double %log1p(double) ;; __builtin_log1p +declare float %log1pf(float) ;; __builtin_log1pf +declare double %log1pl(double) ;; __builtin_log1pl +declare double %log2(double) ;; __builtin_log2 +declare float %log2f(float) ;; __builtin_log2f +declare double %log2l(double) ;; __builtin_log2l +declare double %logb(double) ;; __builtin_logb +declare float %logbf(float) ;; __builtin_logbf +declare double %logbl(double) ;; __builtin_logbl +declare float %logf(float) ;; __builtin_logf +declare double %logl(double) ;; __builtin_logl +declare int %lrint(double) ;; __builtin_lrint +declare int %lrintf(float) ;; __builtin_lrintf +declare int %lrintl(double) ;; __builtin_lrintl +declare int %lround(double) ;; __builtin_lround +declare int %lroundf(float) ;; __builtin_lroundf +declare int %lroundl(double) ;; __builtin_lroundl +declare double %modf(double,double*) ;; __builtin_modf +declare float %modff(float,float*) ;; __builtin_modff +declare double %modfl(double,double*) ;; __builtin_modfl +declare double %nan(sbyte*) ;; __builtin_nan +declare float %nanf(sbyte*) ;; __builtin_nanf +declare double %nanl(sbyte*) ;; __builtin_nanl +declare double %nans(sbyte*) ;; __builtin_nans +declare float %nansf(sbyte*) ;; __builtin_nansf +declare double %nansl(sbyte*) ;; __builtin_nansl +declare double %nearbyint(double) ;; __builtin_nearbyint +declare float %nearbyintf(float) ;; __builtin_nearbyintf +declare double %nearbyintl(double) ;; __builtin_nearbyintl +declare double %nextafter(double,double) ;; __builtin_nextafter +declare float %nextafterf(float,float) ;; __builtin_nextafterf +declare double %nextafterl(double,double) ;; __builtin_nextafterl +declare double %nexttoward(double,double) ;; __builtin_nexttoward +declare float %nexttowardf(float,double) ;; __builtin_nexttowardf +declare double %nexttowardl(double,double) ;; __builtin_nexttowardl +declare double %pow(double,double) ;; __builtin_pow +declare double %pow10(double) ;; __builtin_pow10 +declare float %pow10f(float) ;; __builtin_pow10f +declare double %pow10l(double) ;; __builtin_pow10l +declare float %powf(float,float) ;; __builtin_powf +declare double %powl(double,double) ;; __builtin_powl +declare double %remainder(double,double) ;; __builtin_remainder +declare float %remainderf(float,float) ;; __builtin_remainderf +declare double %remainderl(double,double) ;; __builtin_remainderl +declare double %remquo(double,double,int*) ;; __builtin_remquo +declare float %remquof(float,float,int*) ;; __builtin_remquof +declare double %remquol(double,double,int*) ;; __builtin_remquol +declare double %rint(double) ;; __builtin_rint +declare float %rintf(float) ;; __builtin_rintf +declare double %rintl(double) ;; __builtin_rintl +declare double %round(double) ;; __builtin_round +declare float %roundf(float) ;; __builtin_roundf +declare double %roundl(double) ;; __builtin_roundl +declare double %scalb(double,double) ;; __builtin_scalb +declare float %scalbf(float,float) ;; __builtin_scalbf +declare double %scalbl(double,double) ;; __builtin_scalbl +declare double %scalbln(double,int) ;; __builtin_scalbln +declare float %scalblnf(float,int) ;; __builtin_scalblnf +declare double %scalblnl(double,int) ;; __builtin_scalblnl +declare double %scalbn(double,int) ;; __builtin_scalbn +declare float %scalbnf(float,int) ;; __builtin_scalbnf +declare double %scalbnl(double,int) ;; __builtin_scalbnl +declare double %significand(double) ;; __builtin_significand +declare float %significandf(float) ;; __builtin_significandf +declare double %significandl(double) ;; __builtin_significandl +declare double %sin(double) ;; __builtin_sin +declare void %sincos(double,double*,double*) ;; __builtin_sincos +declare void %sincosf(float,float*,float*) ;; __builtin_sincosf +declare void %sincosl(double,double*,double*) ;; __builtin_sincosl +declare float %sinf(float) ;; __builtin_sinf +declare double %sinh(double) ;; __builtin_sinh +declare float %sinhf(float) ;; __builtin_sinhf +declare double %sinhl(double) ;; __builtin_sinhl +declare double %sinl(double) ;; __builtin_sinl +declare double %sqrt(double) ;; __builtin_sqrt +declare float %sqrtf(float) ;; __builtin_sqrtf +declare double %sqrtl(double) ;; __builtin_sqrtl +declare double %tan(double) ;; __builtin_tan +declare float %tanf(float) ;; __builtin_tanf +declare double %tanh(double) ;; __builtin_tanh +declare float %tanhf(float) ;; __builtin_tanhf +declare double %tanhl(double) ;; __builtin_tanhl +declare double %tanl(double) ;; __builtin_tanl +declare double %tgamma(double) ;; __builtin_tgamma +declare float %tgammaf(float) ;; __builtin_tgammaf +declare double %tgammal(double) ;; __builtin_tgammal +declare double %trunc(double) ;; __builtin_trunc +declare float %truncf(float) ;; __builtin_truncf +declare double %truncl(double) ;; __builtin_truncl +declare double %y0(double) ;; __builtin_y0 +declare float %y0f(float) ;; __builtin_y0f +declare double %y0l(double) ;; __builtin_y0l +declare double %y1(double) ;; __builtin_y1 +declare float %y1f(float) ;; __builtin_y1f +declare double %y1l(double) ;; __builtin_y1l +declare double %yn(int,double) ;; __builtin_yn +declare float %ynf(int,float) ;; __builtin_ynf +declare double %ynl(int,double) ;; __builtin_ynl +declare double %cabs(double,double) ;; __builtin_cabs +declare float %cabsf(float,float) ;; __builtin_cabsf +declare double %cabsl(double,double) ;; __builtin_cabsl +declare void %cacos("complex double"*,double,double) ;; __builtin_cacos +declare void %cacosf("complex float"*,float,float) ;; __builtin_cacosf +declare void %cacosh("complex double"*,double,double) ;; __builtin_cacosh +declare void %cacoshf("complex float"*,float,float) ;; __builtin_cacoshf +declare void %cacoshl("complex long double"*,double,double) ;; __builtin_cacoshl +declare void %cacosl("complex long double"*,double,double) ;; __builtin_cacosl +declare double %carg(double,double) ;; __builtin_carg +declare float %cargf(float,float) ;; __builtin_cargf +declare double %cargl(double,double) ;; __builtin_cargl +declare void %casin("complex double"*,double,double) ;; __builtin_casin +declare void %casinf("complex float"*,float,float) ;; __builtin_casinf +declare void %casinh("complex double"*,double,double) ;; __builtin_casinh +declare void %casinhf("complex float"*,float,float) ;; __builtin_casinhf +declare void %casinhl("complex long double"*,double,double) ;; __builtin_casinhl +declare void %casinl("complex long double"*,double,double) ;; __builtin_casinl +declare void %catan("complex double"*,double,double) ;; __builtin_catan +declare void %catanf("complex float"*,float,float) ;; __builtin_catanf +declare void %catanh("complex double"*,double,double) ;; __builtin_catanh +declare void %catanhf("complex float"*,float,float) ;; __builtin_catanhf +declare void %catanhl("complex long double"*,double,double) ;; __builtin_catanhl +declare void %catanl("complex long double"*,double,double) ;; __builtin_catanl +declare void %ccos("complex double"*,double,double) ;; __builtin_ccos +declare void %ccosf("complex float"*,float,float) ;; __builtin_ccosf +declare void %ccosh("complex double"*,double,double) ;; __builtin_ccosh +declare void %ccoshf("complex float"*,float,float) ;; __builtin_ccoshf +declare void %ccoshl("complex long double"*,double,double) ;; __builtin_ccoshl +declare void %ccosl("complex long double"*,double,double) ;; __builtin_ccosl +declare void %cexp("complex double"*,double,double) ;; __builtin_cexp +declare void %cexpf("complex float"*,float,float) ;; __builtin_cexpf +declare void %cexpl("complex long double"*,double,double) ;; __builtin_cexpl +declare double %cimag(double,double) ;; __builtin_cimag +declare float %cimagf(float,float) ;; __builtin_cimagf +declare double %cimagl(double,double) ;; __builtin_cimagl +declare void %conj("complex double"*,double,double) ;; __builtin_conj +declare void %conjf("complex float"*,float,float) ;; __builtin_conjf +declare void %conjl("complex long double"*,double,double) ;; __builtin_conjl +declare void %cpow("complex double"*,double,double,double,double) ;; __builtin_cpow +declare void %cpowf("complex float"*,float,float,float,float) ;; __builtin_cpowf +declare void %cpowl("complex long double"*,double,double,double,double) ;; __builtin_cpowl +declare void %cproj("complex double"*,double,double) ;; __builtin_cproj +declare void %cprojf("complex float"*,float,float) ;; __builtin_cprojf +declare void %cprojl("complex long double"*,double,double) ;; __builtin_cprojl +declare double %creal(double,double) ;; __builtin_creal +declare float %crealf(float,float) ;; __builtin_crealf +declare double %creall(double,double) ;; __builtin_creall +declare void %csin("complex double"*,double,double) ;; __builtin_csin +declare void %csinf("complex float"*,float,float) ;; __builtin_csinf +declare void %csinh("complex double"*,double,double) ;; __builtin_csinh +declare void %csinhf("complex float"*,float,float) ;; __builtin_csinhf +declare void %csinhl("complex long double"*,double,double) ;; __builtin_csinhl +declare void %csinl("complex long double"*,double,double) ;; __builtin_csinl +declare void %csqrt("complex double"*,double,double) ;; __builtin_csqrt +declare void %csqrtf("complex float"*,float,float) ;; __builtin_csqrtf +declare void %csqrtl("complex long double"*,double,double) ;; __builtin_csqrtl +declare void %ctan("complex double"*,double,double) ;; __builtin_ctan +declare void %ctanf("complex float"*,float,float) ;; __builtin_ctanf +declare void %ctanh("complex double"*,double,double) ;; __builtin_ctanh +declare void %ctanhf("complex float"*,float,float) ;; __builtin_ctanhf +declare void %ctanhl("complex long double"*,double,double) ;; __builtin_ctanhl +declare void %ctanl("complex long double"*,double,double) ;; __builtin_ctanl +declare int %bcmp(sbyte*,sbyte*,uint) ;; __builtin_bcmp +declare void %bcopy(sbyte*,sbyte*,uint) ;; __builtin_bcopy +declare void %bzero(sbyte*,uint) ;; __builtin_bzero +declare int %ffs(int) ;; __builtin_ffs +declare int %ffsl(int) ;; __builtin_ffsl +declare int %ffsll(long) ;; __builtin_ffsll +declare sbyte* %index(sbyte*,int) ;; __builtin_index +declare int %memcmp(sbyte*,sbyte*,uint) ;; __builtin_memcmp +declare sbyte* %memcpy(sbyte*,sbyte*,uint) ;; __builtin_memcpy +declare sbyte* %memmove(sbyte*,sbyte*,uint) ;; __builtin_memmove +declare sbyte* %mempcpy(sbyte*,sbyte*,uint) ;; __builtin_mempcpy +declare sbyte* %memset(sbyte*,int,uint) ;; __builtin_memset +declare sbyte* %rindex(sbyte*,int) ;; __builtin_rindex +declare sbyte* %stpcpy(sbyte*,sbyte*) ;; __builtin_stpcpy +declare sbyte* %strcat(sbyte*,sbyte*) ;; __builtin_strcat +declare sbyte* %strchr(sbyte*,int) ;; __builtin_strchr +declare int %strcmp(sbyte*,sbyte*) ;; __builtin_strcmp +declare sbyte* %strcpy(sbyte*,sbyte*) ;; __builtin_strcpy +declare uint %strcspn(sbyte*,sbyte*) ;; __builtin_strcspn +declare sbyte* %strdup(sbyte*) ;; __builtin_strdup +declare uint %strlen(sbyte*) ;; __builtin_strlen +declare sbyte* %strncat(sbyte*,sbyte*,uint) ;; __builtin_strncat +declare int %strncmp(sbyte*,sbyte*,uint) ;; __builtin_strncmp +declare sbyte* %strncpy(sbyte*,sbyte*,uint) ;; __builtin_strncpy +declare sbyte* %strpbrk(sbyte*,sbyte*) ;; __builtin_strpbrk +declare sbyte* %strrchr(sbyte*,int) ;; __builtin_strrchr +declare uint %strspn(sbyte*,sbyte*) ;; __builtin_strspn +declare sbyte* %strstr(sbyte*,sbyte*) ;; __builtin_strstr +declare int %fprintf(sbyte*,sbyte*, ...) ;; __builtin_fprintf +declare int %fprintf_unlocked(sbyte*,sbyte*, ...) ;; __builtin_fprintf_unlocked +declare int %fputc(int,sbyte*) ;; __builtin_fputc +declare int %fputc_unlocked(int,sbyte*) ;; __builtin_fputc_unlocked +declare int %fputs(sbyte*,sbyte*) ;; __builtin_fputs +declare int %fputs_unlocked(sbyte*,sbyte*) ;; __builtin_fputs_unlocked +declare int %fscanf(sbyte*,sbyte*, ...) ;; __builtin_fscanf +declare uint %fwrite(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite +declare uint %fwrite_unlocked(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite_unlocked +declare int %printf(sbyte*, ...) ;; __builtin_printf +declare int %printf_unlocked(sbyte*, ...) ;; __builtin_printf_unlocked +declare int %putchar(int) ;; __builtin_putchar +declare int %putchar_unlocked(int) ;; __builtin_putchar_unlocked +declare int %puts(sbyte*) ;; __builtin_puts +declare int %puts_unlocked(sbyte*) ;; __builtin_puts_unlocked +declare int %scanf(sbyte*, ...) ;; __builtin_scanf +declare int %snprintf(sbyte*,uint,sbyte*, ...) ;; __builtin_snprintf +declare int %sprintf(sbyte*,sbyte*, ...) ;; __builtin_sprintf +declare int %sscanf(sbyte*,sbyte*, ...) ;; __builtin_sscanf +declare int %vfprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfprintf +declare int %vfscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfscanf +declare int %vprintf(sbyte*,sbyte*) ;; __builtin_vprintf +declare int %vscanf(sbyte*,sbyte*) ;; __builtin_vscanf +declare int %vsnprintf(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_vsnprintf +declare int %vsprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsprintf +declare int %vsscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsscanf +declare void %abort() ;; __builtin_abort +declare int %abs(int) ;; __builtin_abs +declare sbyte* %__builtin_aggregate_incoming_address(...) +declare sbyte* %alloca(uint) ;; __builtin_alloca +declare sbyte* %__builtin_apply(void (...)*,sbyte*,uint) +declare sbyte* %__builtin_apply_args(...) +declare int %__builtin_args_info(int) +declare sbyte* %calloc(uint,uint) ;; __builtin_calloc +declare int %__builtin_classify_type(...) +declare int %__builtin_clz(int) +declare int %__builtin_clzl(int) +declare int %__builtin_clzll(long) +declare int %__builtin_constant_p(...) +declare int %__builtin_ctz(int) +declare int %__builtin_ctzl(int) +declare int %__builtin_ctzll(long) +declare sbyte* %dcgettext(sbyte*,sbyte*,int) ;; __builtin_dcgettext +declare sbyte* %dgettext(sbyte*,sbyte*) ;; __builtin_dgettext +declare sbyte* %__builtin_dwarf_cfa() +declare uint %__builtin_dwarf_sp_column() +declare void %__builtin_eh_return(int,sbyte*) +declare int %__builtin_eh_return_data_regno(int) +declare void %exit(int) ;; __builtin_exit +declare int %__builtin_expect(int,int) +declare sbyte* %__builtin_extract_return_addr(sbyte*) +declare sbyte* %__builtin_frame_address(uint) +declare sbyte* %__builtin_frob_return_addr(sbyte*) +declare sbyte* %gettext(sbyte*) ;; __builtin_gettext +declare long %imaxabs(long) ;; __builtin_imaxabs +declare void %__builtin_init_dwarf_reg_size_table(sbyte*) +declare int %__builtin_isgreater(...) +declare int %__builtin_isgreaterequal(...) +declare int %__builtin_isless(...) +declare int %__builtin_islessequal(...) +declare int %__builtin_islessgreater(...) +declare int %__builtin_isunordered(...) +declare int %labs(int) ;; __builtin_labs +declare long %llabs(long) ;; __builtin_llabs +declare void %__builtin_longjmp(sbyte*,int) +declare sbyte* %malloc(uint) ;; __builtin_malloc +declare sbyte* %__builtin_next_arg(...) +declare int %__builtin_parity(int) +declare int %__builtin_parityl(int) +declare int %__builtin_parityll(long) +declare int %__builtin_popcount(int) +declare int %__builtin_popcountl(int) +declare int %__builtin_popcountll(long) +declare void %__builtin_prefetch(sbyte*, ...) +declare void %__builtin_return(sbyte*) +declare sbyte* %__builtin_return_address(uint) +declare sbyte* %__builtin_saveregs(...) +declare int %__builtin_setjmp(sbyte*) +declare void %__builtin_stdarg_start(sbyte**, ...) +declare int %strfmon(sbyte*,uint,sbyte*, ...) ;; __builtin_strfmon +declare uint %strftime(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_strftime +declare void %__builtin_trap() +declare void %__builtin_unwind_init() +declare void %__builtin_va_copy(sbyte**,sbyte*) +declare void %__builtin_va_end(sbyte**) +declare void %__builtin_va_start(sbyte**, ...) +declare void %_exit(int) ;; __builtin__exit +declare void %_Exit(int) ;; __builtin__Exit + +int %add(int %n, int %y) { +entry: + %n_addr = alloca int ; ty=int* + %y_addr = alloca int ; ty=int* + %result = alloca int ; ty=int* + store int %n, int* %n_addr + store int %y, int* %y_addr + %tmp.0 = load int* %n_addr ; ty=int + %tmp.1 = load int* %y_addr ; ty=int + %tmp.2 = add int %tmp.0, %tmp.1 ; ty=int + store int %tmp.2, int* %result + br label %return +after_ret: + br label %return +return: + %tmp.3 = load int* %result ; ty=int + ret int %tmp.3 +} + +;; Created by "GCC: (GNU) 3.4-llvm 20051104 (LLVM 1.6)" Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/hello.c ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/hello.c Sun Apr 2 13:08:28 2006 @@ -0,0 +1,9 @@ +char *gethellostr() { + return "hello world\n"; +} + +int hello() { + printf(gethellostr()); + return 0; +} + Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/hello.s ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/hello.s Sun Apr 2 13:08:28 2006 @@ -0,0 +1,451 @@ +; GNU C version 3.4-llvm 20051104 (LLVM 1.6) (i686-pc-linux-gnu) +; compiled by GNU C version 3.4.0. +; GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096 +; options passed: -iprefix -mtune=pentiumpro -auxbase +; options enabled: -feliminate-unused-debug-types -fpeephole +; -ffunction-cse -fkeep-static-consts -fpcc-struct-return -fgcse-lm +; -fgcse-sm -fsched-interblock -fsched-spec -fbranch-count-reg -fcommon +; -fgnu-linker -fargument-alias -fzero-initialized-in-bss -fident +; -fmath-errno -ftrapping-math -m80387 -mhard-float -mno-soft-float +; -mieee-fp -mfp-ret-in-387 -maccumulate-outgoing-args -mno-red-zone +; -mtls-direct-seg-refs -mtune=pentiumpro -march=i386 + +target triple = "i686-pc-linux-gnu" +target pointersize = 32 +target endian = little +deplibs = ["c", "crtend"] + +"complex double" = type { double, double } +"complex float" = type { float, float } +"complex long double" = type { double, double } +%.str_1 = internal constant [13 x sbyte] c"hello world\0A\00" + + +implementation +declare double %acos(double) ;; __builtin_acos +declare float %acosf(float) ;; __builtin_acosf +declare double %acosh(double) ;; __builtin_acosh +declare float %acoshf(float) ;; __builtin_acoshf +declare double %acoshl(double) ;; __builtin_acoshl +declare double %acosl(double) ;; __builtin_acosl +declare double %asin(double) ;; __builtin_asin +declare float %asinf(float) ;; __builtin_asinf +declare double %asinh(double) ;; __builtin_asinh +declare float %asinhf(float) ;; __builtin_asinhf +declare double %asinhl(double) ;; __builtin_asinhl +declare double %asinl(double) ;; __builtin_asinl +declare double %atan(double) ;; __builtin_atan +declare double %atan2(double,double) ;; __builtin_atan2 +declare float %atan2f(float,float) ;; __builtin_atan2f +declare double %atan2l(double,double) ;; __builtin_atan2l +declare float %atanf(float) ;; __builtin_atanf +declare double %atanh(double) ;; __builtin_atanh +declare float %atanhf(float) ;; __builtin_atanhf +declare double %atanhl(double) ;; __builtin_atanhl +declare double %atanl(double) ;; __builtin_atanl +declare double %cbrt(double) ;; __builtin_cbrt +declare float %cbrtf(float) ;; __builtin_cbrtf +declare double %cbrtl(double) ;; __builtin_cbrtl +declare double %ceil(double) ;; __builtin_ceil +declare float %ceilf(float) ;; __builtin_ceilf +declare double %ceill(double) ;; __builtin_ceill +declare double %copysign(double,double) ;; __builtin_copysign +declare float %copysignf(float,float) ;; __builtin_copysignf +declare double %copysignl(double,double) ;; __builtin_copysignl +declare double %cos(double) ;; __builtin_cos +declare float %cosf(float) ;; __builtin_cosf +declare double %cosh(double) ;; __builtin_cosh +declare float %coshf(float) ;; __builtin_coshf +declare double %coshl(double) ;; __builtin_coshl +declare double %cosl(double) ;; __builtin_cosl +declare double %drem(double,double) ;; __builtin_drem +declare float %dremf(float,float) ;; __builtin_dremf +declare double %dreml(double,double) ;; __builtin_dreml +declare double %erf(double) ;; __builtin_erf +declare double %erfc(double) ;; __builtin_erfc +declare float %erfcf(float) ;; __builtin_erfcf +declare double %erfcl(double) ;; __builtin_erfcl +declare float %erff(float) ;; __builtin_erff +declare double %erfl(double) ;; __builtin_erfl +declare double %exp(double) ;; __builtin_exp +declare double %exp10(double) ;; __builtin_exp10 +declare float %exp10f(float) ;; __builtin_exp10f +declare double %exp10l(double) ;; __builtin_exp10l +declare double %exp2(double) ;; __builtin_exp2 +declare float %exp2f(float) ;; __builtin_exp2f +declare double %exp2l(double) ;; __builtin_exp2l +declare float %expf(float) ;; __builtin_expf +declare double %expl(double) ;; __builtin_expl +declare double %expm1(double) ;; __builtin_expm1 +declare float %expm1f(float) ;; __builtin_expm1f +declare double %expm1l(double) ;; __builtin_expm1l +declare double %fabs(double) ;; __builtin_fabs +declare float %fabsf(float) ;; __builtin_fabsf +declare double %fabsl(double) ;; __builtin_fabsl +declare double %fdim(double,double) ;; __builtin_fdim +declare float %fdimf(float,float) ;; __builtin_fdimf +declare double %fdiml(double,double) ;; __builtin_fdiml +declare double %floor(double) ;; __builtin_floor +declare float %floorf(float) ;; __builtin_floorf +declare double %floorl(double) ;; __builtin_floorl +declare double %fma(double,double,double) ;; __builtin_fma +declare float %fmaf(float,float,float) ;; __builtin_fmaf +declare double %fmal(double,double,double) ;; __builtin_fmal +declare double %fmax(double,double) ;; __builtin_fmax +declare float %fmaxf(float,float) ;; __builtin_fmaxf +declare double %fmaxl(double,double) ;; __builtin_fmaxl +declare double %fmin(double,double) ;; __builtin_fmin +declare float %fminf(float,float) ;; __builtin_fminf +declare double %fminl(double,double) ;; __builtin_fminl +declare double %fmod(double,double) ;; __builtin_fmod +declare float %fmodf(float,float) ;; __builtin_fmodf +declare double %fmodl(double,double) ;; __builtin_fmodl +declare double %frexp(double,int*) ;; __builtin_frexp +declare float %frexpf(float,int*) ;; __builtin_frexpf +declare double %frexpl(double,int*) ;; __builtin_frexpl +declare double %gamma(double) ;; __builtin_gamma +declare float %gammaf(float) ;; __builtin_gammaf +declare double %gammal(double) ;; __builtin_gammal +declare double %__builtin_huge_val() +declare float %__builtin_huge_valf() +declare double %__builtin_huge_vall() +declare double %hypot(double,double) ;; __builtin_hypot +declare float %hypotf(float,float) ;; __builtin_hypotf +declare double %hypotl(double,double) ;; __builtin_hypotl +declare int %ilogb(double) ;; __builtin_ilogb +declare int %ilogbf(float) ;; __builtin_ilogbf +declare int %ilogbl(double) ;; __builtin_ilogbl +declare double %__builtin_inf() +declare float %__builtin_inff() +declare double %__builtin_infl() +declare double %j0(double) ;; __builtin_j0 +declare float %j0f(float) ;; __builtin_j0f +declare double %j0l(double) ;; __builtin_j0l +declare double %j1(double) ;; __builtin_j1 +declare float %j1f(float) ;; __builtin_j1f +declare double %j1l(double) ;; __builtin_j1l +declare double %jn(int,double) ;; __builtin_jn +declare float %jnf(int,float) ;; __builtin_jnf +declare double %jnl(int,double) ;; __builtin_jnl +declare double %ldexp(double,int) ;; __builtin_ldexp +declare float %ldexpf(float,int) ;; __builtin_ldexpf +declare double %ldexpl(double,int) ;; __builtin_ldexpl +declare double %lgamma(double) ;; __builtin_lgamma +declare float %lgammaf(float) ;; __builtin_lgammaf +declare double %lgammal(double) ;; __builtin_lgammal +declare long %llrint(double) ;; __builtin_llrint +declare long %llrintf(float) ;; __builtin_llrintf +declare long %llrintl(double) ;; __builtin_llrintl +declare long %llround(double) ;; __builtin_llround +declare long %llroundf(float) ;; __builtin_llroundf +declare long %llroundl(double) ;; __builtin_llroundl +declare double %log(double) ;; __builtin_log +declare double %log10(double) ;; __builtin_log10 +declare float %log10f(float) ;; __builtin_log10f +declare double %log10l(double) ;; __builtin_log10l +declare double %log1p(double) ;; __builtin_log1p +declare float %log1pf(float) ;; __builtin_log1pf +declare double %log1pl(double) ;; __builtin_log1pl +declare double %log2(double) ;; __builtin_log2 +declare float %log2f(float) ;; __builtin_log2f +declare double %log2l(double) ;; __builtin_log2l +declare double %logb(double) ;; __builtin_logb +declare float %logbf(float) ;; __builtin_logbf +declare double %logbl(double) ;; __builtin_logbl +declare float %logf(float) ;; __builtin_logf +declare double %logl(double) ;; __builtin_logl +declare int %lrint(double) ;; __builtin_lrint +declare int %lrintf(float) ;; __builtin_lrintf +declare int %lrintl(double) ;; __builtin_lrintl +declare int %lround(double) ;; __builtin_lround +declare int %lroundf(float) ;; __builtin_lroundf +declare int %lroundl(double) ;; __builtin_lroundl +declare double %modf(double,double*) ;; __builtin_modf +declare float %modff(float,float*) ;; __builtin_modff +declare double %modfl(double,double*) ;; __builtin_modfl +declare double %nan(sbyte*) ;; __builtin_nan +declare float %nanf(sbyte*) ;; __builtin_nanf +declare double %nanl(sbyte*) ;; __builtin_nanl +declare double %nans(sbyte*) ;; __builtin_nans +declare float %nansf(sbyte*) ;; __builtin_nansf +declare double %nansl(sbyte*) ;; __builtin_nansl +declare double %nearbyint(double) ;; __builtin_nearbyint +declare float %nearbyintf(float) ;; __builtin_nearbyintf +declare double %nearbyintl(double) ;; __builtin_nearbyintl +declare double %nextafter(double,double) ;; __builtin_nextafter +declare float %nextafterf(float,float) ;; __builtin_nextafterf +declare double %nextafterl(double,double) ;; __builtin_nextafterl +declare double %nexttoward(double,double) ;; __builtin_nexttoward +declare float %nexttowardf(float,double) ;; __builtin_nexttowardf +declare double %nexttowardl(double,double) ;; __builtin_nexttowardl +declare double %pow(double,double) ;; __builtin_pow +declare double %pow10(double) ;; __builtin_pow10 +declare float %pow10f(float) ;; __builtin_pow10f +declare double %pow10l(double) ;; __builtin_pow10l +declare float %powf(float,float) ;; __builtin_powf +declare double %powl(double,double) ;; __builtin_powl +declare double %remainder(double,double) ;; __builtin_remainder +declare float %remainderf(float,float) ;; __builtin_remainderf +declare double %remainderl(double,double) ;; __builtin_remainderl +declare double %remquo(double,double,int*) ;; __builtin_remquo +declare float %remquof(float,float,int*) ;; __builtin_remquof +declare double %remquol(double,double,int*) ;; __builtin_remquol +declare double %rint(double) ;; __builtin_rint +declare float %rintf(float) ;; __builtin_rintf +declare double %rintl(double) ;; __builtin_rintl +declare double %round(double) ;; __builtin_round +declare float %roundf(float) ;; __builtin_roundf +declare double %roundl(double) ;; __builtin_roundl +declare double %scalb(double,double) ;; __builtin_scalb +declare float %scalbf(float,float) ;; __builtin_scalbf +declare double %scalbl(double,double) ;; __builtin_scalbl +declare double %scalbln(double,int) ;; __builtin_scalbln +declare float %scalblnf(float,int) ;; __builtin_scalblnf +declare double %scalblnl(double,int) ;; __builtin_scalblnl +declare double %scalbn(double,int) ;; __builtin_scalbn +declare float %scalbnf(float,int) ;; __builtin_scalbnf +declare double %scalbnl(double,int) ;; __builtin_scalbnl +declare double %significand(double) ;; __builtin_significand +declare float %significandf(float) ;; __builtin_significandf +declare double %significandl(double) ;; __builtin_significandl +declare double %sin(double) ;; __builtin_sin +declare void %sincos(double,double*,double*) ;; __builtin_sincos +declare void %sincosf(float,float*,float*) ;; __builtin_sincosf +declare void %sincosl(double,double*,double*) ;; __builtin_sincosl +declare float %sinf(float) ;; __builtin_sinf +declare double %sinh(double) ;; __builtin_sinh +declare float %sinhf(float) ;; __builtin_sinhf +declare double %sinhl(double) ;; __builtin_sinhl +declare double %sinl(double) ;; __builtin_sinl +declare double %sqrt(double) ;; __builtin_sqrt +declare float %sqrtf(float) ;; __builtin_sqrtf +declare double %sqrtl(double) ;; __builtin_sqrtl +declare double %tan(double) ;; __builtin_tan +declare float %tanf(float) ;; __builtin_tanf +declare double %tanh(double) ;; __builtin_tanh +declare float %tanhf(float) ;; __builtin_tanhf +declare double %tanhl(double) ;; __builtin_tanhl +declare double %tanl(double) ;; __builtin_tanl +declare double %tgamma(double) ;; __builtin_tgamma +declare float %tgammaf(float) ;; __builtin_tgammaf +declare double %tgammal(double) ;; __builtin_tgammal +declare double %trunc(double) ;; __builtin_trunc +declare float %truncf(float) ;; __builtin_truncf +declare double %truncl(double) ;; __builtin_truncl +declare double %y0(double) ;; __builtin_y0 +declare float %y0f(float) ;; __builtin_y0f +declare double %y0l(double) ;; __builtin_y0l +declare double %y1(double) ;; __builtin_y1 +declare float %y1f(float) ;; __builtin_y1f +declare double %y1l(double) ;; __builtin_y1l +declare double %yn(int,double) ;; __builtin_yn +declare float %ynf(int,float) ;; __builtin_ynf +declare double %ynl(int,double) ;; __builtin_ynl +declare double %cabs(double,double) ;; __builtin_cabs +declare float %cabsf(float,float) ;; __builtin_cabsf +declare double %cabsl(double,double) ;; __builtin_cabsl +declare void %cacos("complex double"*,double,double) ;; __builtin_cacos +declare void %cacosf("complex float"*,float,float) ;; __builtin_cacosf +declare void %cacosh("complex double"*,double,double) ;; __builtin_cacosh +declare void %cacoshf("complex float"*,float,float) ;; __builtin_cacoshf +declare void %cacoshl("complex long double"*,double,double) ;; __builtin_cacoshl +declare void %cacosl("complex long double"*,double,double) ;; __builtin_cacosl +declare double %carg(double,double) ;; __builtin_carg +declare float %cargf(float,float) ;; __builtin_cargf +declare double %cargl(double,double) ;; __builtin_cargl +declare void %casin("complex double"*,double,double) ;; __builtin_casin +declare void %casinf("complex float"*,float,float) ;; __builtin_casinf +declare void %casinh("complex double"*,double,double) ;; __builtin_casinh +declare void %casinhf("complex float"*,float,float) ;; __builtin_casinhf +declare void %casinhl("complex long double"*,double,double) ;; __builtin_casinhl +declare void %casinl("complex long double"*,double,double) ;; __builtin_casinl +declare void %catan("complex double"*,double,double) ;; __builtin_catan +declare void %catanf("complex float"*,float,float) ;; __builtin_catanf +declare void %catanh("complex double"*,double,double) ;; __builtin_catanh +declare void %catanhf("complex float"*,float,float) ;; __builtin_catanhf +declare void %catanhl("complex long double"*,double,double) ;; __builtin_catanhl +declare void %catanl("complex long double"*,double,double) ;; __builtin_catanl +declare void %ccos("complex double"*,double,double) ;; __builtin_ccos +declare void %ccosf("complex float"*,float,float) ;; __builtin_ccosf +declare void %ccosh("complex double"*,double,double) ;; __builtin_ccosh +declare void %ccoshf("complex float"*,float,float) ;; __builtin_ccoshf +declare void %ccoshl("complex long double"*,double,double) ;; __builtin_ccoshl +declare void %ccosl("complex long double"*,double,double) ;; __builtin_ccosl +declare void %cexp("complex double"*,double,double) ;; __builtin_cexp +declare void %cexpf("complex float"*,float,float) ;; __builtin_cexpf +declare void %cexpl("complex long double"*,double,double) ;; __builtin_cexpl +declare double %cimag(double,double) ;; __builtin_cimag +declare float %cimagf(float,float) ;; __builtin_cimagf +declare double %cimagl(double,double) ;; __builtin_cimagl +declare void %conj("complex double"*,double,double) ;; __builtin_conj +declare void %conjf("complex float"*,float,float) ;; __builtin_conjf +declare void %conjl("complex long double"*,double,double) ;; __builtin_conjl +declare void %cpow("complex double"*,double,double,double,double) ;; __builtin_cpow +declare void %cpowf("complex float"*,float,float,float,float) ;; __builtin_cpowf +declare void %cpowl("complex long double"*,double,double,double,double) ;; __builtin_cpowl +declare void %cproj("complex double"*,double,double) ;; __builtin_cproj +declare void %cprojf("complex float"*,float,float) ;; __builtin_cprojf +declare void %cprojl("complex long double"*,double,double) ;; __builtin_cprojl +declare double %creal(double,double) ;; __builtin_creal +declare float %crealf(float,float) ;; __builtin_crealf +declare double %creall(double,double) ;; __builtin_creall +declare void %csin("complex double"*,double,double) ;; __builtin_csin +declare void %csinf("complex float"*,float,float) ;; __builtin_csinf +declare void %csinh("complex double"*,double,double) ;; __builtin_csinh +declare void %csinhf("complex float"*,float,float) ;; __builtin_csinhf +declare void %csinhl("complex long double"*,double,double) ;; __builtin_csinhl +declare void %csinl("complex long double"*,double,double) ;; __builtin_csinl +declare void %csqrt("complex double"*,double,double) ;; __builtin_csqrt +declare void %csqrtf("complex float"*,float,float) ;; __builtin_csqrtf +declare void %csqrtl("complex long double"*,double,double) ;; __builtin_csqrtl +declare void %ctan("complex double"*,double,double) ;; __builtin_ctan +declare void %ctanf("complex float"*,float,float) ;; __builtin_ctanf +declare void %ctanh("complex double"*,double,double) ;; __builtin_ctanh +declare void %ctanhf("complex float"*,float,float) ;; __builtin_ctanhf +declare void %ctanhl("complex long double"*,double,double) ;; __builtin_ctanhl +declare void %ctanl("complex long double"*,double,double) ;; __builtin_ctanl +declare int %bcmp(sbyte*,sbyte*,uint) ;; __builtin_bcmp +declare void %bcopy(sbyte*,sbyte*,uint) ;; __builtin_bcopy +declare void %bzero(sbyte*,uint) ;; __builtin_bzero +declare int %ffs(int) ;; __builtin_ffs +declare int %ffsl(int) ;; __builtin_ffsl +declare int %ffsll(long) ;; __builtin_ffsll +declare sbyte* %index(sbyte*,int) ;; __builtin_index +declare int %memcmp(sbyte*,sbyte*,uint) ;; __builtin_memcmp +declare sbyte* %memcpy(sbyte*,sbyte*,uint) ;; __builtin_memcpy +declare sbyte* %memmove(sbyte*,sbyte*,uint) ;; __builtin_memmove +declare sbyte* %mempcpy(sbyte*,sbyte*,uint) ;; __builtin_mempcpy +declare sbyte* %memset(sbyte*,int,uint) ;; __builtin_memset +declare sbyte* %rindex(sbyte*,int) ;; __builtin_rindex +declare sbyte* %stpcpy(sbyte*,sbyte*) ;; __builtin_stpcpy +declare sbyte* %strcat(sbyte*,sbyte*) ;; __builtin_strcat +declare sbyte* %strchr(sbyte*,int) ;; __builtin_strchr +declare int %strcmp(sbyte*,sbyte*) ;; __builtin_strcmp +declare sbyte* %strcpy(sbyte*,sbyte*) ;; __builtin_strcpy +declare uint %strcspn(sbyte*,sbyte*) ;; __builtin_strcspn +declare sbyte* %strdup(sbyte*) ;; __builtin_strdup +declare uint %strlen(sbyte*) ;; __builtin_strlen +declare sbyte* %strncat(sbyte*,sbyte*,uint) ;; __builtin_strncat +declare int %strncmp(sbyte*,sbyte*,uint) ;; __builtin_strncmp +declare sbyte* %strncpy(sbyte*,sbyte*,uint) ;; __builtin_strncpy +declare sbyte* %strpbrk(sbyte*,sbyte*) ;; __builtin_strpbrk +declare sbyte* %strrchr(sbyte*,int) ;; __builtin_strrchr +declare uint %strspn(sbyte*,sbyte*) ;; __builtin_strspn +declare sbyte* %strstr(sbyte*,sbyte*) ;; __builtin_strstr +declare int %fprintf(sbyte*,sbyte*, ...) ;; __builtin_fprintf +declare int %fprintf_unlocked(sbyte*,sbyte*, ...) ;; __builtin_fprintf_unlocked +declare int %fputc(int,sbyte*) ;; __builtin_fputc +declare int %fputc_unlocked(int,sbyte*) ;; __builtin_fputc_unlocked +declare int %fputs(sbyte*,sbyte*) ;; __builtin_fputs +declare int %fputs_unlocked(sbyte*,sbyte*) ;; __builtin_fputs_unlocked +declare int %fscanf(sbyte*,sbyte*, ...) ;; __builtin_fscanf +declare uint %fwrite(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite +declare uint %fwrite_unlocked(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite_unlocked +declare int %printf(sbyte*, ...) ;; __builtin_printf +declare int %printf_unlocked(sbyte*, ...) ;; __builtin_printf_unlocked +declare int %putchar(int) ;; __builtin_putchar +declare int %putchar_unlocked(int) ;; __builtin_putchar_unlocked +declare int %puts(sbyte*) ;; __builtin_puts +declare int %puts_unlocked(sbyte*) ;; __builtin_puts_unlocked +declare int %scanf(sbyte*, ...) ;; __builtin_scanf +declare int %snprintf(sbyte*,uint,sbyte*, ...) ;; __builtin_snprintf +declare int %sprintf(sbyte*,sbyte*, ...) ;; __builtin_sprintf +declare int %sscanf(sbyte*,sbyte*, ...) ;; __builtin_sscanf +declare int %vfprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfprintf +declare int %vfscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfscanf +declare int %vprintf(sbyte*,sbyte*) ;; __builtin_vprintf +declare int %vscanf(sbyte*,sbyte*) ;; __builtin_vscanf +declare int %vsnprintf(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_vsnprintf +declare int %vsprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsprintf +declare int %vsscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsscanf +declare void %abort() ;; __builtin_abort +declare int %abs(int) ;; __builtin_abs +declare sbyte* %__builtin_aggregate_incoming_address(...) +declare sbyte* %alloca(uint) ;; __builtin_alloca +declare sbyte* %__builtin_apply(void (...)*,sbyte*,uint) +declare sbyte* %__builtin_apply_args(...) +declare int %__builtin_args_info(int) +declare sbyte* %calloc(uint,uint) ;; __builtin_calloc +declare int %__builtin_classify_type(...) +declare int %__builtin_clz(int) +declare int %__builtin_clzl(int) +declare int %__builtin_clzll(long) +declare int %__builtin_constant_p(...) +declare int %__builtin_ctz(int) +declare int %__builtin_ctzl(int) +declare int %__builtin_ctzll(long) +declare sbyte* %dcgettext(sbyte*,sbyte*,int) ;; __builtin_dcgettext +declare sbyte* %dgettext(sbyte*,sbyte*) ;; __builtin_dgettext +declare sbyte* %__builtin_dwarf_cfa() +declare uint %__builtin_dwarf_sp_column() +declare void %__builtin_eh_return(int,sbyte*) +declare int %__builtin_eh_return_data_regno(int) +declare void %exit(int) ;; __builtin_exit +declare int %__builtin_expect(int,int) +declare sbyte* %__builtin_extract_return_addr(sbyte*) +declare sbyte* %__builtin_frame_address(uint) +declare sbyte* %__builtin_frob_return_addr(sbyte*) +declare sbyte* %gettext(sbyte*) ;; __builtin_gettext +declare long %imaxabs(long) ;; __builtin_imaxabs +declare void %__builtin_init_dwarf_reg_size_table(sbyte*) +declare int %__builtin_isgreater(...) +declare int %__builtin_isgreaterequal(...) +declare int %__builtin_isless(...) +declare int %__builtin_islessequal(...) +declare int %__builtin_islessgreater(...) +declare int %__builtin_isunordered(...) +declare int %labs(int) ;; __builtin_labs +declare long %llabs(long) ;; __builtin_llabs +declare void %__builtin_longjmp(sbyte*,int) +declare sbyte* %malloc(uint) ;; __builtin_malloc +declare sbyte* %__builtin_next_arg(...) +declare int %__builtin_parity(int) +declare int %__builtin_parityl(int) +declare int %__builtin_parityll(long) +declare int %__builtin_popcount(int) +declare int %__builtin_popcountl(int) +declare int %__builtin_popcountll(long) +declare void %__builtin_prefetch(sbyte*, ...) +declare void %__builtin_return(sbyte*) +declare sbyte* %__builtin_return_address(uint) +declare sbyte* %__builtin_saveregs(...) +declare int %__builtin_setjmp(sbyte*) +declare void %__builtin_stdarg_start(sbyte**, ...) +declare int %strfmon(sbyte*,uint,sbyte*, ...) ;; __builtin_strfmon +declare uint %strftime(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_strftime +declare void %__builtin_trap() +declare void %__builtin_unwind_init() +declare void %__builtin_va_copy(sbyte**,sbyte*) +declare void %__builtin_va_end(sbyte**) +declare void %__builtin_va_start(sbyte**, ...) +declare void %_exit(int) ;; __builtin__exit +declare void %_Exit(int) ;; __builtin__Exit + +sbyte* %gethellostr() { +entry: + %result = alloca sbyte* ; ty=sbyte** + store sbyte* getelementptr ([13 x sbyte]* %.str_1, int 0, int 0), sbyte** %result + br label %return +after_ret: + br label %return +return: + %tmp.1 = load sbyte** %result ; ty=sbyte* + ret sbyte* %tmp.1 +} + + +int %hello() { +entry: + %result = alloca int ; ty=int* + %tmp.2 = call sbyte* (...)* cast (sbyte* ()* %gethellostr to sbyte* (...)*)() ; ty=sbyte* + %tmp.0 = call int (sbyte*, ...)* %printf(sbyte* %tmp.2) ; ty=int + store int 0, int* %result + br label %return +after_ret: + br label %return +return: + %tmp.3 = load int* %result ; ty=int + ret int %tmp.3 +} + +;; Created by "GCC: (GNU) 3.4-llvm 20051104 (LLVM 1.6)" Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/ll_snippet.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/ll_snippet.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,32 @@ +calc = """int %calc(int %n) { + %tmp.0 = call int %add1(int %n) + ret int %tmp.0 +} +declare int %add1(int)""" + +add1 = """int %add1(int %n) { + %tmp.0 = add int %n, 1 + ret int %tmp.0 +}""" + +add1_version2 = """int %add1(int %n) { + %tmp.0 = add int %n, 100 ;used for testing function replacement + ret int %tmp.0 +}""" + +global_int_a_is_100 = """%a = global int 100""" + +add1_to_global_int_a = """ +int %add1_to_global_int_a() { + %tmp.0 = load int* %a + %tmp.1 = add int %tmp.0, 1 + store int %tmp.1, int* %a + ret int %tmp.1 +}""" + +sub10_from_global_int_a = """int %sub10_from_global_int_a() { + %tmp.0 = load int* %a + %tmp.1 = sub int %tmp.0, 10 + store int %tmp.1, int* %a + ret int %tmp.1 +}""" Added: pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Sun Apr 2 13:08:28 2006 @@ -0,0 +1,126 @@ +import py +from pypy.translator.llvm.buildllvm import llvm_is_on_path +if not llvm_is_on_path(): + py.test.skip("llvm not found") + +from pypy.translator.llvm.pythonllvm import pyllvm +from pypy.translator.llvm.pythonllvm.test import ll_snippet + +py.test.skip("WIP") + +def test_execution_engine(): + ee = pyllvm.ExecutionEngine() + ee = pyllvm.ExecutionEngine() + ee = pyllvm.ExecutionEngine() + del ee #XXX not actualy deleted at the moment!!! + ee2 = pyllvm.ExecutionEngine() + ee2 = pyllvm.ExecutionEngine() + ee2 = pyllvm.ExecutionEngine() + +codepath = py.path.local(__file__).dirpath() + +def test_load(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + ee.parse(codepath.join("addnumbers.s").read()) + +def test_functions(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + assert ee.n_functions() == 2 + #TODO + #for function in functions: + # returnId, name, args = function + # assert len(function) == 3 + # assert returnId > 0 + # assert name in ('gethellostr', 'hello') + # assert len(args) == 0 + py.test.raises(Exception, ee.n_functions, 1) + py.test.raises(Exception, ee.n_functions, "string") + +def test_call_parse_once(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + assert ee.call("hello") == 0 + assert ee.call("gethellostr") == "hello world\n" + py.test.raises(Exception, ee.call) + py.test.raises(Exception, ee.call, 1) + py.test.raises(Exception, ee.call, "gethellostrx") + py.test.raises(Exception, ee.call, "gethellostrx", 1) + py.test.raises(Exception, ee.call, "gethellostr", 1) + +def test_call_parse_twice(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + assert ee.call("gethellostr") == "hello world\n" + ee.parse(codepath.join("addnumbers.s").read()) + assert ee.call("add", 10, 32) == 42 + assert ee.call("gethellostr") == "hello world\n" + py.test.raises(Exception, ee.parse) + py.test.raises(Exception, ee.parse, 1) + py.test.raises(Exception, ee.parse, "abc") + +def test_call_between_parsed_code(): + """we parse add1 last on purpose to see if the JIT resolves + the function at execution time. Not sure if we really need this + particular feature. It appears that 'calc' requires a forward + declaration to add1 otherwise a segfault will occur!""" + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert ee.call("add1", 41) == 42 + assert ee.call("calc", 122) == 123 + +def test_replace_function(): + """similar to test_call_between_parsed_code with additional complexity + because we rebind the add1 function to another version after it the + first version already has been used.""" + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert ee.call("add1", 41) == 42 + assert ee.call("calc", 122) == 123 #XXX need recompileAndRelinkFunction somewhere + ee.parse(ll_snippet.add1_version2, "add1") + assert ee.call("add1", 42) == 142 + assert ee.call("calc", 142) == 242 + +def test_share_data_between_parsed_code(): + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.global_int_a_is_100) + ee.parse(ll_snippet.add1_to_global_int_a) + ee.parse(ll_snippet.sub10_from_global_int_a) + assert ee.call("add1_to_global_int_a") == 101 + assert ee.call("sub10_from_global_int_a") == 91 + assert ee.call("add1_to_global_int_a") == 92 + assert ee.call("sub10_from_global_int_a") == 82 + +def test_native_code(): #examine JIT generate native (assembly) code + pyllvm.toggle_print_machineinstrs() + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert ee.call("calc", 41) == 42 + pyllvm.toggle_print_machineinstrs() + +def test_delete_function(): #this will only work if nothing uses Fn of course! + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert ee.n_functions() == 2 + + ee.delete("calc") + assert ee.n_functions() == 1 + assert ee.call("add1", 41) == 42 + + ee.delete("add1") + assert ee.n_functions() == 0 + + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert ee.call("calc", 100) == 101 + +def TODOtest_add_to_function(): + pass + +def TODOtest_optimize_functions(): #add/del/list llvm transformation passes + pass From arigo at codespeak.net Sun Apr 2 14:32:41 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 2 Apr 2006 14:32:41 +0200 (CEST) Subject: [pypy-svn] r25222 - pypy/dist/pypy/objspace/std/test Message-ID: <20060402123241.76A19100C3@code0.codespeak.net> Author: arigo Date: Sun Apr 2 14:32:40 2006 New Revision: 25222 Modified: pypy/dist/pypy/objspace/std/test/test_unicodeobject.py Log: Workaround for inspect.py bug in Python 2.4.2 and 2.4.3. Modified: pypy/dist/pypy/objspace/std/test/test_unicodeobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/test/test_unicodeobject.py (original) +++ pypy/dist/pypy/objspace/std/test/test_unicodeobject.py Sun Apr 2 14:32:40 2006 @@ -33,6 +33,7 @@ assert u'' in 'abc' assert u'bc' in 'abc' assert 'bc' in 'abc' + pass # workaround for inspect.py bug in some Python 2.4s def test_contains(self): assert u'a' in 'abc' From arigo at codespeak.net Sun Apr 2 15:46:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 2 Apr 2006 15:46:05 +0200 (CEST) Subject: [pypy-svn] r25223 - in pypy/dist/pypy: tool/algo tool/algo/test translator/tool Message-ID: <20060402134605.7FB3F100C8@code0.codespeak.net> Author: arigo Date: Sun Apr 2 15:46:04 2006 New Revision: 25223 Added: pypy/dist/pypy/tool/algo/multiweakdict.py pypy/dist/pypy/tool/algo/test/test_multiweakdict.py Modified: pypy/dist/pypy/translator/tool/reftracker.py Log: (arre, pedronis, arigo) * fix the reference tracker to display non-negative id's * a MultiWeakKeyDictionary: like a WeakKeyDictionary but with tuples as keys; when any item of a key is deallocated, the whole entry is removed from the dict. NB: we're not sure at all we'll actually use it, but we had fun writing it anyway. Added: pypy/dist/pypy/tool/algo/multiweakdict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/tool/algo/multiweakdict.py Sun Apr 2 15:46:04 2006 @@ -0,0 +1,45 @@ +import weakref +import UserDict + + +class MultiWeakKeyDictionary(UserDict.DictMixin): + + def __init__(self): + self._bylength = {} + + def __getitem__(self, key): + key = (len(key),) + key + d = self._bylength + for step in key: + d = d[step] + return d + + def __setitem__(self, key, value): + key = (len(key),) + key + d = self._bylength + for step in key[:-1]: + try: + d = d[step] + except KeyError: + d[step] = newd = weakref.WeakKeyDictionary() + d = newd + d[key[-1]] = value + + def __delitem__(self, key): + key = (len(key),) + key + d = self._bylength + for step in key[:-1]: + d = d[step] + del d[key[-1]] + + def keys(self): + result = [] + def enumkeys(initialkey, d, result): + if len(initialkey) == length: + result.append(initialkey) + else: + for key, value in d.iteritems(): + enumkeys(initialkey + (key,), value, result) + for length, d in self._bylength.iteritems(): + enumkeys((), d, result) + return result Added: pypy/dist/pypy/tool/algo/test/test_multiweakdict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/tool/algo/test/test_multiweakdict.py Sun Apr 2 15:46:04 2006 @@ -0,0 +1,52 @@ +import py, gc +from pypy.tool.algo.multiweakdict import MultiWeakKeyDictionary + + +class A(object): + pass + + +def test_simple(): + d = MultiWeakKeyDictionary() + a1 = A() + a2 = A() + a3 = A() + d[a1, a2] = 12 + d[a1,] = 1 + d[a2,] = 2 + d[a2, a1] = 21 + d[a2, a2] = 22 + d[()] = 0 + assert d[a1, a2] == 12 + assert d[a1,] == 1 + assert d[a2,] == 2 + assert d[a2, a1] == 21 + assert d[a2, a2] == 22 + assert d[()] == 0 + assert dict.fromkeys(d.keys()) == {(a1, a2): None, + (a1,): None, + (a2,): None, + (a2, a1): None, + (a2, a2): None, + (): None} + del d[a2,] + assert dict.fromkeys(d.keys()) == {(a1, a2): None, + (a1,): None, + (a2, a1): None, + (a2, a2): None, + (): None} + assert d[a1, a2] == 12 + assert d[a1,] == 1 + assert d[a2, a1] == 21 + assert d[a2, a2] == 22 + assert d[()] == 0 + py.test.raises(KeyError, "d[a2,]") + + del a1 + locals() # obscure fix for CPython -- make sure a1 is no longer in + # the cached f_locals of the frame + gc.collect() # less obscure fix for other Python implementations + assert dict.fromkeys(d.keys()) == {(a2, a2): None, + (): None} + assert d[a2, a2] == 22 + assert d[()] == 0 Modified: pypy/dist/pypy/translator/tool/reftracker.py ============================================================================== --- pypy/dist/pypy/translator/tool/reftracker.py (original) +++ pypy/dist/pypy/translator/tool/reftracker.py Sun Apr 2 15:46:04 2006 @@ -6,6 +6,7 @@ import autopath import gc from pypy.translator.tool.graphpage import GraphPage, DotGen +from pypy.tool.uid import uid MARKER = object() @@ -22,7 +23,7 @@ edges = {} def addedge(o1, o2): - key = (id(o1), id(o2)) + key = (uid(o1), uid(o2)) slst = [] if type(o1) in (list, tuple): for i in range(len(o1)): @@ -36,7 +37,7 @@ for i in range(1, len(objectlist)): s = repr(objectlist[i]) - word = '0x%x' % id(objectlist[i]) + word = '0x%x' % uid(objectlist[i]) if len(s) > 50: self.links[word] = s s = s[:20] + ' ... ' + s[-20:] @@ -45,12 +46,12 @@ s) nodename = 'node%d' % len(nodes) dotgen.emit_node(nodename, label=s, shape="box") - nodes[id(objectlist[i])] = nodename + nodes[uid(objectlist[i])] = nodename for o2 in gc.get_referents(objectlist[i]): if o2 is None: continue addedge(objectlist[i], o2) - id2typename[id(o2)] = type(o2).__name__ + id2typename[uid(o2)] = type(o2).__name__ del o2 for o2 in gc.get_referrers(objectlist[i]): if o2 is None: @@ -58,7 +59,7 @@ if type(o2) is list and o2 and o2[0] is MARKER: continue addedge(o2, objectlist[i]) - id2typename[id(o2)] = type(o2).__name__ + id2typename[uid(o2)] = type(o2).__name__ del o2 for ids, label in edges.items(): @@ -81,10 +82,10 @@ objectlist = self.objectlist for i in range(1, len(objectlist)): for o2 in gc.get_referents(objectlist[i]): - if id(o2) == id1: + if uid(o2) == id1: found = o2 for o2 in gc.get_referrers(objectlist[i]): - if id(o2) == id1: + if uid(o2) == id1: found = o2 if found is not None: objectlist = objectlist + [found] From arigo at codespeak.net Sun Apr 2 16:37:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 2 Apr 2006 16:37:01 +0200 (CEST) Subject: [pypy-svn] r25227 - in pypy/branch/jit-timeshifter-refactoring/timeshifter: . test Message-ID: <20060402143701.561D8100C0@code0.codespeak.net> Author: arigo Date: Sun Apr 2 16:37:00 2006 New Revision: 25227 Added: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py Log: (pedronis, arre, arigo) intermediate. Added: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- (empty file) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Sun Apr 2 16:37:00 2006 @@ -0,0 +1,93 @@ +from pypy.jit.timeshifter import rvalue + + +class AbstractContainer(object): + pass + +# ____________________________________________________________ + +class cachedtype(type): + """Metaclass for classes that should only have one instance per + tuple of arguments given to the constructor.""" + + def __init__(selfcls, name, bases, dict): + super(cachedtype, selfcls).__init__(name, bases, dict) + selfcls._instancecache = {} + + def __call__(selfcls, *args): + d = selfcls._instancecache + try: + return d[args] + except KeyError: + instance = d[args] = selfcls.__new__(selfcls, *args) + instance.__init__(*args) + return instance + + +class StructTypeDesc(object): + __metaclass__ = cachedtype + + def __init__(self, TYPE): + self.TYPE = TYPE + self.PTRTYPE = lltype.Ptr(TYPE) + self.gv_type = rgenop.constTYPE(self.TYPE) + self.gv_ptrtype = rgenop.constTYPE(self.PTRTYPE) + + fielddescs = [] + for name in self.TYPE._names: + FIELDTYPE = getattr(self.TYPE, name) + if isinstance(FIELDTYPE, lltype.ContainerType): + substructdesc = StructTypeDesc(FIELDTYPE) + for subfielddesc in substructdesc.fielddescs: + dottedname = '%s.%s' % (name, subfielddesc.fieldname) + index = len(fielddesc) + fielddescs.append(StructFieldDesc(self.TYPE, dottedname, + index)) + else: + index = len(fielddesc) + fielddesc.append(StructFieldDesc(self.TYPE, name, index)) + self.fielddescs = fielddescs + + def _freeze_(self): + return True + + def compact_repr(self): # goes in ll helper names + return "Desc_%s" % (self.TYPE._short_name(),) + + +class StructFieldDesc(object): + __metaclass__ = cachedtype + + def __init__(self, PTRTYPE, fieldname, index): + assert isinstance(PTRTYPE.TO, lltype.Struct) + RES1 = PTRTYPE.TO + for component in fieldname.split('.'): + LASTSTRUCT = RES1 + RES1 = getattr(RES1, component) + assert not isinstance(RES1, lltype.ContainerType) + self.PTRTYPE = PTRTYPE + self.RESTYPE = RES1 + self.gv_resulttype = rgenop.constTYPE(RESTYPE) + self.fieldname = fieldname + self.fieldname_gv = [rgenop.constFieldName(component) + for component in fieldname.split('.')] + self.fieldindex = index + self.gv_default = rgenop.genconst(RES1._defl()) + self.redboxcls = rvalue.ll_redboxcls(RES1) + self.immutable = LASTSTRUCT._hints.get('immutable', False) + + def _freeze_(self): + return True + + def compact_repr(self): # goes in ll helper names + return "Fld_%s_in_%s" % (self.fieldname.replace('.','_'), + self.PTRTYPE._short_name()) + + +class VirtualStruct(AbstractContainer): + + def __init__(self, typedesc): + self.typedesc = typedesc + self.content_boxes = [desc.redboxcls(desc.gv_resulttype, + desc.gv_default) + for desc in typedesc.fielddescs] Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py Sun Apr 2 16:37:00 2006 @@ -343,6 +343,7 @@ assert insns == {} def test_simple_struct_malloc(): + py.test.skip("blue containers: to be reimplemented") S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), ('world', lltype.Signed)) def ll_function(x): @@ -359,6 +360,7 @@ assert insns == {} def test_inlined_substructure(): + py.test.skip("blue containers: to be reimplemented") S = lltype.Struct('S', ('n', lltype.Signed)) T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) def ll_function(k): From pedronis at codespeak.net Sun Apr 2 17:53:49 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 17:53:49 +0200 (CEST) Subject: [pypy-svn] r25228 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402155349.1E4CA100CE@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 17:53:47 2006 New Revision: 25228 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py Log: (arre, pedronis) fixed getfield/getsubstruct at least for the non-virtual case. plus minor typo fixing. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Sun Apr 2 17:53:47 2006 @@ -1,3 +1,6 @@ +import operator +from pypy.rpython.lltypesystem import lltype +from pypy.rpython import rgenop from pypy.jit.timeshifter import rvalue @@ -40,20 +43,53 @@ substructdesc = StructTypeDesc(FIELDTYPE) for subfielddesc in substructdesc.fielddescs: dottedname = '%s.%s' % (name, subfielddesc.fieldname) - index = len(fielddesc) - fielddescs.append(StructFieldDesc(self.TYPE, dottedname, + index = len(fielddescs) + fielddescs.append(StructFieldDesc(self.PTRTYPE, dottedname, index)) else: - index = len(fielddesc) - fielddesc.append(StructFieldDesc(self.TYPE, name, index)) + index = len(fielddescs) + fielddescs.append(StructFieldDesc(self.PTRTYPE, name, index)) self.fielddescs = fielddescs + def getfielddesc(self, name): + index = operator.indexOf(self.TYPE._names, name) + return self.fielddescs[index] + def _freeze_(self): return True def compact_repr(self): # goes in ll helper names return "Desc_%s" % (self.TYPE._short_name(),) +# XXX basic field descs for now +class FieldDesc(object): + __metaclass__ = cachedtype + + def __init__(self, PTRTYPE, RESTYPE): + self.PTRTYPE = PTRTYPE + if isinstance(RESTYPE, lltype.ContainerType): + RESTYPE = lltype.Ptr(RESTYPE) + self.RESTYPE = RESTYPE + self.gv_resulttype = rgenop.constTYPE(RESTYPE) + self.redboxcls = rvalue.ll_redboxcls(RESTYPE) + self.immutable = PTRTYPE.TO._hints.get('immutable', False) + + def _freeze_(self): + return True + +class NamedFieldDesc(FieldDesc): + + def __init__(self, PTRTYPE, name): + FieldDesc.__init__(self, PTRTYPE, getattr(PTRTYPE.TO, name)) + self.fieldname = name + self.gv_fieldname = rgenop.constFieldName(name) + +class ArrayFieldDesc(FieldDesc): + def __init__(self, PTRTYPE): + assert isinstance(PTRTYPE.TO, lltype.Array) + FieldDesc.__init__(self, PTRTYPE, PTRTYPE.TO.OF) + +# ____________________________________________________________ class StructFieldDesc(object): __metaclass__ = cachedtype @@ -67,7 +103,7 @@ assert not isinstance(RES1, lltype.ContainerType) self.PTRTYPE = PTRTYPE self.RESTYPE = RES1 - self.gv_resulttype = rgenop.constTYPE(RESTYPE) + self.gv_resulttype = rgenop.constTYPE(RES1) self.fieldname = fieldname self.fieldname_gv = [rgenop.constFieldName(component) for component in fieldname.split('.')] Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sun Apr 2 17:53:47 2006 @@ -515,152 +515,152 @@ opdesc.gv_RESULT) return opdesc.redboxcls(opdesc.gv_RESULT, genvar) -class StructTypeDesc(object): - _type_cache = weakref.WeakKeyDictionary() +## class StructTypeDesc(object): +## _type_cache = weakref.WeakKeyDictionary() - def __init__(self, TYPE): - self.TYPE = TYPE - self.PTRTYPE = lltype.Ptr(TYPE) - self.gv_type = rgenop.constTYPE(self.TYPE) - self.gv_ptrtype = rgenop.constTYPE(self.PTRTYPE) - - def setup(self): - self.fielddescs = [StructFieldDesc.make(self.PTRTYPE, name) - for name in self.TYPE._names] - defls = [] - for desc in self.fielddescs: - if desc.inlined_typedesc is not None: - defaultbox = None - else: - defaultvalue = desc.RESTYPE._defl() - defaultbox = rvalue.ll_fromvalue(defaultvalue) - defls.append(defaultbox) - self.default_boxes = defls - - def build_content_boxes(self, parentbox): - # make a'content_boxes' list based on the typedesc's default_boxes, - # building nested SubVirtualRedBoxes for inlined substructs - clist = [] - for i in range(len(self.fielddescs)): - fielddesc = self.fielddescs[i] - if fielddesc.inlined_typedesc: - box = SubVirtualRedBox(parentbox, fielddesc) - else: - box = self.default_boxes[i] - clist.append(box) - return clist - - def compare_content_boxes(self, content_boxes_1, content_boxes_2): - for i in range(len(self.fielddescs)): - fielddesc = self.fielddescs[i] - if fielddesc.inlined_typedesc: - box1 = content_boxes_1[i] - box2 = content_boxes_2[i] - assert isinstance(box1, BigRedBox) - assert isinstance(box2, BigRedBox) - if not fielddesc.inlined_typedesc.compare_content_boxes( - box1.content_boxes, box2.content_boxes): - return False - else: - return True - - def materialize_content(self, jitstate, gv, boxes): - for i in range(len(boxes)): - smallbox = boxes[i] - fielddesc = self.fielddescs[i] - if fielddesc.inlined_typedesc: - op_args = lltype.malloc(VARLIST.TO, 2) - op_args[0] = gv - op_args[1] = fielddesc.gv_fieldname - gv_sub = rgenop.genop(jitstate.curblock, 'getsubstruct', - op_args, fielddesc.gv_resulttype) - assert isinstance(smallbox, SubVirtualRedBox) - subboxes = smallbox.content_boxes - smallbox.content_boxes = None - fielddesc.inlined_typedesc.materialize_content(jitstate, - gv_sub, - subboxes) - else: - op_args = lltype.malloc(VARLIST.TO, 3) - op_args[0] = gv - op_args[1] = fielddesc.gv_fieldname - op_args[2] = smallbox.getgenvar(jitstate) - rgenop.genop(jitstate.curblock, 'setfield', op_args, - gv_Void) - - def make(T): - try: - return StructTypeDesc._type_cache[T] - except KeyError: - desc = StructTypeDesc._type_cache[T] = StructTypeDesc(T) - desc.setup() - return desc - make = staticmethod(make) - - def ll_factory(self): - return VirtualRedBox(self) - - def _freeze_(self): - return True - - def compact_repr(self): # goes in ll helper names - return "Desc_%s" % (self.TYPE._short_name(),) - -class FieldDesc(object): - _fielddesc_cache = weakref.WeakKeyDictionary() - - def __init__(self, PTRTYPE, RESTYPE): - self.PTRTYPE = PTRTYPE - if isinstance(RESTYPE, lltype.ContainerType): - RESTYPE = lltype.Ptr(RESTYPE) - self.RESTYPE = RESTYPE - self.gv_resulttype = rgenop.constTYPE(RESTYPE) - self.redboxcls = rvalue.ll_redboxcls(RESTYPE) - self.immutable = PTRTYPE.TO._hints.get('immutable', False) - - def _freeze_(self): - return True - - def make(cls, PTRTYPE, *args): - T = PTRTYPE.TO - cache = FieldDesc._fielddesc_cache.setdefault(T, {}) - try: - return cache[args] - except KeyError: - fdesc = cache[args] = cls(PTRTYPE, *args) - fdesc.setup() - return fdesc - make = classmethod(make) - -class StructFieldDesc(FieldDesc): - def __init__(self, PTRTYPE, fieldname): - assert isinstance(PTRTYPE.TO, lltype.Struct) - RES1 = getattr(PTRTYPE.TO, fieldname) - FieldDesc.__init__(self, PTRTYPE, RES1) - self.fieldname = fieldname - self.gv_fieldname = rgenop.constFieldName(fieldname) - self.fieldindex = operator.indexOf(PTRTYPE.TO._names, fieldname) - if isinstance(RES1, lltype.Struct): - # inlined substructure - self.inlined_typedesc = StructTypeDesc.make(RES1) -## elif isinstance(RES1, lltype.Array): -## # inlined array XXX in-progress -## self.inlined_typedesc = ArrayTypeDesc.make(RES1) - else: - self.inlined_typedesc = None - - def setup(self): - self.parenttypedesc = StructTypeDesc.make(self.PTRTYPE.TO) - - def compact_repr(self): # goes in ll helper names - return "Fld_%s_in_%s" % (self.fieldname, self.PTRTYPE._short_name()) - -class ArrayFieldDesc(FieldDesc): - def __init__(self, PTRTYPE): - assert isinstance(PTRTYPE.TO, lltype.Array) - FieldDesc.__init__(self, PTRTYPE, PTRTYPE.TO.OF) - def setup(self): - pass +## def __init__(self, TYPE): +## self.TYPE = TYPE +## self.PTRTYPE = lltype.Ptr(TYPE) +## self.gv_type = rgenop.constTYPE(self.TYPE) +## self.gv_ptrtype = rgenop.constTYPE(self.PTRTYPE) + +## def setup(self): +## self.fielddescs = [StructFieldDesc.make(self.PTRTYPE, name) +## for name in self.TYPE._names] +## defls = [] +## for desc in self.fielddescs: +## if desc.inlined_typedesc is not None: +## defaultbox = None +## else: +## defaultvalue = desc.RESTYPE._defl() +## defaultbox = rvalue.ll_fromvalue(defaultvalue) +## defls.append(defaultbox) +## self.default_boxes = defls + +## def build_content_boxes(self, parentbox): +## # make a'content_boxes' list based on the typedesc's default_boxes, +## # building nested SubVirtualRedBoxes for inlined substructs +## clist = [] +## for i in range(len(self.fielddescs)): +## fielddesc = self.fielddescs[i] +## if fielddesc.inlined_typedesc: +## box = SubVirtualRedBox(parentbox, fielddesc) +## else: +## box = self.default_boxes[i] +## clist.append(box) +## return clist + +## def compare_content_boxes(self, content_boxes_1, content_boxes_2): +## for i in range(len(self.fielddescs)): +## fielddesc = self.fielddescs[i] +## if fielddesc.inlined_typedesc: +## box1 = content_boxes_1[i] +## box2 = content_boxes_2[i] +## assert isinstance(box1, BigRedBox) +## assert isinstance(box2, BigRedBox) +## if not fielddesc.inlined_typedesc.compare_content_boxes( +## box1.content_boxes, box2.content_boxes): +## return False +## else: +## return True + +## def materialize_content(self, jitstate, gv, boxes): +## for i in range(len(boxes)): +## smallbox = boxes[i] +## fielddesc = self.fielddescs[i] +## if fielddesc.inlined_typedesc: +## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args[0] = gv +## op_args[1] = fielddesc.gv_fieldname +## gv_sub = rgenop.genop(jitstate.curblock, 'getsubstruct', +## op_args, fielddesc.gv_resulttype) +## assert isinstance(smallbox, SubVirtualRedBox) +## subboxes = smallbox.content_boxes +## smallbox.content_boxes = None +## fielddesc.inlined_typedesc.materialize_content(jitstate, +## gv_sub, +## subboxes) +## else: +## op_args = lltype.malloc(VARLIST.TO, 3) +## op_args[0] = gv +## op_args[1] = fielddesc.gv_fieldname +## op_args[2] = smallbox.getgenvar(jitstate) +## rgenop.genop(jitstate.curblock, 'setfield', op_args, +## gv_Void) + +## def make(T): +## try: +## return StructTypeDesc._type_cache[T] +## except KeyError: +## desc = StructTypeDesc._type_cache[T] = StructTypeDesc(T) +## desc.setup() +## return desc +## make = staticmethod(make) + +## def ll_factory(self): +## return VirtualRedBox(self) + +## def _freeze_(self): +## return True + +## def compact_repr(self): # goes in ll helper names +## return "Desc_%s" % (self.TYPE._short_name(),) + +## class FieldDesc(object): +## _fielddesc_cache = weakref.WeakKeyDictionary() + +## def __init__(self, PTRTYPE, RESTYPE): +## self.PTRTYPE = PTRTYPE +## if isinstance(RESTYPE, lltype.ContainerType): +## RESTYPE = lltype.Ptr(RESTYPE) +## self.RESTYPE = RESTYPE +## self.gv_resulttype = rgenop.constTYPE(RESTYPE) +## self.redboxcls = rvalue.ll_redboxcls(RESTYPE) +## self.immutable = PTRTYPE.TO._hints.get('immutable', False) + +## def _freeze_(self): +## return True + +## def make(cls, PTRTYPE, *args): +## T = PTRTYPE.TO +## cache = FieldDesc._fielddesc_cache.setdefault(T, {}) +## try: +## return cache[args] +## except KeyError: +## fdesc = cache[args] = cls(PTRTYPE, *args) +## fdesc.setup() +## return fdesc +## make = classmethod(make) + +## class StructFieldDesc(FieldDesc): +## def __init__(self, PTRTYPE, fieldname): +## assert isinstance(PTRTYPE.TO, lltype.Struct) +## RES1 = getattr(PTRTYPE.TO, fieldname) +## FieldDesc.__init__(self, PTRTYPE, RES1) +## self.fieldname = fieldname +## self.gv_fieldname = rgenop.constFieldName(fieldname) +## self.fieldindex = operator.indexOf(PTRTYPE.TO._names, fieldname) +## if isinstance(RES1, lltype.Struct): +## # inlined substructure +## self.inlined_typedesc = StructTypeDesc.make(RES1) +## ## elif isinstance(RES1, lltype.Array): +## ## # inlined array XXX in-progress +## ## self.inlined_typedesc = ArrayTypeDesc.make(RES1) +## else: +## self.inlined_typedesc = None + +## def setup(self): +## self.parenttypedesc = StructTypeDesc.make(self.PTRTYPE.TO) + +## def compact_repr(self): # goes in ll helper names +## return "Fld_%s_in_%s" % (self.fieldname, self.PTRTYPE._short_name()) + +## class ArrayFieldDesc(FieldDesc): +## def __init__(self, PTRTYPE): +## assert isinstance(PTRTYPE.TO, lltype.Array) +## FieldDesc.__init__(self, PTRTYPE, PTRTYPE.TO.OF) +## def setup(self): +## pass def ll_generate_getfield(jitstate, fielddesc, argbox): if fielddesc.immutable and argbox.is_constant(): @@ -671,7 +671,7 @@ if argbox.content is None: op_args = lltype.malloc(VARLIST.TO, 2) op_args[0] = argbox.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname + op_args[1] = fielddesc.fieldname_gv[-1] genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, fielddesc.gv_resulttype) return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) @@ -689,7 +689,16 @@ res = getattr(rvalue.ll_getvalue(argbox, fielddesc.PTRTYPE), fielddesc.fieldname) return rvalue.ll_fromvalue(res) - return argbox.op_getsubstruct(jitstate, fielddesc) + assert isinstance(argbox, rvalue.PtrRedBox) + if argbox.content is None: + op_args = lltype.malloc(VARLIST.TO, 2) + op_args[0] = argbox.getgenvar(jitstate) + op_args[1] = fielddesc.gv_fieldname + genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, + fielddesc.gv_resulttype) + return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) + else: + return argbox.content.op_getsubstruct(jitstate, fielddesc) def ll_generate_getarrayitem(jitstate, fielddesc, argbox, indexbox): Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py Sun Apr 2 17:53:47 2006 @@ -10,7 +10,7 @@ from pypy.rpython import rgenop from pypy.jit.hintannotator import model as hintmodel from pypy.jit.hintannotator import container as hintcontainer -from pypy.jit.timeshifter import rtimeshift, rvalue +from pypy.jit.timeshifter import rtimeshift, rvalue, rcontainer class HintTypeSystem(TypeSystem): name = "hinttypesystem" @@ -122,7 +122,7 @@ ts = self.timeshifter v_argbox, c_fieldname = hop.inputargs(self.getredrepr(PTRTYPE), green_void_repr) - fielddesc = rtimeshift.StructFieldDesc.make(PTRTYPE, c_fieldname.value) + fielddesc = rcontainer.StructTypeDesc(PTRTYPE.TO).getfielddesc(c_fieldname.value) c_fielddesc = inputconst(lltype.Void, fielddesc) s_fielddesc = ts.rtyper.annotator.bookkeeper.immutablevalue(fielddesc) v_jitstate = hop.llops.getjitstate() @@ -142,7 +142,7 @@ ts = self.timeshifter v_argbox, v_index = hop.inputargs(self.getredrepr(PTRTYPE), self.getredrepr(lltype.Signed)) - fielddesc = rtimeshift.ArrayFieldDesc.make(PTRTYPE) + fielddesc = rcontainer.ArrayFieldDesc(PTRTYPE) c_fielddesc = inputconst(lltype.Void, fielddesc) s_fielddesc = ts.rtyper.annotator.bookkeeper.immutablevalue(fielddesc) v_jitstate = hop.llops.getjitstate() @@ -163,7 +163,7 @@ green_void_repr, self.getredrepr(VALUETYPE) ) - fielddesc = rtimeshift.StructFieldDesc.make(PTRTYPE, c_fieldname.value) + fielddesc = rcontainer.StructTypeDesc(PTRTYPE.TO).getfielddesc(c_fieldname.value) c_fielddesc = inputconst(lltype.Void, fielddesc) s_fielddesc = ts.rtyper.annotator.bookkeeper.immutablevalue(fielddesc) v_jitstate = hop.llops.getjitstate() @@ -181,7 +181,7 @@ PTRTYPE = originalconcretetype(hop.args_s[0]) v_argbox, c_fieldname = hop.inputargs(self.getredrepr(PTRTYPE), green_void_repr) - fielddesc = rtimeshift.StructFieldDesc.make(PTRTYPE, c_fieldname.value) + fielddesc = rcontainer.NamedFieldDesc(PTRTYPE, c_fieldname.value) # XXX c_fielddesc = inputconst(lltype.Void, fielddesc) s_fielddesc = ts.rtyper.annotator.bookkeeper.immutablevalue(fielddesc) v_jitstate = hop.llops.getjitstate() From pedronis at codespeak.net Sun Apr 2 19:01:07 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 19:01:07 +0200 (CEST) Subject: [pypy-svn] r25229 - pypy/dist/pypy/rpython Message-ID: <20060402170107.B9930100D1@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 19:01:06 2006 New Revision: 25229 Modified: pypy/dist/pypy/rpython/rgenop.py Log: (arre, pedronis) adding some useful type and constant defs to rgenop directly. Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Sun Apr 2 19:01:06 2006 @@ -230,7 +230,24 @@ fields = tuple(zip(fieldnames, lltypes)) LINKPAIR = lltype.GcStruct('tuple2', *fields) +# support constants and types + nullvar = lltype.nullptr(CONSTORVAR.TO) +gv_Void = constTYPE(lltype.Void) + +# VARLIST +def ll_fixed_items(l): + return l + +def ll_fixed_length(l): + return len(l) + +VARLIST = lltype.Ptr(lltype.GcArray(CONSTORVAR, + adtmeths = { + "ll_items": ll_fixed_items, + "ll_length": ll_fixed_length + })) + # helpers def setannotation(func, TYPE): From pedronis at codespeak.net Sun Apr 2 19:03:30 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 19:03:30 +0200 (CEST) Subject: [pypy-svn] r25230 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402170330.0DC9C100D3@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 19:03:28 2006 New Revision: 25230 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: (arre, pedronis) fixed red repr create and added enter_block, op_get/setfield, and build_runtime_container to VirtualStruct. Some small fixes and using of constants moved to rgenop too. More tests pass again. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Sun Apr 2 19:03:28 2006 @@ -3,8 +3,7 @@ from pypy.rpython import rgenop from pypy.jit.timeshifter import rvalue - -class AbstractContainer(object): +class AbstractContainer(rvalue.Memoizable): pass # ____________________________________________________________ @@ -55,6 +54,9 @@ index = operator.indexOf(self.TYPE._names, name) return self.fielddescs[index] + def ll_factory(self): + return rvalue.PtrRedBox(self.gv_type, content=VirtualStruct(self)) + def _freeze_(self): return True @@ -97,8 +99,10 @@ def __init__(self, PTRTYPE, fieldname, index): assert isinstance(PTRTYPE.TO, lltype.Struct) RES1 = PTRTYPE.TO + accessptrtype_gv = self.accessptrtype_gv = [] for component in fieldname.split('.'): LASTSTRUCT = RES1 + accessptrtype_gv.append(rgenop.constTYPE(lltype.Ptr(LASTSTRUCT))) RES1 = getattr(RES1, component) assert not isinstance(RES1, lltype.ContainerType) self.PTRTYPE = PTRTYPE @@ -119,6 +123,20 @@ return "Fld_%s_in_%s" % (self.fieldname.replace('.','_'), self.PTRTYPE._short_name()) + def generate_set(self, jitstate, genvar, box): + gv_sub = genvar + for i in range(len(self.accessptrtype_gv)-1): + op_args = lltype.malloc(rgenop.VARLIST.TO, 2) + op_args[0] = gv_sub + op_args[1] = self.fieldname_gv[i] + gv_sub = rgenop.genop(jitstate.curblock, 'getsubstruct', + op_args, self.accessptrtype_gv[i+1]) + op_args = lltype.malloc(rgenop.VARLIST.TO, 3) + op_args[0] = gv_sub + op_args[1] = self.fieldname_gv[-1] + op_args[2] = box.getgenvar(jitstate) + rgenop.genop(jitstate.curblock, 'setfield', op_args, + rgenop.gv_Void) class VirtualStruct(AbstractContainer): @@ -127,3 +145,32 @@ self.content_boxes = [desc.redboxcls(desc.gv_resulttype, desc.gv_default) for desc in typedesc.fielddescs] + + def enter_block(self, newblock, incoming, memo): + if self not in memo: + memo[self] = None + for box in self.content_boxes: + box.enter_block(newblock, incoming, memo) + + def op_getfield(self, jitstate, fielddesc): + return self.content_boxes[fielddesc.fieldindex] + + def op_setfield(self, jitstate, fielddesc, valuebox): + self.content_boxes[fielddesc.fieldindex] = valuebox + + def build_runtime_container(self, jitstate): + typedesc = self.typedesc + boxes = self.content_boxes + self.content_boxes = None + op_args = lltype.malloc(rgenop.VARLIST.TO, 1) + op_args[0] = typedesc.gv_type + genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, + typedesc.gv_ptrtype) + fielddescs = typedesc.fielddescs + for i in range(len(fielddescs)): + fielddesc = fielddescs[i] + box = boxes[i] + # xxx a bit inefficient + fielddesc.generate_set(jitstate, genvar, box) + + return genvar Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sun Apr 2 19:03:28 2006 @@ -5,24 +5,10 @@ FOLDABLE_OPS = dict.fromkeys(lloperation.enum_foldable_ops()) -# ____________________________________________________________ -# types and adtmeths - -def ll_fixed_items(l): - return l - -def ll_fixed_length(l): - return len(l) - -VARLIST = lltype.Ptr(lltype.GcArray(rgenop.CONSTORVAR, - adtmeths = { - "ll_items": ll_fixed_items, - "ll_length": ll_fixed_length - })) ##def make_types_const(TYPES): ## n = len(TYPES) -## l = lltype.malloc(VARLIST.TO, n) +## l = lltype.malloc(rgenop.VARLIST.TO, n) ## for i in range(n): ## l[i] = rgenop.constTYPE(TYPES[i]) ## return l @@ -61,7 +47,7 @@ ## # generic implementation of some operations ## def op_getfield(self, jitstate, fielddesc): -## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 2) ## op_args[0] = self.getgenvar(jitstate) ## op_args[1] = fielddesc.gv_fieldname ## genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, @@ -69,15 +55,15 @@ ## return VarRedBox(genvar) ## def op_setfield(self, jitstate, fielddesc, valuebox): -## op_args = lltype.malloc(VARLIST.TO, 3) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 3) ## op_args[0] = self.getgenvar(jitstate) ## op_args[1] = fielddesc.gv_fieldname ## op_args[2] = valuebox.getgenvar(jitstate) ## rgenop.genop(jitstate.curblock, 'setfield', op_args, -## gv_Void) +## rgenop.gv_Void) ## def op_getsubstruct(self, jitstate, fielddesc): -## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 2) ## op_args[0] = self.getgenvar(jitstate) ## op_args[1] = fielddesc.gv_fieldname ## genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, @@ -185,7 +171,7 @@ ## typedesc = self.typedesc ## boxes = self.content_boxes ## self.content_boxes = None -## op_args = lltype.malloc(VARLIST.TO, 1) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 1) ## op_args[0] = typedesc.gv_type ## self.genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, ## typedesc.gv_ptrtype) @@ -269,7 +255,7 @@ ## def getgenvar(self, jitstate): ## gv = self.parentbox.getgenvar(jitstate) -## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 2) ## op_args[0] = gv ## op_args[1] = self.fielddesc.gv_fieldname ## genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, @@ -491,7 +477,7 @@ arg = rvalue.ll_getvalue(argbox, ARG0) res = opdesc.llop(RESULT, arg) return rvalue.ll_fromvalue(res) - op_args = lltype.malloc(VARLIST.TO, 1) + op_args = lltype.malloc(rgenop.VARLIST.TO, 1) op_args[0] = argbox.getgenvar(jitstate) genvar = rgenop.genop(jitstate.curblock, opdesc.opname, op_args, opdesc.gv_RESULT) @@ -508,7 +494,7 @@ arg1 = rvalue.ll_getvalue(argbox1, ARG1) res = opdesc.llop(RESULT, arg0, arg1) return rvalue.ll_fromvalue(res) - op_args = lltype.malloc(VARLIST.TO, 2) + op_args = lltype.malloc(rgenop.VARLIST.TO, 2) op_args[0] = argbox0.getgenvar(jitstate) op_args[1] = argbox1.getgenvar(jitstate) genvar = rgenop.genop(jitstate.curblock, opdesc.opname, op_args, @@ -569,7 +555,7 @@ ## smallbox = boxes[i] ## fielddesc = self.fielddescs[i] ## if fielddesc.inlined_typedesc: -## op_args = lltype.malloc(VARLIST.TO, 2) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 2) ## op_args[0] = gv ## op_args[1] = fielddesc.gv_fieldname ## gv_sub = rgenop.genop(jitstate.curblock, 'getsubstruct', @@ -581,12 +567,12 @@ ## gv_sub, ## subboxes) ## else: -## op_args = lltype.malloc(VARLIST.TO, 3) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 3) ## op_args[0] = gv ## op_args[1] = fielddesc.gv_fieldname ## op_args[2] = smallbox.getgenvar(jitstate) ## rgenop.genop(jitstate.curblock, 'setfield', op_args, -## gv_Void) +## rgenop.gv_Void) ## def make(T): ## try: @@ -669,7 +655,7 @@ return rvalue.ll_fromvalue(res) assert isinstance(argbox, rvalue.PtrRedBox) if argbox.content is None: - op_args = lltype.malloc(VARLIST.TO, 2) + op_args = lltype.malloc(rgenop.VARLIST.TO, 2) op_args[0] = argbox.getgenvar(jitstate) op_args[1] = fielddesc.fieldname_gv[-1] genvar = rgenop.genop(jitstate.curblock, 'getfield', op_args, @@ -678,11 +664,17 @@ else: return argbox.content.op_getfield(jitstate, fielddesc) -gv_Void = rgenop.constTYPE(lltype.Void) - def ll_generate_setfield(jitstate, fielddesc, destbox, valuebox): - destbox.op_setfield(jitstate, fielddesc, valuebox) - + assert isinstance(destbox, rvalue.PtrRedBox) + if destbox.content is None: + op_args = lltype.malloc(rgenop.VARLIST.TO, 3) + op_args[0] = destbox.getgenvar(jitstate) + op_args[1] = fielddesc.fieldname_gv[-1] + op_args[2] = valuebox.getgenvar(jitstate) + rgenop.genop(jitstate.curblock, 'setfield', op_args, + rgenop.gv_Void) + else: + destbox.content.op_setfield(jitstate, fielddesc, valuebox) def ll_generate_getsubstruct(jitstate, fielddesc, argbox): if argbox.is_constant(): @@ -691,7 +683,7 @@ return rvalue.ll_fromvalue(res) assert isinstance(argbox, rvalue.PtrRedBox) if argbox.content is None: - op_args = lltype.malloc(VARLIST.TO, 2) + op_args = lltype.malloc(rgenop.VARLIST.TO, 2) op_args[0] = argbox.getgenvar(jitstate) op_args[1] = fielddesc.gv_fieldname genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, @@ -706,7 +698,7 @@ array = rvalue.ll_getvalue(argbox, fielddesc.PTRTYPE) res = array[rvalue.ll_getvalue(indexbox, lltype.Signed)] return rvalue.ll_fromvalue(res) - op_args = lltype.malloc(VARLIST.TO, 2) + op_args = lltype.malloc(rgenop.VARLIST.TO, 2) op_args[0] = argbox.getgenvar(jitstate) op_args[1] = indexbox.getgenvar(jitstate) genvar = rgenop.genop(jitstate.curblock, 'getarrayitem', op_args, @@ -714,7 +706,7 @@ return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) ##def ll_generate_malloc(jitstate, gv_type, gv_resulttype): -## op_args = lltype.malloc(VARLIST.TO, 1) +## op_args = lltype.malloc(rgenop.VARLIST.TO, 1) ## op_args[0] = gv_type ## genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, ## gv_resulttype) @@ -843,7 +835,7 @@ def schedule_return(jitstate, redbox): jitstate.return_queue.append((jitstate.curoutgoinglink, redbox)) -novars = lltype.malloc(VARLIST.TO, 0) +novars = lltype.malloc(rgenop.VARLIST.TO, 0) def dispatch_next(jitstate, outredboxes, gv_return_type): split_queue = jitstate.split_queue Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py Sun Apr 2 19:03:28 2006 @@ -336,7 +336,7 @@ def create(self, hop): if self.typedesc is None: T = self.original_concretetype.TO - self.typedesc = rtimeshift.StructTypeDesc.make(T) + self.typedesc = rcontainer.StructTypeDesc(T) ts = self.timeshifter return hop.llops.genmixlevelhelpercall(self.typedesc.ll_factory, [], [], ts.s_RedBox) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Sun Apr 2 19:03:28 2006 @@ -1,8 +1,10 @@ from pypy.rpython import rgenop from pypy.rpython.lltypesystem import lltype, llmemory +class Memoizable(object): + pass -class RedBox(object): +class RedBox(Memoizable): def __init__(self, gv_type, genvar=rgenop.nullvar): assert rgenop.isconst(gv_type) # temporary? @@ -138,6 +140,7 @@ if not self.genvar: assert self.content self.genvar = self.content.build_runtime_container(jitstate) + self.content = None return self.genvar def enter_block(self, newblock, incoming, memo): @@ -211,8 +214,8 @@ def exactmatch(self, box, outgoingvarboxes, memo): if (box.is_constant() and - rgenop.revealconst(lltype.Double, self.gv_const) == - rgenop.revealconst(lltype.Double, box.genvar)): + rgenop.revealconst(lltype.Float, self.gv_const) == + rgenop.revealconst(lltype.Float, box.genvar)): return True else: outgoingvarboxes.append(box) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Sun Apr 2 19:03:28 2006 @@ -4,7 +4,7 @@ from pypy.annotation import model as annmodel from pypy.annotation import listdef, dictdef from pypy.jit.timeshifter import rvalue -from pypy.jit.timeshifter.rtimeshift import VARLIST, JITState +from pypy.jit.timeshifter.rtimeshift import JITState from pypy.rpython import rmodel, rlist, rdict, rgenop, annlowlevel from pypy.rpython.lltypesystem import rtuple from pypy.jit.timeshifter import rtimeshift From cfbolz at codespeak.net Sun Apr 2 19:05:05 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 2 Apr 2006 19:05:05 +0200 (CEST) Subject: [pypy-svn] r25231 - pypy/dist/pypy/doc Message-ID: <20060402170505.19446100D7@code0.codespeak.net> Author: cfbolz Date: Sun Apr 2 19:05:03 2006 New Revision: 25231 Modified: pypy/dist/pypy/doc/coding-guide.txt Log: superfluous comma Modified: pypy/dist/pypy/doc/coding-guide.txt ============================================================================== --- pypy/dist/pypy/doc/coding-guide.txt (original) +++ pypy/dist/pypy/doc/coding-guide.txt Sun Apr 2 19:05:03 2006 @@ -323,7 +323,7 @@ Integer Types ------------------------- -While implementing the integer type, we stumbled over the problem, that +While implementing the integer type, we stumbled over the problem that integers are quite in flux in CPython right now. Starting on Python 2.2, integers mutate into longs on overflow. However, shifting to the left truncates up to 2.3 but extends to longs as well in 2.4. By contrast, we need From nik at codespeak.net Sun Apr 2 19:20:47 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sun, 2 Apr 2006 19:20:47 +0200 (CEST) Subject: [pypy-svn] r25232 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060402172047.F187910090@code0.codespeak.net> Author: nik Date: Sun Apr 2 19:20:46 2006 New Revision: 25232 Modified: pypy/dist/pypy/translator/backendopt/removenoops.py pypy/dist/pypy/translator/backendopt/test/test_removenoops.py Log: light getting-into-pypy-mood hacking. take remove_same_as backendopt and generalize it to remove_unaryops, to remove any unary low-level operation. to be used to remove e.g. oodowncast/upcast in gensqueak. Modified: pypy/dist/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/removenoops.py Sun Apr 2 19:20:46 2006 @@ -2,38 +2,38 @@ from pypy.objspace.flow.model import traverse from pypy.rpython.lltypesystem.lltype import Void -def remove_same_as(graph): - """Remove all 'same_as' operations. +def remove_unaryops(graph, opnames): + """Removes unary low-level ops with a name appearing in the opnames list. """ - same_as_positions = [] + positions = [] def visit(node): if isinstance(node, Block): for i, op in enumerate(node.operations): - if op.opname == 'same_as': - same_as_positions.append((node, i)) + if op.opname in opnames: + positions.append((node, i)) traverse(visit, graph) - while same_as_positions: - block, index = same_as_positions.pop() - same_as_result = block.operations[index].result - same_as_arg = block.operations[index].args[0] - # replace the new variable (same_as_result) with the old variable + while positions: + block, index = positions.pop() + op_result = block.operations[index].result + op_arg = block.operations[index].args[0] + # replace the new variable (op_result) with the old variable # (from all subsequent positions) for op in block.operations[index:]: if op is not None: for i in range(len(op.args)): - if op.args[i] == same_as_result: - op.args[i] = same_as_arg + if op.args[i] == op_result: + op.args[i] = op_arg for link in block.exits: for i in range(len(link.args)): - if link.args[i] == same_as_result: - link.args[i] = same_as_arg - if block.exitswitch == same_as_result: - if isinstance(same_as_arg, Variable): - block.exitswitch = same_as_arg + if link.args[i] == op_result: + link.args[i] = op_arg + if block.exitswitch == op_result: + if isinstance(op_arg, Variable): + block.exitswitch = op_arg else: - assert isinstance(same_as_arg, Constant) + assert isinstance(op_arg, Constant) newexits = [link for link in block.exits - if link.exitcase == same_as_arg.value] + if link.exitcase == op_arg.value] assert len(newexits) == 1 newexits[0].exitcase = None if hasattr(newexits[0], 'llexitcase'): @@ -42,12 +42,15 @@ block.recloseblock(*newexits) block.operations[index] = None - # remove all same_as operations + # remove all operations def visit(node): if isinstance(node, Block) and node.operations: node.operations[:] = filter(None, node.operations) traverse(visit, graph) +def remove_same_as(graph): + remove_unaryops(graph, ["same_as"]) + def remove_void(translator): for graph in translator.graphs: Modified: pypy/dist/pypy/translator/backendopt/test/test_removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_removenoops.py Sun Apr 2 19:20:46 2006 @@ -1,9 +1,11 @@ -from pypy.translator.backendopt.removenoops import remove_void, remove_same_as +from pypy.translator.backendopt.removenoops import remove_void, remove_same_as, \ + remove_unaryops from pypy.translator.backendopt.inline import inline_function from pypy.translator.translator import TranslationContext, graphof from pypy.translator.test.snippet import simple_method from pypy.objspace.flow.model import checkgraph, flatten, Block -from pypy.rpython.lltypesystem.lltype import Void +from pypy.rpython.lltypesystem.lltype import Void, Signed +from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.llinterp import LLInterpreter import py @@ -67,3 +69,23 @@ interp = LLInterpreter(t.rtyper) result = interp.eval_graph(f_graph, []) assert result == 42 + +def test_remove_unaryops(): + # We really want to use remove_unaryops for things like ooupcast and + # oodowncast in dynamically typed languages, but it's easier to test + # it with operations on ints here. + def f(x): + i = llop.int_invert(Signed, x) + i = llop.int_add(Signed, x, 1) + return llop.int_neg(Signed, i) + t = TranslationContext() + t.buildannotator().build_types(f, [int]) + t.buildrtyper().specialize() + f_graph = graphof(t, f) + remove_unaryops(f_graph, ["int_neg", "int_invert"]) + t.checkgraphs() + + interp = LLInterpreter(t.rtyper) + result = interp.eval_graph(f_graph, [-2]) + assert result == -1 + From nik at codespeak.net Sun Apr 2 20:07:47 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sun, 2 Apr 2006 20:07:47 +0200 (CEST) Subject: [pypy-svn] r25233 - pypy/dist/pypy/translator/squeak Message-ID: <20060402180747.D50B0100BB@code0.codespeak.net> Author: nik Date: Sun Apr 2 20:07:46 2006 New Revision: 25233 Modified: pypy/dist/pypy/translator/squeak/node.py pypy/dist/pypy/translator/squeak/opformatter.py Log: use remove_unaryops in gensqueak to remove noops from function graphs. Modified: pypy/dist/pypy/translator/squeak/node.py ============================================================================== --- pypy/dist/pypy/translator/squeak/node.py (original) +++ pypy/dist/pypy/translator/squeak/node.py Sun Apr 2 20:07:46 2006 @@ -1,5 +1,6 @@ import datetime from pypy.objspace.flow.model import Constant, Variable, c_last_exception +from pypy.translator.backendopt.removenoops import remove_unaryops from pypy.translator.squeak.opformatter import OpFormatter from pypy.translator.squeak.codeformatter import CodeFormatter, Message from pypy.translator.squeak.codeformatter import Field, Assignment, CustomVariable @@ -195,6 +196,9 @@ yield " %s" % line yield "]" + def apply_backendopt(self, graph): + remove_unaryops(graph, OpFormatter.noops) + class MethodNode(CallableNode): def __init__(self, gen, INSTANCE, method_name): @@ -219,6 +223,7 @@ yield self.render_fileout_header( codef.format(self.INSTANCE), "methods") graph = self.INSTANCE._methods[self.name].graph + self.apply_backendopt(graph) self.self = graph.startblock.inputargs[0] for line in self.render_body(graph.startblock): yield line @@ -245,6 +250,7 @@ def render(self): yield self.render_fileout_header( "%s class" % self._class_name, "functions") + self.apply_backendopt(self.graph) for line in self.render_body(self.graph.startblock): yield line Modified: pypy/dist/pypy/translator/squeak/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/squeak/opformatter.py (original) +++ pypy/dist/pypy/translator/squeak/opformatter.py Sun Apr 2 20:07:46 2006 @@ -71,6 +71,11 @@ wrapping_ops = "neg", "invert", "add", "sub", "mul", "lshift" + noops = "ooupcast", "oodowncast", "cast_char_to_int", \ + "cast_unichar_to_int", "cast_int_to_unichar", \ + "cast_int_to_char", "cast_int_to_longlong", \ + "truncate_longlong_to_int" + int_masks = _setup_int_masks() def __init__(self, gen, node): @@ -150,12 +155,6 @@ # Public field access return Message(field_name).send_to(op.args[0], [field_value]) - def noop(self, op): - return Assignment(op.result, op.args[0]) - - op_oodowncast = noop - op_ooupcast = noop - def op_direct_call(self, op): # XXX how do i get rid of this import? from pypy.translator.squeak.node import FunctionNode @@ -170,18 +169,10 @@ def op_cast_bool_to_int(self, op): return self.cast_bool(op, "1", "0") - def op_cast_bool_to_float(self, op): - return self.cast_bool(op, "1.0", "0.0") - op_cast_bool_to_uint = op_cast_bool_to_int - op_cast_char_to_int = noop - op_cast_unichar_to_int = noop - - # NB: behaviour for casts to chars is undefined for too wide ints - op_cast_int_to_char = noop - op_cast_int_to_unichar = noop - op_cast_int_to_longlong = noop + def op_cast_bool_to_float(self, op): + return self.cast_bool(op, "1.0", "0.0") def masking_cast(self, op, mask): cast = self.apply_mask_helper(op.args[0], mask) @@ -193,8 +184,6 @@ def op_cast_uint_to_int(self, op): return self.masking_cast(op, "int") - op_truncate_longlong_to_int = noop - def op_cast_float_to_uint(self, op): truncated = Message("truncated").send_to(op.args[0], []) return Assignment(op.result, self.apply_mask_helper(truncated, "uint")) From nik at codespeak.net Sun Apr 2 20:17:39 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sun, 2 Apr 2006 20:17:39 +0200 (CEST) Subject: [pypy-svn] r25234 - pypy/dist/pypy/translator/tool Message-ID: <20060402181739.BD1C7100BC@code0.codespeak.net> Author: nik Date: Sun Apr 2 20:17:38 2006 New Revision: 25234 Modified: pypy/dist/pypy/translator/tool/graphpage.py Log: fix an import not updated after the jit module refactoring. Modified: pypy/dist/pypy/translator/tool/graphpage.py ============================================================================== --- pypy/dist/pypy/translator/tool/graphpage.py (original) +++ pypy/dist/pypy/translator/tool/graphpage.py Sun Apr 2 20:17:38 2006 @@ -153,8 +153,8 @@ self.annotator.binding_cause_history.get(var, [])) self.binding_history[var.name] = zip(history, cause_history) - from pypy.jit import hintannotator - if isinstance(self.annotator, hintannotator.HintAnnotator): + from pypy.jit.hintannotator.annotator import HintAnnotator + if isinstance(self.annotator, HintAnnotator): return def visit(node): From pedronis at codespeak.net Sun Apr 2 20:27:45 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 20:27:45 +0200 (CEST) Subject: [pypy-svn] r25235 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402182745.1A7F3100B7@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 20:27:44 2006 New Revision: 25235 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Log: (arre, pedronis) first pass at implementing copy, freeze and exactmatch for virtual structures, their frozen version respectively. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Sun Apr 2 20:27:44 2006 @@ -3,7 +3,7 @@ from pypy.rpython import rgenop from pypy.jit.timeshifter import rvalue -class AbstractContainer(rvalue.Memoizable): +class AbstractContainer(object): pass # ____________________________________________________________ @@ -138,6 +138,32 @@ rgenop.genop(jitstate.curblock, 'setfield', op_args, rgenop.gv_Void) +class FrozenVirtualStruct(AbstractContainer): + + def exactmatch(self, vstruct, outgoingvarboxes, memo): + contmemo = memo.containers + if self in contmemo: + return vstruct is contmemo[self] + if vstruct in contmemo: + return contmemo[vstruct] is self + contmemo[self] = vstruct + contmemo[vstruct] = self + self_length = len(self.fz_content_boxes) + vstruct_length = len(vstruct.content_boxes) + if self_length != vstruct_length: + return False + self_boxes = self.fz_content_boxes + vstruct_boxes = vstruct.content_boxes + fullmatch = True + for i in range(self_length): + if not self_boxes[i].exactmatch(vstruct_boxes[i], + outgoingvarboxes, + memo): + fullmatch = False + return fullmatch + + + class VirtualStruct(AbstractContainer): def __init__(self, typedesc): @@ -147,8 +173,9 @@ for desc in typedesc.fielddescs] def enter_block(self, newblock, incoming, memo): - if self not in memo: - memo[self] = None + contmemo = memo.containers + if self not in contmemo: + contmemo[self] = None for box in self.content_boxes: box.enter_block(newblock, incoming, memo) @@ -174,3 +201,25 @@ fielddesc.generate_set(jitstate, genvar, box) return genvar + + def freeze(self, memo): + contmemo = memo.containers + try: + return contmemo[self] + except KeyError: + result = contmemo[self] = FrozenVirtualStruct() + frozens = [box.freeze(memo) for box in self.content_boxes] + result.fz_content_boxes = frozens + return result + + def copy(self, memo): + contmemo = memo.containers + try: + return contmemo[self] + except KeyError: + result = contmemo[self] = VirtualStruct(self.typedesc) + boxcopies = [] + for box in self.content_boxes: + boxcopies.append(box.copy(memo)) + result.content_boxes = boxcopies + return result Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sun Apr 2 20:27:44 2006 @@ -718,9 +718,9 @@ def retrieve_jitstate_for_merge(states_dic, jitstate, key, redboxes): if key not in states_dic: - memo = {} + memo = rvalue.freeze_memo() frozens = [redbox.freeze(memo) for redbox in redboxes] - memo = {} + memo = rvalue.exactmatch_memo() outgoingvarboxes = [] for i in range(len(redboxes)): res = frozens[i].exactmatch(redboxes[i], outgoingvarboxes, memo) @@ -737,7 +737,7 @@ return jitstate frozens, oldblock = states_dic[key] - memo = {} + memo = rvalue.exactmatch_memo() outgoingvarboxes = [] exactmatch = True for i in range(len(redboxes)): @@ -765,7 +765,7 @@ rgenop.closelink(link, linkargs, newblock) jitstate.curblock = newblock #jitstate.curoutgoinglink = lltype.nullptr(rgenop.LINK.TO) - memo = {} + memo = rvalue.freeze_memo() frozens = [redbox.freeze(memo) for redbox in redboxes] states_dic[key] = frozens, newblock return jitstate @@ -793,7 +793,7 @@ def enter_block(jitstate, redboxes): newblock = rgenop.newblock() incoming = [] - memo = {} + memo = rvalue.enter_block_memo() for i in range(len(redboxes)): redboxes[i].enter_block(newblock, incoming, memo) rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) @@ -804,7 +804,7 @@ def dyn_enter_block(jitstate, redboxes): newblock = rgenop.newblock() incoming = [] - memo = {} + memo = rvalue.enter_block_memo() for i in range(len(redboxes)): redboxes[i].enter_block(newblock, incoming, memo) rgenop.closelink(jitstate.curoutgoinglink, incoming, newblock) @@ -827,7 +827,7 @@ later_jitstate = jitstate.copystate() jitstate.curoutgoinglink = true_link later_jitstate.curoutgoinglink = false_link - memo = {} + memo = rvalue.Memo() redboxcopies = [redbox.copy(memo) for redbox in redboxes] jitstate.split_queue.append((exitindex, later_jitstate, redboxcopies)) return True @@ -867,7 +867,7 @@ for link, redbox in return_queue: newblock = rgenop.newblock() incoming = [] - memo = {} + memo = rvalue.enter_block_memo() redbox.enter_block(newblock, incoming, memo) jitstate.curblock = newblock gv_retval = redbox.getgenvar(jitstate) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Sun Apr 2 20:27:44 2006 @@ -1,10 +1,24 @@ from pypy.rpython import rgenop from pypy.rpython.lltypesystem import lltype, llmemory -class Memoizable(object): - pass +class Memo(object): + _annspecialcase_ = 'specialize:ctr_location' -class RedBox(Memoizable): + def __init__(self): + self.boxes = {} + self.containers = {} + +def enter_block_memo(): + return Memo() + +def freeze_memo(): + return Memo() + +def exactmatch_memo(): + return Memo() + + +class RedBox(object): def __init__(self, gv_type, genvar=rgenop.nullvar): assert rgenop.isconst(gv_type) # temporary? @@ -24,6 +38,7 @@ return self.genvar def enter_block(self, newblock, incoming, memo): + memo = memo.boxes if not self.is_constant() and self not in memo: incoming.append(self.genvar) memo[self] = None @@ -58,6 +73,7 @@ "A red box that contains a constant integer-like value." def copy(self, memo): + memo = memo.boxes try: return memo[self] except KeyError: @@ -65,6 +81,7 @@ return result def freeze(self, memo): + memo = memo.boxes try: return memo[self] except KeyError: @@ -80,6 +97,7 @@ "A red box that contains a constant double-precision floating point value." def copy(self, memo): + memo = memo.boxes try: return memo[self] except KeyError: @@ -87,6 +105,7 @@ return result def freeze(self, memo): + memo = memo.boxes try: return memo[self] except KeyError: @@ -111,29 +130,31 @@ return RedBox.__repr__(self) def copy(self, memo): + boxmemo = memo.boxes try: - return memo[self] + return boxmemo[self] except KeyError: result = PtrRedBox(self.gv_type, self.genvar) - memo[self] = result + boxmemo[self] = result if self.content: result.content = self.content.copy(memo) return result def freeze(self, memo): + boxmemo = memo.boxes try: - return memo[self] + return boxmemo[self] except KeyError: if self.content: result = FrozenPtrVirtual() - memo[self] = result + boxmemo[self] = result result.fz_content = self.content.freeze(memo) else: if self.is_constant(): result = FrozenPtrConst(self.genvar) else: result = FrozenPtrVar() - memo[self] = result + boxmemo[self] = result return result def getgenvar(self, jitstate): @@ -189,6 +210,7 @@ return box def exactmatch(self, box, outgoingvarboxes, memo): + memo = memo.boxes if self not in memo: memo[self] = box outgoingvarboxes.append(box) @@ -233,6 +255,7 @@ return box def exactmatch(self, box, outgoingvarboxes, memo): + memo = memo.boxes if self not in memo: memo[self] = box outgoingvarboxes.append(box) @@ -277,6 +300,7 @@ return box def exactmatch(self, box, outgoingvarboxes, memo): + memo = memo.boxes if self not in memo: memo[self] = box outgoingvarboxes.append(box) @@ -304,5 +328,8 @@ outgoingvarboxes.append(box) return False else: - return self.fz_content.exactmatch(box.content, outgoingvarboxes, - memo) + result = self.fz_content.exactmatch(box.content, outgoingvarboxes, + memo) + if not result: + outgoingvarboxes.append(box) + return result From gromit at codespeak.net Sun Apr 2 20:51:20 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Sun, 2 Apr 2006 20:51:20 +0200 (CEST) Subject: [pypy-svn] r25236 - pypy/dist/pypy/doc Message-ID: <20060402185120.0705F100D0@code0.codespeak.net> Author: gromit Date: Sun Apr 2 20:51:18 2006 New Revision: 25236 Modified: pypy/dist/pypy/doc/ctypes-integration.txt Log: CHG: Discussed the alternatives. Added a complete description of the GC related keep alive management. Modified: pypy/dist/pypy/doc/ctypes-integration.txt ============================================================================== --- pypy/dist/pypy/doc/ctypes-integration.txt (original) +++ pypy/dist/pypy/doc/ctypes-integration.txt Sun Apr 2 20:51:18 2006 @@ -45,6 +45,8 @@ - All types are defined at module load time and thus need not be rpython. + This means ctypes' functions like `POINTER` and + `pointer` must not be annotated. - Free unions are not supported, because it is unclear whether they can be properly annotated. @@ -77,8 +79,56 @@ .. [#] This restriction will be lifted in future ctypes versions. -Memory-Layout -------------- + +Memory-Layout as Proposed By Gerald Klix +---------------------------------------- + +Primitive Types +~~~~~~~~~~~~~~~ + +Ctypes' primitive types are mapped directly to the correspondending +PyPy type. + +Structures +~~~~~~~~~~ +Structures will have the following memory layout if they were allocated by ctypes:: + + Ptr( GcStruct( "CtypesGcStructure_ + ( "c_data" + (Struct "C-Data_ + *) ) ) ) + +We will try hard not to expose the "c-data" member of the structure +at rpython level. + +Structures that result form dereferencing a pointer will have the following +layout:: + + Ptr( GcStruct( "CtypesStructure_ + ( "c_data" + Ptr( Struct( "C-Data_ + *) ) ) ) ) + +Pointers +~~~~~~~~ +Pointers pointing to structures allocated by ctypes will have the following memory layout:: + + Ptr( GcStruct( "CtypesGCPointer_ + "contents" Ptr( GcStruct( "CtypesGcStructure_" ... ) ) ) ) + + +Pointers pointing returned from external functions have the follwing layout if the +point to a structure:: + + Ptr( GcStruct( "CtypesPointer_" + "contents" Ptr( Struct( "CtypesStructure_" ... ) ) ) ) + +Currently it is not decided whether assiging a pointers `contents` attribute from +a GC-pointer should be allowed. The other case will only become valid if we implement +structures with mixed memory state. + +Memory-Layout as Proposed by Armin Rigo +--------------------------------------- In Ctypes, all instances are mutable boxes containing either some raw memory with a layout compatible to that of the equivalent C type, or a @@ -193,3 +243,241 @@ ~~~~~~ Arrays behave like structures, but use an Array instead of a Struct in the "c_data" or "c_data_ref" declaration. + + +Use Cases +========= + +This section will discussion various use cases, espcially the use of pointers +in the context of primitive types and structures. + +Pointers to Primitive Types +--------------------------- +This section discusses the memory layout resulting from allocating +pointers to primitives. + +The original direct mapping of ctypes primitive types to PyPy +primitive types does not work with PyPy's notion of pointers. In +PyPy it is not possible to create a pointer to a primitive. +Therefore Armin Rigo's proposal is a better alternative. + +If a pointer is passed to an external function the +garbage collection header must be stripped. This is easy if +the pointer directly passed to an external function. + +If the pointer is contained in an other structure a similar +procedure applies. The `c_data`-part of the structure +must be assigned to the `c_data_ref`-part of the pointer +and the pointer to the garbage collection header must be +assigned to the `keepalive`-field. + +Pointers to Structures +---------------------- +This section discusses the memory layout resulting from allocating +pointers to structures. + +At least the original structure layout will not work, +if pointers to structures are embedded in in other structures. + +The following example illustrates the resulting structure according +to the original layout, if we embed a pointer to a structure instance +of the same type:: + + Ptr( GcStruct( "CtypesGcStructure_ + ( "c_data" + (Struct "C-Data_ + Ptr( GcStruct ( "CtypesGcStructure_ + "c_data" ... ) ) ) ) ) ) + +It is clear that the `c_data` member does not point to a "C-compatible" +structure, but to the garbage collection header. In the case of a linked list +the whole list has to traversed and copied. This operation is O( n ) for +if n is the number of pointers to structures. + +The same structure according to Armin Rigo's proposal will look like this - +at least if I understood it right:: + + Ptr( GcStruct( "CtypesBox_ + ( "c_data_ref" + (Ptr (Struct "C_Data_ + ( "value", Ptr(...) ) ) ) ), + ( "keepalive" + (Ptr (GcStruct (Struct "C_Data_ + ( "value", Ptr(...) ) ) ) ) ) ) ) + +In this case every pointer's `c_data_ref` is accompanied by +a corespondending `keepalive` field. + +Structures containing Structures +---------------------------------- + +Memory Layout +~~~~~~~~~~~~~ +The original layout would embed the inner GcStruct in the outer +structure. Again passing structures with this layout to +external functions is not feasable without copying. + +Armin's approach will lead to the right memory layout, if one +assumes, that the outer structure contains the `c_data` part +of the inner structure. + +Dereferencing The Inner Structure Member +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The original solution has no problems in this case; +it is enough to return the result of the getfield operation. + +Armin's layout must implement the same behaviour than CPython's +ctypes. A new structure instance needs to allocated, because +the field of the outer structure containing the inner +strcuture lacks a garbage collection header. + +Solution +======== +The final solution is close to Armin's approach, but describes the +management of `keepalive` pointers to great detail. + +Guiding Principle +----------------- +The guiding principle behind building "C-compatible" external structures +is to separate the garbage collection object graph from the actual "C-compatible" +layout. This has the big advantage, that the "C-compatible" graph can +be passed to external functions without change. + +Layouts by Type +--------------- +The following sections describe the layout for various types. + +Primitive Types +~~~~~~~~~~~~~~~ +The layout of atomic primitive types like integers or characters is like +proposed by Armin Rigo:: + + Ptr( GcStruct( "CtypesBox_ + ( "c_data" + (Struct "C_Data_ + ( "value", Signed/Float/etc. ) ) ) ) ) + + +`c_char_p` will have the following layout:: + + Ptr( GcStruct( "CtypesBox_Char_p + ( "length" Int ), + ( "c_data" + (Struct "C_Data_Char_p + ( "value", Array( Char ) ) ) ) ) ) + +`c_wchar_p` will have the following layout:: + + Ptr( GcStruct( "CtypesBox_Char_p + ( "length" Int ), + ( "c_data" + (Struct "C_Data_Char_p + ( "value", Array( Int ) ) ) ) ) ) + +The use of `Int` here implies that it is 2 bytes long. + +Structures Without Pointers +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Structures, that do not contain pointers will have the layout +proposed by Armin Rigo:: + + Ptr( GcStruct( "CtypesBox_ + ( "keepalive" Ptr( ... ) ), + ( "c_data" + Struct( "C_Data_ + *) ) ) ) + +The `keepalive`-field is neccessary in the case of structures +resulting from dereferencing a structure contained in another +structure, as explained below. + +Pointers +~~~~~~~~ +Pointers come in two flavors. + +The simple case is a pointer returned by an external function, +because no care must be taken to keep the object pointed to alive. +We can savely assume that the structure was allocated by the +external funcion. [#]_ Therefore we can use a layout similar to +Armin's proposal:: + + Ptr( GcStruct( "CtypesBox_ + ( "c_data" + (Struct "C_Data_ + ( "contents", Ptr(...) ) ) ) ) ) + +Of course this layout can be simplified to:: + + Ptr( GcStruct( "CtypesBox_ + ( "contents", Ptr(...) ) ) ) + +but the layout with the `c_data`-field is similar to the ordinary case +when the object pointed to was allocated by PyPy:: + + Ptr( GcStruct( "CtypesBox_ + ( "keepalive" + GcStruct( "CtypesBox_ + ( "c_data" + Struct( "C_Data_ + ( "value", Signed/Float/etc. ) ) ) ) ), + ( "c_data" + Struct( "C_Data_ + ( "contents", Ptr( + Struct( "C_Data_ + ( "value", Signed/Float/etc. ) ) ) ) ) ) ) + +Of course the pointer's `c_data`-field and the keepalive-fields +`GcStruct` point to the same object. + +.. [#] It may be neccessary to deallocate the structure returned. + In this case it should be simple to call an external deallocation + function, such as `malloc`. + +Structures Containing Pointers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Structures containing pointers are a bit different, because all `keepalives`-fields +need to be moved to one end of the structure [#]_ :: + + Ptr( GcStruct( "CtypesBox_ + ( "container_keepalive" Ptr( ... ) ), + ( "pointer_keepalive" + GcStruct( "C_Keepalive_ + *) ), + ( "c_data" + Struct( "C_Data_ + *) ) ) ) + +Structures conatining pointers and structures that in turn contain +structures with pointers, need the keepalive pointers to the begining +of the structure. + +.. [#] Obviously the start of the structure is a better choice, because it leaves + the structure's end for variable length arrays. + +Structures Created by Dereferencing an Inner Structure +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +In order to be ctypes compatible, structures that are created by dereferencing +a structure inside another structure, need a `container_keepalive`-field. Otherwise +a dangling pointer will result when the outer structure becomes unreachable and +is freed by the garbage collector. + +Arrays +~~~~~~ +Arrays of a primitive objects don't need keepalive-fields. Arrays of pointers +should consist of a "C-compatible" part and a `pointer_keepalive`-part unless they are part +of another structure or array. In the later case the array's `pointer_keepalive`-part must +be contained in the `pointer_keepalive` part of the outer structure or array. + +Management of Container-Keepalive fields +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +For every outermost structure or array the `container_keepalive`-field points +to the `GcStruct` containing the structure or array. Everytime an array or +structure object is created be dereferencing an inner array or structure +its `container_keepalive`-field is set with the value of the outermost array's +or structure's `container_keepalive`-field. + +The same is true when creating a pointer that points to an inner structure or array. +The pointers `keepalive`-field is set from the the `container_keepalive` field of +the outermost structure or array. +When such is a pointer is dereferenced the `container_keepalive` field of the newly +created `GcStruct` must be set from the pointer's `keepalive`-field. From pedronis at codespeak.net Sun Apr 2 23:11:21 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 23:11:21 +0200 (CEST) Subject: [pypy-svn] r25238 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402211121.1A6A1100DD@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 23:11:20 2006 New Revision: 25238 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Log: (arre, pedronis) fix indentation Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py Sun Apr 2 23:11:20 2006 @@ -667,12 +667,12 @@ def ll_generate_setfield(jitstate, fielddesc, destbox, valuebox): assert isinstance(destbox, rvalue.PtrRedBox) if destbox.content is None: - op_args = lltype.malloc(rgenop.VARLIST.TO, 3) - op_args[0] = destbox.getgenvar(jitstate) - op_args[1] = fielddesc.fieldname_gv[-1] - op_args[2] = valuebox.getgenvar(jitstate) - rgenop.genop(jitstate.curblock, 'setfield', op_args, - rgenop.gv_Void) + op_args = lltype.malloc(rgenop.VARLIST.TO, 3) + op_args[0] = destbox.getgenvar(jitstate) + op_args[1] = fielddesc.fieldname_gv[-1] + op_args[2] = valuebox.getgenvar(jitstate) + rgenop.genop(jitstate.curblock, 'setfield', op_args, + rgenop.gv_Void) else: destbox.content.op_setfield(jitstate, fielddesc, valuebox) @@ -683,12 +683,12 @@ return rvalue.ll_fromvalue(res) assert isinstance(argbox, rvalue.PtrRedBox) if argbox.content is None: - op_args = lltype.malloc(rgenop.VARLIST.TO, 2) - op_args[0] = argbox.getgenvar(jitstate) - op_args[1] = fielddesc.gv_fieldname - genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, - fielddesc.gv_resulttype) - return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) + op_args = lltype.malloc(rgenop.VARLIST.TO, 2) + op_args[0] = argbox.getgenvar(jitstate) + op_args[1] = fielddesc.gv_fieldname + genvar = rgenop.genop(jitstate.curblock, 'getsubstruct', op_args, + fielddesc.gv_resulttype) + return fielddesc.redboxcls(fielddesc.gv_resulttype, genvar) else: return argbox.content.op_getsubstruct(jitstate, fielddesc) From pedronis at codespeak.net Sun Apr 2 23:12:04 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 23:12:04 +0200 (CEST) Subject: [pypy-svn] r25239 - pypy/dist/pypy/rpython Message-ID: <20060402211204.01BEC100E0@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 23:12:04 2006 New Revision: 25239 Modified: pypy/dist/pypy/rpython/rgenop.py Log: (arre, pedronis) debugging helpers to check whether a gv_type contains a pointer type Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Sun Apr 2 23:12:04 2006 @@ -15,6 +15,10 @@ # for debugging, sanity checks in non-RPython code reveal = from_opaque_object +def isptrtype(gv_type): + c = from_opaque_object(gv_type) + return isinstance(c.value, lltype.Ptr) + def initblock(opaqueptr): init_opaque_object(opaqueptr, flowmodel.Block([])) @@ -281,6 +285,8 @@ setannotation(closelink, None) setannotation(closereturnlink, None) +setannotation(isptrtype, annmodel.SomeBool()) + # specialize setspecialize(initblock) setspecialize(geninputarg) @@ -293,6 +299,8 @@ setspecialize(closelink) setspecialize(closereturnlink) +setspecialize(isptrtype) + # XXX(for now) void constant constructors setannotation(constFieldName, s_ConstOrVar) setannotation(constTYPE, s_ConstOrVar) From pedronis at codespeak.net Sun Apr 2 23:14:48 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 2 Apr 2006 23:14:48 +0200 (CEST) Subject: [pypy-svn] r25240 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060402211448.697CA100DE@code0.codespeak.net> Author: pedronis Date: Sun Apr 2 23:14:47 2006 New Revision: 25240 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Log: (arre, pedronis) fix ll_factory. op_getsubstruct for virtual structures. make sure that virtual structures are materialized exactly once. apart from the skipped tests all tests in timeshifter/test pass again. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Sun Apr 2 23:14:47 2006 @@ -55,7 +55,7 @@ return self.fielddescs[index] def ll_factory(self): - return rvalue.PtrRedBox(self.gv_type, content=VirtualStruct(self)) + return rvalue.PtrRedBox(self.gv_ptrtype, content=VirtualStruct(self)) def _freeze_(self): return True @@ -171,6 +171,7 @@ self.content_boxes = [desc.redboxcls(desc.gv_resulttype, desc.gv_default) for desc in typedesc.fielddescs] + self.genvar = rgenop.nullvar def enter_block(self, newblock, incoming, memo): contmemo = memo.containers @@ -186,6 +187,8 @@ self.content_boxes[fielddesc.fieldindex] = valuebox def build_runtime_container(self, jitstate): + if self.genvar: + return self.genvar typedesc = self.typedesc boxes = self.content_boxes self.content_boxes = None @@ -193,6 +196,7 @@ op_args[0] = typedesc.gv_type genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, typedesc.gv_ptrtype) + self.genvar = genvar fielddescs = typedesc.fielddescs for i in range(len(fielddescs)): fielddesc = fielddescs[i] @@ -223,3 +227,7 @@ boxcopies.append(box.copy(memo)) result.content_boxes = boxcopies return result + + def op_getsubstruct(self, jitstate, fielddesc): + #assert fielddesc.fieldindex == 0 + return rvalue.PtrRedBox(fielddesc.gv_resulttype, content=self) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Sun Apr 2 23:14:47 2006 @@ -120,6 +120,7 @@ class PtrRedBox(RedBox): def __init__(self, gv_type, genvar=rgenop.nullvar, content=None): + #assert rgenop.isptrtype(gv_type) RedBox.__init__(self, gv_type, genvar) self.content = content # None or an AbstractContainer @@ -160,7 +161,12 @@ def getgenvar(self, jitstate): if not self.genvar: assert self.content - self.genvar = self.content.build_runtime_container(jitstate) + genvar = self.content.build_runtime_container(jitstate) + op_args = lltype.malloc(rgenop.VARLIST.TO, 1) + op_args[0] = genvar + genvar = rgenop.genop(jitstate.curblock, 'cast_pointer', op_args, + self.gv_type) + self.genvar = genvar self.content = None return self.genvar From nik at codespeak.net Mon Apr 3 10:20:20 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 3 Apr 2006 10:20:20 +0200 (CEST) Subject: [pypy-svn] r25245 - pypy/dist/pypy/translator/squeak Message-ID: <20060403082020.89E62100E9@code0.codespeak.net> Author: nik Date: Mon Apr 3 10:20:19 2006 New Revision: 25245 Modified: pypy/dist/pypy/translator/squeak/node.py pypy/dist/pypy/translator/squeak/opformatter.py Log: argh, major thinking error. of course i can't really remove operations from the graph that return a result with a type different from the argument. Modified: pypy/dist/pypy/translator/squeak/node.py ============================================================================== --- pypy/dist/pypy/translator/squeak/node.py (original) +++ pypy/dist/pypy/translator/squeak/node.py Mon Apr 3 10:20:19 2006 @@ -197,7 +197,7 @@ yield "]" def apply_backendopt(self, graph): - remove_unaryops(graph, OpFormatter.noops) + remove_unaryops(graph, ["same_as"]) class MethodNode(CallableNode): Modified: pypy/dist/pypy/translator/squeak/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/squeak/opformatter.py (original) +++ pypy/dist/pypy/translator/squeak/opformatter.py Mon Apr 3 10:20:19 2006 @@ -29,7 +29,6 @@ 'new': 'new', 'runtimenew': 'new', 'classof': 'class', - 'same_as': 'yourself', 'bool_not': 'not', 'cast_int_to_float': 'asFloat', @@ -71,7 +70,7 @@ wrapping_ops = "neg", "invert", "add", "sub", "mul", "lshift" - noops = "ooupcast", "oodowncast", "cast_char_to_int", \ + noops = "same_as", "ooupcast", "oodowncast", "cast_char_to_int", \ "cast_unichar_to_int", "cast_int_to_unichar", \ "cast_int_to_char", "cast_int_to_longlong", \ "truncate_longlong_to_int" @@ -188,3 +187,8 @@ truncated = Message("truncated").send_to(op.args[0], []) return Assignment(op.result, self.apply_mask_helper(truncated, "uint")) + def noop(self, op): + return Assignment(op.result, op.args[0]) + +for opname in OpFormatter.noops: + setattr(OpFormatter, "op_%s" % opname, OpFormatter.noop) From nik at codespeak.net Mon Apr 3 13:06:10 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 3 Apr 2006 13:06:10 +0200 (CEST) Subject: [pypy-svn] r25246 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060403110610.C324A100F3@code0.codespeak.net> Author: nik Date: Mon Apr 3 13:06:01 2006 New Revision: 25246 Added: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (contents, props changed) Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: a first stab at bringing lists to ootype. a list of e.g. integers is of type ootype.List(Signed) in ootypesystem. oo lists have a yet to be fully discovered interface defined in ootype.LIST_METHODS. for now, the interface consists of only a length method that can already be rtyped. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 3 13:06:01 2006 @@ -168,6 +168,27 @@ def __init__(self, args, result): StaticMethod.__init__(self, args, result) + + +class List(OOType): + + def __init__(self, ITEMTYPE): + self._ITEMTYPE = ITEMTYPE + + def __str__(self): + return '%s(%s)' % (self.__class__.__name__, self._ITEMTYPE) + + def _lookup(self, meth_name): + METH = LIST_METHODS.get(meth_name) + meth = None + if METH is not None: + meth = _meth(METH, _name=meth_name, + _callable=getattr(_list, meth_name)) + return self, meth + + def _example(self): + return new(self) + # ____________________________________________________________ class _class(object): @@ -403,7 +424,7 @@ _callable.__init__(self, METHOD, **attrs) def _bound(self, DEFINST, inst): - assert isinstance(inst, _instance) + assert isinstance(inst, _instance) or isinstance(inst, _list) return _bound_meth(DEFINST, inst, self) class _bound_meth(object): @@ -416,8 +437,42 @@ callb, checked_args = self.meth._checkargs(args) return callb(self.inst, *checked_args) -def new(INSTANCE): - return make_instance(INSTANCE) + +# This defines the abstract list interface that backends will have to map to +# their native list implementations. +LIST_METHODS = frozendict({ + # "method name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) + "length": Meth([], Unsigned), +}) + +class _list(object): + + def __init__(self, LIST): + self._TYPE = LIST + self._list = [] + + def __getattribute__(self, name): + TYPE = object.__getattribute__(self, "_TYPE") + _, meth = TYPE._lookup(name) + if meth is not None: + return meth._bound(TYPE, self) + + return object.__getattribute__(self, name) + + # The following are implementations of the abstract list interface for + # use by the llinterpreter and ootype tests. There are NOT_RPYTHON + # because the annotator is not supposed to follow them. + + def length(self): + # NOT_RPYTHON + return len(self._list) + + +def new(TYPE): + if isinstance(TYPE, Instance): + return make_instance(TYPE) + elif isinstance(TYPE, List): + return _list(TYPE) def runtimenew(class_): assert isinstance(class_, _class) Added: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Mon Apr 3 13:06:01 2006 @@ -0,0 +1,12 @@ +from pypy.rpython.ootypesystem.ootype import * + +def test_new(): + LT = List(Signed) + l = new(LT) + assert typeOf(l) == LT + +def test_len(): + LT = List(Signed) + l = new(LT) + assert l.length() == 0 + Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Mon Apr 3 13:06:01 2006 @@ -1,4 +1,4 @@ - +from pypy import conftest from pypy.rpython.ootypesystem.ootype import * from pypy.annotation import model as annmodel from pypy.objspace.flow import FlowObjSpace @@ -8,10 +8,10 @@ def gengraph(f, args=[], viewBefore=False, viewAfter=False): t = TranslationContext() t.buildannotator().build_types(f, args) - if viewBefore: + if viewBefore or conftest.option.view: t.view() t.buildrtyper(type_system="ootype").specialize() - if viewAfter: + if viewAfter or conftest.option.view: t.view() return graphof(t, f) @@ -97,3 +97,14 @@ rettype = g.getreturnvar().concretetype assert rettype == Bool +def test_list_len(): + LT = List(Signed) + + def oof(): + l = new(LT) + return l.length() + + g = gengraph(oof, []) + rettype = g.getreturnvar().concretetype + assert rettype == Signed + From ericvrp at codespeak.net Mon Apr 3 13:28:30 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Mon, 3 Apr 2006 13:28:30 +0200 (CEST) Subject: [pypy-svn] r25248 - in pypy/dist/pypy/translator/llvm: llvmcapi/include llvmcapi/include/ExecutionEngine llvmcapi/include/VMCore llvmcapi/lib llvmcapi/lib/ExecutionEngine llvmcapi/lib/VMCore pythonllvm pythonllvm/test Message-ID: <20060403112830.3AE0C100F4@code0.codespeak.net> Author: ericvrp Date: Mon Apr 3 13:28:26 2006 New Revision: 25248 Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/DerivedTypes.h pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/GenericValue.cpp (contents, props changed) pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/DerivedTypes.cpp (contents, props changed) Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp pypy/dist/pypy/translator/llvm/pythonllvm/helper.py pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Log: More llvm c api support Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h Mon Apr 3 13:28:26 2006 @@ -11,8 +11,10 @@ */ void* ExecutionEngine__create__(void* MP, int ForceInterpreter); void* ExecutionEngine_getModule(void* EE); -int ExecutionEngine_n_functions(void* EE); -int ExecutionEngine_function_exists(void* EE, const char* funcname); +void ExecutionEngine_freeMachineCodeForFunction(void* EE, void* F); + +// return union and takes std::vector actually +int ExecutionEngine_runFunction(void* EE, void* F, int args_vector); #ifdef __cplusplus }; Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h Mon Apr 3 13:28:26 2006 @@ -0,0 +1,14 @@ +#ifndef __GENERICVALUE_H__ +#define __GENERICVALUE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +void* GenericValue__init__(); + +#ifdef __cplusplus +}; +#endif + +#endif Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/DerivedTypes.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/DerivedTypes.h Mon Apr 3 13:28:26 2006 @@ -0,0 +1,16 @@ +#ifndef __DERIVEDTYPES_H__ +#define __DERIVEDTYPES_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +int FunctionType_getNumParams(void* FT); +const void* FunctionType_getParamType(void* FT, int i); +const void* FunctionType_getReturnType(void* FT); + +#ifdef __cplusplus +}; +#endif + +#endif Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Function.h Mon Apr 3 13:28:26 2006 @@ -5,7 +5,8 @@ extern "C" { #endif -// +void Function_eraseFromParent(void* F); +const void* Function_getFunctionType(void* F); #ifdef __cplusplus }; Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Module.h Mon Apr 3 13:28:26 2006 @@ -5,20 +5,23 @@ extern "C" { #endif -void* Module__init__(const char* ModuleID); +void* Module__init__(const char* ModuleID); const char* Module_getModuleIdentifier(void* M); void Module_setModuleIdentifier(void* M, const char* ID); const char* Module_getTargetTriple(void* M); void Module_setTargetTriple(void* M, const char* T); -//const char* Module_getEndianness(void* M); -//void Module_setEndianness(void* M, Endianness E); -//const char* Module_getPointerSize(void* M); -//void Module_setPointerSize(void* M, PointerSize PS); const char* Module_getModuleInlineAsm(void* M); void Module_setModuleInlineAsm(void* M, const char* Asm); +void* Module_getNamedFunction(void* M, const char* fnname); + +// global function that seem to better fit here void Module_ParseAssemblyString(void* M, const char* AsmString); int Module_verifyModule(void* M); +// helpers +int Module_n_functions(void* M); +int Module_function_exists(void* M, const char* fnname); + #ifdef __cplusplus }; #endif Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h Mon Apr 3 13:28:26 2006 @@ -8,7 +8,9 @@ #include "VMCore/Module.h" #include "VMCore/ModuleProvider.h" #include "VMCore/Function.h" +#include "VMCore/DerivedTypes.h" #include "ExecutionEngine/ExecutionEngine.h" +#include "ExecutionEngine/GenericValue.h" void toggle_print_machineinstrs(); Modified: pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp Mon Apr 3 13:28:26 2006 @@ -20,27 +20,15 @@ return &((ExecutionEngine*)EE)->getModule(); } -int ExecutionEngine_n_functions(void* EE) { //move to Module? - Module* mod = &((ExecutionEngine*)EE)->getModule(); - int funccount = 0; - Module::FunctionListType &fns = mod->getFunctionList(); - for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { - if (!(ii->isIntrinsic() || ii->isExternal())) { - funccount += 1; - } - } - return funccount; +void ExecutionEngine_freeMachineCodeForFunction(void* EE, void* F) { + ExecutionEngine* ee = (ExecutionEngine*)EE; + Function* f = (Function*)F; + ee->freeMachineCodeForFunction(f); } -int Module_function_exists(void* EE, const char* funcname) { //move to Module? - Module* mod = &((ExecutionEngine*)EE)->getModule(); - Module::FunctionListType &fns = mod->getFunctionList(); - for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { - if (!(ii->isIntrinsic() || ii->isExternal())) { - if (ii->getName() == funcname) { - return 1; - } - } - } - return 0; +int ExecutionEngine_runFunction(void* EE, void* F, int args_vector) { + ExecutionEngine* ee = (ExecutionEngine*)EE; + Function* f = (Function*)F; + std::vector args; + return ee->runFunction(f, args).IntVal; } Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/GenericValue.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/GenericValue.cpp Mon Apr 3 13:28:26 2006 @@ -0,0 +1,7 @@ +#include "llvm/ExecutionEngine/GenericValue.h" + +using namespace llvm; + +void* GenericValue__init__() { + return new GenericValue(); +} Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/DerivedTypes.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/DerivedTypes.cpp Mon Apr 3 13:28:26 2006 @@ -0,0 +1,18 @@ +#include "llvm/DerivedTypes.h" + +using namespace llvm; + +int FunctionType_getNumParams(void* FT) { + FunctionType* ft = (FunctionType*)FT; + return ft->getNumParams(); +} + +const void* FunctionType_getParamType(void* FT, int i) { + FunctionType* ft = (FunctionType*)FT; + return ft->getParamType(i); +} + +const void* FunctionType_getReturnType(void* FT) { + FunctionType* ft = (FunctionType*)FT; + return ft->getReturnType(); +} Modified: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Function.cpp Mon Apr 3 13:28:26 2006 @@ -2,6 +2,12 @@ using namespace llvm; -//void* Function__init__(void* M) { -// return new Function((Module*)M); -//} +void Function_eraseFromParent(void* F) { + Function* f = (Function*)F; + f->eraseFromParent(); +} + +const void* Function_getFunctionType(void* F) { + Function* f = (Function*)F; + return f->getFunctionType(); +} Modified: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Module.cpp Mon Apr 3 13:28:26 2006 @@ -23,24 +23,6 @@ return ((Module*)M)->setTargetTriple(T); } -/* -const char* Module_getEndianness(void* M) { - return ((Module*)M)->getEndianness().c_str(); -} - -void Module_setEndianness(void* M, Endianness E) { - return ((Module*)M)->setEndianness(E); -} - -const char* Module_getPointerSize(void* M) { - return ((Module*)M)->getPointerSize().c_str(); -} - -void Module_setPointerSize(void* M, PointerSize PS) { - return ((Module*)M)->setPointerSize(PS); -} -*/ - const char* Module_getModuleInlineAsm(void* M) { return ((Module*)M)->getModuleInlineAsm().c_str(); } @@ -49,6 +31,11 @@ ((Module*)M)->setModuleInlineAsm(Asm); } +void* Module_getNamedFunction(void* M, const char* fnname) { + return ((Module*)M)->getNamedFunction(fnname); +} + +// global functions, but they make more sense here void Module_ParseAssemblyString(void* M, const char* AsmString) { //from Assembly/Parser.h ParseAssemblyString(AsmString, (Module*)M); } @@ -56,3 +43,29 @@ int Module_verifyModule(void* M) { //from Analysis/Verifier.h return verifyModule(*(Module*)M, ThrowExceptionAction); } + +// helpers +int Module_n_functions(void* M) { + Module* mod = (Module*)M; + int funccount = 0; + Module::FunctionListType &fns = mod->getFunctionList(); + for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + funccount += 1; + } + } + return funccount; +} + +int Module_function_exists(void* M, const char* fnname) { + Module* mod = (Module*)M; + Module::FunctionListType &fns = mod->getFunctionList(); + for (Module::FunctionListType::const_iterator ii = fns.begin(); ii != fns.end(); ++ii) { + if (!(ii->isIntrinsic() || ii->isExternal())) { + if (ii->getName() == fnname) { + return 1; + } + } + } + return 0; +} Modified: pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp Mon Apr 3 13:28:26 2006 @@ -22,7 +22,9 @@ #include "VMCore/Module.cpp" #include "VMCore/ModuleProvider.cpp" #include "VMCore/Function.cpp" +#include "VMCore/DerivedTypes.cpp" #include "ExecutionEngine/ExecutionEngine.cpp" +#include "ExecutionEngine/GenericValue.cpp" void toggle_print_machineinstrs() { Modified: pypy/dist/pypy/translator/llvm/pythonllvm/helper.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/helper.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/helper.py Mon Apr 3 13:28:26 2006 @@ -10,12 +10,11 @@ def __call__(self, *args): a = [self.instance] - #for arg in args: - # if isinstance(arg, Wrapper): #pass on value to actual C (not Python) object - # a.append(arg.instance) - # else: - # a.append(arg) - a += args + for arg in args: + if isinstance(arg, Wrapper): #pass on value to actual C (not Python) object + a.append(arg.instance) + else: + a.append(arg) return apply(self.method, a) class Wrapper(object): Modified: pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py Mon Apr 3 13:28:26 2006 @@ -17,10 +17,52 @@ ExecutionEngine_getModule = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (ExecutionEngine_getModule) -def ExecutionEngine_n_functions(EE): +def ExecutionEngine_freeMachineCodeForFunction(EE, F): # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 14 - return ExecutionEngine_n_functions._api_(EE) -ExecutionEngine_n_functions = cdecl(c_int, 'llvmcapi', [c_void_p]) (ExecutionEngine_n_functions) + return ExecutionEngine_freeMachineCodeForFunction._api_(EE, F) +ExecutionEngine_freeMachineCodeForFunction = cdecl(None, 'llvmcapi', [c_void_p, c_void_p]) (ExecutionEngine_freeMachineCodeForFunction) + + +def ExecutionEngine_runFunction(EE, F, args_vector): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 17 + return ExecutionEngine_runFunction._api_(EE, F, args_vector) +ExecutionEngine_runFunction = cdecl(c_int, 'llvmcapi', [c_void_p, c_void_p, c_int]) (ExecutionEngine_runFunction) + + +def GenericValue__init__(): + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 8 + return GenericValue__init__._api_() +GenericValue__init__ = cdecl(c_void_p, 'llvmcapi', []) (GenericValue__init__) + + +def FunctionType_getNumParams(FT): + # ../llvmcapi/include/VMCore/DerivedTypes.h 8 + return FunctionType_getNumParams._api_(FT) +FunctionType_getNumParams = cdecl(c_int, 'llvmcapi', [c_void_p]) (FunctionType_getNumParams) + + +def FunctionType_getParamType(FT, i): + # ../llvmcapi/include/VMCore/DerivedTypes.h 9 + return FunctionType_getParamType._api_(FT, i) +FunctionType_getParamType = cdecl(c_void_p, 'llvmcapi', [c_void_p, c_int]) (FunctionType_getParamType) + + +def FunctionType_getReturnType(FT): + # ../llvmcapi/include/VMCore/DerivedTypes.h 10 + return FunctionType_getReturnType._api_(FT) +FunctionType_getReturnType = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (FunctionType_getReturnType) + + +def Function_eraseFromParent(F): + # ../llvmcapi/include/VMCore/Function.h 8 + return Function_eraseFromParent._api_(F) +Function_eraseFromParent = cdecl(None, 'llvmcapi', [c_void_p]) (Function_eraseFromParent) + + +def Function_getFunctionType(F): + # ../llvmcapi/include/VMCore/Function.h 9 + return Function_getFunctionType._api_(F) +Function_getFunctionType = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (Function_getFunctionType) def Module__init__(ModuleID): @@ -54,29 +96,47 @@ def Module_getModuleInlineAsm(M): - # ../llvmcapi/include/VMCore/Module.h 17 + # ../llvmcapi/include/VMCore/Module.h 13 return Module_getModuleInlineAsm._api_(M) Module_getModuleInlineAsm = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getModuleInlineAsm) def Module_setModuleInlineAsm(M, Asm): - # ../llvmcapi/include/VMCore/Module.h 18 + # ../llvmcapi/include/VMCore/Module.h 14 return Module_setModuleInlineAsm._api_(M, Asm) Module_setModuleInlineAsm = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setModuleInlineAsm) +def Module_getNamedFunction(M, fnname): + # ../llvmcapi/include/VMCore/Module.h 15 + return Module_getNamedFunction._api_(M, fnname) +Module_getNamedFunction = cdecl(c_void_p, 'llvmcapi', [c_void_p, STRING]) (Module_getNamedFunction) + + def Module_ParseAssemblyString(M, AsmString): - # ../llvmcapi/include/VMCore/Module.h 19 + # ../llvmcapi/include/VMCore/Module.h 18 return Module_ParseAssemblyString._api_(M, AsmString) Module_ParseAssemblyString = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_ParseAssemblyString) def Module_verifyModule(M): - # ../llvmcapi/include/VMCore/Module.h 20 + # ../llvmcapi/include/VMCore/Module.h 19 return Module_verifyModule._api_(M) Module_verifyModule = cdecl(c_int, 'llvmcapi', [c_void_p]) (Module_verifyModule) +def Module_n_functions(M): + # ../llvmcapi/include/VMCore/Module.h 22 + return Module_n_functions._api_(M) +Module_n_functions = cdecl(c_int, 'llvmcapi', [c_void_p]) (Module_n_functions) + + +def Module_function_exists(M, fnname): + # ../llvmcapi/include/VMCore/Module.h 23 + return Module_function_exists._api_(M, fnname) +Module_function_exists = cdecl(c_int, 'llvmcapi', [c_void_p, STRING]) (Module_function_exists) + + def ExistingModuleProvider__init__(M): # ../llvmcapi/include/VMCore/ModuleProvider.h 8 return ExistingModuleProvider__init__._api_(M) @@ -84,7 +144,7 @@ def toggle_print_machineinstrs(): - # ../llvmcapi/include/llvmcapi.h 13 + # ../llvmcapi/include/llvmcapi.h 15 return toggle_print_machineinstrs._api_() toggle_print_machineinstrs = cdecl(None, 'llvmcapi', []) (toggle_print_machineinstrs) Modified: pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py Mon Apr 3 13:28:26 2006 @@ -1,29 +1,89 @@ from helper import * + class Module(Wrapper): def __init__(self, ModuleID='mymodule'): self.instance = Module__init__(ModuleID) -class ModuleProvider(Wrapper): - pass + def getNamedFunction(self, fnname): + f = object.__new__(Function) + f.instance = Module_getNamedFunction(self.instance, fnname) + return f + -class ExistingModuleProvider(ModuleProvider): +class ExistingModuleProvider(Wrapper): def __init__(self, M=None): if not M: M = Module() self.instance = ExistingModuleProvider__init__(M.instance) +global ee_hack + class ExecutionEngine(Wrapper): def __init__(self, MP=None, ForceInterpreter=False): if not MP: MP = ExistingModuleProvider(); self.instance = ExecutionEngine__create__(MP.instance, ForceInterpreter) + global ee_hack + ee_hack = self.instance + # XXX cast to actual Python Module (can't we automate this?) def getModule(self): m = object.__new__(Module) m.instance = ExecutionEngine_getModule(self.instance) return m + # helpers def parse(self, llcode): - self.getModule().ParseAssemblyString(llcode) - self.getModule().verifyModule() + mod = self.getModule() + mod.ParseAssemblyString(llcode) + mod.verifyModule() + + def delete(self, fnname): + mod = self.getModule() + f = mod.getNamedFunction(fnname) # XXX handle fnname not found? + self.freeMachineCodeForFunction(f) # still no-op on march 27th 2006 + f.eraseFromParent() + + +def to_llvm_value(pythonvalue, type_): + # XXX use the GenericValue union instead + return pythonvalue + + +def to_python_value(llvmvalue, type_): + # XXX use the GenericValue union instead + return llvmvalue + + +class Function(Wrapper): + def __init__(self): + self.instance = Function__init__() #XXX this get annoying quickly + + def __call__(self, *args): + print 'calling %s(%s)' % ('Function', ','.join([str(arg) for arg in args])) + ft = Function_getFunctionType(self.instance) + argcount = FunctionType_getNumParams(ft) + print 'argcount = ',argcount + if argcount != len(args): + raise Exception("incorrect number of parameters") + llvmvalues = GenericValue__init__() + for i, arg in enumerate(args): + llvmvalues.append( to_llvm_value(arg, FunctionType_getParamType(ft, i))) + llvmreturnvalue = ExecutionEngine_runFunction(ee_hack, self.instance, llvmvalues) + return to_python_value(llvmreturnvalue, FunctionType_getReturnType(ft)) + + +class GenericValue(Wrapper): + def __init(self): + self.instance = GenericValue__init__() + + +#class Instruction(Wrapper): +# def __init__(self): +# self.instance = Instruction__init__() +# +# +#class BasicBlock(Wrapper): +# def __init__(self): +# self.instance = BasicBlock__init__() Modified: pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Mon Apr 3 13:28:26 2006 @@ -27,7 +27,8 @@ def test_functions(): ee = pyllvm.ExecutionEngine() ee.parse(codepath.join("hello.s").read()) - assert ee.n_functions() == 2 + mod = ee.getModule() + assert mod.n_functions() == 2 #TODO #for function in functions: # returnId, name, args = function @@ -35,27 +36,26 @@ # assert returnId > 0 # assert name in ('gethellostr', 'hello') # assert len(args) == 0 - py.test.raises(Exception, ee.n_functions, 1) - py.test.raises(Exception, ee.n_functions, "string") + py.test.raises(Exception, mod.n_functions, 1) + py.test.raises(Exception, mod.n_functions, "string") def test_call_parse_once(): ee = pyllvm.ExecutionEngine() ee.parse(codepath.join("hello.s").read()) - assert ee.call("hello") == 0 - assert ee.call("gethellostr") == "hello world\n" - py.test.raises(Exception, ee.call) - py.test.raises(Exception, ee.call, 1) - py.test.raises(Exception, ee.call, "gethellostrx") - py.test.raises(Exception, ee.call, "gethellostrx", 1) - py.test.raises(Exception, ee.call, "gethellostr", 1) + f = ee.getModule().getNamedFunction + hello = f("hello") + gethellostr = f("gethellostr") + assert hello() == 0 + assert gethellostr() == "hello world\n" def test_call_parse_twice(): ee = pyllvm.ExecutionEngine() ee.parse(codepath.join("hello.s").read()) - assert ee.call("gethellostr") == "hello world\n" + f = ee.getModule().getNamedFunction + assert f("gethellostr")() == "hello world\n" ee.parse(codepath.join("addnumbers.s").read()) - assert ee.call("add", 10, 32) == 42 - assert ee.call("gethellostr") == "hello world\n" + assert f("add")(10, 32) == 42 + assert f("gethellostr")() == "hello world\n" py.test.raises(Exception, ee.parse) py.test.raises(Exception, ee.parse, 1) py.test.raises(Exception, ee.parse, "abc") @@ -68,8 +68,9 @@ ee = pyllvm.ExecutionEngine() ee.parse(ll_snippet.calc) ee.parse(ll_snippet.add1) - assert ee.call("add1", 41) == 42 - assert ee.call("calc", 122) == 123 + f = ee.getModule().getNamedFunction + assert f("add1")(41) == 42 + assert f("calc")(122) == 123 def test_replace_function(): """similar to test_call_between_parsed_code with additional complexity @@ -78,46 +79,51 @@ ee = pyllvm.ExecutionEngine() ee.parse(ll_snippet.calc) ee.parse(ll_snippet.add1) - assert ee.call("add1", 41) == 42 - assert ee.call("calc", 122) == 123 #XXX need recompileAndRelinkFunction somewhere + f = ee.getModule().getNamedFunction + assert f("add1")(41) == 42 + assert f("calc")(122) == 123 #XXX need recompileAndRelinkFunction somewhere ee.parse(ll_snippet.add1_version2, "add1") - assert ee.call("add1", 42) == 142 - assert ee.call("calc", 142) == 242 + assert f("add1")(42) == 142 + assert f("calc")(142) == 242 def test_share_data_between_parsed_code(): ee = pyllvm.ExecutionEngine() ee.parse(ll_snippet.global_int_a_is_100) ee.parse(ll_snippet.add1_to_global_int_a) ee.parse(ll_snippet.sub10_from_global_int_a) - assert ee.call("add1_to_global_int_a") == 101 - assert ee.call("sub10_from_global_int_a") == 91 - assert ee.call("add1_to_global_int_a") == 92 - assert ee.call("sub10_from_global_int_a") == 82 + f = ee.getModule().getNamedFunction + assert f("add1_to_global_int_a")() == 101 + assert f("sub10_from_global_int_a")() == 91 + assert f("add1_to_global_int_a")() == 92 + assert f("sub10_from_global_int_a")() == 82 def test_native_code(): #examine JIT generate native (assembly) code pyllvm.toggle_print_machineinstrs() ee = pyllvm.ExecutionEngine() ee.parse(ll_snippet.calc) ee.parse(ll_snippet.add1) - assert ee.call("calc", 41) == 42 + f = ee.getModule().getNamedFunction + assert f("calc")(41) == 42 pyllvm.toggle_print_machineinstrs() def test_delete_function(): #this will only work if nothing uses Fn of course! ee = pyllvm.ExecutionEngine() + mod = ee.getModule() ee.parse(ll_snippet.calc) ee.parse(ll_snippet.add1) - assert ee.n_functions() == 2 + assert mod.n_functions() == 2 ee.delete("calc") - assert ee.n_functions() == 1 - assert ee.call("add1", 41) == 42 + assert mod.n_functions() == 1 + f = ee.getModule().getNamedFunction + assert f("add1")(41) == 42 ee.delete("add1") - assert ee.n_functions() == 0 + assert mod.n_functions() == 0 ee.parse(ll_snippet.calc) ee.parse(ll_snippet.add1) - assert ee.call("calc", 100) == 101 + assert f("calc")(100) == 101 def TODOtest_add_to_function(): pass From arigo at codespeak.net Mon Apr 3 14:39:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 14:39:36 +0200 (CEST) Subject: [pypy-svn] r25251 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060403123936.787A8100EC@code0.codespeak.net> Author: arigo Date: Mon Apr 3 14:39:30 2006 New Revision: 25251 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Log: (pedronis, arigo) Fixes for exactmatch(). Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Mon Apr 3 14:39:30 2006 @@ -91,8 +91,6 @@ assert isinstance(PTRTYPE.TO, lltype.Array) FieldDesc.__init__(self, PTRTYPE, PTRTYPE.TO.OF) -# ____________________________________________________________ - class StructFieldDesc(object): __metaclass__ = cachedtype @@ -138,24 +136,35 @@ rgenop.genop(jitstate.curblock, 'setfield', op_args, rgenop.gv_Void) +# ____________________________________________________________ + class FrozenVirtualStruct(AbstractContainer): - def exactmatch(self, vstruct, outgoingvarboxes, memo): + def __init__(self, typedesc): + self.typedesc = typedesc + #self.fz_content initialized later + + def exactmatch(self, box, outgoingvarboxes, memo): + vstruct = box.content contmemo = memo.containers if self in contmemo: - return vstruct is contmemo[self] + ok = vstruct is contmemo[self] + if not ok: + outgoingvarboxes.append(box) + return ok if vstruct in contmemo: - return contmemo[vstruct] is self + assert contmemo[vstruct] is not self + outgoingvarboxes.append(box) + return False contmemo[self] = vstruct contmemo[vstruct] = self - self_length = len(self.fz_content_boxes) - vstruct_length = len(vstruct.content_boxes) - if self_length != vstruct_length: + if self.typedesc is not vstruct.typedesc: + outgoingvarboxes.append(box) return False self_boxes = self.fz_content_boxes vstruct_boxes = vstruct.content_boxes fullmatch = True - for i in range(self_length): + for i in range(len(self_boxes)): if not self_boxes[i].exactmatch(vstruct_boxes[i], outgoingvarboxes, memo): @@ -180,12 +189,6 @@ for box in self.content_boxes: box.enter_block(newblock, incoming, memo) - def op_getfield(self, jitstate, fielddesc): - return self.content_boxes[fielddesc.fieldindex] - - def op_setfield(self, jitstate, fielddesc, valuebox): - self.content_boxes[fielddesc.fieldindex] = valuebox - def build_runtime_container(self, jitstate): if self.genvar: return self.genvar @@ -211,7 +214,7 @@ try: return contmemo[self] except KeyError: - result = contmemo[self] = FrozenVirtualStruct() + result = contmemo[self] = FrozenVirtualStruct(self.typedesc) frozens = [box.freeze(memo) for box in self.content_boxes] result.fz_content_boxes = frozens return result @@ -228,6 +231,12 @@ result.content_boxes = boxcopies return result + def op_getfield(self, jitstate, fielddesc): + return self.content_boxes[fielddesc.fieldindex] + + def op_setfield(self, jitstate, fielddesc, valuebox): + self.content_boxes[fielddesc.fieldindex] = valuebox + def op_getsubstruct(self, jitstate, fielddesc): #assert fielddesc.fieldindex == 0 return rvalue.PtrRedBox(fielddesc.gv_resulttype, content=self) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Mon Apr 3 14:39:30 2006 @@ -334,8 +334,4 @@ outgoingvarboxes.append(box) return False else: - result = self.fz_content.exactmatch(box.content, outgoingvarboxes, - memo) - if not result: - outgoingvarboxes.append(box) - return result + return self.fz_content.exactmatch(box, outgoingvarboxes, memo) From gromit at codespeak.net Mon Apr 3 14:41:23 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Mon, 3 Apr 2006 14:41:23 +0200 (CEST) Subject: [pypy-svn] r25252 - pypy/extradoc/sprintinfo/leysin-winter-2006 Message-ID: <20060403124123.B1641100F8@code0.codespeak.net> Author: gromit Date: Mon Apr 3 14:41:21 2006 New Revision: 25252 Modified: pypy/extradoc/sprintinfo/leysin-winter-2006/people.txt Log: Removed myslef from the Leysin people list. Modified: pypy/extradoc/sprintinfo/leysin-winter-2006/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/leysin-winter-2006/people.txt (original) +++ pypy/extradoc/sprintinfo/leysin-winter-2006/people.txt Mon Apr 3 14:41:21 2006 @@ -22,7 +22,6 @@ Niklaus Haldimann 2/4 - 5/4 Ermina Aurelien Campeas 3/4 - 7/4 Ermina Alexandre Fayolle 3/4 - 7/4 Ermina -Gerald Klix 8/4? - 9/4 Ermina (if the short time makes sense) ==================== ============== ===================== People on the following list were present at previous sprints: From nik at codespeak.net Mon Apr 3 15:05:28 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 3 Apr 2006 15:05:28 +0200 (CEST) Subject: [pypy-svn] r25253 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060403130528.A4B12100E8@code0.codespeak.net> Author: nik Date: Mon Apr 3 15:05:20 2006 New Revision: 25253 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: add append method to the ootype list interface (to have a bit more to work with for repr tests). Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 3 15:05:20 2006 @@ -175,11 +175,19 @@ def __init__(self, ITEMTYPE): self._ITEMTYPE = ITEMTYPE + # This defines the abstract list interface that backends will have + # to map to their native list implementations. + self._METHODS = frozendict({ + # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) + "length": Meth([], Unsigned), + "append": Meth([ITEMTYPE], Void), + }) + def __str__(self): return '%s(%s)' % (self.__class__.__name__, self._ITEMTYPE) def _lookup(self, meth_name): - METH = LIST_METHODS.get(meth_name) + METH = self._METHODS.get(meth_name) meth = None if METH is not None: meth = _meth(METH, _name=meth_name, @@ -438,13 +446,6 @@ return callb(self.inst, *checked_args) -# This defines the abstract list interface that backends will have to map to -# their native list implementations. -LIST_METHODS = frozendict({ - # "method name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) - "length": Meth([], Unsigned), -}) - class _list(object): def __init__(self, LIST): @@ -467,6 +468,10 @@ # NOT_RPYTHON return len(self._list) + def append(self, item): + # NOT_RPYTHON + assert typeOf(item) == self._TYPE._ITEMTYPE + self._list.append(item) def new(TYPE): if isinstance(TYPE, Instance): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Mon Apr 3 15:05:20 2006 @@ -10,3 +10,8 @@ l = new(LT) assert l.length() == 0 +def test_append(): + LT = List(Signed) + l = new(LT) + l.append(1) + assert l.length() == 1 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Mon Apr 3 15:05:20 2006 @@ -108,3 +108,15 @@ rettype = g.getreturnvar().concretetype assert rettype == Signed +def test_list_append(): + LT = List(Signed) + + def oof(): + l = new(LT) + l.append(1) + return l.length() + + g = gengraph(oof, []) + rettype = g.getreturnvar().concretetype + assert rettype == Signed + From hpk at codespeak.net Mon Apr 3 15:46:11 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 3 Apr 2006 15:46:11 +0200 (CEST) Subject: [pypy-svn] r25254 - pypy/extradoc/sprintinfo/leysin-winter-2006 Message-ID: <20060403134611.3C17A100FA@code0.codespeak.net> Author: hpk Date: Mon Apr 3 15:46:10 2006 New Revision: 25254 Added: pypy/extradoc/sprintinfo/leysin-winter-2006/planning.txt (contents, props changed) Log: first rough planning of tasks and days Added: pypy/extradoc/sprintinfo/leysin-winter-2006/planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/leysin-winter-2006/planning.txt Mon Apr 3 15:46:10 2006 @@ -0,0 +1,44 @@ + + +work and ski days +--------------------- + +monday: some start of hacking +tuesday: (jacob, arre, niklaus, alexandre, aurelien, michael) + 09:00 AM starting time for half-day ski-ing + 15:00 sprinting time + +wednesday-friday: sprinting, maybe more half-day ski-ing + +tasks tackled during this sprint +-------------------------------------- + +- progress on logic object space + (alexandre, aurelien, Anders) + - integrate constraint-solver + - implement stackless-thread-cloning + +- stackless as a CFG transformation + - heading towards thread pickling + (holger, michael) + +- move rctypes towards something usable + (Armin, Arre) + +- GC works + - finalize explicit-exceptions + (and GC cleanup) branch + - weakrefs + - GC performance + - support moving GCs + +- play with core optimizations + - type impls + - tagged integers + - lazy slicing / concatentation + +- progress on JIT + +- progress on ootype / gensqueak + (niklaus, samuele) + From auc at codespeak.net Mon Apr 3 16:47:22 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 3 Apr 2006 16:47:22 +0200 (CEST) Subject: [pypy-svn] r25256 - pypy/dist/pypy/objspace/logic Message-ID: <20060403144722.04160100FB@code0.codespeak.net> Author: auc Date: Mon Apr 3 16:47:22 2006 New Revision: 25256 Added: pypy/dist/pypy/objspace/logic/ Log: create new space for logic space... From arigo at codespeak.net Mon Apr 3 17:03:34 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 17:03:34 +0200 (CEST) Subject: [pypy-svn] r25257 - pypy/dist/pypy/rpython Message-ID: <20060403150334.A1BD8100FC@code0.codespeak.net> Author: arigo Date: Mon Apr 3 17:03:33 2006 New Revision: 25257 Modified: pypy/dist/pypy/rpython/rlist.py Log: We don't remember why this was added. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Mon Apr 3 17:03:33 2006 @@ -415,8 +415,7 @@ if r_lst1.listitem is None or r_lst2.listitem is None: return NotImplemented if r_lst1.listitem is not r_lst2.listitem: - if r_lst1.lowleveltype != r_lst2.lowleveltype: - return NotImplemented + return NotImplemented return v def rtype_is_((r_lst1, r_lst2), hop): From arigo at codespeak.net Mon Apr 3 17:04:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 17:04:18 +0200 (CEST) Subject: [pypy-svn] r25258 - in pypy/dist/pypy/rpython: . test Message-ID: <20060403150418.13E7810100@code0.codespeak.net> Author: arigo Date: Mon Apr 3 17:04:17 2006 New Revision: 25258 Modified: pypy/dist/pypy/rpython/rpbc.py pypy/dist/pypy/rpython/test/test_rpbc.py Log: Converting None pbcs to Multiple pbcs (getting the test right is fun). Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Mon Apr 3 17:04:17 2006 @@ -558,12 +558,13 @@ return pair(r_from.r_im_self, r_to.r_im_self).convert_from_to(v, llops) # __ None ____________________________________________________ -class NoneFrozenPBCRepr(SingleFrozenPBCRepr): - +class NoneFrozenPBCRepr(Repr): + lowleveltype = Void + def rtype_is_true(self, hop): return Constant(False, Bool) -none_frozen_pbc_repr = NoneFrozenPBCRepr(None) +none_frozen_pbc_repr = NoneFrozenPBCRepr() class __extend__(pairtype(Repr, NoneFrozenPBCRepr)): @@ -582,11 +583,6 @@ def rtype_is_((rnone1, robj2), hop): return hop.rtyper.type_system.rpbc.rtype_is_None( robj2, rnone1, hop, pos=1) - -class __extend__(pairtype(NoneFrozenPBCRepr, robject.PyObjRepr)): - - def convert_from_to(_, v, llops): - return inputconst(robject.pyobj_repr, None) # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Mon Apr 3 17:04:17 2006 @@ -1180,6 +1180,22 @@ res = interpret(f, [], type_system=self.ts) assert res == 165 * 1066 + def test_convert_none_to_frozen_pbc(self): + fr1 = Freezing(); fr1.x = 65 + fr2 = Freezing(); fr2.y = 65 + def g(fr): + return fr.x + def identity(z): + return z + def f(n): # NB. this crashes with n == 0 + if n == 0: + fr = identity(None) + else: + fr = fr1 + return g(fr) + res = interpret(f, [1], type_system=self.ts) + assert res == 65 + def test_call_from_list(): # Don't test with ootype, since it doesn't support lists in a From arigo at codespeak.net Mon Apr 3 17:08:03 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 17:08:03 +0200 (CEST) Subject: [pypy-svn] r25259 - pypy/dist/pypy/rpython Message-ID: <20060403150803.73504100DC@code0.codespeak.net> Author: arigo Date: Mon Apr 3 17:08:01 2006 New Revision: 25259 Modified: pypy/dist/pypy/rpython/llinterp.py Log: Saner warning message. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Mon Apr 3 17:08:01 2006 @@ -406,7 +406,7 @@ if hasattr(obj, 'graph'): assert obj.graph in graphs else: - print "this should ideally not happen", f, graphs, args + log.warn("op_indirect_call with graphs=None:", f) return self.op_direct_call(f, *args) def op_malloc(self, obj): From nik at codespeak.net Mon Apr 3 17:11:09 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 3 Apr 2006 17:11:09 +0200 (CEST) Subject: [pypy-svn] r25260 - in pypy/dist/pypy/rpython: . lltypesystem memory/test ootypesystem/test test Message-ID: <20060403151109.27C04100E5@code0.codespeak.net> Author: nik Date: Mon Apr 3 17:11:01 2006 New Revision: 25260 Added: pypy/dist/pypy/rpython/lltypesystem/rlist.py - copied, changed from r25252, pypy/dist/pypy/rpython/rlist.py Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/rtuple.py pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/rrange.py pypy/dist/pypy/rpython/rstr.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/rpython/test/test_llinterp.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/rpython/typesystem.py Log: split rlist into an lltype and an ootype version, touching a lot of code since there are many references to rlist. make basic rtyping of lists work for ootypesystem. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Mon Apr 3 17:11:01 2006 @@ -122,7 +122,7 @@ return lltype.typeOf(addr) == llmemory.Address def checkinst(inst): - return isinstance(lltype.typeOf(inst), ootype.Instance) + return isinstance(lltype.typeOf(inst), (ootype.Instance, ootype.List)) class LLFrame(object): def __init__(self, graph, args, llinterpreter, f_back=None): @@ -796,7 +796,7 @@ #Operation of ootype def op_new(self, INST): - assert isinstance(INST, ootype.Instance) + assert isinstance(INST, (ootype.Instance, ootype.List)) return ootype.new(INST) def op_runtimenew(self, class_): Modified: pypy/dist/pypy/rpython/lltypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rtuple.py Mon Apr 3 17:11:01 2006 @@ -39,7 +39,7 @@ return malloc(self.lowleveltype.TO) def rtype_bltn_list(self, hop): - from pypy.rpython import rlist + from pypy.rpython.lltypesystem import rlist nitems = len(self.items_r) vtup = hop.inputarg(self, 0) LIST = hop.r_result.lowleveltype.TO Modified: pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py Mon Apr 3 17:11:01 2006 @@ -3,7 +3,6 @@ from pypy.rpython.lltypesystem.lltype import typeOf, Ptr, PyObject from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.llinterp import LLInterpreter, LLException,log -from pypy.rpython.rlist import * from pypy.rpython.rint import signed_repr from pypy.rpython import rstr from pypy.annotation.model import lltype_to_annotation Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Mon Apr 3 17:11:01 2006 @@ -1,3 +1,4 @@ +from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.ootypesystem.ootype import * def test_new(): @@ -15,3 +16,14 @@ l = new(LT) l.append(1) assert l.length() == 1 + + +class TestInterpreted: + + def test_append_length(self): + def f(x): + l = [] + l.append(x) + return len(l) + res = interpret(f, [2], type_system="ootype") + assert res == 1 Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Mon Apr 3 17:11:01 2006 @@ -1,36 +1,7 @@ -from pypy.annotation.pairtype import pairtype, pair from pypy.annotation import model as annmodel -from pypy.objspace.flow.model import Constant -from pypy.rpython.error import TyperError -from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst -from pypy.rpython.rmodel import IteratorRepr, externalvsinternal -from pypy.rpython.rslice import SliceRepr -from pypy.rpython.rslice import startstop_slice_repr, startonly_slice_repr -from pypy.rpython.rslice import minusone_slice_repr -from pypy.rpython.lltypesystem. lltype import \ - GcForwardReference, Ptr, GcArray, GcStruct, \ - Void, Signed, malloc, typeOf, Primitive, \ - Bool, nullptr, typeMethod -from pypy.rpython import rstr +from pypy.rpython.rmodel import Repr, inputconst from pypy.rpython import robject -# ____________________________________________________________ -# -# Concrete implementation of RPython lists: -# -# struct list { -# int length; -# items_array *items; -# } -# -# 'items' points to a C-like array in memory preceded by a 'length' header, -# where each item contains a primitive value or pointer to the actual list -# item. -# -# or for fixed-size lists an array is directly used: -# -# item_t list_items[] -# class __extend__(annmodel.SomeList): def rtyper_makerepr(self, rtyper): @@ -44,936 +15,21 @@ else: # cannot do the rtyper.getrepr() call immediately, for the case # of recursive structures -- i.e. if the listdef contains itself + rlist = rtyper.type_system.rlist if self.listdef.listitem.resized: - return ListRepr(rtyper, lambda: rtyper.getrepr(listitem.s_value), - listitem) + return rlist.ListRepr(rtyper, + lambda: rtyper.getrepr(listitem.s_value), listitem) else: - return FixedSizeListRepr(rtyper, lambda: rtyper.getrepr(listitem.s_value), - listitem) + return rlist.FixedSizeListRepr(rtyper, + lambda: rtyper.getrepr(listitem.s_value), listitem) def rtyper_makekey(self): return self.__class__, self.listdef.listitem -class BaseListRepr(Repr): +class AbstractListRepr(Repr): - def __init__(self, rtyper, item_repr, listitem=None): - self.rtyper = rtyper - self.LIST = GcForwardReference() - self.lowleveltype = Ptr(self.LIST) - if not isinstance(item_repr, Repr): # not computed yet, done by setup() - assert callable(item_repr) - self._item_repr_computer = item_repr - else: - self.external_item_repr, self.item_repr = externalvsinternal(rtyper, item_repr) - self.listitem = listitem - self.list_cache = {} - # setup() needs to be called to finish this initialization - self.list_builder = ListBuilder() - - def _setup_repr_final(self): - self.list_builder.setup(self) - - def recast(self, llops, v): - return llops.convertvar(v, self.item_repr, self.external_item_repr) - - def convert_const(self, listobj): - # get object from bound list method - #listobj = getattr(listobj, '__self__', listobj) - if listobj is None: - return nullptr(self.LIST) - if not isinstance(listobj, list): - raise TyperError("expected a list: %r" % (listobj,)) - try: - key = Constant(listobj) - return self.list_cache[key] - except KeyError: - self.setup() - n = len(listobj) - result = self.prepare_const(n) - self.list_cache[key] = result - r_item = self.item_repr - if r_item.lowleveltype is not Void: - items = result.ll_items() - for i in range(n): - x = listobj[i] - items[i] = r_item.convert_const(x) - return result - - def prepare_const(self, nitems): - raise NotImplementedError - - def get_eqfunc(self): - return inputconst(Void, self.item_repr.get_ll_eq_function()) - - def rtype_bltn_list(self, hop): - v_lst = hop.inputarg(self, 0) - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - return hop.gendirectcall(ll_copy, cRESLIST, v_lst) - - def rtype_len(self, hop): - v_lst, = hop.inputargs(self) - return hop.gendirectcall(ll_len, v_lst) - - def rtype_is_true(self, hop): - v_lst, = hop.inputargs(self) - return hop.gendirectcall(ll_list_is_true, v_lst) - - def rtype_method_index(self, hop): - v_lst, v_value = hop.inputargs(self, self.item_repr) - hop.has_implicit_exception(ValueError) # record that we know about it - hop.exception_is_here() - return hop.gendirectcall(ll_listindex, v_lst, v_value, self.get_eqfunc()) - - def rtype_method_reverse(self, hop): - v_lst, = hop.inputargs(self) - hop.exception_cannot_occur() - hop.gendirectcall(ll_reverse,v_lst) - - def make_iterator_repr(self): - return ListIteratorRepr(self) - - def ll_str(self, l): - items = l.ll_items() - length = l.ll_length() - item_repr = self.item_repr - - temp = malloc(TEMP, length) - i = 0 - while i < length: - temp[i] = item_repr.ll_str(items[i]) - i += 1 - - return rstr.ll_strconcat( - rstr.list_str_open_bracket, - rstr.ll_strconcat(rstr.ll_join(rstr.list_str_sep, - length, - temp), - rstr.list_str_close_bracket)) - -class ListBuilder(object): - """Interface to allow lazy list building by the JIT.""" - # This should not keep a reference to the RTyper, even indirectly via - # the list_repr. - - def setup(self, list_repr): - # Precompute the c_newitem and c_setitem_nonneg function pointers, - # needed below. - if list_repr.rtyper is None: - return # only for test_rlist, which doesn't need this anyway - - LIST = list_repr.LIST - LISTPTR = list_repr.lowleveltype - ITEM = list_repr.item_repr.lowleveltype - self.LIST = LIST - self.LISTPTR = LISTPTR - - argtypes = [Signed] - fnptr = list_repr.rtyper.annotate_helper_fn(LIST.ll_newlist, argtypes) - self.newlist_ptr = fnptr - - bk = list_repr.rtyper.annotator.bookkeeper - argtypes = [bk.immutablevalue(dum_nocheck), LISTPTR, Signed, ITEM] - fnptr = list_repr.rtyper.annotate_helper_fn(ll_setitem_nonneg, argtypes) - self.setitem_nonneg_ptr = fnptr - #self.c_dum_nocheck = inputconst(Void, dum_nocheck) - #self.c_LIST = inputconst(Void, self.LIST) - - def __call__(self, builder, items_v): - """Make the operations that would build a list containing the - provided items.""" - from pypy.rpython import rgenop - c_newlist = builder.genconst(self.newlist_ptr) - c_len = builder.genconst(len(items_v)) - v_result = builder.genop('direct_call', - [c_newlist, rgenop.placeholder(self.LIST), c_len], - self.LISTPTR) - c_setitem_nonneg = builder.genconst(self.setitem_nonneg_ptr) - for i, v in enumerate(items_v): - c_i = builder.genconst(i) - builder.genop('direct_call', [c_setitem_nonneg, - rgenop.placeholder(dum_nocheck), - v_result, c_i, v], - Void) - return v_result - - def __eq__(self, other): - return self.LISTPTR == other.LISTPTR - - def __ne__(self, other): - return not (self == other) - - def __hash__(self): - return 1 # bad but not used alone - - -class ListRepr(BaseListRepr): - - def _setup_repr(self): - if 'item_repr' not in self.__dict__: - self.external_item_repr, self.item_repr = externalvsinternal(self.rtyper, self._item_repr_computer()) - if isinstance(self.LIST, GcForwardReference): - ITEM = self.item_repr.lowleveltype - ITEMARRAY = GcArray(ITEM) - # XXX we might think of turning length stuff into Unsigned - self.LIST.become(GcStruct("list", ("length", Signed), - ("items", Ptr(ITEMARRAY)), - adtmeths = { - "ll_newlist": ll_newlist, - "ll_length": ll_length, - "ll_items": ll_items, - "list_builder": self.list_builder, - "ITEM": ITEM, - }) - ) - - def compact_repr(self): - return 'ListR %s' % (self.item_repr.compact_repr(),) - - def prepare_const(self, n): - result = malloc(self.LIST, immortal=True) - result.length = n - result.items = malloc(self.LIST.items.TO, n) - return result - - def rtype_method_append(self, hop): - v_lst, v_value = hop.inputargs(self, self.item_repr) - hop.exception_cannot_occur() - hop.gendirectcall(ll_append, v_lst, v_value) - - def rtype_method_insert(self, hop): - v_lst, v_index, v_value = hop.inputargs(self, Signed, self.item_repr) - arg1 = hop.args_s[1] - args = v_lst, v_index, v_value - if arg1.is_constant() and arg1.const == 0: - llfn = ll_prepend - args = v_lst, v_value - elif arg1.nonneg: - llfn = ll_insert_nonneg - else: - raise TyperError("insert() index must be proven non-negative") - hop.exception_cannot_occur() - hop.gendirectcall(llfn, *args) - - def rtype_method_extend(self, hop): - v_lst1, v_lst2 = hop.inputargs(*hop.args_r) - hop.exception_cannot_occur() - hop.gendirectcall(ll_extend, v_lst1, v_lst2) - - def rtype_method_pop(self, hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - if hop.nb_args == 2: - args = hop.inputargs(self, Signed) - assert hasattr(args[1], 'concretetype') - arg1 = hop.args_s[1] - if arg1.is_constant() and arg1.const == 0: - llfn = ll_pop_zero - args = args[:1] - elif hop.args_s[1].nonneg: - llfn = ll_pop_nonneg - else: - llfn = ll_pop - else: - args = hop.inputargs(self) - llfn = ll_pop_default - hop.exception_is_here() - v_res = hop.gendirectcall(llfn, v_func, *args) - return self.recast(hop.llops, v_res) - -class FixedSizeListRepr(BaseListRepr): - - def _setup_repr(self): - if 'item_repr' not in self.__dict__: - self.external_item_repr, self.item_repr = externalvsinternal(self.rtyper, self._item_repr_computer()) - if isinstance(self.LIST, GcForwardReference): - ITEM = self.item_repr.lowleveltype - ITEMARRAY = GcArray(ITEM, - adtmeths = { - "ll_newlist": ll_fixed_newlist, - "ll_length": ll_fixed_length, - "ll_items": ll_fixed_items, - "list_builder": self.list_builder, - "ITEM": ITEM, - }) - - self.LIST.become(ITEMARRAY) - - def compact_repr(self): - return 'FixedSizeListR %s' % (self.item_repr.compact_repr(),) - - def prepare_const(self, n): - result = malloc(self.LIST, n, immortal=True) - return result - -class __extend__(pairtype(BaseListRepr, Repr)): - - def rtype_contains((r_lst, _), hop): - v_lst, v_any = hop.inputargs(r_lst, r_lst.item_repr) - return hop.gendirectcall(ll_listcontains, v_lst, v_any, r_lst.get_eqfunc()) - - -class __extend__(pairtype(BaseListRepr, IntegerRepr)): - - def rtype_getitem((r_lst, r_int), hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index = hop.inputargs(r_lst, Signed) - if hop.args_s[1].nonneg: - llfn = ll_getitem_nonneg - else: - llfn = ll_getitem - hop.exception_is_here() - v_res = hop.gendirectcall(llfn, v_func, v_lst, v_index) - return r_lst.recast(hop.llops, v_res) - - def rtype_setitem((r_lst, r_int), hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index, v_item = hop.inputargs(r_lst, Signed, r_lst.item_repr) - if hop.args_s[1].nonneg: - llfn = ll_setitem_nonneg - else: - llfn = ll_setitem - hop.exception_is_here() - return hop.gendirectcall(llfn, v_func, v_lst, v_index, v_item) - - def rtype_mul((r_lst, r_int), hop): - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - v_lst, v_factor = hop.inputargs(r_lst, Signed) - return hop.gendirectcall(ll_mul, cRESLIST, v_lst, v_factor) - -class __extend__(pairtype(ListRepr, IntegerRepr)): - - def rtype_delitem((r_lst, r_int), hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index = hop.inputargs(r_lst, Signed) - if hop.args_s[1].nonneg: - llfn = ll_delitem_nonneg - else: - llfn = ll_delitem - hop.exception_is_here() - return hop.gendirectcall(llfn, v_func, v_lst, v_index) - - def rtype_inplace_mul((r_lst, r_int), hop): - v_lst, v_factor = hop.inputargs(r_lst, Signed) - return hop.gendirectcall(ll_inplace_mul, v_lst, v_factor) - -class __extend__(pairtype(BaseListRepr, SliceRepr)): - - def rtype_getitem((r_lst, r_slic), hop): - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - if r_slic == startonly_slice_repr: - v_lst, v_start = hop.inputargs(r_lst, startonly_slice_repr) - return hop.gendirectcall(ll_listslice_startonly, cRESLIST, v_lst, v_start) - if r_slic == startstop_slice_repr: - v_lst, v_slice = hop.inputargs(r_lst, startstop_slice_repr) - return hop.gendirectcall(ll_listslice, cRESLIST, v_lst, v_slice) - if r_slic == minusone_slice_repr: - v_lst, v_ignored = hop.inputargs(r_lst, minusone_slice_repr) - return hop.gendirectcall(ll_listslice_minusone, cRESLIST, v_lst) - raise TyperError('getitem does not support slices with %r' % (r_slic,)) - - def rtype_setitem((r_lst, r_slic), hop): - #if r_slic == startonly_slice_repr: - # not implemented - if r_slic == startstop_slice_repr: - v_lst, v_slice, v_lst2 = hop.inputargs(r_lst, startstop_slice_repr, - hop.args_r[2]) - hop.gendirectcall(ll_listsetslice, v_lst, v_slice, v_lst2) - return - raise TyperError('setitem does not support slices with %r' % (r_slic,)) - -class __extend__(pairtype(ListRepr, SliceRepr)): - - def rtype_delitem((r_lst, r_slic), hop): - if r_slic == startonly_slice_repr: - v_lst, v_start = hop.inputargs(r_lst, startonly_slice_repr) - hop.gendirectcall(ll_listdelslice_startonly, v_lst, v_start) - return - if r_slic == startstop_slice_repr: - v_lst, v_slice = hop.inputargs(r_lst, startstop_slice_repr) - hop.gendirectcall(ll_listdelslice, v_lst, v_slice) - return - raise TyperError('delitem does not support slices with %r' % (r_slic,)) - -class __extend__(pairtype(BaseListRepr, BaseListRepr)): - def convert_from_to((r_lst1, r_lst2), v, llops): - if r_lst1.listitem is None or r_lst2.listitem is None: - return NotImplemented - if r_lst1.listitem is not r_lst2.listitem: - return NotImplemented - return v - - def rtype_is_((r_lst1, r_lst2), hop): - if r_lst1.lowleveltype != r_lst2.lowleveltype: - # obscure logic, the is can be true only if both are None - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - return hop.gendirectcall(ll_both_none, v_lst1, v_lst2) - - return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) - - def rtype_add((r_lst1, r_lst2), hop): - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - return hop.gendirectcall(ll_concat, cRESLIST, v_lst1, v_lst2) - - def rtype_eq((r_lst1, r_lst2), hop): - assert r_lst1.item_repr == r_lst2.item_repr - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - return hop.gendirectcall(ll_listeq, v_lst1, v_lst2, r_lst1.get_eqfunc()) - - def rtype_ne((r_lst1, r_lst2), hop): - assert r_lst1.item_repr == r_lst2.item_repr - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - flag = hop.gendirectcall(ll_listeq, v_lst1, v_lst2, r_lst1.get_eqfunc()) - return hop.genop('bool_not', [flag], resulttype=Bool) - -def ll_both_none(lst1, lst2): - return not lst1 and not lst2 - -class __extend__(pairtype(ListRepr, BaseListRepr)): - - def rtype_inplace_add((r_lst1, r_lst2), hop): - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - hop.gendirectcall(ll_extend, v_lst1, v_lst2) - return v_lst1 - - -# ____________________________________________________________ -# -# Low-level methods. These can be run for testing, but are meant to -# be direct_call'ed from rtyped flow graphs, which means that they will -# get flowed and annotated, mostly with SomePtr. - -# adapted C code - -def _ll_list_resize_really(l, newsize): - """ - Ensure ob_item has room for at least newsize elements, and set - ob_size to newsize. If newsize > ob_size on entry, the content - of the new slots at exit is undefined heap trash; it's the caller's - responsiblity to overwrite them with sane values. - The number of allocated elements may grow, shrink, or stay the same. - Failure is impossible if newsize <= self.allocated on entry, although - that partly relies on an assumption that the system realloc() never - fails when passed a number of bytes <= the number of bytes last - allocated (the C standard doesn't guarantee this, but it's hard to - imagine a realloc implementation where it wouldn't be true). - Note that self->ob_item may change, and even if newsize is less - than ob_size on entry. - """ - allocated = len(l.items) - - # This over-allocates proportional to the list size, making room - # for additional growth. The over-allocation is mild, but is - # enough to give linear-time amortized behavior over a long - # sequence of appends() in the presence of a poorly-performing - # system realloc(). - # The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... - ## (newsize < 9 ? 3 : 6) - if newsize < 9: - some = 3 - else: - some = 6 - new_allocated = (newsize >> 3) + some + newsize - if newsize == 0: - new_allocated = 0 - # XXX consider to have a real realloc - items = l.items - newitems = malloc(typeOf(l).TO.items.TO, new_allocated) - if allocated < new_allocated: - p = allocated - 1 - else: - p = new_allocated - 1 - while p >= 0: - newitems[p] = items[p] - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - items[p] = nullptr(ITEM.TO) - p -= 1 - l.length = newsize - l.items = newitems - -# this common case was factored out of _ll_list_resize -# to see if inlining it gives some speed-up. - -def _ll_list_resize(l, newsize): - # Bypass realloc() when a previous overallocation is large enough - # to accommodate the newsize. If the newsize falls lower than half - # the allocated size, then proceed with the realloc() to shrink the list. - allocated = len(l.items) - if allocated >= newsize and newsize >= ((allocated >> 1) - 5): - l.length = newsize - else: - _ll_list_resize_really(l, newsize) - -def _ll_list_resize_ge(l, newsize): - if len(l.items) >= newsize: - l.length = newsize - else: - _ll_list_resize_really(l, newsize) - -def _ll_list_resize_le(l, newsize): - if newsize >= (len(l.items) >> 1) - 5: - l.length = newsize - else: - _ll_list_resize_really(l, newsize) - - -def ll_copy(RESLIST, l): - items = l.ll_items() - length = l.ll_length() - new_lst = RESLIST.ll_newlist(length) - i = 0 - new_items = new_lst.ll_items() - while i < length: - new_items[i] = items[i] - i += 1 - return new_lst -ll_copy.oopspec = 'list.copy(l)' - -def ll_len(l): - return l.ll_length() -ll_len.oopspec = 'list.len(l)' - -def ll_list_is_true(l): - # check if a list is True, allowing for None - return bool(l) and l.ll_length() != 0 -ll_list_is_true.oopspec = 'list.nonzero(l)' - -def ll_append(l, newitem): - length = l.length - _ll_list_resize_ge(l, length+1) - l.items[length] = newitem -ll_append.oopspec = 'list.append(l, newitem)' - -# this one is for the special case of insert(0, x) -def ll_prepend(l, newitem): - length = l.length - _ll_list_resize_ge(l, length+1) - items = l.items - dst = length - while dst > 0: - src = dst - 1 - items[dst] = items[src] - dst = src - items[0] = newitem -ll_prepend.oopspec = 'list.insert(l, 0, newitem)' - -def ll_insert_nonneg(l, index, newitem): - length = l.length - _ll_list_resize_ge(l, length+1) - items = l.items - dst = length - while dst > index: - src = dst - 1 - items[dst] = items[src] - dst = src - items[index] = newitem -ll_insert_nonneg.oopspec = 'list.insert(l, index, newitem)' - -def dum_checkidx(): pass -def dum_nocheck(): pass - -def ll_pop_nonneg(func, l, index): - if func is dum_checkidx and (index >= l.length): - raise IndexError - res = l.items[index] - ll_delitem_nonneg(dum_nocheck, l, index) - return res -ll_pop_nonneg.oopspec = 'list.pop(l, index)' - -def ll_pop_default(func, l): - length = l.length - if func is dum_checkidx and (length == 0): - raise IndexError - index = length - 1 - newlength = index - items = l.items - res = items[index] - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - items[index] = nullptr(ITEM.TO) - _ll_list_resize_le(l, newlength) - return res -ll_pop_default.oopspec = 'list.pop(l)' - -def ll_pop_zero(func, l): - length = l.length - if func is dum_checkidx and (length == 0): - raise IndexError - newlength = length - 1 - res = l.items[0] - j = 0 - items = l.items - j1 = j+1 - while j < newlength: - items[j] = items[j1] - j = j1 - j1 += 1 - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - items[newlength] = nullptr(ITEM.TO) - _ll_list_resize_le(l, newlength) - return res -ll_pop_zero.oopspec = 'list.pop(l, 0)' - -def ll_pop(func, l, index): - length = l.length - if index < 0: - index += length - if func is dum_checkidx and (index < 0 or index >= length): - raise IndexError - res = l.items[index] - ll_delitem_nonneg(dum_nocheck, l, index) - return res -ll_pop.oopspec = 'list.pop(l, index)' - -def ll_reverse(l): - length = l.ll_length() - i = 0 - items = l.ll_items() - length_1_i = length-1-i - while i < length_1_i: - tmp = items[i] - items[i] = items[length_1_i] - items[length_1_i] = tmp - i += 1 - length_1_i -= 1 -ll_reverse.oopspec = 'list.reverse(l)' - -def ll_getitem_nonneg(func, l, index): - if func is dum_checkidx and (index >= l.ll_length()): - raise IndexError - return l.ll_items()[index] -ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' - -def ll_getitem(func, l, index): - length = l.ll_length() - if index < 0: - index += length - if func is dum_checkidx and (index < 0 or index >= length): - raise IndexError - return l.ll_items()[index] -ll_getitem.oopspec = 'list.getitem(l, index)' - -def ll_setitem_nonneg(func, l, index, newitem): - if func is dum_checkidx and (index >= l.ll_length()): - raise IndexError - l.ll_items()[index] = newitem -ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' - -def ll_setitem(func, l, index, newitem): - length = l.ll_length() - if index < 0: - index += length - if func is dum_checkidx and (index < 0 or index >= length): - raise IndexError - l.ll_items()[index] = newitem -ll_setitem.oopspec = 'list.setitem(l, index, newitem)' - -def ll_delitem_nonneg(func, l, index): - length = l.length - if func is dum_checkidx and (index >= length): - raise IndexError - newlength = length - 1 - j = index - items = l.items - j1 = j+1 - while j < newlength: - items[j] = items[j1] - j = j1 - j1 += 1 - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - items[newlength] = nullptr(ITEM.TO) - _ll_list_resize_le(l, newlength) -ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' - -def ll_delitem(func, l, i): - length = l.length - if i < 0: - i += length - if func is dum_checkidx and (i < 0 or i >= length): - raise IndexError - ll_delitem_nonneg(dum_nocheck, l, i) -ll_delitem.oopspec = 'list.delitem(l, i)' - -def ll_concat(RESLIST, l1, l2): - len1 = l1.ll_length() - len2 = l2.ll_length() - newlength = len1 + len2 - l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() - j = 0 - source = l1.ll_items() - while j < len1: - newitems[j] = source[j] - j += 1 - i = 0 - source = l2.ll_items() - while i < len2: - newitems[j] = source[i] - i += 1 - j += 1 - return l -ll_concat.oopspec = 'list.concat(l1, l2)' - -def ll_extend(l1, l2): - len1 = l1.length - len2 = l2.ll_length() - newlength = len1 + len2 - _ll_list_resize_ge(l1, newlength) - items = l1.items - source = l2.ll_items() - i = 0 - j = len1 - while i < len2: - items[j] = source[i] - i += 1 - j += 1 - -def ll_listslice_startonly(RESLIST, l1, start): - len1 = l1.ll_length() - newlength = len1 - start - l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() - j = 0 - source = l1.ll_items() - i = start - while i < len1: - newitems[j] = source[i] - i += 1 - j += 1 - return l - -def ll_listslice(RESLIST, l1, slice): - start = slice.start - stop = slice.stop - length = l1.ll_length() - if stop > length: - stop = length - newlength = stop - start - l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() - j = 0 - source = l1.ll_items() - i = start - while i < stop: - newitems[j] = source[i] - i += 1 - j += 1 - return l - -def ll_listslice_minusone(RESLIST, l1): - newlength = l1.ll_length() - 1 - assert newlength >= 0 - l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() - j = 0 - source = l1.ll_items() - while j < newlength: - newitems[j] = source[j] - j += 1 - return l - -def ll_listdelslice_startonly(l, start): - newlength = start - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - j = l.length - 1 - items = l.items - while j >= newlength: - items[j] = nullptr(ITEM.TO) - j -= 1 - _ll_list_resize_le(l, newlength) - -def ll_listdelslice(l, slice): - start = slice.start - stop = slice.stop - if stop > l.length: - stop = l.length - newlength = l.length - (stop-start) - j = start - items = l.items - i = stop - while j < newlength: - items[j] = items[i] - i += 1 - j += 1 - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - j = l.length - 1 - while j >= newlength: - items[j] = nullptr(ITEM.TO) - j -= 1 - _ll_list_resize_le(l, newlength) - -def ll_listsetslice(l1, slice, l2): - count = l2.ll_length() - assert count == slice.stop - slice.start, ( - "setslice cannot resize lists in RPython") - # XXX but it should be easy enough to support, soon - start = slice.start - j = start - items1 = l1.ll_items() - items2 = l2.ll_items() - i = 0 - while i < count: - items1[j] = items2[i] - i += 1 - j += 1 - -# ____________________________________________________________ -# -# Comparison. - -def ll_listeq(l1, l2, eqfn): - if not l1 and not l2: - return True - if not l1 or not l2: - return False - len1 = l1.ll_length() - len2 = l2.ll_length() - if len1 != len2: - return False - j = 0 - items1 = l1.ll_items() - items2 = l2.ll_items() - while j < len1: - if eqfn is None: - if items1[j] != items2[j]: - return False - else: - if not eqfn(items1[j], items2[j]): - return False - j += 1 - return True - -def ll_listcontains(lst, obj, eqfn): - items = lst.ll_items() - lng = lst.ll_length() - j = 0 - while j < lng: - if eqfn is None: - if items[j] == obj: - return True - else: - if eqfn(items[j], obj): - return True - j += 1 - return False - -def ll_listindex(lst, obj, eqfn): - items = lst.ll_items() - lng = lst.ll_length() - j = 0 - while j < lng: - if eqfn is None: - if items[j] == obj: - return j - else: - if eqfn(items[j], obj): - return j - j += 1 - raise ValueError # can't say 'list.index(x): x not in list' - -TEMP = GcArray(Ptr(rstr.STR)) - -def ll_inplace_mul(l, factor): - length = l.ll_length() - if factor < 0: - factor = 0 - resultlen = length * factor - res = l - _ll_list_resize(res, resultlen) - j = length - source = l.ll_items() - target = res.ll_items() - while j < resultlen: - i = 0 - while i < length: - p = j + i - target[p] = source[i] - i += 1 - j += length - return res - - -def ll_mul(RESLIST, l, factor): - length = l.ll_length() - if factor < 0: - factor = 0 - resultlen = length * factor - res = RESLIST.ll_newlist(resultlen) - j = 0 - source = l.ll_items() - target = res.ll_items() - while j < resultlen: - i = 0 - while i < length: - p = j + i - target[p] = source[i] - i += 1 - j += length - return res - - -# ____________________________________________________________ -# -# Irregular operations. - -def ll_newlist(LIST, length): - l = malloc(LIST) - l.length = length - l.items = malloc(LIST.items.TO, length) - return l -ll_newlist = typeMethod(ll_newlist) -ll_newlist.oopspec = 'newlist(length)' - -def ll_length(l): - return l.length - -def ll_items(l): - return l.items - -# fixed size versions - -def ll_fixed_newlist(LIST, length): - l = malloc(LIST, length) - return l -ll_fixed_newlist = typeMethod(ll_fixed_newlist) -ll_fixed_newlist.oopspec = 'newlist(length)' - -def ll_fixed_length(l): - return len(l) - -def ll_fixed_items(l): - return l - -def newlist(llops, r_list, items_v): - LIST = r_list.LIST - cno = inputconst(Signed, len(items_v)) - v_result = llops.gendirectcall(LIST.ll_newlist, cno) - v_func = inputconst(Void, dum_nocheck) - for i, v_item in enumerate(items_v): - ci = inputconst(Signed, i) - llops.gendirectcall(ll_setitem_nonneg, v_func, v_result, ci, v_item) - return v_result + pass def rtype_newlist(hop): nb_args = hop.nb_args @@ -989,61 +45,9 @@ return v_result r_listitem = r_list.item_repr items_v = [hop.inputarg(r_listitem, arg=i) for i in range(nb_args)] - return newlist(hop.llops, r_list, items_v) + return hop.rtyper.type_system.rlist.newlist(hop.llops, r_list, items_v) -def ll_alloc_and_set(LIST, count, item): - if count < 0: - count = 0 - l = LIST.ll_newlist(count) - if item: # as long as malloc it is known to zero the allocated memory avoid zeroing twice - i = 0 - items = l.ll_items() - while i < count: - items[i] = item - i += 1 - return l -ll_alloc_and_set.oopspec = 'newlist(count, item)' -def rtype_alloc_and_set(hop): - r_list = hop.r_result - v_count, v_item = hop.inputargs(Signed, r_list.item_repr) - cLIST = hop.inputconst(Void, r_list.LIST) - return hop.gendirectcall(ll_alloc_and_set, cLIST, v_count, v_item) - -# ____________________________________________________________ -# -# Iteration. - -class ListIteratorRepr(IteratorRepr): - - def __init__(self, r_list): - self.r_list = r_list - self.lowleveltype = Ptr(GcStruct('listiter', - ('list', r_list.lowleveltype), - ('index', Signed))) - - def newiter(self, hop): - v_lst, = hop.inputargs(self.r_list) - citerptr = hop.inputconst(Void, self.lowleveltype) - return hop.gendirectcall(ll_listiter, citerptr, v_lst) - - def rtype_next(self, hop): - v_iter, = hop.inputargs(self) - hop.has_implicit_exception(StopIteration) # record that we know about it - hop.exception_is_here() - v_res = hop.gendirectcall(ll_listnext, v_iter) - return self.r_list.recast(hop.llops, v_res) - -def ll_listiter(ITERPTR, lst): - iter = malloc(ITERPTR.TO) - iter.list = lst - iter.index = 0 - return iter +def dum_checkidx(): pass +def dum_nocheck(): pass -def ll_listnext(iter): - l = iter.list - index = iter.index - if index >= l.ll_length(): - raise StopIteration - iter.index = index + 1 - return l.ll_items()[index] Modified: pypy/dist/pypy/rpython/rrange.py ============================================================================== --- pypy/dist/pypy/rpython/rrange.py (original) +++ pypy/dist/pypy/rpython/rrange.py Mon Apr 3 17:11:01 2006 @@ -50,7 +50,6 @@ class __extend__(pairtype(RangeRepr, IntegerRepr)): def rtype_getitem((r_rng, r_int), hop): - from pypy.rpython.rlist import dum_nocheck, dum_checkidx if hop.has_implicit_exception(IndexError): spec = dum_checkidx else: Modified: pypy/dist/pypy/rpython/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/rstr.py (original) +++ pypy/dist/pypy/rpython/rstr.py Mon Apr 3 17:11:01 2006 @@ -173,8 +173,7 @@ if hop.s_result.is_constant(): return inputconst(string_repr, hop.s_result.const) r_lst = hop.args_r[1] - from pypy.rpython.rlist import BaseListRepr - if not isinstance(r_lst, BaseListRepr): + if not isinstance(r_lst, hop.rtyper.type_system.rlist.BaseListRepr): raise TyperError("string.join of non-list: %r" % r_lst) v_str, v_lst = hop.inputargs(string_repr, r_lst) LIST = r_lst.lowleveltype.TO Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Mon Apr 3 17:11:01 2006 @@ -504,13 +504,13 @@ # __________ irregular operations __________ def translate_op_newlist(self, hop): - return rlist.rtype_newlist(hop) + return self.type_system.rlist.rtype_newlist(hop) def translate_op_newdict(self, hop): return rdict.rtype_newdict(hop) def translate_op_alloc_and_set(self, hop): - return rlist.rtype_alloc_and_set(hop) + return self.type_system.rlist.rtype_alloc_and_set(hop) def translate_op_newtuple(self, hop): return self.type_system.rtuple.rtype_newtuple(hop) @@ -842,7 +842,7 @@ from pypy.rpython import robject from pypy.rpython import rint, rbool, rfloat from pypy.rpython import rslice, rrange -from pypy.rpython import rlist, rstr, rdict +from pypy.rpython import rstr, rdict from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase from pypy.rpython import rexternalobj from pypy.rpython import rptr Modified: pypy/dist/pypy/rpython/test/test_llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_llinterp.py (original) +++ pypy/dist/pypy/rpython/test/test_llinterp.py Mon Apr 3 17:11:01 2006 @@ -5,9 +5,9 @@ from pypy.rpython.llinterp import LLInterpreter, LLException, log from pypy.rpython.rmodel import inputconst from pypy.translator.translator import TranslationContext -from pypy.rpython.rlist import * from pypy.rpython.rint import signed_repr from pypy.rpython import rstr +from pypy.annotation import model as annmodel from pypy.annotation.model import lltype_to_annotation from pypy.rpython.rarithmetic import r_uint, ovfcheck from pypy import conftest Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Mon Apr 3 17:11:01 2006 @@ -1,7 +1,7 @@ import sys from pypy.translator.translator import TranslationContext from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.rlist import * +from pypy.rpython.lltypesystem.rlist import * from pypy.rpython.rslice import ll_newslice from pypy.rpython.rint import signed_repr from pypy.rpython.test.test_llinterp import interpret, interpret_raises Modified: pypy/dist/pypy/rpython/typesystem.py ============================================================================== --- pypy/dist/pypy/rpython/typesystem.py (original) +++ pypy/dist/pypy/rpython/typesystem.py Mon Apr 3 17:11:01 2006 @@ -20,7 +20,8 @@ None, None, ['__doc__']) except ImportError: return None - if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'exceptiondata'): + if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', + 'exceptiondata'): mod = load(name) if mod is not None: setattr(self, name, mod) From nik at codespeak.net Mon Apr 3 17:24:31 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 3 Apr 2006 17:24:31 +0200 (CEST) Subject: [pypy-svn] r25261 - in pypy/dist/pypy: jit/timeshifter translator/c translator/llvm Message-ID: <20060403152431.4755110103@code0.codespeak.net> Author: nik Date: Mon Apr 3 17:24:29 2006 New Revision: 25261 Modified: pypy/dist/pypy/jit/timeshifter/timeshift.py pypy/dist/pypy/translator/c/extfunc.py pypy/dist/pypy/translator/llvm/extfunchelper.py Log: fix some references to rlist that i forgot in the last checkin. Modified: pypy/dist/pypy/jit/timeshifter/timeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/timeshift.py (original) +++ pypy/dist/pypy/jit/timeshifter/timeshift.py Mon Apr 3 17:24:29 2006 @@ -6,7 +6,8 @@ from pypy.jit.timeshifter.rtimeshift import VARLIST, RedBox, VarRedBox from pypy.jit.timeshifter.rtimeshift import ConstRedBox, JITState from pypy.jit.timeshifter.rtimeshift import make_types_const -from pypy.rpython import rmodel, rlist, rdict, rgenop, annlowlevel +from pypy.rpython import rmodel, rdict, rgenop, annlowlevel +from pypy.rpython.lltypesystem import rlist from pypy.rpython.lltypesystem import rtuple from pypy.jit.timeshifter import rtimeshift from pypy.jit.timeshifter.rtyper import HintRTyper, originalconcretetype Modified: pypy/dist/pypy/translator/c/extfunc.py ============================================================================== --- pypy/dist/pypy/translator/c/extfunc.py (original) +++ pypy/dist/pypy/translator/c/extfunc.py Mon Apr 3 17:24:29 2006 @@ -3,7 +3,8 @@ from pypy.rpython.lltypesystem import lltype from pypy.translator.c.support import cdecl from pypy.rpython.rstr import STR -from pypy.rpython import rlist, rstr +from pypy.rpython import rstr +from pypy.rpython.lltypesystem import rlist from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack from pypy.module.thread.rpython import ll_thread Modified: pypy/dist/pypy/translator/llvm/extfunchelper.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfunchelper.py (original) +++ pypy/dist/pypy/translator/llvm/extfunchelper.py Mon Apr 3 17:24:29 2006 @@ -3,7 +3,8 @@ from pypy.rpython.lltypesystem import lltype from pypy.translator.c.support import cdecl from pypy.rpython.rstr import STR -from pypy.rpython import rlist, rstr +from pypy.rpython import rstr +from pypy.rpython.lltypesystem import rlist from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack from pypy.module.thread.rpython import ll_thread From arigo at codespeak.net Mon Apr 3 17:35:27 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 17:35:27 +0200 (CEST) Subject: [pypy-svn] r25262 - in pypy/dist/pypy: annotation rpython rpython/test Message-ID: <20060403153527.8455F10104@code0.codespeak.net> Author: arigo Date: Mon Apr 3 17:35:26 2006 New Revision: 25262 Modified: pypy/dist/pypy/annotation/description.py pypy/dist/pypy/rpython/rpbc.py pypy/dist/pypy/rpython/test/test_rpbc.py Log: (pedronis, arigo) Yet another obscure rpbc case, this time requiring some more information from the annotator. Modified: pypy/dist/pypy/annotation/description.py ============================================================================== --- pypy/dist/pypy/annotation/description.py (original) +++ pypy/dist/pypy/annotation/description.py Mon Apr 3 17:35:26 2006 @@ -628,10 +628,18 @@ super(FrozenDesc, self).__init__(bookkeeper, pyobj) if read_attribute is None: read_attribute = lambda attr: getattr(pyobj, attr) - self.read_attribute = read_attribute + self._read_attribute = read_attribute + self.attrcache = {} self.knowntype = new_or_old_class(pyobj) assert bool(pyobj), "__nonzero__ unsupported on frozen PBC %r" %(pyobj,) + def read_attribute(self, attr): + try: + return self.attrcache[attr] + except KeyError: + result = self.attrcache[attr] = self._read_attribute(attr) + return result + def s_read_attribute(self, attr): try: value = self.read_attribute(attr) @@ -648,13 +656,7 @@ pass else: raise AssertionError("name clash: %r" % (name,)) - def extended_read_attribute(attr): - if attr == name: - return value - else: - return previous_read_attribute(attr) - previous_read_attribute = self.read_attribute - self.read_attribute = extended_read_attribute + self.attrcache[name] = value class MethodOfFrozenDesc(Desc): Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Mon Apr 3 17:35:26 2006 @@ -446,8 +446,8 @@ if r_value.lowleveltype is Void: continue try: - thisattrvalue = frozendesc.read_attribute(attr) - except AttributeError: + thisattrvalue = frozendesc.attrcache[attr] + except KeyError: warning("Desc %r has no attribute %r" % (frozendesc, attr)) continue llvalue = r_value.convert_const(thisattrvalue) Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Mon Apr 3 17:35:26 2006 @@ -967,6 +967,20 @@ assert res.item0 == f(i)[0] assert res.item1 == f(i)[1] + def test_pbc_imprecise_attrfamily(self): + fr1 = Freezing(); fr1.x = 5; fr1.y = [8] + fr2 = Freezing(); fr2.x = 6; fr2.y = ["string"] + def head(fr): + return fr.y[0] + def f(n): + if n == 1: + fr = fr1 + else: + fr = fr2 + return head(fr1) + fr.x + res = interpret(f, [2], type_system=self.ts) + assert res == 8 + 6 + def test_multiple_specialized_functions(self): def myadder(x, y): # int,int->int or str,str->str return x+y From arigo at codespeak.net Mon Apr 3 17:36:13 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 17:36:13 +0200 (CEST) Subject: [pypy-svn] r25263 - in pypy/branch/jit-timeshifter-refactoring/timeshifter: . test Message-ID: <20060403153613.D3BF810106@code0.codespeak.net> Author: arigo Date: Mon Apr 3 17:36:12 2006 New Revision: 25263 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py Log: (pedronis, arigo) A quick fix. (insert smiley here) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py Mon Apr 3 17:36:12 2006 @@ -28,6 +28,7 @@ class StructTypeDesc(object): __metaclass__ = cachedtype + firstsubstructdesc = None def __init__(self, TYPE): self.TYPE = TYPE @@ -40,6 +41,9 @@ FIELDTYPE = getattr(self.TYPE, name) if isinstance(FIELDTYPE, lltype.ContainerType): substructdesc = StructTypeDesc(FIELDTYPE) + assert name == self.TYPE._names[0], ( + "unsupported: inlined substructures not as first field") + self.firstsubstructdesc = substructdesc for subfielddesc in substructdesc.fielddescs: dottedname = '%s.%s' % (name, subfielddesc.fieldname) index = len(fielddescs) @@ -55,7 +59,15 @@ return self.fielddescs[index] def ll_factory(self): - return rvalue.PtrRedBox(self.gv_ptrtype, content=VirtualStruct(self)) + vstruct = VirtualStruct(self) + vstruct.substruct_boxes = [] + typedesc = self + while typedesc is not None: + box = rvalue.PtrRedBox(typedesc.gv_ptrtype, + content=vstruct) + vstruct.substruct_boxes.append(box) + typedesc = typedesc.firstsubstructdesc + return vstruct.substruct_boxes[0] def _freeze_(self): return True @@ -83,6 +95,12 @@ def __init__(self, PTRTYPE, name): FieldDesc.__init__(self, PTRTYPE, getattr(PTRTYPE.TO, name)) + self.structdepth = 0 + T = self.PTRTYPE.TO + while (T._names and + isinstance(getattr(T, T._names[0]), lltype.ContainerType)): + self.structdepth += 1 + T = getattr(T, T._names[0]) self.fieldname = name self.gv_fieldname = rgenop.constFieldName(name) @@ -144,23 +162,22 @@ self.typedesc = typedesc #self.fz_content initialized later - def exactmatch(self, box, outgoingvarboxes, memo): - vstruct = box.content + def exactmatch(self, vstruct, outgoingvarboxes, memo): contmemo = memo.containers if self in contmemo: ok = vstruct is contmemo[self] if not ok: - outgoingvarboxes.append(box) + outgoingvarboxes.extend(vstruct.substruct_boxes) return ok if vstruct in contmemo: assert contmemo[vstruct] is not self - outgoingvarboxes.append(box) + outgoingvarboxes.extend(vstruct.substruct_boxes) return False - contmemo[self] = vstruct - contmemo[vstruct] = self if self.typedesc is not vstruct.typedesc: - outgoingvarboxes.append(box) + outgoingvarboxes.extend(vstruct.substruct_boxes) return False + contmemo[self] = vstruct + contmemo[vstruct] = self self_boxes = self.fz_content_boxes vstruct_boxes = vstruct.content_boxes fullmatch = True @@ -180,7 +197,7 @@ self.content_boxes = [desc.redboxcls(desc.gv_resulttype, desc.gv_default) for desc in typedesc.fielddescs] - self.genvar = rgenop.nullvar + #self.substruct_boxes = ... def enter_block(self, newblock, incoming, memo): contmemo = memo.containers @@ -189,9 +206,7 @@ for box in self.content_boxes: box.enter_block(newblock, incoming, memo) - def build_runtime_container(self, jitstate): - if self.genvar: - return self.genvar + def force_runtime_container(self, jitstate): typedesc = self.typedesc boxes = self.content_boxes self.content_boxes = None @@ -199,7 +214,15 @@ op_args[0] = typedesc.gv_type genvar = rgenop.genop(jitstate.curblock, 'malloc', op_args, typedesc.gv_ptrtype) - self.genvar = genvar + # force all the boxes pointing to this VirtualStruct + for box in self.substruct_boxes: + # XXX using getsubstruct would be nicer + op_args = lltype.malloc(rgenop.VARLIST.TO, 1) + op_args[0] = genvar + box.genvar = rgenop.genop(jitstate.curblock, 'cast_pointer', + op_args, box.gv_type) + box.content = None + self.substruct_boxes = None fielddescs = typedesc.fielddescs for i in range(len(fielddescs)): fielddesc = fielddescs[i] @@ -207,8 +230,6 @@ # xxx a bit inefficient fielddesc.generate_set(jitstate, genvar, box) - return genvar - def freeze(self, memo): contmemo = memo.containers try: @@ -225,10 +246,10 @@ return contmemo[self] except KeyError: result = contmemo[self] = VirtualStruct(self.typedesc) - boxcopies = [] - for box in self.content_boxes: - boxcopies.append(box.copy(memo)) - result.content_boxes = boxcopies + result.content_boxes = [box.copy(memo) + for box in self.content_boxes] + result.substruct_boxes = [box.copy(memo) + for box in self.substruct_boxes] return result def op_getfield(self, jitstate, fielddesc): @@ -239,4 +260,4 @@ def op_getsubstruct(self, jitstate, fielddesc): #assert fielddesc.fieldindex == 0 - return rvalue.PtrRedBox(fielddesc.gv_resulttype, content=self) + return self.substruct_boxes[-fielddesc.structdepth] Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py Mon Apr 3 17:36:12 2006 @@ -161,13 +161,8 @@ def getgenvar(self, jitstate): if not self.genvar: assert self.content - genvar = self.content.build_runtime_container(jitstate) - op_args = lltype.malloc(rgenop.VARLIST.TO, 1) - op_args[0] = genvar - genvar = rgenop.genop(jitstate.curblock, 'cast_pointer', op_args, - self.gv_type) - self.genvar = genvar - self.content = None + self.content.force_runtime_container(jitstate) + assert self.genvar return self.genvar def enter_block(self, newblock, incoming, memo): @@ -334,4 +329,5 @@ outgoingvarboxes.append(box) return False else: - return self.fz_content.exactmatch(box, outgoingvarboxes, memo) + return self.fz_content.exactmatch(box.content, outgoingvarboxes, + memo) Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/test/test_timeshift.py Mon Apr 3 17:36:12 2006 @@ -436,6 +436,26 @@ assert parentindex == 's' assert parent.n == 3.25 +def test_degenerated_via_substructure(): + S = lltype.GcStruct('S', ('n', lltype.Signed)) + T = lltype.GcStruct('T', ('s', S), ('n', lltype.Float)) + + def ll_function(flag): + t = lltype.malloc(T) + t.s.n = 3 + s = lltype.malloc(S) + s.n = 7 + if flag: + pass + else: + s = t.s + t.s.n += 1 + return s.n * t.s.n + insns, res = timeshift(ll_function, [1], []) + assert res == 7 * 4 + insns, res = timeshift(ll_function, [0], []) + assert res == 4 * 4 + def test_plus_minus_all_inlined(): def ll_plus_minus(s, x, y): acc = x From goden at codespeak.net Mon Apr 3 17:59:00 2006 From: goden at codespeak.net (goden at codespeak.net) Date: Mon, 3 Apr 2006 17:59:00 +0200 (CEST) Subject: [pypy-svn] r25264 - in pypy/dist/pypy/rpython: . rctypes Message-ID: <20060403155900.EB7B410109@code0.codespeak.net> Author: goden Date: Mon Apr 3 17:58:53 2006 New Revision: 25264 Modified: pypy/dist/pypy/rpython/extregistry.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py Log: more work on ctypes pointer specialization Modified: pypy/dist/pypy/rpython/extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/extregistry.py (original) +++ pypy/dist/pypy/rpython/extregistry.py Mon Apr 3 17:58:53 2006 @@ -8,7 +8,8 @@ def get_annotation(self, type, func=None): assert func is not None from pypy.annotation import model as annmodel - return annmodel.SomeBuiltin(self.compute_result_annotation, methodname=func.__name__) + return annmodel.SomeBuiltin(self.compute_result_annotation, + methodname=func.__name__) class ExtRegistryInstance(object): def __init__(self, compute_annotation, specialize_call, get_repr): Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 3 17:58:53 2006 @@ -1,20 +1,39 @@ from pypy.rpython.rmodel import Repr from pypy.rpython import extregistry +from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel -from ctypes import POINTER, c_int +from ctypes import POINTER class PointerRepr(Repr): """XXX: todo """ - def __init__(self, rtyper, ctype, ref_type): - pass + def __init__(self, rtyper, s_pointer, s_contents): + self.s_pointer = s_pointer + self.s_contents = s_contents + self.ctype = s_pointer.knowntype + self.ref_ctype = s_contents.knowntype + + if not extregistry.is_registered_type(self.ref_ctype): + raise TypeError("Unregistered referenced " + "type: %s" % (self.ref_ctype.__name__,)) + + ref_entry = extregistry.lookup_type(self.ref_ctype) + contents_repr = ref_entry.get_repr(rtyper, self.s_contents) + + ll_contents = lltype.Ptr(contents_repr.c_data_type) + + self.lowleveltype = lltype.Ptr( + lltype.GcStruct("CtypesBox_%s" % (self.ctype.__name__,), + ("c_data", + lltype.Struct("C_Data_%s" % (self.ctype.__name__), + ('value', ll_contents) + ) + ), + ("keepalive", contents_repr.lowleveltype) + ) + ) -def get_repr(rtyper, s_pointer): - """XXX: todo - """ - raise RuntimeError("foo") - def registerPointerType(ptrtype): """Adds a new pointer type to the extregistry. @@ -26,29 +45,34 @@ which, when retrieved, in effect dereferences the pointer and returns the referenced value. """ - def compute_result_annotation(s_arg): + def compute_result_annotation(s_self, s_arg): return annmodel.SomeCTypesObject(ptrtype, annmodel.SomeCTypesObject.OWNSMEMORY) def specialize_call(hop): raise RuntimeError('foo') + contentsType = annmodel.SomeCTypesObject(ptrtype._type_, + annmodel.SomeCTypesObject.MEMORYALIAS) + + def get_repr(rtyper, s_pointer): + return PointerRepr(rtyper, s_pointer, contentsType) + type_entry = extregistry.register_type(ptrtype, specialize_call=specialize_call, get_repr=get_repr) - contentsType = annmodel.SomeCTypesObject(ptrtype._type_, - annmodel.SomeCTypesObject.MEMORYALIAS) type_entry.fields_s = {'contents': contentsType} return extregistry.register_value(ptrtype, - compute_result_annotation=compute_result_annotation) + compute_result_annotation=compute_result_annotation, + specialize_call=specialize_call) -def POINTER_compute_annotation(metatype, the_type): +def pointer_compute_annotation(metatype, the_type): """compute the annotation of POINTER() calls to create a ctypes pointer for the given type """ - def POINTER_compute_result_annotation(s_arg): + def pointer_compute_result_annotation(s_arg): """Called to compute the result annotation of POINTER(). This happens to return a new class which itself is treated as SomeBuiltin because when @@ -65,23 +89,23 @@ else: entry = extregistry.lookup(ptrtype) + s_self = annmodel.SomeCTypesObject(ptrtype, + annmodel.SomeCTypesObject.OWNSMEMORY) + + return annmodel.SomeBuiltin(entry.compute_result_annotation, - methodname=ptrtype.__name__) + s_self=s_self, + methodname=ptrtype.__name__) # annotation of POINTER (not the call) is SomeBuitin which provides # a way of computing the result annotation of POINTER() - return annmodel.SomeBuiltin(POINTER_compute_result_annotation, + return annmodel.SomeBuiltin(pointer_compute_result_annotation, methodname=the_type.__name__) -# handles POINTER() calls -value_entry = extregistry.register_value(POINTER, - compute_annotation=POINTER_compute_annotation) - -def POINTER_specialize_call(hop): +def pointer_specialize_call(hop): raise RuntimeError("foo") -extregistry.register_type(POINTER, specialize_call=POINTER_specialize_call) - -#type_entry = extregistry.register_type(ptrmeta, -# compute_annotation=compute_annotation, -# get_repr=get_repr) +# handles POINTER() calls +value_entry = extregistry.register_value(POINTER, + compute_annotation=pointer_compute_annotation, + specialize_call=pointer_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Mon Apr 3 17:58:53 2006 @@ -28,11 +28,12 @@ def __init__(self, rtyper, ctype, ll_type): self.ctype = ctype self.ll_type = ll_type + self.c_data_type = lltype.Struct('C_Data_%s' % (ctype.__name__,), + ('value', ll_type) ) + self.lowleveltype = lltype.Ptr( lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data", lltype.Struct('C_Data_%s' % (ctype.__name__,), - ('value', ll_type) ) - ) + ( "c_data", self.c_data_type ) ) ) From nik at codespeak.net Mon Apr 3 18:59:48 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 3 Apr 2006 18:59:48 +0200 (CEST) Subject: [pypy-svn] r25265 - in pypy/dist/pypy: annotation rpython/ootypesystem rpython/ootypesystem/test Message-ID: <20060403165948.7AF50100F6@code0.codespeak.net> Author: nik Date: Mon Apr 3 18:59:46 2006 New Revision: 25265 Modified: pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: implement getitem/setitem for ootype lists (ignoring exception issues and negative indices for now). fix a subtle bug in annotation of bound methods. Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Mon Apr 3 18:59:46 2006 @@ -561,7 +561,7 @@ # annotation of low-level types from pypy.annotation.model import SomePtr, SomeLLADTMeth from pypy.annotation.model import SomeOOInstance, SomeOOBoundMeth, SomeOOStaticMeth -from pypy.annotation.model import ll_to_annotation, annotation_to_lltype +from pypy.annotation.model import ll_to_annotation, lltype_to_annotation, annotation_to_lltype class __extend__(SomePtr): @@ -619,8 +619,8 @@ def simple_call(m, *args_s): llargs = [annotation_to_lltype(arg_s)._example() for arg_s in args_s] inst = m.ootype._example() - v = getattr(inst, m.name)(*llargs) - return ll_to_annotation(v) + RESULT = ootype.typeOf(m.ootype._lookup(m.name)[1]).RESULT + return lltype_to_annotation(RESULT) class __extend__(SomeOOStaticMeth): def simple_call(m, *args_s): Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 3 18:59:46 2006 @@ -181,6 +181,8 @@ # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) "length": Meth([], Unsigned), "append": Meth([ITEMTYPE], Void), + "getitem": Meth([Signed], ITEMTYPE), + "setitem": Meth([Signed, ITEMTYPE], Void), }) def __str__(self): @@ -473,6 +475,17 @@ assert typeOf(item) == self._TYPE._ITEMTYPE self._list.append(item) + def getitem(self, index): + # NOT_RPYTHON + assert typeOf(index) == Signed + return self._list[index] + + def setitem(self, index, item): + # NOT_RPYTHON + assert typeOf(item) == self._TYPE._ITEMTYPE + assert typeOf(index) == Signed + self._list[index] = item + def new(TYPE): if isinstance(TYPE, Instance): return make_instance(TYPE) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Mon Apr 3 18:59:46 2006 @@ -17,6 +17,13 @@ l.append(1) assert l.length() == 1 +def test_setitem_getitem(): + LT = List(Signed) + l = new(LT) + l.append(2) + assert l.getitem(0) == 2 + l.setitem(0, 3) + assert l.getitem(0) == 3 class TestInterpreted: @@ -27,3 +34,13 @@ return len(l) res = interpret(f, [2], type_system="ootype") assert res == 1 + + def test_setitem_getitem(self): + def f(x): + l = [] + l.append(3) + l[0] = x + return l[0] + res = interpret(f, [2], type_system="ootype") + assert res == 2 + Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Mon Apr 3 18:59:46 2006 @@ -106,7 +106,7 @@ g = gengraph(oof, []) rettype = g.getreturnvar().concretetype - assert rettype == Signed + assert rettype == Unsigned def test_list_append(): LT = List(Signed) @@ -118,5 +118,19 @@ g = gengraph(oof, []) rettype = g.getreturnvar().concretetype + assert rettype == Unsigned + +def test_list_getitem_setitem(): + # XXX need to test exceptions + LT = List(Signed) + + def oof(): + l = new(LT) + l.append(1) + l.setitem(0, 2) + return l.getitem(0) + + g = gengraph(oof, []) + rettype = g.getreturnvar().concretetype assert rettype == Signed From hpk at codespeak.net Mon Apr 3 19:42:08 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 3 Apr 2006 19:42:08 +0200 (CEST) Subject: [pypy-svn] r25266 - pypy/dist/pypy/translator/c Message-ID: <20060403174208.01B2210110@code0.codespeak.net> Author: hpk Date: Mon Apr 3 19:42:08 2006 New Revision: 25266 Modified: pypy/dist/pypy/translator/c/stackless.py Log: clarify code a bit while trying to understand it Modified: pypy/dist/pypy/translator/c/stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/stackless.py (original) +++ pypy/dist/pypy/translator/c/stackless.py Mon Apr 3 19:42:08 2006 @@ -69,25 +69,20 @@ for caller, callee in translator.callgraph.values(): # ignore calls issued suggested_primitives -- they are not # compiled in, and they typically contain pseudo-recursions - try: - suggprim = caller.func.suggested_primitive - except AttributeError: - suggprim = False - if not suggprim: + if not getattr(caller.func, 'suggested_primitive', False): callers[caller].append(callee) # check all callees if they can reach unwind - seen = self.can_reach_unwind + can_reach_unwind = self.can_reach_unwind pending = {} - ext = self.database.externalfuncs def check_unwind(graph): if graph in pending: - seen[graph] = True + can_reach_unwind[graph] = True return True pending[graph] = graph for callee in callers[graph]: - if callee in seen: - ret = seen[callee] + if callee in can_reach_unwind: + ret = can_reach_unwind[callee] else: ret = check_unwind(callee) if ret: @@ -95,9 +90,11 @@ else: ret = graph.func in self.stackless_roots del pending[graph] - seen[graph] = ret + can_reach_unwind[graph] = ret return ret - [check_unwind(caller) for caller in callers if caller not in seen] + for caller in callers: + if caller not in can_reach_unwind: + check_unwind(caller) def registerunwindable(self, functionname, FUNC, resume_points): if resume_points >= 1: From arigo at codespeak.net Mon Apr 3 19:49:08 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 3 Apr 2006 19:49:08 +0200 (CEST) Subject: [pypy-svn] r25267 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060403174908.417E110111@code0.codespeak.net> Author: arigo Date: Mon Apr 3 19:49:06 2006 New Revision: 25267 Added: pypy/dist/pypy/rpython/rctypes/rmodel.py Modified: pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: (arre, goden, arigo) Introduced a base class CTypesRepr handling the common parts of the ll structs. Not all Reprs have been converted to inherit from it so far. Removed the rpointer.py support for calling POINTER() at run-time. Registering meta-classes instead :-) Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Mon Apr 3 19:49:06 2006 @@ -652,7 +652,7 @@ else: if extregistry.is_registered_type(cto.knowntype): entry = extregistry.lookup_type(cto.knowntype) - s_value = entry.fields_s[attr] + s_value = entry.get_field_annotation(cto, attr) return s_value else: atype = cto.knowntype._fields_def_[attr] Added: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Mon Apr 3 19:49:06 2006 @@ -0,0 +1,72 @@ +from pypy.rpython.rmodel import Repr, inputconst +from pypy.rpython.error import TyperError +from pypy.rpython.lltypesystem import lltype +from pypy.annotation.model import SomeCTypesObject + + + +class CTypesRepr(Repr): + "Base class for the Reprs representing ctypes object." + + def __init__(self, rtyper, s_ctypesobject, ll_type): + # s_ctypesobject: the annotation to represent + # ll_type: the low-level type representing the raw + # data, which is then embedded in a box. + ctype = s_ctypesobject.knowntype + memorystate = s_ctypesobject.memorystate + + self.rtyper = rtyper + self.ctype = ctype + self.ll_type = ll_type + if memorystate == SomeCTypesObject.OWNSMEMORY: + self.ownsmemory = True + elif memorystate == SomeCTypesObject.MEMORYALIAS: + self.ownsmemory = False + else: + raise TyperError("unsupported ctypes memorystate %r" % memorystate) + + if isinstance(ll_type, lltype.ContainerType): + self.c_data_type = ll_type + else: + self.c_data_type = lltype.Struct('C_Data_%s' % (ctype.__name__,), + ('value', ll_type) ) + + if self.ownsmemory: + self.lowleveltype = lltype.Ptr( + lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), + ( "c_data", self.c_data_type ) + ) + ) + else: + self.lowleveltype = lltype.Ptr( + lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), + ( "c_data_ref", lltype.Ptr(self.c_data_type) ) + ) + ) + # XXX keepalives... + self.const_cache = {} # store generated const values+original value + + def get_c_data(self, llops, v_box): + if self.ownsmemory: + inputargs = [v_box, inputconst(lltype.Void, "c_data")] + return llops.genop('getsubstruct', inputargs, + lltype.Ptr(self.c_data_type) ) + else: + inputargs = [v_box, inputconst(lltype.Void, "c_data_ref")] + return llops.genop('getfield', inputargs, + lltype.Ptr(self.c_data_type) ) + + def setvalue(self, llops, v_box, v_value): + """Writes the 'value' field of the raw data + (only if ll_type is not a container type)""" + v_c_data = self.get_c_data(llops, v_box) + cname = inputconst(lltype.Void, 'value') + llops.genop('setfield', [v_c_data, cname, v_value]) + + def getvalue(self, llops, v_box): + """Reads the 'value' field of the raw data + (only if ll_type is not a container type)""" + v_c_data = self.get_c_data(llops, v_box) + cname = inputconst(lltype.Void, 'value') + return llops.genop('getfield', [v_c_data, cname], + resulttype=self.ll_type) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 3 19:49:06 2006 @@ -3,7 +3,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel -from ctypes import POINTER +from ctypes import POINTER, c_int class PointerRepr(Repr): """XXX: todo @@ -34,78 +34,112 @@ ) ) -def registerPointerType(ptrtype): - """Adds a new pointer type to the extregistry. - - Since pointers can be created to primitive ctypes objects, arrays, - structs and structs are not predefined each new pointer type is - registered in the extregistry as it is identified. - - The new pointers that are created have a "contents" attribute - which, when retrieved, in effect dereferences the pointer and - returns the referenced value. - """ - def compute_result_annotation(s_self, s_arg): - return annmodel.SomeCTypesObject(ptrtype, +#def registerPointerType(ptrtype): +# """Adds a new pointer type to the extregistry. +# +# Since pointers can be created to primitive ctypes objects, arrays, +# structs and structs are not predefined each new pointer type is +# registered in the extregistry as it is identified. +# +# The new pointers that are created have a "contents" attribute +# which, when retrieved, in effect dereferences the pointer and +# returns the referenced value. +# """ +# def compute_result_annotation(s_self, s_arg): +# return annmodel.SomeCTypesObject(ptrtype, +# annmodel.SomeCTypesObject.OWNSMEMORY) +# +# def specialize_call(hop): +# raise RuntimeError('foo') +# +# contentsType = annmodel.SomeCTypesObject(ptrtype._type_, +# annmodel.SomeCTypesObject.MEMORYALIAS) +# +# def get_repr(rtyper, s_pointer): +# return PointerRepr(rtyper, s_pointer, contentsType) +# +# type_entry = extregistry.register_type(ptrtype, +# specialize_call=specialize_call, +# get_repr=get_repr) +# type_entry.get_field_annotation = {'contents': contentsType}.__getitem__ +# +# return extregistry.register_value(ptrtype, +# compute_result_annotation=compute_result_annotation, +# specialize_call=specialize_call) +# +#def pointer_compute_annotation(metatype, the_type): +# """compute the annotation of POINTER() calls to create a ctypes +# pointer for the given type +# """ +# +# def pointer_compute_result_annotation(s_arg): +# """Called to compute the result annotation of +# POINTER(). This happens to return a new +# class which itself is treated as SomeBuiltin because when +# called it creates a new pointer. +# +# NOTE: To handle a myriad of possible pointer types, each +# ctypes type that is passed to POINTER() calls is itself +# registered if it isn't already. +# """ +# ptrtype = POINTER(s_arg.const) +# +# if not extregistry.is_registered_type(ptrtype): +# entry = registerPointerType(ptrtype) +# else: +# entry = extregistry.lookup(ptrtype) +# +# s_self = annmodel.SomeCTypesObject(ptrtype, +# annmodel.SomeCTypesObject.OWNSMEMORY) +# +# +# return annmodel.SomeBuiltin(entry.compute_result_annotation, +# s_self=s_self, +# methodname=ptrtype.__name__) +# +# # annotation of POINTER (not the call) is SomeBuitin which provides +# # a way of computing the result annotation of POINTER() +# return annmodel.SomeBuiltin(pointer_compute_result_annotation, +# methodname=the_type.__name__) +# +#def pointer_specialize_call(hop): +# raise RuntimeError("foo") +# +# handles POINTER() calls +#value_entry = extregistry.register_value(POINTER, +# compute_annotation=pointer_compute_annotation, +# specialize_call=pointer_specialize_call) + +def pointertype_compute_annotation(metatype, type): + def compute_result_annotation(*arg_s): + return annmodel.SomeCTypesObject(type, annmodel.SomeCTypesObject.OWNSMEMORY) + return annmodel.SomeBuiltin(compute_result_annotation, + methodname=type.__name__) - def specialize_call(hop): - raise RuntimeError('foo') +def pointertype_specialize_call(hop): + xxx #... - contentsType = annmodel.SomeCTypesObject(ptrtype._type_, - annmodel.SomeCTypesObject.MEMORYALIAS) - - def get_repr(rtyper, s_pointer): - return PointerRepr(rtyper, s_pointer, contentsType) - - type_entry = extregistry.register_type(ptrtype, - specialize_call=specialize_call, - get_repr=get_repr) - type_entry.fields_s = {'contents': contentsType} - - return extregistry.register_value(ptrtype, - compute_result_annotation=compute_result_annotation, - specialize_call=specialize_call) - -def pointer_compute_annotation(metatype, the_type): - """compute the annotation of POINTER() calls to create a ctypes - pointer for the given type - """ - - def pointer_compute_result_annotation(s_arg): - """Called to compute the result annotation of - POINTER(). This happens to return a new - class which itself is treated as SomeBuiltin because when - called it creates a new pointer. - - NOTE: To handle a myriad of possible pointer types, each - ctypes type that is passed to POINTER() calls is itself - registered if it isn't already. - """ - ptrtype = POINTER(s_arg.const) - - if not extregistry.is_registered_type(ptrtype): - entry = registerPointerType(ptrtype) - else: - entry = extregistry.lookup(ptrtype) - - s_self = annmodel.SomeCTypesObject(ptrtype, - annmodel.SomeCTypesObject.OWNSMEMORY) - - - return annmodel.SomeBuiltin(entry.compute_result_annotation, - s_self=s_self, - methodname=ptrtype.__name__) - - # annotation of POINTER (not the call) is SomeBuitin which provides - # a way of computing the result annotation of POINTER() - return annmodel.SomeBuiltin(pointer_compute_result_annotation, - methodname=the_type.__name__) - -def pointer_specialize_call(hop): - raise RuntimeError("foo") - -# handles POINTER() calls -value_entry = extregistry.register_value(POINTER, - compute_annotation=pointer_compute_annotation, - specialize_call=pointer_specialize_call) +def pointerinstance_compute_annotation(type, instance): + return annmodel.SomeCTypesObject(type, + annmodel.SomeCTypesObject.OWNSMEMORY) + +def pointerinstance_field_annotation(s_pointer, fieldname): + assert fieldname == "contents" + ptrtype = s_pointer.knowntype + return annmodel.SomeCTypesObject(ptrtype._type_, + annmodel.SomeCTypesObject.MEMORYALIAS) + +def pointerinstance_get_repr(rtyper, s_pointer): + s_contents = pointerinstance_field_annotation(s_pointer, "contents") + return PointerRepr(rtyper, s_pointer, s_contents) + +PointerType = type(POINTER(c_int)) +extregistry.register_type(PointerType, + compute_annotation=pointertype_compute_annotation, + specialize_call=pointertype_specialize_call) + +entry = extregistry.register_metatype(PointerType, + compute_annotation=pointerinstance_compute_annotation, + get_repr=pointerinstance_get_repr) +entry.get_field_annotation = pointerinstance_field_annotation Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Mon Apr 3 19:49:06 2006 @@ -7,6 +7,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr +from pypy.rpython.rctypes.rmodel import CTypesRepr ctypes_annotation_list = [ (c_char, lltype.Char), @@ -24,37 +25,7 @@ (c_double, lltype.Float), ] -class PrimitiveRepr(Repr): - def __init__(self, rtyper, ctype, ll_type): - self.ctype = ctype - self.ll_type = ll_type - self.c_data_type = lltype.Struct('C_Data_%s' % (ctype.__name__,), - ('value', ll_type) ) - - self.lowleveltype = lltype.Ptr( - lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data", self.c_data_type ) - ) - ) - - self.const_cache = {} # store generated const values+original value - - def get_c_data(self, llops, v_primitive): - inputargs = [v_primitive, inputconst(lltype.Void, "c_data")] - return llops.genop('getsubstruct', inputargs, - lltype.Ptr(self.lowleveltype.TO.c_data) ) - - def setfield(self, llops, v_primitive, v_value): - v_c_data = self.get_c_data(llops, v_primitive) - cname = inputconst(lltype.Void, 'value') - llops.genop('setfield', [v_c_data, cname, v_value]) - - def getfield(self, llops, v_primitive): - v_c_data = self.get_c_data(llops, v_primitive) - - cname = inputconst(lltype.Void, 'value') - return llops.genop('getfield', [v_c_data, cname], - resulttype=self.ll_type) +class PrimitiveRepr(CTypesRepr): def convert_const(self, ctype_value): assert isinstance(ctype_value, self.ctype) @@ -73,7 +44,7 @@ assert s_attr.is_constant() assert s_attr.const == 'value' v_primitive = hop.inputarg(self, 0) - return self.getfield(hop.llops, v_primitive) + return self.getvalue(hop.llops, v_primitive) def rtype_setattr(self, hop): s_attr = hop.args_s[1] @@ -81,7 +52,7 @@ assert s_attr.const == 'value' v_primitive, v_attr, v_value = hop.inputargs(self, lltype.Void, self.ll_type) - self.setfield(hop.llops, v_primitive, v_value) + self.setvalue(hop.llops, v_primitive, v_value) # need to extend primitive repr to implement convert_from_to() for various # conversions, firstly the conversion from c_long() to Signed @@ -90,7 +61,7 @@ def convert_from_to((r_from, r_to), v, llops): assert r_from.ll_type == r_to.lowleveltype - return r_from.getfield(llops, v) + return r_from.getvalue(llops, v) def primitive_specialize_call(hop): r_primitive = hop.r_result @@ -98,7 +69,7 @@ v_result = hop.genop("malloc", [c1], resulttype=r_primitive.lowleveltype) if len(hop.args_s): v_value, = hop.inputargs(r_primitive.ll_type) - r_primitive.setfield(hop.llops, v_result, v_value) + r_primitive.setvalue(hop.llops, v_result, v_value) return v_result def do_register(the_type, ll_type): @@ -116,13 +87,16 @@ annmodel.SomeCTypesObject.OWNSMEMORY) def primitive_get_repr(rtyper, s_primitive): - return PrimitiveRepr(rtyper, s_primitive.knowntype, ll_type) + return PrimitiveRepr(rtyper, s_primitive, ll_type) entry = extregistry.register_type(the_type, compute_annotation=compute_prebuilt_instance_annotation, get_repr=primitive_get_repr, ) - entry.fields_s = {'value': annmodel.lltype_to_annotation(ll_type)} + def primitive_get_field_annotation(s_primitive, fieldname): + assert fieldname == 'value' + return annmodel.lltype_to_annotation(ll_type) + entry.get_field_annotation = primitive_get_field_annotation entry.lowleveltype = ll_type for the_type, ll_type in ctypes_annotation_list: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Mon Apr 3 19:49:06 2006 @@ -18,9 +18,9 @@ assert res.value == ptrres.contents.value def test_annotate_c_int_ptr(self): + ptrtype = POINTER(c_int) def func(): res = c_int(42) - ptrtype = POINTER(c_int) ptrres = ptrtype(res) return ptrres.contents.value @@ -34,9 +34,9 @@ assert s.knowntype == int def test_annotate_c_float_ptr(self): + ptrtype = POINTER(c_float) def func(): res = c_float(4.2) - ptrtype = POINTER(c_float) ptrres = ptrtype(res) return ptrres.contents.value @@ -50,14 +50,13 @@ assert s.knowntype == float class Test_specialization: - def x_test_specialize_c_int_ptr(self): + def test_specialize_c_int_ptr(self): + py.test.skip("in-progress") + ptrtype = POINTER(c_int) def func(): res = c_int(42) - ptrtype = POINTER(c_int) - ptrres = ptrtype(res) - + ptrres = ptrtype(res) return ptrres.contents.value res = interpret(func, []) - assert res == 42 From tismer at codespeak.net Mon Apr 3 23:34:41 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Mon, 3 Apr 2006 23:34:41 +0200 (CEST) Subject: [pypy-svn] r25269 - pypy/dist/pypy/annotation Message-ID: <20060403213441.C82E01011A@code0.codespeak.net> Author: tismer Date: Mon Apr 3 23:34:39 2006 New Revision: 25269 Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: when someobjects are allowed and we cannot build a PBC, don't crash but make a SomeObject() Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Mon Apr 3 23:34:39 2006 @@ -390,7 +390,13 @@ if result is None: result = SomeObject() else: - result = SomePBC([self.getdesc(x)]) + try: + result = SomePBC([self.getdesc(x)]) + except AssertionError, e: + if not self.annotator.policy.allow_someobjects: + raise + result = SomeObject() + elif hasattr(x, '__class__') \ and x.__class__.__module__ != '__builtin__': # user-defined classes can define a method _freeze_(), which From tismer at codespeak.net Tue Apr 4 02:51:41 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 02:51:41 +0200 (CEST) Subject: [pypy-svn] r25275 - pypy/dist/pypy/annotation Message-ID: <20060404005141.8909910121@code0.codespeak.net> Author: tismer Date: Tue Apr 4 02:51:38 2006 New Revision: 25275 Modified: pypy/dist/pypy/annotation/builtin.py Log: dropped builtin_long Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Tue Apr 4 02:51:38 2006 @@ -96,9 +96,6 @@ def builtin_float(s_obj): return constpropagate(float, [s_obj], SomeFloat()) -def builtin_long(s_obj): - return SomeObject() # XXX go away - def builtin_chr(s_int): return constpropagate(chr, [s_int], SomeChar()) @@ -190,6 +187,7 @@ add_knowntypedata(r.knowntypedata, True, variables, bk.valueoftype(typ)) return r +# note that this one either needs to be constant, or we will create SomeObject def builtin_hasattr(s_obj, s_attr): if not s_attr.is_constant() or not isinstance(s_attr.const, str): getbookkeeper().warning('hasattr(%r, %r) is not RPythonic enough' % From tismer at codespeak.net Tue Apr 4 02:54:25 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 02:54:25 +0200 (CEST) Subject: [pypy-svn] r25276 - in pypy/dist/pypy/rpython: lltypesystem test Message-ID: <20060404005425.7A8B110122@code0.codespeak.net> Author: tismer Date: Tue Apr 4 02:54:22 2006 New Revision: 25276 Modified: pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py pypy/dist/pypy/rpython/test/test_rbuiltin.py Log: support for builtin hasattr (PyObject and const only) Modified: pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py Tue Apr 4 02:54:22 2006 @@ -56,10 +56,20 @@ return hop.genop('cast_pointer', [v_inst], # v_type implicit in r_result resulttype = hop.r_result.lowleveltype) - classdef = s_class.descriptions.keys()[0].getuniqueclassdef() return rclass.rtype_new_instance(hop.rtyper, classdef, hop.llops) +def rtype_builtin_hasattr(hop): + if hop.s_result.is_constant(): + return hop.inputconst(lltype.Bool, hop.s_result.const) + if hop.args_r[0] == pyobj_repr: + v_obj, v_typ = hop.inputargs(pyobj_repr, pyobj_repr) + c = hop.inputconst(pyobj_repr, hasattr) + v = hop.genop('simple_call', [c, v_obj, v_typ], resulttype = pyobj_repr) + return hop.llops.convertvar(v, pyobj_repr, bool_repr) + raise TyperError("hasattr is only suported on a constant or on PyObject") + BUILTIN_TYPER = {} BUILTIN_TYPER[objectmodel.instantiate] = rtype_instantiate BUILTIN_TYPER[isinstance] = rtype_builtin_isinstance +BUILTIN_TYPER[hasattr] = rtype_builtin_hasattr Modified: pypy/dist/pypy/rpython/test/test_rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rbuiltin.py (original) +++ pypy/dist/pypy/rpython/test/test_rbuiltin.py Tue Apr 4 02:54:22 2006 @@ -236,6 +236,20 @@ res = interpret(f, [1]) assert res is False +def test_hasattr(): + class A(object): + def __init__(self): + self.x = 42 + def f(i): + a = A() + if i==0: return int(hasattr(A, '__init__')) + if i==1: return int(hasattr(A, 'y')) + if i==2: return int(hasattr(42, 'x')) + for x, y in zip(range(3), (1, 0, 0)): + res = interpret(f, [x]) + assert res._obj.value == y + # hmm, would like to test against PyObj, is this the wrong place/way? + def test_we_are_translated(): def f(): return we_are_translated() From tismer at codespeak.net Tue Apr 4 02:57:22 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 02:57:22 +0200 (CEST) Subject: [pypy-svn] r25277 - pypy/dist/pypy/annotation Message-ID: <20060404005722.9F8CF10124@code0.codespeak.net> Author: tismer Date: Tue Apr 4 02:57:20 2006 New Revision: 25277 Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: support more SomeObjects, without breaking anything (I hope, since there were lots of broken tests when I checked in) Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Tue Apr 4 02:57:20 2006 @@ -390,13 +390,13 @@ if result is None: result = SomeObject() else: - try: + # note that the print support functions are __builtin__ + if hasattr(x, '__module__') and x.__module__ != '__builtin__' \ + or tp in (types.FunctionType, types.MethodType): result = SomePBC([self.getdesc(x)]) - except AssertionError, e: - if not self.annotator.policy.allow_someobjects: - raise + else: + # a builtin that we don't handle specially result = SomeObject() - elif hasattr(x, '__class__') \ and x.__class__.__module__ != '__builtin__': # user-defined classes can define a method _freeze_(), which From goden at codespeak.net Tue Apr 4 04:29:17 2006 From: goden at codespeak.net (goden at codespeak.net) Date: Tue, 4 Apr 2006 04:29:17 +0200 (CEST) Subject: [pypy-svn] r25278 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060404022917.C049B10127@code0.codespeak.net> Author: goden Date: Tue Apr 4 04:29:14 2006 New Revision: 25278 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rpointer.py Log: refactored the rctypes pointer and array representations to use the CTypesRepr base class from rmodel.py Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Tue Apr 4 04:29:14 2006 @@ -5,50 +5,43 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr +from pypy.rpython.rctypes.rmodel import CTypesRepr ArrayType = type(ARRAY(c_int, 10)) -class ArrayRepr(Repr): - def __init__(self, rtyper, type): +class ArrayRepr(CTypesRepr): + def __init__(self, rtyper, s_array): + array_ctype = s_array.knowntype - item_ctype = type._type_ - self.length = type._length_ + item_ctype = array_ctype._type_ + self.length = array_ctype._length_ - entry = extregistry.lookup_type(item_ctype) - # XXX: goden: So at this point item_ctype is a ctypes object. - # I *think* we need to send a "SomeCTypesObject" box - # to get the PrimitiveRepr instead of the ctypes object - # itself. - self.r_item = entry.get_repr(rtyper, SomeCTypesObject(item_ctype, - SomeCTypesObject.OWNSMEMORY)) + # Find the low-level repr of items using the extregistry + item_entry = extregistry.lookup_type(item_ctype) + self.r_item = item_entry.get_repr(rtyper, SomeCTypesObject(item_ctype, + SomeCTypesObject.OWNSMEMORY)) + + c_data_type = lltype.Array(self.r_item.ll_type, + hints={"nolength": True}) # Array elements are of the low-level type (Signed, etc) and not # of the boxed low level type (Ptr(GcStruct(...))) - self.lowleveltype = lltype.Ptr( - lltype.GcStruct( "CtypesGcArray_%s" % type.__name__, - ( "c_data", lltype.Array(self.r_item.ll_type, - hints={"nolength": True}) - ) - ) - ) + + super(ArrayRepr, self).__init__(rtyper, s_array, c_data_type) class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item.ll_type) inputargs = [v_array, hop.inputconst(lltype.Void, "c_data")] - v_c_data = hop.genop('getsubstruct', - inputargs, - lltype.Ptr(r_array.lowleveltype.TO.c_data) ) + v_c_data = r_array.get_c_data(hop.llops, v_array) hop.genop('setarrayitem', [v_c_data, v_index, v_item]) def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) inputargs = [v_array, hop.inputconst(lltype.Void, "c_data")] - v_c_data = hop.genop('getsubstruct', - inputargs, - lltype.Ptr(r_array.lowleveltype.TO.c_data) ) + v_c_data = r_array.get_c_data(hop.llops, v_array) return hop.genop('getarrayitem', [v_c_data, v_index], r_array.r_item.ll_type) @@ -70,6 +63,6 @@ specialize_call=arraytype_specialize_call) def arraytype_get_repr(rtyper, s_array): - return ArrayRepr(rtyper, s_array.knowntype) + return ArrayRepr(rtyper, s_array) extregistry.register_metatype(ArrayType, get_repr=arraytype_get_repr) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 4 04:29:14 2006 @@ -2,16 +2,14 @@ from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel +from pypy.rpython.rctypes.rmodel import CTypesRepr from ctypes import POINTER, c_int -class PointerRepr(Repr): - """XXX: todo - """ +class PointerRepr(CTypesRepr): def __init__(self, rtyper, s_pointer, s_contents): self.s_pointer = s_pointer self.s_contents = s_contents - self.ctype = s_pointer.knowntype self.ref_ctype = s_contents.knowntype if not extregistry.is_registered_type(self.ref_ctype): @@ -23,16 +21,7 @@ ll_contents = lltype.Ptr(contents_repr.c_data_type) - self.lowleveltype = lltype.Ptr( - lltype.GcStruct("CtypesBox_%s" % (self.ctype.__name__,), - ("c_data", - lltype.Struct("C_Data_%s" % (self.ctype.__name__), - ('value', ll_contents) - ) - ), - ("keepalive", contents_repr.lowleveltype) - ) - ) + super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) #def registerPointerType(ptrtype): # """Adds a new pointer type to the extregistry. From tismer at codespeak.net Tue Apr 4 07:19:53 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 07:19:53 +0200 (CEST) Subject: [pypy-svn] r25279 - in pypy/dist/pypy/objspace/flow: . test Message-ID: <20060404051953.A464610129@code0.codespeak.net> Author: tismer Date: Tue Apr 4 07:19:50 2006 New Revision: 25279 Modified: pypy/dist/pypy/objspace/flow/objspace.py pypy/dist/pypy/objspace/flow/test/test_objspace.py Log: do_imports_immediately support is broken. needed to re-enable AttributeError and ImportError for flow space. Maybe there is a better solution, then let me know. I'm wondering why geninterp is still working... Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Tue Apr 4 07:19:50 2006 @@ -49,7 +49,7 @@ self.w_tuple = Constant(tuple) self.concrete_mode = 0 for exc in [KeyError, ValueError, IndexError, StopIteration, - AssertionError, TypeError]: + AssertionError, TypeError, AttributeError, ImportError]: clsname = exc.__name__ setattr(self, 'w_'+clsname, Constant(exc)) # the following exceptions are the ones that should not show up Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_objspace.py Tue Apr 4 07:19:50 2006 @@ -636,6 +636,26 @@ assert not call_args +class TestFlowObjSpaceDelay: + def setup_class(cls): + cls.space = FlowObjSpace() + cls.space.do_imports_immediately = False + + def codetest(self, func): + import inspect + try: + func = func.im_func + except AttributeError: + pass + #name = func.func_name + graph = self.space.build_flow(func) + graph.source = inspect.getsource(func) + return graph + + def test_import_something(self): + def f(): + from some.unknown.module import stuff + self.codetest(f) DATA = {'x': 5, 'y': 6} From nik at codespeak.net Tue Apr 4 08:52:57 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 08:52:57 +0200 (CEST) Subject: [pypy-svn] r25281 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060404065257.3C00E1012B@code0.codespeak.net> Author: nik Date: Tue Apr 4 08:52:56 2006 New Revision: 25281 Added: pypy/dist/pypy/rpython/ootypesystem/rlist.py (contents, props changed) Log: ooops. forgot to checkin this rather important file during the rlist refactoring. sorry. Added: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Tue Apr 4 08:52:56 2006 @@ -0,0 +1,71 @@ +from pypy.annotation.pairtype import pairtype +from pypy.rpython.rlist import AbstractListRepr, rtype_newlist +from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst, externalvsinternal +from pypy.rpython.ootypesystem import ootype + +class BaseListRepr(AbstractListRepr): + + def __init__(self, rtyper, item_repr, listitem=None): + self.rtyper = rtyper + # XXX do we need something like this for ootypes? + #self.LIST = GcForwardReference() + if not isinstance(item_repr, Repr): # not computed yet, done by setup() + assert callable(item_repr) + self._item_repr_computer = item_repr + else: + self.lowleveltype = ootype.List(item_repr.lowleveltype) + self.external_item_repr, self.item_repr = \ + externalvsinternal(rtyper, item_repr) + self.listitem = listitem + self.list_cache = {} + # setup() needs to be called to finish this initialization + + def _setup_repr(self): + if 'item_repr' not in self.__dict__: + self.external_item_repr, self.item_repr = \ + externalvsinternal(self.rtyper, self._item_repr_computer()) + self.lowleveltype = ootype.List(self.item_repr.lowleveltype) + + def send_message(self, hop, message, can_raise=False): + v_args = hop.inputargs(self, *hop.args_r[1:]) + c_name = hop.inputconst(ootype.Void, message) + if can_raise: + hop.exception_is_here() + return hop.genop("oosend", [c_name] + v_args, + resulttype=hop.r_result.lowleveltype) + + def rtype_len(self, hop): + return self.send_message(hop, "length") + + def rtype_method_append(self, hop): + return self.send_message(hop, "append") + +ListRepr = BaseListRepr +FixedSizeListRepr = BaseListRepr + +class __extend__(pairtype(BaseListRepr, IntegerRepr)): + + def rtype_getitem((r_list, r_int), hop): + # XXX must handle exceptions + #if hop.has_implicit_exception(IndexError): + # XXX must handle negative indices + return r_list.send_message(hop, "getitem", can_raise=True) + + def rtype_setitem((r_list, r_int), hop): + # XXX must handle exceptions + #if hop.has_implicit_exception(IndexError): + # XXX must handle negative indices + #if hop.has_implicit_exception(IndexError): + return r_list.send_message(hop, "setitem", can_raise=True) + +def newlist(llops, r_list, items_v): + c1 = inputconst(ootype.Void, r_list.lowleveltype) + v_result = llops.genop("new", [c1], resulttype=r_list.lowleveltype) + #LIST = r_list.LIST + #cno = inputconst(Signed, len(items_v)) + #v_result = llops.gendirectcall(LIST.ll_newlist, cno) + #v_func = inputconst(Void, dum_nocheck) + #for i, v_item in enumerate(items_v): + # ci = inputconst(Signed, i) + # llops.gendirectcall(ll_setitem_nonneg, v_func, v_result, ci, v_item) + return v_result From auc at codespeak.net Tue Apr 4 14:59:40 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 4 Apr 2006 14:59:40 +0200 (CEST) Subject: [pypy-svn] r25287 - in pypy/dist/pypy/objspace: . logic/test Message-ID: <20060404125940.0CDDC10126@code0.codespeak.net> Author: auc Date: Tue Apr 4 14:59:38 2006 New Revision: 25287 Added: pypy/dist/pypy/objspace/logic/test/ pypy/dist/pypy/objspace/logic/test/targetsmallstuff.py pypy/dist/pypy/objspace/logic/test/test.py Modified: pypy/dist/pypy/objspace/logic.py Log: remove all prints ... add some testing stuff Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Tue Apr 4 14:59:38 2006 @@ -7,6 +7,7 @@ from pypy.objspace.std.listobject import W_ListObject, W_TupleObject from pypy.objspace.std.dictobject import W_DictObject from pypy.objspace.std.objectobject import W_ObjectObject +from pypy.objspace.std.intobject import W_IntObject from pypy.objspace.std.model import StdObjSpaceMultiMethod #-- THE BUILTINS ---------------------------------------------------------------------- @@ -203,7 +204,7 @@ need_waiters = schedule_state.pop_blocked_byneed_on(w_alias) w_alias.w_needed = True for waiter in need_waiters: - print " :byneed waiter", waiter, "awaken on", w_alias + #print " :byneed waiter", waiter, "awaken on", w_alias schedule_state.add_to_runnable(waiter) # set curr thread to blocked, switch to runnable thread current = get_current_coroutine() @@ -211,7 +212,7 @@ while schedule_state.have_runnable_threads(): next_coro = schedule_state.pop_runnable_thread() if next_coro.is_alive(): - print " :waiter is switching" + #print " :waiter is switching" next_coro.switch() #print " waiter is back" # hope there is a value here now @@ -234,7 +235,7 @@ def wait_needed__Var(space, w_self): while 1: - print " :needed", w_self + #print " :needed", w_self if space.is_true(space.is_free(w_self)): if w_self.w_needed: break # we're done @@ -249,7 +250,7 @@ while schedule_state.have_runnable_threads(): next_coro = schedule_state.pop_runnable_thread() if next_coro.is_alive(): - print " :needed is switching" + #print " :needed is switching" next_coro.switch() #print " byneed is back" # there might be some need right now @@ -282,6 +283,7 @@ def is_free(space, w_obj): return space.is_free(w_obj) +app_is_free = gateway.interp2app(is_free) def is_free__Root(space, w_obj): return space.newbool(False) @@ -289,7 +291,6 @@ def is_free__Var(space, w_var): return space.newbool(isinstance(w_var.w_bound_to, W_Var)) -app_is_free = gateway.interp2app(is_free) is_free_mm = StdObjSpaceMultiMethod('is_free', 1) is_free_mm.register(is_free__Root, W_Root) is_free_mm.register(is_free__Var, W_Var) @@ -297,6 +298,7 @@ def is_bound(space, w_obj): return space.is_bound(w_obj) +app_is_bound = gateway.interp2app(is_bound) def is_bound__Root(space, w_obj): return space.newbool(True) @@ -304,7 +306,6 @@ def is_bound__Var(space, w_var): return space.newbool(not isinstance(w_var.w_bound_to, W_Var)) -app_is_bound = gateway.interp2app(is_bound) is_bound_mm = StdObjSpaceMultiMethod('is_bound', 1) is_bound_mm.register(is_bound__Root, W_Root) is_bound_mm.register(is_bound__Var, W_Var) @@ -314,8 +315,6 @@ def alias_of(space, w_var1, w_var2): # FIXME: appears to block assert space.is_true(space.is_free(w_var1)) assert space.is_true(space.is_free(w_var2)) - # w_var2 could be a right-alias of w_var2 - # or the other way around w_curr = w_var1 while 1: w_next = w_curr.w_bound_to @@ -330,15 +329,15 @@ #-- HELPERS ---------------------- -def disp(space, w_var): - print w_var -app_disp = gateway.interp2app(disp) - -def disp_aliases(space, w_var): - print "Aliases of ", w_var, "are", - for w_al in aliases(space, w_var): - print w_al, - print +## def disp(space, w_var): +## print w_var +## app_disp = gateway.interp2app(disp) + +## def disp_aliases(space, w_var): +## print "Aliases of ", w_var, "are", +## for w_al in aliases(space, w_var): +## print w_al, +## print def deref(space, w_var): """gets the value of a bound variable @@ -371,9 +370,9 @@ def fail(space, w_obj1, w_obj2): """raises a specific exception for bind/unify""" #FIXME : really raise some specific exception - print "failed to bind/unify" + #print "failed to bind/unify" raise OperationError(space.w_RuntimeError, - space.wrap("UnificationFailure")) + space.wrap("Unification failure")) def check_and_memoize_pair(space, w_x, w_y): pass @@ -415,14 +414,14 @@ 2. assign bound var to unbound var 3. assign value to unbound var """ - print " :bind", w_var, w_obj + #print " :bind", w_var, w_obj space.bind(w_var, w_obj) app_bind = gateway.interp2app(bind) def bind__Var_Var(space, w_var, w_obj): if space.is_true(space.is_bound(w_var)): if space.is_true(space.is_bound(w_obj)): - return unify(space, + return unify(space, #FIXME: we could just raise deref(space, w_var), deref(space, w_obj)) # 2. a (obj unbound, var bound) @@ -438,7 +437,7 @@ # 3. var and value if space.is_true(space.is_free(w_v1)): return _assign(space, w_v1, w_v2) - print "uh !" + #print "uh !" fail(space, w_v1, w_v2) bind_mm = StdObjSpaceMultiMethod('bind', 2) @@ -447,13 +446,13 @@ all_mms['bind'] = bind_mm def _assign(space, w_var, w_val): - print " :assign", w_var, w_val, '[', + #print " :assign", w_var, w_val, '[', w_curr = w_var ass_count = 0 while 1: w_next = w_curr.w_bound_to w_curr.w_bound_to = w_val - print w_curr, + #print w_curr, ass_count += 1 # notify the blocked threads to_awake = schedule_state.pop_blocked_on(w_curr) @@ -463,13 +462,13 @@ break # switch to next w_curr = w_next - print "] (to", ass_count, "aliases)" + #print "] (to", ass_count, "aliases)" return space.w_None def _alias(space, w_v1, w_v2): """appends one var to the alias chain of another user must ensure freeness of both vars""" - print " :alias", w_v1, w_v2 + #print " :alias", w_v1, w_v2 if space.is_true(space.is_nb_(w_v1, w_v2)): return space.w_None if space.is_true(is_aliased(space, w_v1)): @@ -484,14 +483,14 @@ return space.w_None def _add_to_aliases(space, w_v1, w_v2): - print " :add to aliases", w_v1, w_v2 + #print " :add to aliases", w_v1, w_v2 w_tail = w_v1.w_bound_to w_v1.w_bound_to = w_v2 w_v2.w_bound_to = w_tail return space.w_None def _merge_aliases(space, w_v1, w_v2): - print " :merge aliases", w_v1, w_v2 + #print " :merge aliases", w_v1, w_v2 w_tail1 = get_ring_tail(space, w_v1) w_tail2 = get_ring_tail(space, w_v2) w_tail1.w_bound_to = w_v2 @@ -501,22 +500,27 @@ #-- UNIFY ------------------------- def unify(space, w_x, w_y): - print ":unify ", w_x, w_y + #print ":unify ", w_x, w_y return space.unify(w_x, w_y) app_unify = gateway.interp2app(unify) -def unify__Root_Root(space, w_x, w_y): +def unify__Int_Int(space, w_x, w_y): if not space.eq_w(w_x, w_y): - w_d1 = w_x.getdict() - w_d2 = w_y.getdict() - if None not in (w_d1, w_d2): - return space.unify(w_d1, w_d2) - else: fail(space, w_x, w_y) return space.w_None + +## def unify__Root_Root(space, w_x, w_y): +## if not space.eq_w(w_x, w_y): +## w_d1 = w_x.getdict() +## w_d2 = w_y.getdict() +## if None not in (w_d1, w_d2): +## return space.unify(w_d1, w_d2) +## else: +## fail(space, w_x, w_y) +## return space.w_None def unify__Var_Var(space, w_x, w_y): - print " :unify of two vars" + #print " :unify of two vars" if space.is_true(space.is_bound(w_x)): if space.is_true(space.is_bound(w_y)): return space.unify(deref(space, w_x), @@ -527,7 +531,7 @@ return bind(space, w_x, w_y) def unify__Var_Root(space, w_x, w_y): - print " :unify var and value" + #print " :unify var and value" if space.is_true(space.is_bound(w_x)): return space.unify(deref(space, w_x), w_y) return bind(space, w_x, w_y) @@ -542,7 +546,7 @@ return _unify_iterables(space, w_x, w_y) def _unify_iterables(space, w_i1, w_i2): - print " :unify iterables", w_i1, w_i2 + #print " :unify iterables", w_i1, w_i2 if len(w_i1.wrappeditems) != len(w_i2.wrappeditems): fail(space, w_i1, w_i2) idx, top = (-1, space.int_w(space.len(w_i1))-1) @@ -555,7 +559,7 @@ unify(space, w_xi, w_yi) def unify__Dict_Dict(space, w_m1, w_m2): - print " :unify mappings", w_m1, w_m2 + #print " :unify mappings", w_m1, w_m2 for w_xk in w_m1.content.keys(): w_xi = space.getitem(w_m1, w_xk) w_yi = space.getitem(w_m2, w_xk) @@ -565,13 +569,16 @@ unify_mm = StdObjSpaceMultiMethod('unify', 2) -unify_mm.register(unify__Root_Root, W_Root, W_Root) -unify_mm.register(unify__Var_Var, W_Var, W_Var) -unify_mm.register(unify__Var_Root, W_Var, W_Root) -unify_mm.register(unify__Root_Var, W_Root, W_Var) -unify_mm.register(unify__Tuple_Tuple, W_TupleObject, W_TupleObject) -unify_mm.register(unify__List_List, W_ListObject, W_ListObject) -unify_mm.register(unify__Dict_Dict, W_DictObject, W_DictObject) +#unify_mm.register(unify__Root_Root, W_Root, W_Root) + +unify_mm.register(unify__Int_Int, W_IntObject, W_IntObject) + +## unify_mm.register(unify__Var_Var, W_Var, W_Var) +## unify_mm.register(unify__Var_Root, W_Var, W_Root) +## unify_mm.register(unify__Root_Var, W_Root, W_Var) +## unify_mm.register(unify__Tuple_Tuple, W_TupleObject, W_TupleObject) +## unify_mm.register(unify__List_List, W_ListObject, W_ListObject) +## unify_mm.register(unify__Dict_Dict, W_DictObject, W_DictObject) all_mms['unify'] = unify_mm @@ -688,6 +695,9 @@ #-- THE SPACE --------------------------------------- +#class UnificationError(w_RuntimeError): +# pass + from pypy.objspace.std import stdtypedef from pypy.tool.sourcetools import func_with_new_name @@ -712,7 +722,6 @@ return func(space, *args) return func_with_new_name(boundmethod, 'boundmethod_'+name) boundmethod = make_boundmethod() - print boundmethod setattr(space, name, boundmethod) # store into 'space' instance # /multimethod hack @@ -734,8 +743,8 @@ space.wrap(app_alias_of)) space.setitem(space.builtin.w_dict, space.wrap('is_aliased'), space.wrap(app_is_aliased)) - space.setitem(space.builtin.w_dict, space.wrap('disp'), - space.wrap(app_disp)) +## space.setitem(space.builtin.w_dict, space.wrap('disp'), +## space.wrap(app_disp)) space.setitem(space.builtin.w_dict, space.wrap('bind'), space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), Added: pypy/dist/pypy/objspace/logic/test/targetsmallstuff.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/logic/test/targetsmallstuff.py Tue Apr 4 14:59:38 2006 @@ -0,0 +1,76 @@ +import os, sys +from pypy.translator.test import rpystone +from pypy.translator.goal import richards +import pypy.interpreter.gateway # needed before sys, order of imports !!! +from pypy.module.sys.version import svn_revision + +# __________ Entry point __________ + +VERSION = svn_revision() + +# note that we have %f but no length specifiers in RPython + +def pystones_main(loops): + benchtime, stones = rpystone.pystones(abs(loops)) + s = '' # annotator happiness + if loops >= 0: + s = ("RPystone(%s) time for %d passes = %f" % + (VERSION, loops, benchtime) + '\n' + ( + "This machine benchmarks at %f pystones/second" % stones)) + os.write(1, s) + if loops == 12345: + pystones_main(loops-1) + +def richards_main(iterations): + s = "Richards benchmark (RPython) starting...\n" + os.write(1, s) + result, startTime, endTime = richards.entry_point(iterations) + if not result: + os.write(2, "Incorrect results!\n") + return + os.write(1, "finished.\n") + total_s = endTime - startTime + avg = total_s * 1000 / iterations + os.write(1, "Total time for %d iterations: %f secs\n" %(iterations, total_s)) + os.write(1, "Average time per iteration: %f ms\n" %(avg)) + +DEF_PYSTONE = 10000000 +DEF_RICHARDS = 1000 + +def entry_point(argv): + proc = pystones_main + default = DEF_PYSTONE + n = 0 + for s in argv[1:]: + s = s.lower() + if 'pystone'.startswith(s): + proc = pystones_main + default = DEF_PYSTONE + elif 'richards'.startswith(s): + proc = richards_main + default = DEF_RICHARDS + else: + try: + n = abs(int(s)) + except ValueError: + os.write(2, '"%s" is neither a valid option (pystone, richards)' + ' nor an integer\n' % s) + return 1 + if not n: + n = default + proc(n) + return 0 + +# _____ Define and setup target ___ + +def target(*args): + return entry_point, None + +""" +Why is this a stand-alone target? + +The above target specifies None as the argument types list. +This is a case treated specially in the driver.py . If the list +of input types is empty, it is meant to be a list of strings, +actually implementing argv of the executable. +""" \ No newline at end of file Added: pypy/dist/pypy/objspace/logic/test/test.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/logic/test/test.py Tue Apr 4 14:59:38 2006 @@ -0,0 +1,43 @@ +from pypy.annotation.model import SomeTuple, SomeInteger +from pypy.objspace.logic import Space, unify, newvar + +space = Space(nofaking=True, + compiler="ast", # interpreter/astcompiler + translating=True, + usemodules=[], + geninterp=False) +# geninterp=not getattr(driver.options, 'lowmem', False)) + +def cfunc(pyfunc, annotations): + from pypy.translator.interactive import Translation + from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy + compiled = None + try: + print "translating" + t = Translation(pyfunc, policy=PyPyAnnotatorPolicy(space)) + print 'translated' + compiled = t.compile_c(annotations) + print 'compiled' + except Exception, e: + print e.__class__.__name__, e + t.view() + raise + return compiled + +## compiledFunc = cfunc(self.filterFunc, +## [SomeTuple([SomeInteger(), SomeInteger()])] * len(variables)) + + + +def unify_42(): + #X = newvar(space) + space.unify(space.newint(42), + space.newint(42)) + return space.newint(42) + + +def test_unify_var_val(): + compiled = cfunc(unify_42, []) + if compiled: + assert compiled() == space.newint(42) + From ale at codespeak.net Tue Apr 4 15:00:31 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 4 Apr 2006 15:00:31 +0200 (CEST) Subject: [pypy-svn] r25288 - pypy/dist/pypy/translator/goal Message-ID: <20060404130031.CB5E010130@code0.codespeak.net> Author: ale Date: Tue Apr 4 15:00:31 2006 New Revision: 25288 Added: pypy/dist/pypy/translator/goal/targetlogic.py Log: Intermediate checkin of a taget the logicobjectspace. This file will proabably goaway soon Added: pypy/dist/pypy/translator/goal/targetlogic.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetlogic.py Tue Apr 4 15:00:31 2006 @@ -0,0 +1,73 @@ +import os, sys +from pypy.objspace.logic import Space, newvar, unify +# XXX from pypy.annotation.model import * +# since we are execfile()'ed this would pull some +# weird objects into the globals, which we would try to pickle. +from pypy.annotation.model import SomeList, SomeString +from pypy.annotation.listdef import ListDef +from pypy.interpreter import gateway +from pypy.interpreter.error import OperationError +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy + +# WARNING: this requires the annotator. +# There is no easy way to build all caches manually, +# but the annotator can do it for us for free. + +try: + this_dir = os.path.dirname(__file__) +except NameError: + this_dir = os.path.dirname(sys.argv[0]) + +def debug(msg): + os.write(2, "debug: " + msg + '\n') + +# __________ Entry point __________ + +def entry_point(argv): + X = newvar(space) + unify(space, X, space.newint(42)) + return X + +# _____ Define and setup target ___ + +def target(driver, args): + options = driver.options + + global space, w_entry_point + + geninterp = not getattr(options, 'lowmem', False) + + # obscure hack to stuff the translation options into the translated PyPy + import pypy.module.sys + wrapstr = 'space.wrap(%r)' % (options.__dict__) + pypy.module.sys.Module.interpleveldefs['pypy_translation_info'] = wrapstr + + # disable translation of the whole of classobjinterp.py + Space.setup_old_style_classes = lambda self: None + # XXX threads are not working right now! + #if options.gc == 'boehm': + # #print "disabling thread with boehm for stabilitiy (combination not tested)" + # #print "trying threads and boehm" + # usemodules = [] + #else: + # usemodules = ['thread'] + usemodules = [] + + space = Space(nofaking=True, + compiler="ast", # interpreter/astcompiler + translating=True, + usemodules=usemodules, + geninterp=geninterp) + # manually imports app_main.py + #filename = os.path.join(this_dir, 'app_main.py') + #w_dict = space.newdict([]) + #space.exec_(open(filename).read(), w_dict, w_dict) + #w_entry_point = space.getitem(w_dict, space.wrap('entry_point')) + + # sanity-check: call the entry point + #res = entry_point(["pypy", "app_basic_example.py"]) + #assert res == 0 + + return entry_point, None, PyPyAnnotatorPolicy() + + From arigo at codespeak.net Tue Apr 4 15:30:26 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 15:30:26 +0200 (CEST) Subject: [pypy-svn] r25291 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060404133026.56B3010133@code0.codespeak.net> Author: arigo Date: Tue Apr 4 15:30:25 2006 New Revision: 25291 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py Log: (arre, arigo) Split some of the CTypesRepr functionality in two subclasses. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Tue Apr 4 15:30:25 2006 @@ -5,11 +5,11 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr -from pypy.rpython.rctypes.rmodel import CTypesRepr +from pypy.rpython.rctypes.rmodel import CTypesRefRepr ArrayType = type(ARRAY(c_int, 10)) -class ArrayRepr(CTypesRepr): +class ArrayRepr(CTypesRefRepr): def __init__(self, rtyper, s_array): array_ctype = s_array.knowntype @@ -21,6 +21,7 @@ self.r_item = item_entry.get_repr(rtyper, SomeCTypesObject(item_ctype, SomeCTypesObject.OWNSMEMORY)) + # Here, self.c_data_type == self.ll_type c_data_type = lltype.Array(self.r_item.ll_type, hints={"nolength": True}) @@ -33,14 +34,12 @@ def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item.ll_type) - inputargs = [v_array, hop.inputconst(lltype.Void, "c_data")] v_c_data = r_array.get_c_data(hop.llops, v_array) hop.genop('setarrayitem', [v_c_data, v_index, v_item]) def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) - inputargs = [v_array, hop.inputconst(lltype.Void, "c_data")] v_c_data = r_array.get_c_data(hop.llops, v_array) return hop.genop('getarrayitem', [v_c_data, v_index], r_array.r_item.ll_type) Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 4 15:30:25 2006 @@ -8,6 +8,24 @@ class CTypesRepr(Repr): "Base class for the Reprs representing ctypes object." + # Attributes that are types: + # + # * 'ctype' is the ctypes type. + # + # * 'll_type' is the low-level type representing the raw C data, + # like Signed or Array(...). + # + # * 'c_data_type' is a low-level container type that also represents + # the raw C data; the difference is that we can take + # an lltype pointer to it. For primitives or pointers + # this is a Struct with a single field 'value' of + # type 'll_type'. Otherwise, c_data_type == ll_type. + # + # * 'lowleveltype' is the Repr's choosen low-level type for the RPython + # variables. It's a Ptr to a GcStruct. This is a box + # traked by our GC around the raw 'c_data_type'-shaped + # data. + def __init__(self, rtyper, s_ctypesobject, ll_type): # s_ctypesobject: the annotation to represent # ll_type: the low-level type representing the raw @@ -25,11 +43,7 @@ else: raise TyperError("unsupported ctypes memorystate %r" % memorystate) - if isinstance(ll_type, lltype.ContainerType): - self.c_data_type = ll_type - else: - self.c_data_type = lltype.Struct('C_Data_%s' % (ctype.__name__,), - ('value', ll_type) ) + self.c_data_type = self.get_c_data_type(ll_type) if self.ownsmemory: self.lowleveltype = lltype.Ptr( @@ -56,16 +70,32 @@ return llops.genop('getfield', inputargs, lltype.Ptr(self.c_data_type) ) + +class CTypesRefRepr(CTypesRepr): + """Base class for ctypes repr that have some kind of by-reference + semantics, like structures and arrays.""" + + def get_c_data_type(self, ll_type): + assert isinstance(ll_type, lltype.ContainerType) + return ll_type + + +class CTypesValueRepr(CTypesRepr): + """Base class for ctypes repr that have some kind of by-value + semantics, like primitives and pointers.""" + + def get_c_data_type(self, ll_type): + return lltype.Struct('C_Data_%s' % (self.ctype.__name__,), + ('value', ll_type) ) + def setvalue(self, llops, v_box, v_value): - """Writes the 'value' field of the raw data - (only if ll_type is not a container type)""" + """Writes to the 'value' field of the raw data.""" v_c_data = self.get_c_data(llops, v_box) cname = inputconst(lltype.Void, 'value') llops.genop('setfield', [v_c_data, cname, v_value]) def getvalue(self, llops, v_box): - """Reads the 'value' field of the raw data - (only if ll_type is not a container type)""" + """Reads from the 'value' field of the raw data.""" v_c_data = self.get_c_data(llops, v_box) cname = inputconst(lltype.Void, 'value') return llops.genop('getfield', [v_c_data, cname], Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 4 15:30:25 2006 @@ -2,11 +2,11 @@ from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel -from pypy.rpython.rctypes.rmodel import CTypesRepr +from pypy.rpython.rctypes.rmodel import CTypesValueRepr from ctypes import POINTER, c_int -class PointerRepr(CTypesRepr): +class PointerRepr(CTypesValueRepr): def __init__(self, rtyper, s_pointer, s_contents): self.s_pointer = s_pointer self.s_contents = s_contents Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Tue Apr 4 15:30:25 2006 @@ -7,7 +7,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr -from pypy.rpython.rctypes.rmodel import CTypesRepr +from pypy.rpython.rctypes.rmodel import CTypesValueRepr ctypes_annotation_list = [ (c_char, lltype.Char), @@ -25,7 +25,7 @@ (c_double, lltype.Float), ] -class PrimitiveRepr(CTypesRepr): +class PrimitiveRepr(CTypesValueRepr): def convert_const(self, ctype_value): assert isinstance(ctype_value, self.ctype) From nik at codespeak.net Tue Apr 4 15:31:41 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 15:31:41 +0200 (CEST) Subject: [pypy-svn] r25292 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060404133141.1DF3010134@code0.codespeak.net> Author: nik Date: Tue Apr 4 15:31:39 2006 New Revision: 25292 Modified: pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: (pedronis, nik) tackle exception for ootype in general (all oosends can raise an exception) and test it for list getitem/setitem in particular. Modified: pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py Tue Apr 4 15:31:39 2006 @@ -1,13 +1,17 @@ from pypy.rpython.exceptiondata import AbstractExceptionData from pypy.rpython.ootypesystem import rclass +from pypy.rpython.ootypesystem import ootype from pypy.rpython.annlowlevel import annotate_lowlevel_helper from pypy.annotation import model as annmodel +from pypy.annotation.classdef import FORCE_ATTRIBUTES_INTO_CLASSES class ExceptionData(AbstractExceptionData): """Public information for the code generators to help with exceptions.""" def make_helpers(self, rtyper): self.fn_exception_match = self.make_exception_matcher(rtyper) + self.fn_pyexcclass2exc = self.make_pyexcclass2exc(rtyper) + self.fn_type_of_exc_inst = self.make_type_of_exc_inst(rtyper) def make_exception_matcher(self, rtyper): # ll_exception_matcher(real_exception_meta, match_exception_meta) @@ -15,3 +19,75 @@ helper_graph = annotate_lowlevel_helper( rtyper.annotator, rclass.ll_issubclass, [s_classtype, s_classtype]) return rtyper.getcallable(helper_graph) + + def make_type_of_exc_inst(self, rtyper): + # ll_type_of_exc_inst(exception_instance) -> exception_vtable + s_excinst = annmodel.SomeOOInstance(self.lltype_of_exception_value) + helper_graph = annotate_lowlevel_helper( + rtyper.annotator, rclass.ll_inst_type, [s_excinst]) + return rtyper.getcallable(helper_graph) + + def make_pyexcclass2exc(self, rtyper): + # ll_pyexcclass2exc(python_exception_class) -> exception_instance + table = {} + Exception_def = rtyper.annotator.bookkeeper.getuniqueclassdef(Exception) + for clsdef in rtyper.class_reprs: + if (clsdef and clsdef is not Exception_def + and clsdef.issubclass(Exception_def)): + if not hasattr(clsdef.classdesc, 'pyobj'): + continue + cls = clsdef.classdesc.pyobj + if cls in self.standardexceptions and cls not in FORCE_ATTRIBUTES_INTO_CLASSES: + is_standard = True + assert not clsdef.attrs, ( + "%r should not have grown attributes" % (cls,)) + else: + is_standard = (cls.__module__ == 'exceptions' + and not clsdef.attrs) + if is_standard: + r_inst = rclass.getinstancerepr(rtyper, clsdef) + r_inst.setup() + r_class = rclass.getclassrepr(rtyper, clsdef) + r_class.setup() + example = ootype.new(r_inst.lowleveltype) + example = ootype.ooupcast(self.lltype_of_exception_value, example) + example.meta = r_class.get_meta_instance() + table[cls] = example + r_inst = rclass.getinstancerepr(rtyper, None) + r_inst.setup() + r_class = rclass.getclassrepr(rtyper, None) + r_class.setup() + default_excinst = ootype.new(self.lltype_of_exception_value) + default_excinst.meta = r_class.get_meta_instance() + + # build the table in order base classes first, subclasses last + sortedtable = [] + def add_class(cls): + if cls in table: + for base in cls.__bases__: + add_class(base) + sortedtable.append((cls, table[cls])) + del table[cls] + for cls in table.keys(): + add_class(cls) + assert table == {} + + initial_value_of_i = len(sortedtable) - 1 + def pyexcclass2exc(python_exception_class): + python_exception_class = python_exception_class._obj.value + i = initial_value_of_i + while i >= 0: + if issubclass(python_exception_class, sortedtable[i][0]): + return sortedtable[i][1] + i -= 1 + return default_excinst + + # This function will only be used by the llinterpreter which usually + # expects a low-level callable (_meth, _static_meth), so we just + # fake it here. + FakeCallableType = ootype.OOType() + class fake_callable(object): + def __init__(self, fn): + self._TYPE = FakeCallableType + self._callable = fn + return fake_callable(pyexcclass2exc) Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Tue Apr 4 15:31:39 2006 @@ -46,16 +46,11 @@ class __extend__(pairtype(BaseListRepr, IntegerRepr)): def rtype_getitem((r_list, r_int), hop): - # XXX must handle exceptions - #if hop.has_implicit_exception(IndexError): # XXX must handle negative indices return r_list.send_message(hop, "getitem", can_raise=True) def rtype_setitem((r_list, r_int), hop): - # XXX must handle exceptions - #if hop.has_implicit_exception(IndexError): # XXX must handle negative indices - #if hop.has_implicit_exception(IndexError): return r_list.send_message(hop, "setitem", can_raise=True) def newlist(llops, r_list, items_v): Modified: pypy/dist/pypy/rpython/ootypesystem/rootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rootype.py Tue Apr 4 15:31:39 2006 @@ -68,6 +68,7 @@ def rtype_simple_call(self, hop): vlist = hop.inputargs(self, *hop.args_r[1:]) cname = hop.inputconst(Void, self.name) + hop.exception_is_here() return hop.genop("oosend", [cname]+vlist, resulttype = hop.r_result.lowleveltype) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 4 15:31:39 2006 @@ -1,3 +1,4 @@ +import py from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.ootypesystem.ootype import * @@ -25,6 +26,12 @@ l.setitem(0, 3) assert l.getitem(0) == 3 +def test_setitem_indexerror(): + LT = List(Signed) + l = new(LT) + py.test.raises(IndexError, l.getitem, 0) + py.test.raises(IndexError, l.setitem, 0, 1) + class TestInterpreted: def test_append_length(self): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Tue Apr 4 15:31:39 2006 @@ -121,7 +121,6 @@ assert rettype == Unsigned def test_list_getitem_setitem(): - # XXX need to test exceptions LT = List(Signed) def oof(): @@ -134,3 +133,16 @@ rettype = g.getreturnvar().concretetype assert rettype == Signed +def test_list_getitem_exceptions(): + LT = List(Signed) + + def oof(): + l = new(LT) + try: + l.getitem(0) + except IndexError: + return -1 + return 0 + + res = interpret(oof, [], type_system='ootype') + assert res is -1 From nik at codespeak.net Tue Apr 4 15:47:38 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 15:47:38 +0200 (CEST) Subject: [pypy-svn] r25293 - pypy/dist/pypy/rpython/ootypesystem/test Message-ID: <20060404134738.1FCEC10136@code0.codespeak.net> Author: nik Date: Tue Apr 4 15:47:37 2006 New Revision: 25293 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: a test to prove that exceptions actually work at RPython level for ootypesystem. Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 4 15:47:37 2006 @@ -51,3 +51,14 @@ res = interpret(f, [2], type_system="ootype") assert res == 2 + def test_getitem_exception(self): + def f(x): + l = [] + l.append(x) + try: + return l[1] + except IndexError: + return -1 + res = interpret(f, [2], type_system="ootype") + assert res == -1 + From mwh at codespeak.net Tue Apr 4 15:57:40 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 4 Apr 2006 15:57:40 +0200 (CEST) Subject: [pypy-svn] r25294 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060404135740.EC6481013C@code0.codespeak.net> Author: mwh Date: Tue Apr 4 15:57:40 2006 New Revision: 25294 Modified: pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py Log: (mwh, hpk) a test that shows the problems of not inserting keepalives during the exception transform. needs a debug build of python. Modified: pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py Tue Apr 4 15:57:40 2006 @@ -6,6 +6,11 @@ from pypy.rpython.test.test_llinterp import get_interpreter from pypy.translator.tool.cbuild import skip_missing_compiler from pypy import conftest +import sys + +def check_debug_build(): + if not hasattr(sys, 'gettotalrefcount'): + py.test.skip("test needs a debug build of Python") def transform_func(fn, inputtypes): t = TranslationContext() @@ -149,3 +154,22 @@ f(0) py.test.raises(ValueError, f, 1) +def test_needs_keepalive(): + check_debug_build() + from pypy.rpython.lltypesystem import lltype + X = lltype.GcStruct("X", + ('y', lltype.Struct("Y", ('z', lltype.Signed)))) + def can_raise(n): + if n: + raise Exception + else: + return 1 + def foo(n): + x = lltype.malloc(X) + y = x.y + y.z = 42 + r = can_raise(n) + return r + y.z + f = compile_func(foo, [int]) + res = f(0) + assert res == 43 From arigo at codespeak.net Tue Apr 4 16:11:14 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 16:11:14 +0200 (CEST) Subject: [pypy-svn] r25295 - pypy/branch/jit-timeshifter-refactoring/timeshifter Message-ID: <20060404141114.6A2F810140@code0.codespeak.net> Author: arigo Date: Tue Apr 4 16:11:13 2006 New Revision: 25295 Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Log: Fixed imports. Modified: pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py ============================================================================== --- pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py (original) +++ pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Tue Apr 4 16:11:13 2006 @@ -5,8 +5,8 @@ from pypy.annotation import listdef, dictdef from pypy.jit.timeshifter import rvalue from pypy.jit.timeshifter.rtimeshift import JITState -from pypy.rpython import rmodel, rlist, rdict, rgenop, annlowlevel -from pypy.rpython.lltypesystem import rtuple +from pypy.rpython import rmodel, rdict, rgenop, annlowlevel +from pypy.rpython.lltypesystem import rtuple, rlist from pypy.jit.timeshifter import rtimeshift from pypy.jit.timeshifter.rtyper import HintRTyper, originalconcretetype from pypy.jit.timeshifter.rtyper import GreenRepr, RedRepr, HintLowLevelOpList From arigo at codespeak.net Tue Apr 4 16:12:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 16:12:05 +0200 (CEST) Subject: [pypy-svn] r25296 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060404141205.A1E5210139@code0.codespeak.net> Author: arigo Date: Tue Apr 4 16:12:04 2006 New Revision: 25296 Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: (arre, arigo) Passing the specialization test for rpointer. Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 4 16:12:04 2006 @@ -2,7 +2,7 @@ from pypy.rpython.error import TyperError from pypy.rpython.lltypesystem import lltype from pypy.annotation.model import SomeCTypesObject - +from pypy.annotation.pairtype import pairtype class CTypesRepr(Repr): @@ -70,6 +70,34 @@ return llops.genop('getfield', inputargs, lltype.Ptr(self.c_data_type) ) + def allocate_instance(self, llops): + c1 = inputconst(lltype.Void, self.lowleveltype.TO) + return llops.genop("malloc", [c1], resulttype=self.lowleveltype) + + def allocate_instance_ref(self, llops, v_c_data): + """Only if self.ownsmemory is false. This allocates a new instance + and initialize its c_data_ref field.""" + if self.ownsmemory: + raise TyperError("allocate_instance_ref: %r owns its memory" % ( + self,)) + v_box = self.allocate_instance(llops) + inputargs = [v_box, inputconst(lltype.Void, "c_data_ref"), v_c_data] + llops.genop('setfield', inputargs) + return v_box + + +class __extend__(pairtype(CTypesRepr, CTypesRepr)): + + def convert_from_to((r_from, r_to), v, llops): + """Transparent conversion from the memory-owned to the memory-aliased + version of the same ctypes repr.""" + if (r_from.ctype == r_to.ctype and + r_from.ownsmemory and not r_to.ownsmemory): + v_c_data = r_from.get_c_data(llops, v) + return r_to.allocate_instance_ref(llops, v_c_data) + else: + return NotImplemented + class CTypesRefRepr(CTypesRepr): """Base class for ctypes repr that have some kind of by-reference Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 4 16:12:04 2006 @@ -11,18 +11,19 @@ self.s_pointer = s_pointer self.s_contents = s_contents self.ref_ctype = s_contents.knowntype - - if not extregistry.is_registered_type(self.ref_ctype): - raise TypeError("Unregistered referenced " - "type: %s" % (self.ref_ctype.__name__,)) - - ref_entry = extregistry.lookup_type(self.ref_ctype) - contents_repr = ref_entry.get_repr(rtyper, self.s_contents) - - ll_contents = lltype.Ptr(contents_repr.c_data_type) + self.r_contents = rtyper.getrepr(s_contents) + ll_contents = lltype.Ptr(self.r_contents.c_data_type) super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) + def rtype_getattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'contents' + v_ptr = hop.inputarg(self, 0) + v_value = self.getvalue(hop.llops, v_ptr) + return self.r_contents.allocate_instance_ref(hop.llops, v_value) + #def registerPointerType(ptrtype): # """Adds a new pointer type to the extregistry. # @@ -107,7 +108,13 @@ methodname=type.__name__) def pointertype_specialize_call(hop): - xxx #... + r_ptr = hop.r_result + v_result = r_ptr.allocate_instance(hop.llops) + if len(hop.args_s): + v_contentsbox, = hop.inputargs(r_ptr.r_contents) + v_c_data = r_ptr.r_contents.get_c_data(hop.llops, v_contentsbox) + r_ptr.setvalue(hop.llops, v_result, v_c_data) + return v_result def pointerinstance_compute_annotation(type, instance): return annmodel.SomeCTypesObject(type, Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Tue Apr 4 16:12:04 2006 @@ -65,8 +65,7 @@ def primitive_specialize_call(hop): r_primitive = hop.r_result - c1 = hop.inputconst(lltype.Void, r_primitive.lowleveltype.TO) - v_result = hop.genop("malloc", [c1], resulttype=r_primitive.lowleveltype) + v_result = r_primitive.allocate_instance(hop.llops) if len(hop.args_s): v_value, = hop.inputargs(r_primitive.ll_type) r_primitive.setvalue(hop.llops, v_result, v_value) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Tue Apr 4 16:12:04 2006 @@ -51,7 +51,6 @@ class Test_specialization: def test_specialize_c_int_ptr(self): - py.test.skip("in-progress") ptrtype = POINTER(c_int) def func(): res = c_int(42) @@ -60,3 +59,15 @@ res = interpret(func, []) assert res == 42 + + def test_specialize_mutate_via_pointer(self): + ptrtype = POINTER(c_int) + def func(): + res = c_int(6) + p1 = ptrtype(res) + p2 = ptrtype(p1.contents) + p2.contents.value *= 7 + return res.value + + res = interpret(func, []) + assert res == 42 From cfbolz at codespeak.net Tue Apr 4 16:13:46 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 4 Apr 2006 16:13:46 +0200 (CEST) Subject: [pypy-svn] r25297 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060404141346.51B0410141@code0.codespeak.net> Author: cfbolz Date: Tue Apr 4 16:13:44 2006 New Revision: 25297 Modified: pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py Log: failing test due to missing keep_alives Modified: pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py Tue Apr 4 16:13:44 2006 @@ -83,6 +83,30 @@ fn = compile_func(g, []) assert fn() == 42 +def test_missing_keepalive_exception(): + py.test.skip("failing due to missing keep_alives") + A = lltype.Array(lltype.Signed) + S = lltype.GcStruct("S", ('a', A)) + def f(x): + if not x: + raise ValueError + return x + 1 + def g(x): + s = lltype.malloc(S, x) + a = s.a + a[x - 1] = 42 + i = x - 2 + try: + i = f(i) + except ValueError: + i = x - 1 + # need to allocate another instance, which will probably be in + # the same place as the previous one + s = lltype.malloc(S, x) + return a[i] + fn = compile_func(g, [int]) + assert fn(6) == 42 + def test_multiply_passed_var(): S = lltype.GcStruct("S", ('x', lltype.Signed)) def f(x): @@ -313,3 +337,4 @@ fn = self.getcompiled(f) res = fn() assert res == 43 + 1000000 + From nik at codespeak.net Tue Apr 4 16:36:41 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 16:36:41 +0200 (CEST) Subject: [pypy-svn] r25298 - in pypy/dist/pypy/rpython: . test Message-ID: <20060404143641.493D21013E@code0.codespeak.net> Author: nik Date: Tue Apr 4 16:36:39 2006 New Revision: 25298 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/test/test_llinterp.py Log: (pedronis, nik) exceptions that escape out of the llinterpreter were only working with the lltypesystem. generalize this for ootypesystem. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Tue Apr 4 16:36:39 2006 @@ -23,7 +23,14 @@ extra = extra.replace('\n', '\n | ') + '\n `------' else: extra = '' - return '' % (''.join(etype.name).rstrip('\x00'), extra) + return '' % (type_name(etype), extra) + +def type_name(etype): + if isinstance(lltype.typeOf(etype), lltype.Ptr): + return ''.join(etype.name).rstrip('\x00') + else: + # ootype! + return etype.class_._INSTANCE._name.split(".")[-1] class LLInterpreter(object): """ low level interpreter working with concrete values. """ @@ -94,14 +101,19 @@ assert isinstance(exc, LLException) import exceptions klass, inst = exc.args[0], exc.args[1] - # indirect way to invoke fn_pyexcclass2exc, for memory/test/test_llinterpsim - f = self.typer.getexceptiondata().fn_pyexcclass2exc - obj = self.typer.type_system.deref(f) - ll_pyexcclass2exc_graph = obj.graph - for cls in exceptions.__dict__.values(): - if type(cls) is type(Exception): - if self.eval_graph(ll_pyexcclass2exc_graph, [lltype.pyobjectptr(cls)]).typeptr == klass: - return cls + exdata = self.typer.getexceptiondata() + frame = LLFrame(None, [], self) + old_active_frame = self.active_frame + try: + for cls in exceptions.__dict__.values(): + if type(cls) is type(Exception): + evalue = frame.op_direct_call(exdata.fn_pyexcclass2exc, + lltype.pyobjectptr(cls)) + etype = frame.op_direct_call(exdata.fn_type_of_exc_inst, evalue) + if etype == klass: + return cls + finally: + self.active_frame = old_active_frame raise ValueError, "couldn't match exception" @@ -126,7 +138,7 @@ class LLFrame(object): def __init__(self, graph, args, llinterpreter, f_back=None): - assert isinstance(graph, FunctionGraph) + assert not graph or isinstance(graph, FunctionGraph) self.graph = graph self.args = args self.llinterpreter = llinterpreter Modified: pypy/dist/pypy/rpython/test/test_llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_llinterp.py (original) +++ pypy/dist/pypy/rpython/test/test_llinterp.py Tue Apr 4 16:36:39 2006 @@ -128,6 +128,8 @@ assert res == 41 interpret_raises(IndexError, raise_exception, [42]) interpret_raises(ValueError, raise_exception, [43]) + interpret_raises(IndexError, raise_exception, [42], type_system="ootype") + interpret_raises(ValueError, raise_exception, [43], type_system="ootype") def test_call_raise(): res = interpret(call_raise, [41]) From arigo at codespeak.net Tue Apr 4 16:44:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 16:44:48 +0200 (CEST) Subject: [pypy-svn] r25299 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060404144448.8640B10144@code0.codespeak.net> Author: arigo Date: Tue Apr 4 16:44:47 2006 New Revision: 25299 Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: (arre, arigo) Keepalives for pointers and for non-memory-owning boxes. Seems to be right, but not obvious. Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 4 16:44:47 2006 @@ -25,6 +25,9 @@ # variables. It's a Ptr to a GcStruct. This is a box # traked by our GC around the raw 'c_data_type'-shaped # data. + # + # * 'owner_lowleveltype' is the lowleveltype of the repr for the same + # ctype but for ownsmemory=True. def __init__(self, rtyper, s_ctypesobject, ll_type): # s_ctypesobject: the annotation to represent @@ -44,22 +47,30 @@ raise TyperError("unsupported ctypes memorystate %r" % memorystate) self.c_data_type = self.get_c_data_type(ll_type) + content_keepalives = self.get_content_keepalives() - if self.ownsmemory: - self.lowleveltype = lltype.Ptr( + self.owner_lowleveltype = lltype.Ptr( lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data", self.c_data_type ) + ( "c_data", self.c_data_type ), + *content_keepalives ) ) + if self.ownsmemory: + self.lowleveltype = self.owner_lowleveltype else: self.lowleveltype = lltype.Ptr( lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data_ref", lltype.Ptr(self.c_data_type) ) + ( "c_data_ref", lltype.Ptr(self.c_data_type) ), + ( "c_data_owner_keepalive", self.owner_lowleveltype ), + *content_keepalives ) ) - # XXX keepalives... self.const_cache = {} # store generated const values+original value + def get_content_keepalives(self): + "Return extra keepalive fields used for the content of this object." + return [] + def get_c_data(self, llops, v_box): if self.ownsmemory: inputargs = [v_box, inputconst(lltype.Void, "c_data")] @@ -70,11 +81,20 @@ return llops.genop('getfield', inputargs, lltype.Ptr(self.c_data_type) ) + def get_c_data_owner(self, llops, v_box): + if self.ownsmemory: + return v_box + else: + inputargs = [v_box, inputconst(lltype.Void, + "c_data_owner_keepalive")] + return llops.genop('getfield', inputargs, + self.owner_lowleveltype) + def allocate_instance(self, llops): c1 = inputconst(lltype.Void, self.lowleveltype.TO) return llops.genop("malloc", [c1], resulttype=self.lowleveltype) - def allocate_instance_ref(self, llops, v_c_data): + def allocate_instance_ref(self, llops, v_c_data, v_c_data_owner=None): """Only if self.ownsmemory is false. This allocates a new instance and initialize its c_data_ref field.""" if self.ownsmemory: @@ -83,6 +103,12 @@ v_box = self.allocate_instance(llops) inputargs = [v_box, inputconst(lltype.Void, "c_data_ref"), v_c_data] llops.genop('setfield', inputargs) + if v_c_data_owner is not None: + assert v_c_data_owner.concretetype == self.owner_lowleveltype + inputargs = [v_box, + inputconst(lltype.Void, "c_data_owner_keepalive"), + v_c_data_owner] + llops.genop('setfield', inputargs) return v_box @@ -94,7 +120,7 @@ if (r_from.ctype == r_to.ctype and r_from.ownsmemory and not r_to.ownsmemory): v_c_data = r_from.get_c_data(llops, v) - return r_to.allocate_instance_ref(llops, v_c_data) + return r_to.allocate_instance_ref(llops, v_c_data, v) else: return NotImplemented Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 4 16:44:47 2006 @@ -1,4 +1,4 @@ -from pypy.rpython.rmodel import Repr +from pypy.rpython.rmodel import Repr, inputconst from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel @@ -16,6 +16,15 @@ super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) + def get_content_keepalives(self): + "Return an extra keepalive field used for the pointer's contents." + return [('keepalive_contents', self.r_contents.owner_lowleveltype)] + + def setkeepalive(self, llops, v_box, v_owner): + inputargs = [v_box, inputconst(lltype.Void, 'keepalive_contents'), + v_owner] + llops.genop('setfield', inputargs) + def rtype_getattr(self, hop): s_attr = hop.args_s[1] assert s_attr.is_constant() @@ -113,7 +122,9 @@ if len(hop.args_s): v_contentsbox, = hop.inputargs(r_ptr.r_contents) v_c_data = r_ptr.r_contents.get_c_data(hop.llops, v_contentsbox) + v_owner = r_ptr.r_contents.get_c_data_owner(hop.llops, v_contentsbox) r_ptr.setvalue(hop.llops, v_result, v_c_data) + r_ptr.setkeepalive(hop.llops, v_result, v_owner) return v_result def pointerinstance_compute_annotation(type, instance): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Tue Apr 4 16:44:47 2006 @@ -71,3 +71,20 @@ res = interpret(func, []) assert res == 42 + + def test_keepalive(self): + ptrtype = POINTER(c_int) + def func(n): + if n == 1: + x = c_int(6) + p1 = ptrtype(x) + else: + y = c_int(7) + p1 = ptrtype(y) + # x or y risk being deallocated by the time we get there, + # unless p1 correctly keeps them alive. The llinterpreter + # detects this error exactly. + return p1.contents.value + + res = interpret(func, [3]) + assert res == 7 From arigo at codespeak.net Tue Apr 4 16:47:53 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 16:47:53 +0200 (CEST) Subject: [pypy-svn] r25300 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060404144753.8E38310145@code0.codespeak.net> Author: arigo Date: Tue Apr 4 16:47:52 2006 New Revision: 25300 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: (arre, arigo) Added a test, passing. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Tue Apr 4 16:47:52 2006 @@ -88,3 +88,25 @@ res = interpret(func, [3]) assert res == 7 + + def test_keepalive_2(self): + ptrtype = POINTER(c_int) + ptrptrtype = POINTER(ptrtype) + def func(n): + if n == 1: + x = c_int(6) + p1 = ptrtype(x) + p2 = ptrptrtype(p1) + else: + if n == 2: + y = c_int(7) + p1 = ptrtype(y) + else: + z = c_int(8) + p1 = ptrtype(z) + p2 = ptrptrtype(p1) + p2.contents.contents.value *= 6 + return p2.contents.contents.value + + res = interpret(func, [2]) + assert res == 42 From auc at codespeak.net Tue Apr 4 17:06:47 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 4 Apr 2006 17:06:47 +0200 (CEST) Subject: [pypy-svn] r25301 - pypy/dist/pypy/objspace/logic Message-ID: <20060404150647.D28CB10147@code0.codespeak.net> Author: auc Date: Tue Apr 4 17:06:47 2006 New Revision: 25301 Added: pypy/dist/pypy/objspace/logic/syntax_proposal.py Log: discussion around syntax for logic variables Added: pypy/dist/pypy/objspace/logic/syntax_proposal.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/logic/syntax_proposal.py Tue Apr 4 17:06:47 2006 @@ -0,0 +1,93 @@ +def foo_explicit(x,y): + """straightforward version. Just a new builtin to add The name of + the builtin can be discussed (current implementation uses + 'newvar') + """ + X = logicvar() + Y = logicvar() + unify(X, x) + unify(Y, y) + return X+Y + + +def foo_let(x, y): + """requires adding a new node in the AST + the scoping rules need to be clarified + """ + let X, Y: + unify(X, x) + unify(Y, y) + xx = X + return X+Y + print X #-> raises name error + print xx # no NameError (?) + +def foo_logic_kw(x,y): + """needs a new keyword + the logic keyword does not introduce a new scope (compare with let above)""" + logic X,Y + unify(X, x) + unify(Y, y) + return X+Y + +def foo_with(x,y): + """requires python 2.5 + we need logic to be a built in context manager + scoping rules are those of with""" + with logic(X, Y): + unify(X, x) + unify(Y, y) + return X+Y + print X #-> raises name error ??? + + at logic('X','Y') +def foo_decorator(x,y): + """can this work at all ? The decorator would add local variables + to the function with the provided names, initialized as logic + variable""" + unify(X, x) + unify(Y, y) + return X+Y + + +def foo_question_mark(x,y): + """introduce a new symbol '?' + """ + X, Y = ?, ? + unify(X, x) + unify(Y, y) + return X+Y + +def foo_singleton(x,y): + """similar to foo_question_mark, but Unknown can clash with + existing code, and is longer to type""" + X, Y = Unknown, Unknown + unify(X, x) + unify(Y, y) + return X+Y + +def foo_equal_question_mark(x,y): + """introduces a new token '=?' without a right hand part + """ + X, Y =? + unify(X, x) + unify(Y, y) + return X+Y + + +def foo_visual_basic(x,y): + """for completeness, april fool was just a few days ago, please + bear with us""" + Dim X, Y as LogicalVariables + unify(X, x) + unify(Y, y) + return X+Y + +def foo_strange_question_mark(x,y): # ????? + """semantics are not totally clear yet, and auc feels this is not + going to match real life use cases. """ + x = ?W + ?Z + unify(?X, x) + unify(?Y, y) + return ?X + ?Y From arigo at codespeak.net Tue Apr 4 17:08:06 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 17:08:06 +0200 (CEST) Subject: [pypy-svn] r25302 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060404150806.4027510149@code0.codespeak.net> Author: arigo Date: Tue Apr 4 17:08:05 2006 New Revision: 25302 Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: (arre, arigo) Support for calling pointer(). Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 4 17:08:05 2006 @@ -4,7 +4,7 @@ from pypy.annotation import model as annmodel from pypy.rpython.rctypes.rmodel import CTypesValueRepr -from ctypes import POINTER, c_int +from ctypes import POINTER, pointer, c_int class PointerRepr(CTypesValueRepr): def __init__(self, rtyper, s_pointer, s_contents): @@ -150,3 +150,16 @@ compute_annotation=pointerinstance_compute_annotation, get_repr=pointerinstance_get_repr) entry.get_field_annotation = pointerinstance_field_annotation + +def pointerfn_compute_annotation(s_arg): + assert isinstance(s_arg, annmodel.SomeCTypesObject) + ctype = s_arg.knowntype + result_ctype = POINTER(ctype) + return annmodel.SomeCTypesObject(result_ctype, + annmodel.SomeCTypesObject.OWNSMEMORY) + +extregistry.register_value(pointer, + compute_result_annotation=pointerfn_compute_annotation, + # same rtyping for calling pointer() or calling a specific instance + # of PointerType: + specialize_call=pointertype_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Tue Apr 4 17:08:05 2006 @@ -7,7 +7,7 @@ from pypy import conftest from pypy.rpython.test.test_llinterp import interpret -from ctypes import c_int, c_float, POINTER +from ctypes import c_int, c_float, POINTER, pointer import pypy.rpython.rctypes.implementation @@ -49,6 +49,21 @@ assert s.knowntype == float + def test_annotate_pointer_fn(self): + def func(): + p = pointer(c_int(123)) + return p.contents.value + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + + if conftest.option.view: + t.view() + + assert s.knowntype == int + + class Test_specialization: def test_specialize_c_int_ptr(self): ptrtype = POINTER(c_int) @@ -94,19 +109,28 @@ ptrptrtype = POINTER(ptrtype) def func(n): if n == 1: - x = c_int(6) - p1 = ptrtype(x) + p1 = ptrtype(c_int(6)) p2 = ptrptrtype(p1) else: if n == 2: - y = c_int(7) - p1 = ptrtype(y) + p1 = ptrtype(c_int(7)) else: - z = c_int(8) - p1 = ptrtype(z) + p1 = ptrtype(c_int(8)) p2 = ptrptrtype(p1) - p2.contents.contents.value *= 6 + del p1 + p3 = ptrptrtype(p2.contents) + p3.contents.contents.value *= 6 return p2.contents.contents.value + assert func(2) == 42 res = interpret(func, [2]) assert res == 42 + + def test_specialize_pointer_fn(self): + def func(): + p = pointer(c_int(123)) + return p.contents.value + + assert func() == 123 + res = interpret(func, []) + assert res == 123 From nik at codespeak.net Tue Apr 4 17:18:01 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 17:18:01 +0200 (CEST) Subject: [pypy-svn] r25303 - in pypy/dist/pypy/rpython: ootypesystem ootypesystem/test test Message-ID: <20060404151801.D74F210057@code0.codespeak.net> Author: nik Date: Tue Apr 4 17:17:59 2006 New Revision: 25303 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/test/test_rpbc.py Log: (nik, pedronis around) skip a new rpbc test for ootypesystem, it uses lists and strings which we don't fully support, yet. but this test uncovered that we are missing a null list, added that. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 4 17:17:59 2006 @@ -174,6 +174,7 @@ def __init__(self, ITEMTYPE): self._ITEMTYPE = ITEMTYPE + self._null = _null_list(self) # This defines the abstract list interface that backends will have # to map to their native list implementations. @@ -199,6 +200,9 @@ def _example(self): return new(self) + def _defl(self): + return self._null + # ____________________________________________________________ class _class(object): @@ -277,34 +281,36 @@ return 0 # for all null instances -class _null_instance(_instance): +def _null_mixin(klass): + class mixin(object): - def __init__(self, INSTANCE): - self.__dict__["_TYPE"] = INSTANCE - - def __str__(self): - return '%r null inst' % (self._TYPE._name,) + def __str__(self): + return '%r null inst' % (self._TYPE._name,) - def __getattribute__(self, name): - if name.startswith("_"): - return object.__getattribute__(self, name) - - self._TYPE._check_field(name) + def __getattribute__(self, name): + if name.startswith("_"): + return object.__getattribute__(self, name) - raise RuntimeError("Access to field in null object") + raise RuntimeError("Access to field in null object") - def __setattr__(self, name, value): - _instance.__setattr__(self, name, value) + def __setattr__(self, name, value): + klass.__setattr__(self, name, value) - raise RuntimeError("Assignment to field in null object") + raise RuntimeError("Assignment to field in null object") - def __nonzero__(self): - return False + def __nonzero__(self): + return False - def __eq__(self, other): - if not isinstance(other, _instance): - raise TypeError("comparing an _instance with %r" % (other,)) - return not other + def __eq__(self, other): + if not isinstance(other, klass): + raise TypeError("comparing an %s with %r" % (klass.__name__, other)) + return not other + return mixin + +class _null_instance(_null_mixin(_instance), _instance): + + def __init__(self, INSTANCE): + self.__dict__["_TYPE"] = INSTANCE class _view(object): @@ -486,6 +492,11 @@ assert typeOf(index) == Signed self._list[index] = item +class _null_list(_null_mixin(_list), _list): + + def __init__(self, LIST): + self.__dict__["_TYPE"] = LIST + def new(TYPE): if isinstance(TYPE, Instance): return make_instance(TYPE) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 4 17:17:59 2006 @@ -32,6 +32,11 @@ py.test.raises(IndexError, l.getitem, 0) py.test.raises(IndexError, l.setitem, 0, 1) +def test_null(): + LT = List(Signed) + n = null(LT) + py.test.raises(RuntimeError, "n.append(0)") + class TestInterpreted: def test_append_length(self): Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Tue Apr 4 17:17:59 2006 @@ -968,6 +968,8 @@ assert res.item1 == f(i)[1] def test_pbc_imprecise_attrfamily(self): + if self.ts == "ootype": + py.test.skip("fix me if ootype fully supports lists") fr1 = Freezing(); fr1.x = 5; fr1.y = [8] fr2 = Freezing(); fr2.x = 6; fr2.y = ["string"] def head(fr): From mwh at codespeak.net Tue Apr 4 17:29:49 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 4 Apr 2006 17:29:49 +0200 (CEST) Subject: [pypy-svn] r25304 - in pypy/branch/explicit-exceptions/translator/c: . test Message-ID: <20060404152949.C067A1013D@code0.codespeak.net> Author: mwh Date: Tue Apr 4 17:29:48 2006 New Revision: 25304 Modified: pypy/branch/explicit-exceptions/translator/c/exceptiontransform.py pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py Log: (hpk, mwh) insert keepalives as needed (we think!) in the exception transformer. Modified: pypy/branch/explicit-exceptions/translator/c/exceptiontransform.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/exceptiontransform.py (original) +++ pypy/branch/explicit-exceptions/translator/c/exceptiontransform.py Tue Apr 4 17:29:48 2006 @@ -5,7 +5,7 @@ c_last_exception, SpaceOperation, checkgraph, FunctionGraph from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.memory.lladdress import NULL -from pypy.rpython.memory.gctransform import varoftype +from pypy.rpython.memory.gctransform import varoftype, var_needsgc from pypy.rpython import rclass from pypy.rpython.rarithmetic import r_uint, r_longlong, r_ulonglong from pypy.annotation import model as annmodel @@ -29,6 +29,19 @@ return Constant(lltype.nullptr(T.TO), T) assert 0, "not implemented yet" +def insert_keepalives_along(translator, link, vars): + link.args.extend(vars) + newvars = [copyvar(translator, v) for v in vars] + block = link.target + block.inputargs.extend(newvars) + block.operations[0:0] = [SpaceOperation('keepalive', [v], + varoftype(lltype.Void)) + for v in newvars] + +def vars_to_keepalive(block): + # XXX make cleverer + return [v for v in block.getvariables() if var_needsgc(v)] + class ExceptionTransformer(object): def __init__(self, translator): self.translator = translator @@ -153,7 +166,10 @@ if lastblock is block: lastblock = afterblock - self.gen_exc_check(block, graph.returnblock) + insert_keepalives_along(self.translator, block.exits[0], + vars_to_keepalive(block)) + + self.gen_exc_check(block, graph.returnblock) #non-exception case block.exits[0].exitcase = block.exits[0].llexitcase = False Modified: pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py Tue Apr 4 17:29:48 2006 @@ -9,7 +9,10 @@ import sys def check_debug_build(): - if not hasattr(sys, 'gettotalrefcount'): + # the 'not conftest.option.view' is because debug builds rarely + # have pygame, so if you want to see the graphs pass --view and + # don't be surprised when the test then passes when it shouldn't. + if not hasattr(sys, 'gettotalrefcount') and not conftest.option.view: py.test.skip("test needs a debug build of Python") def transform_func(fn, inputtypes): From pedronis at codespeak.net Tue Apr 4 17:43:07 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 4 Apr 2006 17:43:07 +0200 (CEST) Subject: [pypy-svn] r25305 - pypy/dist/pypy/translator/backendopt/test Message-ID: <20060404154307.815B31014F@code0.codespeak.net> Author: pedronis Date: Tue Apr 4 17:43:06 2006 New Revision: 25305 Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py Log: case were the current keepalive insertion by inlining is insufficient. Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Tue Apr 4 17:43:06 2006 @@ -469,4 +469,24 @@ t.view() assert hasattr(ggraph.startblock.operations[0], "cleanup") - +def test_keepalive_hard_case(): + py.test.skip("fix this :(") + from pypy.rpython.lltypesystem import lltype + Y = lltype.Struct('y', ('n', lltype.Signed)) + X = lltype.GcStruct('x', ('y', Y)) + def g(x): + if x: + return 3 + else: + return 4 + def f(): + x = lltype.malloc(X) + x.y.n = 2 + y = x.y + z1 = g(y.n) + z = y.n + return z+z1 + eval_func = check_inline(g, f, []) + res = eval_func([]) + assert res == 5 + From arigo at codespeak.net Tue Apr 4 18:04:07 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 18:04:07 +0200 (CEST) Subject: [pypy-svn] r25306 - in pypy/dist/pypy: annotation rpython/rctypes/test Message-ID: <20060404160407.2C16A1014E@code0.codespeak.net> Author: arigo Date: Tue Apr 4 18:04:05 2006 New Revision: 25306 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: (arre, arigo) Re-enabled a test. Reorganized a little bit things although the test passed at once. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Tue Apr 4 18:04:05 2006 @@ -745,6 +745,9 @@ def getitem((s_cto, s_index)): if s_index.is_constant(): + # check that the constant index is valid, just because we + # are nice (users should really catch such errors by + # testing their programs!) idx = s_index.const knowntype = s_cto.knowntype try: @@ -758,18 +761,20 @@ raise IndexError( "invalid index" ) elif idx < 0: raise IndexError( "invalid index" ) - try: - # This is the case for unboxed values, which - # are those having memorystate NOMEMORY - return s_cto.knowntype._type_.annotator_type - except AttributeError: - if extregistry.is_registered_type(s_cto.knowntype._type_): - entry = extregistry.lookup_type(s_cto.knowntype._type_) - return lltype_to_annotation(entry.lowleveltype) - + + # Note: The following works for index either pointers and arrays, + # because both have a _type_ attribute that contains the type of the + # object pointed to or in the case of an array the element type. + assert hasattr(s_cto.knowntype, '_type_') + assert extregistry.is_registered_type(s_cto.knowntype._type_) + entry = extregistry.lookup_type(s_cto.knowntype._type_) + if hasattr(entry, 'lowleveltype'): + # special case for reading primitives out of arrays + return lltype_to_annotation(entry.lowleveltype) + else: return SomeCTypesObject( - s_cto.knowntype._type_, - memorystate=s_cto.memorystate) + s_cto.knowntype._type_, + memorystate=s_cto.memorystate) class __extend__(pairtype(SomeCTypesObject, SomeSlice)): def setitem((s_cto, s_slice), s_iterable): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Tue Apr 4 18:04:05 2006 @@ -15,10 +15,9 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int, ARRAY, POINTER +from ctypes import c_int, ARRAY, POINTER, pointer c_int_10 = ARRAY(c_int,10) -c_int_p_test = POINTER(c_int) #py.test.skip("Reworking primitive types") @@ -50,22 +49,15 @@ if conftest.option.view: t.view() - def x_test_annotate_pointer_access_as_array(self): + def test_annotate_pointer_access_as_array(self): """ - Make sure that pointers work the same way as arrays, for - ctypes compatibility. - - :Note: This works because pointer and array classes both - have a _type_ attribute, that contains the type of the - object pointed to or in the case of an array the element type. + Make sure that pointers work the same way as arrays. """ def access_array(): # Never run this function! - # See test_annotate_pointer_access_as_array_or_whatever - # for the weird reasons why this gets annotated - my_pointer = c_int_p_test(10) + my_pointer = pointer(c_int(10)) my_pointer[0] = c_int(1) - my_pointer[1] = 2 + my_pointer[1] = 2 # <== because of this return my_pointer[0] @@ -73,7 +65,9 @@ a = t.buildannotator() s = a.build_types(access_array, []) assert s.knowntype == int - #d#t.view() + + if conftest.option.view: + t.view() def x_test_annotate_array_slice_access(self): def slice_access(): From ale at codespeak.net Tue Apr 4 18:29:46 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 4 Apr 2006 18:29:46 +0200 (CEST) Subject: [pypy-svn] r25307 - pypy/dist/pypy/translator/goal Message-ID: <20060404162946.7F22C10152@code0.codespeak.net> Author: ale Date: Tue Apr 4 18:29:45 2006 New Revision: 25307 Modified: pypy/dist/pypy/translator/goal/targetlogicstandalone.py Log: Make targetlogicstandalone use stackless by default Modified: pypy/dist/pypy/translator/goal/targetlogicstandalone.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetlogicstandalone.py (original) +++ pypy/dist/pypy/translator/goal/targetlogicstandalone.py Tue Apr 4 18:29:45 2006 @@ -44,6 +44,8 @@ # _____ Define and setup target ___ +opt_defaults = {'stackless' : True} + def target(driver, args): options = driver.options From mwh at codespeak.net Tue Apr 4 18:40:33 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 4 Apr 2006 18:40:33 +0200 (CEST) Subject: [pypy-svn] r25308 - pypy/branch/explicit-exceptions/translator/c/src Message-ID: <20060404164033.3D4F310153@code0.codespeak.net> Author: mwh Date: Tue Apr 4 18:40:31 2006 New Revision: 25308 Modified: pypy/branch/explicit-exceptions/translator/c/src/support.h Log: change the CFAIL macro to make using it less likely to be a syntax error Modified: pypy/branch/explicit-exceptions/translator/c/src/support.h ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/src/support.h (original) +++ pypy/branch/explicit-exceptions/translator/c/src/support.h Tue Apr 4 18:40:31 2006 @@ -16,7 +16,7 @@ #define FAIL_OVF(msg) FAIL_EXCEPTION(PyExc_OverflowError, msg) #define FAIL_VAL(msg) FAIL_EXCEPTION(PyExc_ValueError, msg) #define FAIL_ZER(msg) FAIL_EXCEPTION(PyExc_ZeroDivisionError, msg) -#define CFAIL() { RPyConvertExceptionFromCPython(); } +#define CFAIL() RPyConvertExceptionFromCPython() #define PyString_FromLLCharArrayAndSize(itemsarray, size) \ PyString_FromStringAndSize(itemsarray->items, size) From auc at codespeak.net Tue Apr 4 18:59:06 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 4 Apr 2006 18:59:06 +0200 (CEST) Subject: [pypy-svn] r25309 - pypy/dist/pypy/objspace Message-ID: <20060404165906.F267710156@code0.codespeak.net> Author: auc Date: Tue Apr 4 18:59:06 2006 New Revision: 25309 Modified: pypy/dist/pypy/objspace/logic.py Log: placate logic objspace with assert isinstance ... Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Tue Apr 4 18:59:06 2006 @@ -8,6 +8,7 @@ from pypy.objspace.std.dictobject import W_DictObject from pypy.objspace.std.objectobject import W_ObjectObject from pypy.objspace.std.intobject import W_IntObject +from pypy.objspace.std.stringobject import W_StringObject from pypy.objspace.std.model import StdObjSpaceMultiMethod #-- THE BUILTINS ---------------------------------------------------------------------- @@ -50,9 +51,11 @@ return key def add_to_runnable(self, uthread): + assert isinstance(uthread, GreenletCoroutine) self.runnable_uthreads[uthread] = True def remove_from_runnable(self, uthread): + assert isinstance(uthread, GreenletCoroutine) del self.runnable_uthreads[uthread] def have_runnable_threads(self): @@ -62,6 +65,8 @@ return bool(self.uthreads_blocked_on) def add_to_blocked(self, w_var, uthread): + assert isinstance(w_var, W_Var) + assert isinstance(uthread, GreenletCoroutine) if w_var in self.uthreads_blocked_on: blocked = self.uthreads_blocked_on[w_var] else: @@ -70,6 +75,7 @@ blocked.append(uthread) def pop_blocked_on(self, w_var): + assert isinstance(w_var, W_Var) if w_var not in self.uthreads_blocked_on: return [] blocked = self.uthreads_blocked_on[w_var] @@ -77,6 +83,8 @@ return blocked def add_to_blocked_byneed(self, w_var, uthread): + assert isinstance(w_var, W_Var) + assert isinstance(uthread, GreenletCoroutine) #print " adding", uthread, "to byneed on", w_var if w_var in self.uthreads_blocked_byneed: blocked = self.uthreads_blocked_byneed[w_var] @@ -86,6 +94,7 @@ blocked.append(uthread) def pop_blocked_byneed_on(self, w_var): + assert isinstance(w_var, W_Var) if w_var not in self.uthreads_blocked_byneed: #print " there was nobody to remove for", w_var return [] @@ -128,9 +137,11 @@ return hash(self.greenlet) def __eq__(self, other): + assert isinstance(other, GreenletCoroutine) return self.greenlet == other.greenlet def __ne__(self, other): + assert isinstance(other, GreenletCoroutine) return not (self == other) def __repr__(self): @@ -170,9 +181,8 @@ #-- VARIABLE --------------------- -class W_Var(baseobjspace.W_Root, object): +class W_Var(W_Root, object): def __init__(w_self): - w_self.typedef = 'Variable' w_self.w_bound_to = w_self w_self.w_needed = False @@ -224,6 +234,7 @@ return w_self.w_bound_to def wait(space, w_obj): + assert isinstance(w_obj, W_Root) return space.wait(w_obj) app_wait = gateway.interp2app(wait) @@ -264,6 +275,7 @@ space.wrap("wait_needed only supported on unbound variables")) def wait_needed(space, w_var): + assert isinstance(w_var, W_Var) return space.wait_needed(w_var) app_wait_needed = gateway.interp2app(wait_needed) @@ -282,6 +294,7 @@ app_is_aliased = gateway.interp2app(is_aliased) def is_free(space, w_obj): + assert isinstance(w_obj, W_Root) return space.is_free(w_obj) app_is_free = gateway.interp2app(is_free) @@ -297,6 +310,7 @@ all_mms['is_free'] = is_free_mm def is_bound(space, w_obj): + assert isinstance(w_obj, W_Root) return space.is_bound(w_obj) app_is_bound = gateway.interp2app(is_bound) @@ -313,6 +327,8 @@ def alias_of(space, w_var1, w_var2): # FIXME: appears to block + assert isinstance(w_var1, W_Var) + assert isinstance(w_var2, W_Var) assert space.is_true(space.is_free(w_var1)) assert space.is_true(space.is_free(w_var2)) w_curr = w_var1 @@ -347,6 +363,7 @@ def aliases(space, w_var): """return the aliases of a var, including itself""" + assert isinstance(w_var, W_Var) al = [] w_curr = w_var while 1: @@ -359,6 +376,7 @@ def get_ring_tail(space, w_start): """returns the last var of a ring of aliases""" + assert isinstance(w_start, W_Var) w_curr = w_start while 1: w_next = w_curr.w_bound_to @@ -371,6 +389,8 @@ """raises a specific exception for bind/unify""" #FIXME : really raise some specific exception #print "failed to bind/unify" + assert isinstance(w_obj1, W_Root) + assert isinstance(w_obj2, W_Root) raise OperationError(space.w_RuntimeError, space.wrap("Unification failure")) @@ -381,31 +401,36 @@ pass def prettyfy_id(a_str): + assert isinstance(a_str, W_StringObject) l = len(a_str) - 1 return a_str[l-3:l] #FIXME : does not work at all, # even a pure applevel version ... -def _sleep(space, w_var, w_barrier): - wait(space, w_var) - bind(space, w_barrier, space.newint(1)) - -def wait_two(space, w_1, w_2): - """waits until one out of two logic variables - becomes bound, then tells which one, - with a bias toward the first if both are - suddenly bound""" - w_barrier = newvar(space) - uthread(space, space.wrap(_sleep), - argument.Arguments(space, [w_1, w_barrier])) - uthread(space, space.wrap(_sleep), - argument.Arguments(space, [w_2, w_barrier])) - wait(space, w_barrier) - if space.is_true(space.is_free(w_2)): - return space.newint(1) - return space.newint(2) -app_wait_two = gateway.interp2app(wait_two) +## def _sleep(space, w_var, w_barrier): +## assert isinstance(w_var, W_Var) +## assert isinstance(w_barrier, W_Var) +## wait(space, w_var) +## bind(space, w_barrier, space.newint(1)) + +## def wait_two(space, w_v1, w_v2): +## """waits until one out of two logic variables +## becomes bound, then tells which one, +## with a bias toward the first if both are +## suddenly bound""" +## assert isinstance(w_v1, W_Var) +## assert isinstance(w_v2, W_Var) +## w_barrier = newvar(space) +## uthread(space, space.wrap(_sleep), +## argument.Arguments(space, [w_v1, w_barrier])) +## uthread(space, space.wrap(_sleep), +## argument.Arguments(space, [w_v2, w_barrier])) +## wait(space, w_barrier) +## if space.is_true(space.is_free(w_v2)): +## return space.newint(1) +## return space.newint(2) +## app_wait_two = gateway.interp2app(wait_two) #-- BIND ----------------------------- @@ -415,6 +440,8 @@ 3. assign value to unbound var """ #print " :bind", w_var, w_obj + assert isinstance(w_var, W_Var) + assert isinstance(w_obj, W_Root) space.bind(w_var, w_obj) app_bind = gateway.interp2app(bind) @@ -446,6 +473,8 @@ all_mms['bind'] = bind_mm def _assign(space, w_var, w_val): + assert isinstance(w_var, W_Var) + assert isinstance(w_val, W_Root) #print " :assign", w_var, w_val, '[', w_curr = w_var ass_count = 0 @@ -468,6 +497,8 @@ def _alias(space, w_v1, w_v2): """appends one var to the alias chain of another user must ensure freeness of both vars""" + assert isinstance(w_v1, W_Var) + assert isinstance(w_v2, W_Var) #print " :alias", w_v1, w_v2 if space.is_true(space.is_nb_(w_v1, w_v2)): return space.w_None @@ -500,24 +531,21 @@ #-- UNIFY ------------------------- def unify(space, w_x, w_y): + assert isinstance(w_x, W_Root) + assert isinstance(w_y, W_Root) #print ":unify ", w_x, w_y return space.unify(w_x, w_y) app_unify = gateway.interp2app(unify) -def unify__Int_Int(space, w_x, w_y): +def unify__Root_Root(space, w_x, w_y): if not space.eq_w(w_x, w_y): + w_d1 = w_x.getdict() + w_d2 = w_y.getdict() + if None not in (w_d1, w_d2): + return space.unify(w_d1, w_d2) + else: fail(space, w_x, w_y) return space.w_None - -## def unify__Root_Root(space, w_x, w_y): -## if not space.eq_w(w_x, w_y): -## w_d1 = w_x.getdict() -## w_d2 = w_y.getdict() -## if None not in (w_d1, w_d2): -## return space.unify(w_d1, w_d2) -## else: -## fail(space, w_x, w_y) -## return space.w_None def unify__Var_Var(space, w_x, w_y): #print " :unify of two vars" @@ -546,6 +574,8 @@ return _unify_iterables(space, w_x, w_y) def _unify_iterables(space, w_i1, w_i2): + assert isinstance(w_i1, W_TupleObject) or isinstance(w_i1, W_ListObject) + assert isinstance(w_i2, W_TupleObject) or isinstance(w_i2, W_ListObject) #print " :unify iterables", w_i1, w_i2 if len(w_i1.wrappeditems) != len(w_i2.wrappeditems): fail(space, w_i1, w_i2) @@ -559,6 +589,8 @@ unify(space, w_xi, w_yi) def unify__Dict_Dict(space, w_m1, w_m2): + assert isinstance(w_m1, W_DictObject) + assert isinstance(w_m2, W_DictObject) #print " :unify mappings", w_m1, w_m2 for w_xk in w_m1.content.keys(): w_xi = space.getitem(w_m1, w_xk) @@ -569,16 +601,13 @@ unify_mm = StdObjSpaceMultiMethod('unify', 2) -#unify_mm.register(unify__Root_Root, W_Root, W_Root) - -unify_mm.register(unify__Int_Int, W_IntObject, W_IntObject) - -## unify_mm.register(unify__Var_Var, W_Var, W_Var) -## unify_mm.register(unify__Var_Root, W_Var, W_Root) -## unify_mm.register(unify__Root_Var, W_Root, W_Var) -## unify_mm.register(unify__Tuple_Tuple, W_TupleObject, W_TupleObject) -## unify_mm.register(unify__List_List, W_ListObject, W_ListObject) -## unify_mm.register(unify__Dict_Dict, W_DictObject, W_DictObject) +unify_mm.register(unify__Root_Root, W_Root, W_Root) +unify_mm.register(unify__Var_Var, W_Var, W_Var) +unify_mm.register(unify__Var_Root, W_Var, W_Root) +unify_mm.register(unify__Root_Var, W_Root, W_Var) +unify_mm.register(unify__Tuple_Tuple, W_TupleObject, W_TupleObject) +unify_mm.register(unify__List_List, W_ListObject, W_ListObject) +unify_mm.register(unify__Dict_Dict, W_DictObject, W_DictObject) all_mms['unify'] = unify_mm @@ -621,6 +650,8 @@ def eqproxy(space, parentfn): def eq(w_obj1, w_obj2): + assert isinstance(w_obj1, W_Root) + assert isinstance(w_obj2, W_Root) if space.is_true(space.is_nb_(w_obj1, w_obj2)): return space.newbool(True) if space.is_true(space.is_free(w_obj1)): @@ -632,6 +663,8 @@ def isproxy(space, parentfn): def is_(w_obj1, w_obj2): + assert isinstance(w_obj1, W_Root) + assert isinstance(w_obj2, W_Root) if space.is_true(space.is_nb_(w_obj1, w_obj2)): return space.newbool(True) return parentfn(wait(space, w_obj1), wait(space, w_obj2)) @@ -639,6 +672,8 @@ def cmpproxy(space, parentfn): def cmp(w_obj1, w_obj2): + assert isinstance(w_obj1, W_Root) + assert isinstance(w_obj2, W_Root) if space.is_true(space.is_nb_(w_obj1, w_obj2)): return space.newbool(0) if space.is_true(space.is_free(w_obj1)): @@ -650,6 +685,8 @@ def neproxy(space, parentfn): def ne(w_obj1, w_obj2): + assert isinstance(w_obj1, W_Root) + assert isinstance(w_obj2, W_Root) if space.is_true(space.is_nb_(w_obj1, w_obj2)): return space.newbool(False) if space.is_true(space.is_free(w_obj1)): From nik at codespeak.net Tue Apr 4 19:00:21 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 19:00:21 +0200 (CEST) Subject: [pypy-svn] r25310 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060404170021.432E510157@code0.codespeak.net> Author: nik Date: Tue Apr 4 19:00:20 2006 New Revision: 25310 Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: code and test for lists with initial items in ootypesystem. Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Tue Apr 4 19:00:20 2006 @@ -54,13 +54,12 @@ return r_list.send_message(hop, "setitem", can_raise=True) def newlist(llops, r_list, items_v): - c1 = inputconst(ootype.Void, r_list.lowleveltype) - v_result = llops.genop("new", [c1], resulttype=r_list.lowleveltype) - #LIST = r_list.LIST - #cno = inputconst(Signed, len(items_v)) - #v_result = llops.gendirectcall(LIST.ll_newlist, cno) - #v_func = inputconst(Void, dum_nocheck) - #for i, v_item in enumerate(items_v): - # ci = inputconst(Signed, i) - # llops.gendirectcall(ll_setitem_nonneg, v_func, v_result, ci, v_item) + c_1ist = inputconst(ootype.Void, r_list.lowleveltype) + v_result = llops.genop("new", [c_1ist], resulttype=r_list.lowleveltype) + c_append = inputconst(ootype.Void, "append") + # This is very inefficient for a large amount of initial items ... + for v_item in items_v: + llops.genop("oosend", [c_append, v_result, v_item], + resulttype=ootype.Void) return v_result + Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 4 19:00:20 2006 @@ -67,3 +67,11 @@ res = interpret(f, [2], type_system="ootype") assert res == -1 + def test_initialize(self): + def f(x): + l = [1, 2] + l.append(x) + return l[2] + res = interpret(f, [3], type_system="ootype") + assert res == 3 + From mwh at codespeak.net Tue Apr 4 19:10:16 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 4 Apr 2006 19:10:16 +0200 (CEST) Subject: [pypy-svn] r25311 - pypy/branch/explicit-exceptions/translator Message-ID: <20060404171016.5D36510159@code0.codespeak.net> Author: mwh Date: Tue Apr 4 19:10:14 2006 New Revision: 25311 Modified: pypy/branch/explicit-exceptions/translator/unsimplify.py Log: add some shouting to the docstring of split_block Modified: pypy/branch/explicit-exceptions/translator/unsimplify.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/unsimplify.py (original) +++ pypy/branch/explicit-exceptions/translator/unsimplify.py Tue Apr 4 19:10:14 2006 @@ -36,7 +36,9 @@ return newblock def split_block(translator, graph, block, index): - """split a block in two, inserting a proper link between the new blocks""" + """split a block in two, inserting a proper link between the new + blocks. if you call this after rtyping, you WILL need to worry + about keepalives.""" assert 0 <= index <= len(block.operations) if block.exitswitch == c_last_exception: assert index < len(block.operations) From nik at codespeak.net Tue Apr 4 19:10:49 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 4 Apr 2006 19:10:49 +0200 (CEST) Subject: [pypy-svn] r25312 - pypy/dist/pypy/rpython/test Message-ID: <20060404171049.634C91015E@code0.codespeak.net> Author: nik Date: Tue Apr 4 19:10:48 2006 New Revision: 25312 Modified: pypy/dist/pypy/rpython/test/test_rlist.py Log: start migrating rlist tests to ootypesystem. Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Tue Apr 4 19:10:48 2006 @@ -160,40 +160,42 @@ # ____________________________________________________________ -def test_simple(): - def dummyfn(): - l = [10, 20, 30] - return l[2] - res = interpret(dummyfn, []) - assert res == 30 - -def test_append(): - def dummyfn(): - l = [] - l.append(50) - l.append(60) - l.append(70) - l.append(80) - l.append(90) - return len(l), l[0], l[-1] - res = interpret(dummyfn, []) - assert res.item0 == 5 - assert res.item1 == 50 - assert res.item2 == 90 +class BaseTestListRtyping: -def test_len(): - def dummyfn(): - l = [5, 10] - return len(l) - res = interpret(dummyfn, []) - assert res == 2 - - def dummyfn(): - l = [5] - l.append(6) - return len(l) - res = interpret(dummyfn, []) - assert res == 2 + def test_simple(self): + def dummyfn(): + l = [10, 20, 30] + return l[2] + res = interpret(dummyfn, [], type_system=self.ts) + assert res == 30 + + def test_append(self): + def dummyfn(): + l = [] + l.append(50) + l.append(60) + l.append(70) + l.append(80) + l.append(90) + return len(l), l[0], l[-1] + res = interpret(dummyfn, [], type_system=self.ts) + assert res.item0 == 5 + assert res.item1 == 50 + assert res.item2 == 90 + + def test_len(self): + def dummyfn(): + l = [5, 10] + return len(l) + res = interpret(dummyfn, [], type_system=self.ts) + assert res == 2 + + def dummyfn(): + l = [5] + l.append(6) + return len(l) + res = interpret(dummyfn, []) + assert res == 2 def test_iterate(): def dummyfn(): @@ -1045,3 +1047,13 @@ assert isinstance(r_B_list, ListRepr) assert r_A_list.lowleveltype == r_B_list.lowleveltype + + +class TestLltypeRtyping(BaseTestListRtyping): + + ts = "lltype" + +class TestOotypeRtyping(BaseTestListRtyping): + + ts = "ootype" + From mwh at codespeak.net Tue Apr 4 19:11:57 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 4 Apr 2006 19:11:57 +0200 (CEST) Subject: [pypy-svn] r25313 - pypy/branch/explicit-exceptions/rpython/memory Message-ID: <20060404171157.93B331015B@code0.codespeak.net> Author: mwh Date: Tue Apr 4 19:11:56 2006 New Revision: 25313 Modified: pypy/branch/explicit-exceptions/rpython/memory/gctransform.py Log: make the FrameworkGCTransformer work on the branch. much easier than i expected! Modified: pypy/branch/explicit-exceptions/rpython/memory/gctransform.py ============================================================================== --- pypy/branch/explicit-exceptions/rpython/memory/gctransform.py (original) +++ pypy/branch/explicit-exceptions/rpython/memory/gctransform.py Tue Apr 4 19:11:56 2006 @@ -103,12 +103,12 @@ # had to solve there. for i, op in enumerate(block.operations): num_ops_after_exc_raising = 0 - ops = self.replacement_operations(op, livevars, block) + ops, index = self.replacement_operations(op, livevars, block) if not ops: continue # may happen when we eat gc_protect/gc_unprotect. newops.extend(ops) origname = op.opname - op = ops[-1] + op = ops[index] if var_needsgc(op.result): if op.opname not in ('direct_call', 'indirect_call') and not var_ispyobj(op.result): lst = list(self.push_alive(op.result)) @@ -139,10 +139,13 @@ def replacement_operations(self, op, livevars, block): m = getattr(self, 'replace_' + op.opname, None) if m: - return m(op, livevars, block) + r = m(op, livevars, block) + if not isinstance(r, tuple): + return r, -1 + else: + return r else: - return [op] - + return [op], 0 def push_alive(self, var): if var_ispyobj(var): @@ -935,8 +938,10 @@ else: print "block which needs conservative livevar calculation found" newops = list(self.push_roots(livevars)) + index = len(newops) newops.append(op) - return newops, tuple(self.pop_roots(livevars)) + newops.extend(self.pop_roots(livevars)) + return newops, index replace_direct_call = protect_roots replace_indirect_call = protect_roots @@ -972,11 +977,11 @@ args = [self.malloc_varsize_ptr, newop0.result, c_type_id, v_length, c_size, c_varitemsize, c_ofstolength] newop = SpaceOperation("direct_call", args, v) - ops, finally_ops = self.protect_roots(newop, livevars, block, - block.operations.index(op)) + ops, index = self.protect_roots(newop, livevars, block, + block.operations.index(op)) ops.insert(0, newop0) ops.append(SpaceOperation("cast_adr_to_ptr", [v], op.result)) - return ops, finally_ops, 1 + return ops replace_malloc_varsize = replace_malloc From arigo at codespeak.net Tue Apr 4 19:15:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 19:15:48 +0200 (CEST) Subject: [pypy-svn] r25314 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060404171548.9793D1015F@code0.codespeak.net> Author: arigo Date: Tue Apr 4 19:15:47 2006 New Revision: 25314 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: (arre, arigo) Some refactoring. Added a (skipped) test and started to work on it. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Tue Apr 4 19:15:47 2006 @@ -774,7 +774,7 @@ else: return SomeCTypesObject( s_cto.knowntype._type_, - memorystate=s_cto.memorystate) + memorystate=SomeCTypesObject.MEMORYALIAS) class __extend__(pairtype(SomeCTypesObject, SomeSlice)): def setitem((s_cto, s_slice), s_iterable): Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Tue Apr 4 19:15:47 2006 @@ -16,10 +16,9 @@ item_ctype = array_ctype._type_ self.length = array_ctype._length_ - # Find the low-level repr of items using the extregistry - item_entry = extregistry.lookup_type(item_ctype) - self.r_item = item_entry.get_repr(rtyper, SomeCTypesObject(item_ctype, - SomeCTypesObject.OWNSMEMORY)) + # Find the repr and low-level type of items from their ctype + self.r_item = rtyper.getrepr(SomeCTypesObject(item_ctype, + SomeCTypesObject.MEMORYALIAS)) # Here, self.c_data_type == self.ll_type c_data_type = lltype.Array(self.r_item.ll_type, Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 4 19:15:47 2006 @@ -111,6 +111,14 @@ llops.genop('setfield', inputargs) return v_box + def return_c_data(self, llops, v_c_data): + """Turn a raw C pointer to the data into a memory-alias box. + Used when the data is returned from an operation or C function call. + Special-cased in PrimitiveRepr. + """ + # XXX add v_c_data_owner + return self.allocate_instance_ref(llops, v_c_data) + class __extend__(pairtype(CTypesRepr, CTypesRepr)): @@ -142,15 +150,21 @@ return lltype.Struct('C_Data_%s' % (self.ctype.__name__,), ('value', ll_type) ) - def setvalue(self, llops, v_box, v_value): - """Writes to the 'value' field of the raw data.""" - v_c_data = self.get_c_data(llops, v_box) + def getvalue_from_c_data(self, llops, v_c_data): + cname = inputconst(lltype.Void, 'value') + return llops.genop('getfield', [v_c_data, cname], + resulttype=self.ll_type) + + def setvalue_inside_c_data(self, llops, v_c_data, v_value): cname = inputconst(lltype.Void, 'value') llops.genop('setfield', [v_c_data, cname, v_value]) def getvalue(self, llops, v_box): """Reads from the 'value' field of the raw data.""" v_c_data = self.get_c_data(llops, v_box) - cname = inputconst(lltype.Void, 'value') - return llops.genop('getfield', [v_c_data, cname], - resulttype=self.ll_type) + return self.getvalue_from_c_data(llops, v_c_data) + + def setvalue(self, llops, v_box, v_value): + """Writes to the 'value' field of the raw data.""" + v_c_data = self.get_c_data(llops, v_box) + self.setvalue_inside_c_data(llops, v_c_data, v_value) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 4 19:15:47 2006 @@ -1,17 +1,21 @@ -from pypy.rpython.rmodel import Repr, inputconst +from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel +from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesValueRepr from ctypes import POINTER, pointer, c_int class PointerRepr(CTypesValueRepr): - def __init__(self, rtyper, s_pointer, s_contents): - self.s_pointer = s_pointer - self.s_contents = s_contents - self.ref_ctype = s_contents.knowntype - self.r_contents = rtyper.getrepr(s_contents) + def __init__(self, rtyper, s_pointer): + ptr_ctype = s_pointer.knowntype + ref_ctype = ptr_ctype._type_ + + # Find the repr and low-level type of the contents from its ctype + self.r_contents = rtyper.getrepr(annmodel.SomeCTypesObject(ref_ctype, + annmodel.SomeCTypesObject.MEMORYALIAS)) + ll_contents = lltype.Ptr(self.r_contents.c_data_type) super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) @@ -30,8 +34,30 @@ assert s_attr.is_constant() assert s_attr.const == 'contents' v_ptr = hop.inputarg(self, 0) - v_value = self.getvalue(hop.llops, v_ptr) - return self.r_contents.allocate_instance_ref(hop.llops, v_value) + v_c_ptr = self.getvalue(hop.llops, v_ptr) + return self.r_contents.allocate_instance_ref(hop.llops, v_c_ptr) + + +class __extend__(pairtype(PointerRepr, IntegerRepr)): + + def rtype_getitem((r_ptr, _), hop): + self = r_ptr + v_ptr, v_index = hop.inputargs(self, lltype.Signed) + if hop.args_s[1].is_constant() and hop.args_s[1].const == 0: + v_c_ptr = self.getvalue(hop. llops, v_ptr) + return self.r_contents.return_c_data(hop.llops, v_c_ptr) + else: + raise NotImplementedError("XXX: pointer[non-zero-index]") + + def rtype_setitem((r_ptr, _), hop): + self = r_ptr + v_ptr, v_index, v_newvalue = hop.inputargs(self, lltype.Signed, XXX) + + if hop.args_s[1].is_constant() and hop.args_s[1].const == 0: + v_c_ptr = self.getvalue(hop. llops, v_ptr) + XXX + else: + raise NotImplementedError("XXX: pointer[non-zero-index] = value") #def registerPointerType(ptrtype): # """Adds a new pointer type to the extregistry. @@ -138,8 +164,7 @@ annmodel.SomeCTypesObject.MEMORYALIAS) def pointerinstance_get_repr(rtyper, s_pointer): - s_contents = pointerinstance_field_annotation(s_pointer, "contents") - return PointerRepr(rtyper, s_pointer, s_contents) + return PointerRepr(rtyper, s_pointer) PointerType = type(POINTER(c_int)) extregistry.register_type(PointerType, Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Tue Apr 4 19:15:47 2006 @@ -27,6 +27,12 @@ class PrimitiveRepr(CTypesValueRepr): + def return_c_data(self, llops, v_c_data): + """Read out the atomic data from a raw C pointer. + Used when the data is returned from an operation or C function call. + """ + return self.getvalue_from_c_data(llops, v_c_data) + def convert_const(self, ctype_value): assert isinstance(ctype_value, self.ctype) key = id(ctype_value) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Tue Apr 4 19:15:47 2006 @@ -49,26 +49,6 @@ if conftest.option.view: t.view() - def test_annotate_pointer_access_as_array(self): - """ - Make sure that pointers work the same way as arrays. - """ - def access_array(): - # Never run this function! - my_pointer = pointer(c_int(10)) - my_pointer[0] = c_int(1) - my_pointer[1] = 2 # <== because of this - - return my_pointer[0] - - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(access_array, []) - assert s.knowntype == int - - if conftest.option.view: - t.view() - def x_test_annotate_array_slice_access(self): def slice_access(): my_array = c_int_10() Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Tue Apr 4 19:15:47 2006 @@ -63,6 +63,26 @@ assert s.knowntype == int + def test_annotate_pointer_access_as_array(self): + """ + Make sure that pointers work the same way as arrays. + """ + def access_array(): + # Never run this function! + my_pointer = pointer(c_int(10)) + my_pointer[0] = c_int(1) + my_pointer[1] = 2 # <== because of this + + return my_pointer[0] + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(access_array, []) + assert s.knowntype == int + + if conftest.option.view: + t.view() + class Test_specialization: def test_specialize_c_int_ptr(self): @@ -134,3 +154,21 @@ assert func() == 123 res = interpret(func, []) assert res == 123 + + def test_specialize_pointer_access_as_array(self): + """ + Make sure that pointers work the same way as arrays. + """ + py.test.skip("in-progress") + def access_array(): + my_pointer = pointer(c_int(11)) + x = my_pointer[0] + my_pointer[0] = c_int(7) + y = my_pointer[0] + my_pointer[0] = 5 + z = my_pointer.contents.value + return x * y * z + + assert access_array() == 5 * 7 * 11 + res = interpret(access_array, []) + assert res == 5 * 7 * 11 From arigo at codespeak.net Tue Apr 4 19:44:41 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 4 Apr 2006 19:44:41 +0200 (CEST) Subject: [pypy-svn] r25318 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060404174441.D89DC10161@code0.codespeak.net> Author: arigo Date: Tue Apr 4 19:44:40 2006 New Revision: 25318 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: (arre, arigo) added FixedSizeArray, a low-level type similar to a Struct with n fields item0, item1, etc. but allowing __getitem__-based access as well. Usage: rctypes. This will crash various things along the line of translation, up to the back-ends, to be fixed as we discover the problems. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Tue Apr 4 19:44:40 2006 @@ -297,8 +297,8 @@ else: self.OF = Struct("", *fields) self._anonym_struct = True - if isinstance(self.OF, GcStruct): - raise TypeError("cannot have a GC structure as array item type") + if isinstance(self.OF, GC_CONTAINER): + raise TypeError("cannot have a GC container as array item type") self.OF._inline_is_varsize(False) self._install_extras(**kwds) @@ -342,6 +342,37 @@ def _inline_is_varsize(self, last): raise TypeError("cannot inline a GC array inside a structure") + +class FixedSizeArray(Struct): + # behaves more or less like a Struct with fields item0, item1, ... + # but also supports __getitem__(), __setitem__(), __len__(). + + def __init__(self, OF, length, **kwds): + fields = [('item%d' % i, OF) for i in range(length)] + super(FixedSizeArray, self).__init__('array%d' % length, *fields, + **kwds) + self.OF = OF + self.length = length + if isinstance(self.OF, GC_CONTAINER): + raise TypeError("cannot have a GC container as array item type") + self.OF._inline_is_varsize(False) + + def _str_fields(self): + return str(self.OF) + _str_fields = saferecursive(_str_fields, '...') + + def __str__(self): + return "%s of %d %s " % (self.__class__.__name__, + self.length, + self._str_fields(),) + + def _short_name(self): + return "%s %d %s" % (self.__class__.__name__, + self.length, + self.OF._short_name(),) + _short_name = saferecursive(_short_name, '...') + + class FuncType(ContainerType): __name__ = 'func' def __init__(self, args, result): @@ -735,6 +766,11 @@ raise IndexError("array index out of bounds") o = self._obj.items[i] return _expose(o, self._solid) + if isinstance(self._T, FixedSizeArray): + if not (0 <= i < self._T.length): + raise IndexError("fixed-size array index out of bounds") + o = getattr(self._obj, 'item%d' % i) + return _expose(o, self._solid) raise TypeError("%r instance is not an array" % (self._T,)) def __setitem__(self, i, val): @@ -751,6 +787,19 @@ raise IndexError("array index out of bounds") self._obj.items[i] = val return + if isinstance(self._T, FixedSizeArray): + T1 = self._T.OF + if isinstance(T1, ContainerType): + raise TypeError("cannot directly assign to container array items") + T2 = typeOf(val) + if T2 != T1: + raise TypeError("%r items:\n" + "expect %r\n" + " got %r" % (self._T, T1, T2)) + if not (0 <= i < self._T.length): + raise IndexError("fixed-size array index out of bounds") + setattr(self._obj, 'item%d' % i, val) + return raise TypeError("%r instance is not an array" % (self._T,)) def __len__(self): @@ -760,7 +809,8 @@ (self._T,)) return len(self._obj.items) - + if isinstance(self._T, FixedSizeArray): + return self._T.length raise TypeError("%r instance is not an array" % (self._T,)) def __repr__(self): Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Tue Apr 4 19:44:40 2006 @@ -552,3 +552,28 @@ memo = {} assert list(dissect_ll_instance(r, None, memo)) == r_expected assert list(dissect_ll_instance(b, None, memo)) == b_expected + +def test_fixedsizearray(): + A = FixedSizeArray(Signed, 5) + assert A.OF == Signed + assert A.length == 5 + assert A.item0 == A.item1 == A.item2 == A.item3 == A.item4 == Signed + assert A._names == ('item0', 'item1', 'item2', 'item3', 'item4') + a = malloc(A, immortal=True) + a[0] = 5 + a[4] = 83 + assert a[0] == 5 + assert a[4] == 83 + assert a.item4 == 83 + py.test.raises(IndexError, "a[5] = 183") + py.test.raises(IndexError, "a[-1]") + assert len(a) == 5 + + S = GcStruct('S', ('n1', Signed), + ('a', A), + ('n2', Signed)) + s = malloc(S) + s.a[3] = 17 + assert s.a[3] == 17 + assert len(s.a) == 5 + py.test.raises(TypeError, "s.a = a") From pedronis at codespeak.net Tue Apr 4 19:54:12 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 4 Apr 2006 19:54:12 +0200 (CEST) Subject: [pypy-svn] r25319 - in pypy/dist/pypy: annotation annotation/test translator/c/test Message-ID: <20060404175412.D9DE410163@code0.codespeak.net> Author: pedronis Date: Tue Apr 4 19:54:07 2006 New Revision: 25319 Modified: pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/test/test_annrpython.py pypy/dist/pypy/translator/c/test/test_genc.py Log: 25277 broke some annotation cases in a hard way (contains invariant broken)! for example: test_inline.py test_inline_var_exception Fix for that. The specific situation now is tested in test_annotate_type. Added a test in test_genc for what I suppose (no tests) was the behavior wanted by 25277. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Tue Apr 4 19:54:07 2006 @@ -390,13 +390,14 @@ if result is None: result = SomeObject() else: - # note that the print support functions are __builtin__ - if hasattr(x, '__module__') and x.__module__ != '__builtin__' \ - or tp in (types.FunctionType, types.MethodType): - result = SomePBC([self.getdesc(x)]) - else: - # a builtin that we don't handle specially + if (self.annotator.policy.allow_someobjects + and getattr(x, '__module__', None) == '__builtin__' + # XXX note that the print support functions are __builtin__ + and tp not in (types.FunctionType, types.MethodType)): result = SomeObject() + result.knowntype = tp # at least for types this needs to be correct + else: + result = SomePBC([self.getdesc(x)]) elif hasattr(x, '__class__') \ and x.__class__.__module__ != '__builtin__': # user-defined classes can define a method _freeze_(), which Modified: pypy/dist/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/test/test_annrpython.py (original) +++ pypy/dist/pypy/annotation/test/test_annrpython.py Tue Apr 4 19:54:07 2006 @@ -1945,6 +1945,22 @@ graph = tgraphof(t, A.__del__.im_func) assert graph.startblock in a.annotated + def test_annotate_type(self): + class A: + pass + x = [A(), A()] + def witness(t): + return type(t) + def get(i): + return x[i] + def f(i): + witness(None) + return witness(get(i)) + + a = self.RPythonAnnotator() + s = a.build_types(f, [int]) + assert s.__class__ == annmodel.SomeObject + assert s.knowntype == type def g(n): return [0,1,2,n] Modified: pypy/dist/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_genc.py (original) +++ pypy/dist/pypy/translator/c/test/test_genc.py Tue Apr 4 19:54:07 2006 @@ -398,4 +398,12 @@ before = g(obj) f(2, obj) after = g(obj) - assert before == after \ No newline at end of file + assert before == after + +def test_long_pyobj(): + def f(x): + return long(x) + f = compile(f, [int]) + res = f(2) + assert isinstance(res, long) + assert res == 2L From tismer at codespeak.net Tue Apr 4 21:36:34 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 21:36:34 +0200 (CEST) Subject: [pypy-svn] r25324 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060404193634.E55A51016A@code0.codespeak.net> Author: tismer Date: Tue Apr 4 21:36:33 2006 New Revision: 25324 Modified: pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py Log: added support for builtin_import Modified: pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py Tue Apr 4 21:36:33 2006 @@ -63,13 +63,19 @@ if hop.s_result.is_constant(): return hop.inputconst(lltype.Bool, hop.s_result.const) if hop.args_r[0] == pyobj_repr: - v_obj, v_typ = hop.inputargs(pyobj_repr, pyobj_repr) + v_obj, v_name = hop.inputargs(pyobj_repr, pyobj_repr) c = hop.inputconst(pyobj_repr, hasattr) - v = hop.genop('simple_call', [c, v_obj, v_typ], resulttype = pyobj_repr) + v = hop.genop('simple_call', [c, v_obj, v_name], resulttype = pyobj_repr) return hop.llops.convertvar(v, pyobj_repr, bool_repr) raise TyperError("hasattr is only suported on a constant or on PyObject") +def rtype_builtin___import__(hop): + args_v = hop.inputargs(*[pyobj_repr for ign in hop.args_r]) + c = hop.inputconst(pyobj_repr, __import__) + return hop.genop('simple_call', [c] + args_v, resulttype = pyobj_repr) + BUILTIN_TYPER = {} BUILTIN_TYPER[objectmodel.instantiate] = rtype_instantiate BUILTIN_TYPER[isinstance] = rtype_builtin_isinstance BUILTIN_TYPER[hasattr] = rtype_builtin_hasattr +BUILTIN_TYPER[__import__] = rtype_builtin___import__ From tismer at codespeak.net Tue Apr 4 21:38:43 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 21:38:43 +0200 (CEST) Subject: [pypy-svn] r25325 - in pypy/dist/pypy/translator/c: . src Message-ID: <20060404193843.EA8E41016C@code0.codespeak.net> Author: tismer Date: Tue Apr 4 21:38:40 2006 New Revision: 25325 Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/src/module.h Log: support for exporting simple objects, preparations for post-installation code Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Tue Apr 4 21:38:40 2006 @@ -56,10 +56,14 @@ pfname = db.get(pf) self.exports[self.entrypoint.func_name] = pf for obj in exports: + objname = None + if type(obj) is tuple: + objname, obj = obj po = self.getentrypointptr(obj) - poname = objname = db.get(po) + poname = db.get(po) if hasattr(obj, '__name__'): objname = obj.__name__ + objname = objname or poname if objname in self.exports: raise NameError, 'duplicate name in export: %s is %s and %s' % ( objname, db.get(self.exports[objname]), poname) @@ -667,6 +671,7 @@ wrapper_name)) print >> f, '\t{ NULL }\t/* Sentinel */' print >> f, '};' + print >> f, 'static globalfunctiondef_t *globalfunctiondefsptr = &globalfunctiondefs[0];' print >> f print >> f, '/***********************************************************/' print >> f, '/*** Frozen Python bytecode: the initialization code ***/' Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Tue Apr 4 21:38:40 2006 @@ -16,8 +16,6 @@ # Should this be registered with the annotator? from pypy.interpreter.baseobjspace import ObjSpace -from pypy.annotation.registry import IMPORT_HINTS - class PyObjMaker: """Handles 'PyObject*'; factored out from LowLevelDatabase. This class contains all the nameof_xxx() methods that allow a wild variety @@ -38,7 +36,8 @@ # objects self.debugstack = () # linked list of nested nameof() self.wrappers = {} # {'pycfunctionvariable': ('name', 'wrapperfn')} - self.import_hints = IMPORT_HINTS + self.import_hints = {} # I don't seem to need it any longer. + # leaving the import support intact, doesn't hurt. self.instantiators = instantiators def nameof(self, obj, debug=None): @@ -169,10 +168,7 @@ def skipped_function(self, func): # debugging only! Generates a placeholder for missing functions # that raises an exception when called. - - # XXX this is broken after the translationcontext change! - # the frozen attribute is gone. What to do? - if self.translator.frozen: + if self.translator.annotator.frozen: warning = 'NOT GENERATING' else: warning = 'skipped' Modified: pypy/dist/pypy/translator/c/src/module.h ============================================================================== --- pypy/dist/pypy/translator/c/src/module.h (original) +++ pypy/dist/pypy/translator/c/src/module.h Tue Apr 4 21:38:40 2006 @@ -57,6 +57,10 @@ PyMethodDef ml; } globalfunctiondef_t; +/* helper-hook for post-setup */ +static globalfunctiondef_t *globalfunctiondefsptr; +static PyObject *postsetup_get_type_dict(PyObject *tp); +static PyObject *postsetup_build_method(int funcidx); /* implementations */ @@ -140,4 +144,21 @@ return 0; } +static PyObject *postsetup_get_type_dict(PyObject *tp) +{ + PyTypeObject *type = (PyTypeObject *)tp; + PyObject *ret; + + ret = type->tp_dict; + Py_INCREF(ret); + return ret; +} + +static PyObject *postsetup_build_method(int funcidx, PyObject *type) +{ + globalfunctiondef_t *gfuncdef = &globalfunctiondefsptr[funcidx]; + + return PyDescr_NewMethod((PyTypeObject *)type, &gfuncdef->ml); +} + #endif /* PYPY_NOT_MAIN_FILE */ From tismer at codespeak.net Tue Apr 4 21:40:28 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 21:40:28 +0200 (CEST) Subject: [pypy-svn] r25326 - pypy/dist/pypy/annotation Message-ID: <20060404194028.974021016D@code0.codespeak.net> Author: tismer Date: Tue Apr 4 21:40:26 2006 New Revision: 25326 Removed: pypy/dist/pypy/annotation/registry.py Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: got rid of registry.py, after all. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Tue Apr 4 21:40:26 2006 @@ -354,8 +354,6 @@ for ek, ev in x.iteritems(): result.dictdef.generalize_key(self.immutablevalue(ek)) result.dictdef.generalize_value(self.immutablevalue(ev)) - elif ishashable(x) and x in DEFINED_SOMEOBJECTS: # sys by default - return SomeObject() elif ishashable(x) and x in BUILTIN_ANALYZERS: _module = getattr(x,"__module__","unknown") result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) @@ -365,8 +363,6 @@ result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) elif hasattr(tp, "compute_annotation"): result = tp.compute_annotation() - elif tp in DEFINED_SOMEOBJECTS: - return SomeObject() elif tp in EXTERNAL_TYPE_ANALYZERS: result = SomeExternalObject(tp) elif isinstance(x, lltype._ptr): @@ -701,8 +697,7 @@ def delayed_imports(): # import ordering hack - global BUILTIN_ANALYZERS, EXTERNAL_TYPE_ANALYZERS, DEFINED_SOMEOBJECTS + global BUILTIN_ANALYZERS, EXTERNAL_TYPE_ANALYZERS from pypy.annotation.builtin import BUILTIN_ANALYZERS from pypy.annotation.builtin import EXTERNAL_TYPE_ANALYZERS - from pypy.annotation.registry import DEFINED_SOMEOBJECTS From bea at codespeak.net Tue Apr 4 21:49:39 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Tue, 4 Apr 2006 21:49:39 +0200 (CEST) Subject: [pypy-svn] r25327 - pypy/extradoc/talk Message-ID: <20060404194939.F291110162@code0.codespeak.net> Author: bea Date: Tue Apr 4 21:49:34 2006 New Revision: 25327 Added: pypy/extradoc/talk/pypypaper_xp2006.sxw (contents, props changed) Log: the paper behind the pdf for the xp2006 paper, this needs to adhere to the springer requirements for camera ready submissions....there are req for txt files so it does not have to be a open office doc...deadline for resubmission 6th april - I will make final changes tomorrow evening... Added: pypy/extradoc/talk/pypypaper_xp2006.sxw ============================================================================== Binary file. No diff available. From tismer at codespeak.net Tue Apr 4 23:23:24 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 23:23:24 +0200 (CEST) Subject: [pypy-svn] r25329 - pypy/dist/pypy/translator/c Message-ID: <20060404212324.9497710170@code0.codespeak.net> Author: tismer Date: Tue Apr 4 23:23:22 2006 New Revision: 25329 Modified: pypy/dist/pypy/translator/c/stackless.py Log: do not rely on graphs always having a 'func' attribute Modified: pypy/dist/pypy/translator/c/stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/stackless.py (original) +++ pypy/dist/pypy/translator/c/stackless.py Tue Apr 4 23:23:22 2006 @@ -69,7 +69,9 @@ for caller, callee in translator.callgraph.values(): # ignore calls issued suggested_primitives -- they are not # compiled in, and they typically contain pseudo-recursions - if not getattr(caller.func, 'suggested_primitive', False): + func = getattr(caller, 'func') + # there are graphs without funcs + if not getattr(func, 'suggested_primitive', False): callers[caller].append(callee) # check all callees if they can reach unwind can_reach_unwind = self.can_reach_unwind @@ -88,7 +90,8 @@ if ret: break else: - ret = graph.func in self.stackless_roots + func = getattr(graph, 'func') + ret = func in self.stackless_roots del pending[graph] can_reach_unwind[graph] = ret return ret From tismer at codespeak.net Tue Apr 4 23:33:10 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 4 Apr 2006 23:33:10 +0200 (CEST) Subject: [pypy-svn] r25330 - pypy/dist/pypy/translator/c Message-ID: <20060404213310.C266E10171@code0.codespeak.net> Author: tismer Date: Tue Apr 4 23:33:08 2006 New Revision: 25330 Modified: pypy/dist/pypy/translator/c/stackless.py Log: argh. When do I learn that getattr without default is not helpful! Modified: pypy/dist/pypy/translator/c/stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/stackless.py (original) +++ pypy/dist/pypy/translator/c/stackless.py Tue Apr 4 23:33:08 2006 @@ -69,7 +69,7 @@ for caller, callee in translator.callgraph.values(): # ignore calls issued suggested_primitives -- they are not # compiled in, and they typically contain pseudo-recursions - func = getattr(caller, 'func') + func = getattr(caller, 'func', None) # there are graphs without funcs if not getattr(func, 'suggested_primitive', False): callers[caller].append(callee) @@ -90,7 +90,7 @@ if ret: break else: - func = getattr(graph, 'func') + func = getattr(graph, 'func', None) ret = func in self.stackless_roots del pending[graph] can_reach_unwind[graph] = ret From mwh at codespeak.net Wed Apr 5 00:11:49 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 00:11:49 +0200 (CEST) Subject: [pypy-svn] r25331 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060404221149.A7AC61016F@code0.codespeak.net> Author: mwh Date: Wed Apr 5 00:11:41 2006 New Revision: 25331 Removed: pypy/branch/explicit-exceptions/translator/c/test/test_notype.py Log: remove test_notype. if you want to translate non-rtyped graphs, writing another genc is probably easier than continuing to try to support both rtyped and non-rtyped graphs in the same translator. From mwh at codespeak.net Wed Apr 5 00:12:37 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 00:12:37 +0200 (CEST) Subject: [pypy-svn] r25332 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060404221237.005301018C@code0.codespeak.net> Author: mwh Date: Wed Apr 5 00:12:31 2006 New Revision: 25332 Removed: pypy/branch/explicit-exceptions/translator/c/test/test_operation.py Log: this is also hopeless in the non-rtyped world From mwh at codespeak.net Wed Apr 5 00:27:00 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 00:27:00 +0200 (CEST) Subject: [pypy-svn] r25333 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060404222700.AF33D1017A@code0.codespeak.net> Author: mwh Date: Wed Apr 5 00:26:56 2006 New Revision: 25333 Removed: pypy/branch/explicit-exceptions/translator/c/test/test_annotated.py Modified: pypy/branch/explicit-exceptions/translator/c/test/test_typed.py Log: delete test_annotated, move all of the meaningful contents to test_typed. i think all genc tests pass now, not sure yet though Modified: pypy/branch/explicit-exceptions/translator/c/test/test_typed.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_typed.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_typed.py Wed Apr 5 00:26:56 2006 @@ -1,16 +1,288 @@ import autopath import sys import py + from py.test import raises + +from pypy import conftest from pypy.translator.test import snippet -from pypy.rpython.rarithmetic import r_uint, r_longlong, intmask -from pypy.translator.c.test.test_annotated import TestAnnotatedTestCase as _TestAnnotatedTestCase +from pypy.translator.tool.cbuild import skip_missing_compiler +from pypy.translator.translator import TranslationContext +from pypy.rpython.rarithmetic import r_uint, r_ulonglong, r_longlong, intmask + +# XXX this tries to make compiling faster for full-scale testing +from pypy.translator.tool import cbuild +cbuild.enable_fast_compilation() + +class TestTypedTestCase: + + def annotatefunc(self, func): + t = TranslationContext(simplifying=True) + # builds starting-types from func_defs + argstypelist = [] + if func.func_defaults: + for spec in func.func_defaults: + if isinstance(spec, tuple): + spec = spec[0] # use the first type only for the tests + argstypelist.append(spec) + a = t.buildannotator() + a.build_types(func, argstypelist) + a.simplify() + return t + + def compilefunc(self, t, func): + from pypy.translator.c import genc + builder = genc.CExtModuleBuilder(t, func) + builder.generate_source() + skip_missing_compiler(builder.compile) + builder.import_module() + return builder.get_entry_point() + + def getcompiled(self, func, view=False): + t = self.annotatefunc(func) + self.process(t) + if view or conftest.option.view: + t.view() + t.checkgraphs() + return self.compilefunc(t, func) + + def test_set_attr(self): + set_attr = self.getcompiled(snippet.set_attr) + assert set_attr() == 2 + + def test_inheritance2(self): + inheritance2 = self.getcompiled(snippet.inheritance2) + assert inheritance2() == ((-12, -12), (3, "world")) + + def test_factorial2(self): + factorial2 = self.getcompiled(snippet.factorial2) + assert factorial2(5) == 120 + + def test_factorial(self): + factorial = self.getcompiled(snippet.factorial) + assert factorial(5) == 120 + + def test_simple_method(self): + simple_method = self.getcompiled(snippet.simple_method) + assert simple_method(55) == 55 + + def test_sieve_of_eratosthenes(self): + sieve_of_eratosthenes = self.getcompiled(snippet.sieve_of_eratosthenes) + assert sieve_of_eratosthenes() == 1028 + + def test_nested_whiles(self): + nested_whiles = self.getcompiled(snippet.nested_whiles) + assert nested_whiles(5,3) == '!!!!!' + + def test_call_five(self): + call_five = self.getcompiled(snippet.call_five) + result = call_five() + assert result == [5] + # -- currently result isn't a real list, but a pseudo-array + # that can't be inspected from Python. + #self.assertEquals(result.__class__.__name__[:8], "list of ") + + def test_call_unpack_56(self): + call_unpack_56 = self.getcompiled(snippet.call_unpack_56) + result = call_unpack_56() + assert result == (2, 5, 6) + + def test_class_defaultattr(self): + class K: + n = "hello" + def class_defaultattr(): + k = K() + k.n += " world" + return k.n + fn = self.getcompiled(class_defaultattr) + assert fn() == "hello world" + + def test_tuple_repr(self): + def tuple_repr(x=int, y=object): + z = x, y + while x: + x = x-1 + return z + fn = self.getcompiled(tuple_repr) + assert fn(6,'a') == (6,'a') + + def test_classattribute(self): + fn = self.getcompiled(snippet.classattribute) + assert fn(1) == 123 + assert fn(2) == 456 + assert fn(3) == 789 + assert fn(4) == 789 + assert fn(5) == 101112 + + def test_get_set_del_slice(self): + fn = self.getcompiled(snippet.get_set_del_slice) + l = list('abcdefghij') + result = fn(l) + assert l == [3, 'c', 8, 11, 'h', 9] + assert result == ([3, 'c'], [9], [11, 'h']) + + def test_slice_long(self): + def slice_long(l=list, n=long): + return l[:n] + fn = self.getcompiled(slice_long) + l = list('abc') + result = fn(l, 2**32) + assert result == list('abc') + result = fn(l, 2**64) + assert result == list('abc') + + def test_type_conversion(self): + # obfuscated test case specially for typer.insert_link_conversions() + def type_conversion(n=int): + if n > 3: + while n > 0: + n = n-1 + if n == 5: + n += 3.1416 + return n + fn = self.getcompiled(type_conversion) + assert fn(3) == 3 + assert fn(5) == 0 + assert abs(fn(7) + 0.8584) < 1E-5 + + def test_do_try_raise_choose(self): + fn = self.getcompiled(snippet.try_raise_choose) + result = [] + for n in [-1,0,1,2]: + result.append(fn(n)) + assert result == [-1,0,1,2] + + def test_is_perfect_number(self): + fn = self.getcompiled(snippet.is_perfect_number) + for i in range(1, 33): + perfect = fn(i) + assert perfect is (i in (6,28)) + + def test_prime(self): + fn = self.getcompiled(snippet.prime) + result = [fn(i) for i in range(1, 21)] + assert result == [False, True, True, False, True, False, True, False, + False, False, True, False, True, False, False, False, + True, False, True, False] + + def test_mutate_global(self): + class Stuff: + pass + g1 = Stuff(); g1.value = 1 + g2 = Stuff(); g2.value = 2 + g3 = Stuff(); g3.value = 3 + g1.next = g3 + g2.next = g3 + g3.next = g3 + def do_things(): + g1.next = g1 + g2.next = g1 + g3.next = g2 + return g3.next.next.value + fn = self.getcompiled(do_things) + assert fn() == 1 + + def test_float_ops(self): + def f(x=float): + return abs((-x) ** 3 + 1) + fn = self.getcompiled(f) + assert fn(-4.5) == 92.125 + assert fn(4.5) == 90.125 + + def test_memoryerror(self): + def f(i=int): + lst = [0]*i + lst[-1] = 5 + return lst[0] + fn = self.getcompiled(f) + assert fn(1) == 5 + assert fn(2) == 0 + py.test.raises(MemoryError, fn, sys.maxint//2+1) + py.test.raises(MemoryError, fn, sys.maxint) + + def test_chr(self): + def f(x=int): + try: + return 'Yes ' + chr(x) + except ValueError: + return 'No' + fn = self.getcompiled(f) + assert fn(65) == 'Yes A' + assert fn(256) == 'No' + assert fn(-1) == 'No' + def test_unichr(self): + def f(x=int): + try: + return ord(unichr(x)) + except ValueError: + return -42 + fn = self.getcompiled(f) + assert fn(65) == 65 + assert fn(-12) == -42 + assert fn(sys.maxint) == -42 -class TestTypedTestCase(_TestAnnotatedTestCase): + def test_list_indexerror(self): + def f(i=int): + lst = [123, 456] + try: + lst[i] = 789 + except IndexError: + return 42 + return lst[0] + fn = self.getcompiled(f) + assert fn(1) == 123 + assert fn(2) == 42 + assert fn(-2) == 789 + assert fn(-3) == 42 + + def test_long_long(self): + def f(i=r_ulonglong): + return 4*i + fn = self.getcompiled(f, view=False) + assert fn(sys.maxint) == 4*sys.maxint + + def g(i=r_longlong): + return 4*i + gn = self.getcompiled(g, view=False) + assert gn(sys.maxint) == 4*sys.maxint + + def test_specializing_int_functions(self): + def f(i): + return i + 1 + f._annspecialcase_ = "specialize:argtype(0)" + def g(n=int): + if n > 0: + return f(r_longlong(0)) + else: + return f(0) + + fn = self.getcompiled(g) + assert g(0) == 1 + assert g(1) == 1 + + def test_downcast_int(self): + def f(i=r_longlong): + return int(i) + fn = self.getcompiled(f) + assert fn(0) == 0 + + def test_function_ptr(self): + def f1(): + return 1 + def f2(): + return 2 + def g(i=int): + if i: + f = f1 + else: + f = f2 + return f() + fn = self.getcompiled(g) + assert fn(0) == 2 + assert fn(1) == 1 def process(self, t): - _TestAnnotatedTestCase.process(self, t) t.buildrtyper().specialize() #raisingop2direct_call(t) From mwh at codespeak.net Wed Apr 5 00:42:18 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 00:42:18 +0200 (CEST) Subject: [pypy-svn] r25334 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060404224218.E893C10189@code0.codespeak.net> Author: mwh Date: Wed Apr 5 00:42:17 2006 New Revision: 25334 Modified: pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py Log: this test turned out not to be valid Modified: pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py Wed Apr 5 00:42:17 2006 @@ -83,30 +83,6 @@ fn = compile_func(g, []) assert fn() == 42 -def test_missing_keepalive_exception(): - py.test.skip("failing due to missing keep_alives") - A = lltype.Array(lltype.Signed) - S = lltype.GcStruct("S", ('a', A)) - def f(x): - if not x: - raise ValueError - return x + 1 - def g(x): - s = lltype.malloc(S, x) - a = s.a - a[x - 1] = 42 - i = x - 2 - try: - i = f(i) - except ValueError: - i = x - 1 - # need to allocate another instance, which will probably be in - # the same place as the previous one - s = lltype.malloc(S, x) - return a[i] - fn = compile_func(g, [int]) - assert fn(6) == 42 - def test_multiply_passed_var(): S = lltype.GcStruct("S", ('x', lltype.Signed)) def f(x): From tismer at codespeak.net Wed Apr 5 06:54:56 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 5 Apr 2006 06:54:56 +0200 (CEST) Subject: [pypy-svn] r25335 - in pypy/dist/pypy: annotation rpython translator translator/c/test translator/c/winproj/extension Message-ID: <20060405045456.6AE941018B@code0.codespeak.net> Author: tismer Date: Wed Apr 5 06:54:47 2006 New Revision: 25335 Modified: pypy/dist/pypy/annotation/policy.py pypy/dist/pypy/rpython/annlowlevel.py pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj pypy/dist/pypy/translator/translator.py Log: a few changes were necessary to enable non-immediate imports. This makes it very convenient to use a mix of PyPy objects and external or builtin things. No more magic needed. Modified: pypy/dist/pypy/annotation/policy.py ============================================================================== --- pypy/dist/pypy/annotation/policy.py (original) +++ pypy/dist/pypy/annotation/policy.py Wed Apr 5 06:54:47 2006 @@ -26,6 +26,10 @@ callback() del annotator.bookkeeper.pending_specializations[:] + def _adjust_space_config(self, space): + # allow to override space options. + if getattr(self, 'override_do_imports_immediately', None) is not None: + space.do_imports_immediately = self.override_do_imports_immediately class AnnotatorPolicy(BasicAnnotatorPolicy): """ Modified: pypy/dist/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/dist/pypy/rpython/annlowlevel.py (original) +++ pypy/dist/pypy/rpython/annlowlevel.py Wed Apr 5 06:54:47 2006 @@ -43,6 +43,8 @@ class LowLevelAnnotatorPolicy(AnnotatorPolicy): allow_someobjects = False + # if this exists and is boolean, then it always wins: + override_do_imports_immediately = True def default_specialize(pol, funcdesc, args_s): if hasattr(funcdesc, 'pyobj') and hasattr(funcdesc.pyobj, 'llresult'): Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Wed Apr 5 06:54:47 2006 @@ -7,6 +7,8 @@ from pypy.rpython.objectmodel import instantiate from pypy.rpython import robject, rclass +import sys + P = False # debug printing def get_annotation(func): @@ -21,7 +23,7 @@ return missing + argstypelist def get_compiled_module(func, view=conftest.option.view, inline_threshold=0*1, - use_boehm=False, exports=[]): + use_boehm=False, exports=None): from pypy.translator.translator import TranslationContext from pypy.translator.backendopt.all import backend_optimizations @@ -29,12 +31,17 @@ from pypy.translator.c.genc import CExtModuleBuilder global t # allow us to view later - t = TranslationContext() + t = TranslationContext(do_imports_immediately=False) do_register(t) t.buildannotator() rtyper = t.buildrtyper() bk = rtyper.annotator.bookkeeper instantiators = {} + t.annotator.build_types(func, get_annotation(func)) + if not exports: + exports = [] + all = [obj.__name__ for obj in exports] + exports = exports + [('__all__', all)] for obj in exports: if isinstance(obj, type): cls = obj @@ -42,11 +49,12 @@ obj = instantiate(cls) return obj make.__name__ = cls.__name__ + '__new__' - t.annotator.build_types(make, [], complete_now=False) + t.annotator.build_types(make, []) instantiators[cls] = make clsdef = bk.getuniqueclassdef(cls) rtyper.add_wrapper(clsdef) - t.annotator.build_types(func, get_annotation(func)) + elif callable(obj): + t.annotator.build_types(obj, get_annotation(obj)) if view: t.viewcg() rtyper.specialize() @@ -267,8 +275,9 @@ DemoClass.__init__(self, b, a) self.c = c - def demo(self): - return float(DemoClass.demo(self)) +# def demo(self, *other): + #if other: print other + # return float(DemoClass.demo(self)) def otherdemo(self): return 'this is the DemoSubclass', self.a, self.b @@ -319,6 +328,61 @@ self2 = DemoSubclass(a, b, 42) return self +# _______________________________________________ +# creating our own setup function for the module + +def do_the_import(): + from twisted.internet import reactor + return reactor + +def rtype_wraptest(hop): + #v_obj, = hop.inputargs((robject.pyobj_repr, )) + from pypy.objspace.flow.model import Constant + v_obj = Constant(hop.args_s[0].const) + v = hop.genop('simple_call', [v_obj], resulttype = robject.pyobj_repr) + return v + +def wraptest(obj): + return obj +wraptest.compute_result_annotation = lambda *args: annmodel.SomeObject() +wraptest.specialize = rtype_wraptest + +# not sure what to do with the above. +# use genpickle facility to produce a plain function? +# create a space and run geninterp on it? +# tweak flow space to delay the imports for this func? + +def setup_new_module(mod, modname): + # note the name clash with py.test + import sys + from __builtin__ import type + print type(modname) + m = type(sys)(modname) + print 'hallo', 42 + dummy = long(42) + allobjs = mod.__dict__.values() + funcs = eval('[]') + print 'alive' + from twisted.internet import reactor + print dir(reactor) + for obj in allobjs: + print obj, dir(obj) + if hasattr(42, 'func_name'): + funcs.append( (obj.func_name, obj) ) + print 'funcs=', funcs + funcs.sort() + for name, func in funcs: + print name, func + for name in mod.__all__: + obj = getattr(mod, name) + #if isinstance(obj, type): + # careful, this gives a class! + if hasattr(mod, '__bases__'): + print name + setattr(m, name, obj) + return m + + # creating an object, wrapping, unwrapping, call function, check whether __del__ is called def test_wrap_call_dtor(): f = getcompiled(democlass_helper, use_boehm=not True, exports=[DemoSubclass]) @@ -329,7 +393,11 @@ # exposing and using classes from a generasted extension module def test_expose_classes(): m = get_compiled_module(democlass_helper2, use_boehm=not True, exports=[ - DemoClass, DemoSubclass, DemoNotAnnotated]) + DemoClass, DemoSubclass, DemoNotAnnotated, setup_new_module]) obj = m.DemoClass(2, 3) res = obj.demo() assert res == DemoClass(2, 3).demo() + +if __name__=='__main__': + test_expose_classes() + \ No newline at end of file Modified: pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj ============================================================================== --- pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj (original) +++ pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Wed Apr 5 06:54:47 2006 @@ -208,7 +208,7 @@ + RelativePath="..\..\..\..\..\..\..\Documents and Settings\ctismer\Local Settings\Temp\usession-1751\testing_1\testing_1.c"> Modified: pypy/dist/pypy/translator/translator.py ============================================================================== --- pypy/dist/pypy/translator/translator.py (original) +++ pypy/dist/pypy/translator/translator.py Wed Apr 5 06:54:47 2006 @@ -46,6 +46,8 @@ log.start(nice_repr_for_func(func)) space = FlowObjSpace() space.__dict__.update(self.flags) # xxx push flags there + if self.annotator: + self.annotator.policy._adjust_space_config(space) graph = space.build_flow(func) if self.flags.get('simplifying'): simplify_graph(graph) From tismer at codespeak.net Wed Apr 5 07:18:46 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 5 Apr 2006 07:18:46 +0200 (CEST) Subject: [pypy-svn] r25336 - pypy/dist/pypy/translator/c/test Message-ID: <20060405051846.D21F31018C@code0.codespeak.net> Author: tismer Date: Wed Apr 5 07:18:44 2006 New Revision: 25336 Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py Log: weeh. importing twisted works like a charm. going home, need thinking Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Wed Apr 5 07:18:44 2006 @@ -352,19 +352,17 @@ # create a space and run geninterp on it? # tweak flow space to delay the imports for this func? +### XXX write up the rules how to use this ### + def setup_new_module(mod, modname): - # note the name clash with py.test - import sys - from __builtin__ import type - print type(modname) - m = type(sys)(modname) - print 'hallo', 42 - dummy = long(42) + # note the name clash with py.test on setup_module + from types import module + m = module(modname) allobjs = mod.__dict__.values() - funcs = eval('[]') - print 'alive' + funcs = eval('[]') # or import list from __builtin__ from twisted.internet import reactor print dir(reactor) + #whow this works for obj in allobjs: print obj, dir(obj) if hasattr(42, 'func_name'): From ericvrp at codespeak.net Wed Apr 5 09:15:34 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 09:15:34 +0200 (CEST) Subject: [pypy-svn] r25337 - in pypy/dist/pypy/translator/llvm: llvmcapi/include llvmcapi/include/ExecutionEngine llvmcapi/include/VMCore llvmcapi/lib llvmcapi/lib/ExecutionEngine llvmcapi/lib/VMCore pythonllvm pythonllvm/test Message-ID: <20060405071534.4E0E410190@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 09:15:29 2006 New Revision: 25337 Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Type.h pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Type.cpp (contents, props changed) Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py pypy/dist/pypy/translator/llvm/pythonllvm/helper.py pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Log: most pythonllvm tests passing again Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/ExecutionEngine.h Wed Apr 5 09:15:29 2006 @@ -14,7 +14,7 @@ void ExecutionEngine_freeMachineCodeForFunction(void* EE, void* F); // return union and takes std::vector actually -int ExecutionEngine_runFunction(void* EE, void* F, int args_vector); +long long ExecutionEngine_runFunction(void* EE, void* F, void* A); #ifdef __cplusplus }; Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/ExecutionEngine/GenericValue.h Wed Apr 5 09:15:29 2006 @@ -5,6 +5,35 @@ extern "C" { #endif +//#include "llvm/ExecutionEngine/GenericValue.h" + +// taken from llvm/ExecutionEngine/GenericValue.h +#define bool signed char // C has no boolean datatype +#define uint64_t unsigned long long +#define int64_t signed long long +#define PointerTy void* + +// XXX GenericValue_ grew an underscore to avoid a conflict with llvm's GenericValue +// which has additional stuff like constructors and the like which ctypes +// codewriter does not swallow at the moment. +// Idealy we would rewrite/extend codewriter ( +union GenericValue_ { + bool BoolVal; + unsigned char UByteVal; + signed char SByteVal; + unsigned short UShortVal; + signed short ShortVal; + unsigned int UIntVal; + signed int IntVal; + uint64_t ULongVal; + int64_t LongVal; + double DoubleVal; + float FloatVal; + struct { unsigned int first; unsigned int second; } UIntPairVal; + PointerTy PointerVal; + unsigned char Untyped[8]; +}; + void* GenericValue__init__(); #ifdef __cplusplus Added: pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Type.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/VMCore/Type.h Wed Apr 5 09:15:29 2006 @@ -0,0 +1,40 @@ +#ifndef __TYPE_H__ +#define __TYPE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +// XXX can ctypes codewriter this directly from llvm/Type.h? +enum TypeID { + // PrimitiveTypes .. make sure LastPrimitiveTyID stays up to date + VoidTyID = 0 , BoolTyID, // 0, 1: Basics... + UByteTyID , SByteTyID, // 2, 3: 8 bit types... + UShortTyID , ShortTyID, // 4, 5: 16 bit types... + UIntTyID , IntTyID, // 6, 7: 32 bit types... + ULongTyID , LongTyID, // 8, 9: 64 bit types... + FloatTyID , DoubleTyID, // 10,11: Floating point types... + LabelTyID , // 12 : Labels... + + // Derived types... see DerivedTypes.h file... + // Make sure FirstDerivedTyID stays up to date!!! + FunctionTyID , StructTyID, // Functions... Structs... + ArrayTyID , PointerTyID, // Array... pointer... + OpaqueTyID, // Opaque type instances... + PackedTyID, // SIMD 'packed' format... + //... + + NumTypeIDs, // Must remain as last defined ID + LastPrimitiveTyID = LabelTyID, + FirstDerivedTyID = FunctionTyID + }; + +TypeID Type_getTypeID(void* T); +const void* Type_getContainedType(void* T, int n); //return a Type* +const char* Type_getDescription(void* T); + +#ifdef __cplusplus +}; +#endif + +#endif Modified: pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/include/llvmcapi.h Wed Apr 5 09:15:29 2006 @@ -9,6 +9,7 @@ #include "VMCore/ModuleProvider.h" #include "VMCore/Function.h" #include "VMCore/DerivedTypes.h" +#include "VMCore/Type.h" #include "ExecutionEngine/ExecutionEngine.h" #include "ExecutionEngine/GenericValue.h" Modified: pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/ExecutionEngine/ExecutionEngine.cpp Wed Apr 5 09:15:29 2006 @@ -26,9 +26,18 @@ ee->freeMachineCodeForFunction(f); } -int ExecutionEngine_runFunction(void* EE, void* F, int args_vector) { - ExecutionEngine* ee = (ExecutionEngine*)EE; - Function* f = (Function*)F; +long long ExecutionEngine_runFunction(void* EE, void* F, void* A) { + ExecutionEngine* ee = (ExecutionEngine*)EE; + Function* f = (Function*)F; + long long* pArgs = (long long*)A; + + int n_params = f->getFunctionType()->getNumParams(); std::vector args; - return ee->runFunction(f, args).IntVal; + GenericValue gv; + for (int i=0;i < n_params;i++) { + gv.LongVal = pArgs[i]; + args.push_back(gv); + } + + return ee->runFunction(f, args).LongVal; } Added: pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Type.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/VMCore/Type.cpp Wed Apr 5 09:15:29 2006 @@ -0,0 +1,18 @@ +#include "llvm/Type.h" + +using namespace llvm; + +TypeID Type_getTypeID(void* T) { + Type* t = (Type*)T; + return (TypeID)t->getTypeID(); +} + +const void* Type_getContainedType(void* T, int n) { + Type* t = (Type*)T; + return t->getContainedType(n); +} + +const char* Type_getDescription(void* T) { + Type* t = (Type*)T; + return t->getDescription().c_str(); +} Modified: pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp ============================================================================== --- pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp (original) +++ pypy/dist/pypy/translator/llvm/llvmcapi/lib/llvmcapi.cpp Wed Apr 5 09:15:29 2006 @@ -23,6 +23,7 @@ #include "VMCore/ModuleProvider.cpp" #include "VMCore/Function.cpp" #include "VMCore/DerivedTypes.cpp" +#include "VMCore/Type.cpp" #include "ExecutionEngine/ExecutionEngine.cpp" #include "ExecutionEngine/GenericValue.cpp" Modified: pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/create_llvmcapi.py Wed Apr 5 09:15:29 2006 @@ -4,11 +4,14 @@ WRAPDIR = "/Users/eric/projects/ctypes/ctypes/wrap" #XXX get rid of this hardcoded path if os.path.exists(WRAPDIR + "/h2xml.py"): - print "creating llvmcapi.py" - os.system("python " + WRAPDIR + "/h2xml.py ../llvmcapi/include/llvmcapi.h -q -I . -o llvmcapi.xml") + print "h2xml.py" + os.system("python " + WRAPDIR + "/h2xml.py ../llvmcapi/include/llvmcapi.h -q -I. -I/opt/local/include -D_GNU_SOURCE -D__STDC_LIMIT_MACROS -o llvmcapi.xml") + print "xml2py.py" os.system("python " + WRAPDIR + "/xml2py.py llvmcapi.xml -l ../llvmcapi/llvmcapi.so -o llvmcapi.tmp") + print "massage output" os.system("sed -e s/from\ ctypes\ import/from\ cc\ import/ llvmcapi.tmp > llvmcapi.py") + print "cleanup some files" os.system("rm -f llvmcapi.tmp llvmcapi.xml") - print "done" + print "created llvmcapi.py" else: print "skipping llvmcapi.py creation" Modified: pypy/dist/pypy/translator/llvm/pythonllvm/helper.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/helper.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/helper.py Wed Apr 5 09:15:29 2006 @@ -21,3 +21,41 @@ def __getattr__(self, name): global_funcname = self.__class__.__name__ + "_" + name return Method(self.instance, globals()[global_funcname]) + + +def to_llvm_value(pythonvalue, type_): + value = GenericValue_() + value.LongVal = 0 + + # XXX store in correct value. here + value.IntVal = pythonvalue + return value + + +def to_python_value(llvmvalue, type_): + value = GenericValue_() + value.LongVal = llvmvalue # XXX convert llvmvalue from long long + id_ = type_.getTypeID() + if id_ == PointerTyID and type_.getContainedType(0).getTypeID() == SByteTyID: + return STRING(value.PointerVal).value + elif id_ == VoidTyID: + return None + elif id_ == BoolTyID: + return value.BoolVal + elif id_ == UByteTyID: + return value.UByteVal + elif id_ == SByteTyID: + return value.SByteVal + elif id_ == UShortTyID: + return value.UShortVal + elif id_ == ShortTyID: + return value.ShortVal + elif id_ == UIntTyID: + return value.UIntVal + elif id_ == IntTyID: + return value.IntVal + elif id_ == ULongTyID: + return value.ULongVal + elif id_ == LongTyID: + return value.ULongVal + raise Exception("don't know how to convert llvmtype '%s' to python" % type_.getDescription()) Modified: pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/llvmcapi.py Wed Apr 5 09:15:29 2006 @@ -4,6 +4,28 @@ STRING = c_char_p +UShortTyID = 4 +FirstDerivedTyID = 13 +FunctionTyID = 13 +ShortTyID = 5 +LabelTyID = 12 +UIntTyID = 6 +ArrayTyID = 15 +IntTyID = 7 +PointerTyID = 16 +ULongTyID = 8 +LongTyID = 9 +BoolTyID = 1 +StructTyID = 14 +PackedTyID = 18 +FloatTyID = 10 +NumTypeIDs = 19 +UByteTyID = 2 +VoidTyID = 0 +SByteTyID = 3 +OpaqueTyID = 17 +LastPrimitiveTyID = 12 +DoubleTyID = 11 def ExecutionEngine__create__(MP, ForceInterpreter): # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 12 @@ -23,14 +45,44 @@ ExecutionEngine_freeMachineCodeForFunction = cdecl(None, 'llvmcapi', [c_void_p, c_void_p]) (ExecutionEngine_freeMachineCodeForFunction) -def ExecutionEngine_runFunction(EE, F, args_vector): +def ExecutionEngine_runFunction(EE, F, A): # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 17 - return ExecutionEngine_runFunction._api_(EE, F, args_vector) -ExecutionEngine_runFunction = cdecl(c_int, 'llvmcapi', [c_void_p, c_void_p, c_int]) (ExecutionEngine_runFunction) + return ExecutionEngine_runFunction._api_(EE, F, A) +ExecutionEngine_runFunction = cdecl(c_longlong, 'llvmcapi', [c_void_p, c_void_p, c_void_p]) (ExecutionEngine_runFunction) +# ../llvmcapi/include/ExecutionEngine/GenericValue.h 20 +class GenericValue_(Union): + pass +# ../llvmcapi/include/ExecutionEngine/GenericValue.h 32 +class N13GenericValue_3DOLLAR_0E(Structure): + pass +N13GenericValue_3DOLLAR_0E._fields_ = [ + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 32 + ('first', c_uint), + ('second', c_uint), +] +assert sizeof(N13GenericValue_3DOLLAR_0E) == 8, sizeof(N13GenericValue_3DOLLAR_0E) +GenericValue_._fields_ = [ + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 20 + ('BoolVal', c_byte), + ('UByteVal', c_ubyte), + ('SByteVal', c_byte), + ('UShortVal', c_ushort), + ('ShortVal', c_short), + ('UIntVal', c_uint), + ('IntVal', c_int), + ('ULongVal', c_ulonglong), + ('LongVal', c_longlong), + ('DoubleVal', c_double), + ('FloatVal', c_float), + ('UIntPairVal', N13GenericValue_3DOLLAR_0E), + ('PointerVal', c_void_p), + ('Untyped', c_ubyte * 8), +] +assert sizeof(GenericValue_) == 8, sizeof(GenericValue_) def GenericValue__init__(): - # ../llvmcapi/include/ExecutionEngine/GenericValue.h 8 + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 37 return GenericValue__init__._api_() GenericValue__init__ = cdecl(c_void_p, 'llvmcapi', []) (GenericValue__init__) @@ -143,8 +195,29 @@ ExistingModuleProvider__init__ = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (ExistingModuleProvider__init__) +# values for enumeration 'TypeID' +TypeID = c_int # enum + +def Type_getTypeID(T): + # ../llvmcapi/include/VMCore/Type.h 32 + return Type_getTypeID._api_(T) +Type_getTypeID = cdecl(TypeID, 'llvmcapi', [c_void_p]) (Type_getTypeID) + + +def Type_getContainedType(T, n): + # ../llvmcapi/include/VMCore/Type.h 33 + return Type_getContainedType._api_(T, n) +Type_getContainedType = cdecl(c_void_p, 'llvmcapi', [c_void_p, c_int]) (Type_getContainedType) + + +def Type_getDescription(T): + # ../llvmcapi/include/VMCore/Type.h 34 + return Type_getDescription._api_(T) +Type_getDescription = cdecl(STRING, 'llvmcapi', [c_void_p]) (Type_getDescription) + + def toggle_print_machineinstrs(): - # ../llvmcapi/include/llvmcapi.h 15 + # ../llvmcapi/include/llvmcapi.h 16 return toggle_print_machineinstrs._api_() toggle_print_machineinstrs = cdecl(None, 'llvmcapi', []) (toggle_print_machineinstrs) Modified: pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/pyllvm.py Wed Apr 5 09:15:29 2006 @@ -25,7 +25,7 @@ MP = ExistingModuleProvider(); self.instance = ExecutionEngine__create__(MP.instance, ForceInterpreter) global ee_hack - ee_hack = self.instance + ee_hack = self.instance #XXX how to get to the executionengine from a function? # XXX cast to actual Python Module (can't we automate this?) def getModule(self): @@ -34,7 +34,7 @@ return m # helpers - def parse(self, llcode): + def parse(self, llcode, fnname=None): mod = self.getModule() mod.ParseAssemblyString(llcode) mod.verifyModule() @@ -46,32 +46,29 @@ f.eraseFromParent() -def to_llvm_value(pythonvalue, type_): - # XXX use the GenericValue union instead - return pythonvalue - - -def to_python_value(llvmvalue, type_): - # XXX use the GenericValue union instead - return llvmvalue - - class Function(Wrapper): def __init__(self): self.instance = Function__init__() #XXX this get annoying quickly def __call__(self, *args): - print 'calling %s(%s)' % ('Function', ','.join([str(arg) for arg in args])) - ft = Function_getFunctionType(self.instance) - argcount = FunctionType_getNumParams(ft) - print 'argcount = ',argcount + #print 'calling %s(%s)' % ('Function', ','.join([str(arg) for arg in args])) + ft = self.getFunctionType() + argcount = ft.getNumParams() + #print 'argcount = ',argcounT if argcount != len(args): raise Exception("incorrect number of parameters") - llvmvalues = GenericValue__init__() + ParamType = c_longlong * argcount + llvmvalues = ParamType() for i, arg in enumerate(args): - llvmvalues.append( to_llvm_value(arg, FunctionType_getParamType(ft, i))) - llvmreturnvalue = ExecutionEngine_runFunction(ee_hack, self.instance, llvmvalues) - return to_python_value(llvmreturnvalue, FunctionType_getReturnType(ft)) + llvmvalues[i] = to_llvm_value(arg, ft.getParamType(i)).LongVal + pLlvmValues = cast(llvmvalues, c_void_p) #would like to cast to c_longlong_p + llvmreturnvalue = ExecutionEngine_runFunction(ee_hack, self.instance, pLlvmValues) + return to_python_value(llvmreturnvalue, ft.getReturnType()) + + def getFunctionType(self): + ft = object.__new__(FunctionType) + ft.instance = Function_getFunctionType(self.instance) + return ft class GenericValue(Wrapper): @@ -79,11 +76,26 @@ self.instance = GenericValue__init__() -#class Instruction(Wrapper): -# def __init__(self): -# self.instance = Instruction__init__() -# -# -#class BasicBlock(Wrapper): -# def __init__(self): -# self.instance = BasicBlock__init__() +class Type(Wrapper): + def __init(self): + self.instance = Type__init__() + + def getContainedType(self, n): + t = object.__new__(Type) + t.instance = Type_getContainedType(self.instance, n) + return t + + +class FunctionType(Wrapper): + def __init(self): + self.instance = FunctionType__init__() + + def getReturnType(self): + t = object.__new__(Type) + t.instance = FunctionType_getReturnType(self.instance) + return t + + def getParamType(self, i): + t = object.__new__(Type) + t.instance = FunctionType_getParamType(self.instance, i) + return t Modified: pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Wed Apr 5 09:15:29 2006 @@ -6,7 +6,6 @@ from pypy.translator.llvm.pythonllvm import pyllvm from pypy.translator.llvm.pythonllvm.test import ll_snippet -py.test.skip("WIP") def test_execution_engine(): ee = pyllvm.ExecutionEngine() @@ -49,13 +48,16 @@ assert gethellostr() == "hello world\n" def test_call_parse_twice(): + py.test.skip("WIP") ee = pyllvm.ExecutionEngine() ee.parse(codepath.join("hello.s").read()) f = ee.getModule().getNamedFunction - assert f("gethellostr")() == "hello world\n" + f1 = f("gethellostr") + assert f1() == "hello world\n" ee.parse(codepath.join("addnumbers.s").read()) - assert f("add")(10, 32) == 42 - assert f("gethellostr")() == "hello world\n" + f2 = f("add") + assert f2(10, 32) == 42 + assert f1() == "hello world\n" py.test.raises(Exception, ee.parse) py.test.raises(Exception, ee.parse, 1) py.test.raises(Exception, ee.parse, "abc") @@ -73,6 +75,7 @@ assert f("calc")(122) == 123 def test_replace_function(): + py.test.skip("WIP") """similar to test_call_between_parsed_code with additional complexity because we rebind the add1 function to another version after it the first version already has been used.""" @@ -98,6 +101,7 @@ assert f("sub10_from_global_int_a")() == 82 def test_native_code(): #examine JIT generate native (assembly) code + py.test.skip("WIP") pyllvm.toggle_print_machineinstrs() ee = pyllvm.ExecutionEngine() ee.parse(ll_snippet.calc) @@ -107,6 +111,7 @@ pyllvm.toggle_print_machineinstrs() def test_delete_function(): #this will only work if nothing uses Fn of course! + py.test.skip("WIP") ee = pyllvm.ExecutionEngine() mod = ee.getModule() ee.parse(ll_snippet.calc) From ale at codespeak.net Wed Apr 5 09:25:12 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Wed, 5 Apr 2006 09:25:12 +0200 (CEST) Subject: [pypy-svn] r25338 - pypy/dist/pypy/objspace Message-ID: <20060405072512.B629510191@code0.codespeak.net> Author: ale Date: Wed Apr 5 09:25:10 2006 New Revision: 25338 Modified: pypy/dist/pypy/objspace/logic.py Log: Small change to make logic object space translateable Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 09:25:10 2006 @@ -11,6 +11,7 @@ from pypy.objspace.std.stringobject import W_StringObject from pypy.objspace.std.model import StdObjSpaceMultiMethod + #-- THE BUILTINS ---------------------------------------------------------------------- # this collects all multimethods to be made part of the Space @@ -183,6 +184,7 @@ class W_Var(W_Root, object): def __init__(w_self): + #w_self.typedef = 'Variable' w_self.w_bound_to = w_self w_self.w_needed = False From arigo at codespeak.net Wed Apr 5 11:00:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 11:00:36 +0200 (CEST) Subject: [pypy-svn] r25339 - pypy/dist/pypy/translator/c/src Message-ID: <20060405090036.1081D10194@code0.codespeak.net> Author: arigo Date: Wed Apr 5 11:00:35 2006 New Revision: 25339 Modified: pypy/dist/pypy/translator/c/src/module.h Log: Argh! This doesn't compile at all under gcc. Modified: pypy/dist/pypy/translator/c/src/module.h ============================================================================== --- pypy/dist/pypy/translator/c/src/module.h (original) +++ pypy/dist/pypy/translator/c/src/module.h Wed Apr 5 11:00:35 2006 @@ -60,7 +60,7 @@ /* helper-hook for post-setup */ static globalfunctiondef_t *globalfunctiondefsptr; static PyObject *postsetup_get_type_dict(PyObject *tp); -static PyObject *postsetup_build_method(int funcidx); +static PyObject *postsetup_build_method(int funcidx, PyObject *type); /* implementations */ From nik at codespeak.net Wed Apr 5 11:04:51 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 11:04:51 +0200 (CEST) Subject: [pypy-svn] r25340 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem rpython/ootypesystem rpython/ootypesystem/test rpython/test Message-ID: <20060405090451.9B75E10196@code0.codespeak.net> Author: nik Date: Wed Apr 5 11:04:49 2006 New Revision: 25340 Modified: pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py Log: make list iterators work in ootypesystem. this included some other fixes along the way: - make sure list types are cached so we can rely on their identity hash - make annotation of lists in ll helpers correct Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Wed Apr 5 11:04:49 2006 @@ -561,7 +561,7 @@ except TypeError: s = None # unhashable T, e.g. a Ptr(GcForwardReference()) if s is None: - if isinstance(T, ootype.Instance): + if isinstance(T, (ootype.Instance, ootype.List)): return SomeOOInstance(T) elif isinstance(T, ootype.StaticMethod): return SomeOOStaticMeth(T) Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Wed Apr 5 11:04:49 2006 @@ -3,8 +3,9 @@ from pypy.objspace.flow.model import Constant from pypy.rpython.error import TyperError from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst -from pypy.rpython.rmodel import IteratorRepr, externalvsinternal -from pypy.rpython.rlist import AbstractListRepr, rtype_newlist +from pypy.rpython.rmodel import externalvsinternal +from pypy.rpython.rlist import AbstractListRepr, AbstractListIteratorRepr, \ + rtype_newlist from pypy.rpython.rlist import dum_nocheck, dum_checkidx from pypy.rpython.rslice import SliceRepr from pypy.rpython.rslice import startstop_slice_repr, startonly_slice_repr @@ -53,9 +54,6 @@ def _setup_repr_final(self): self.list_builder.setup(self) - def recast(self, llops, v): - return llops.convertvar(v, self.item_repr, self.external_item_repr) - def convert_const(self, listobj): # get object from bound list method #listobj = getattr(listobj, '__self__', listobj) @@ -974,25 +972,15 @@ # # Iteration. -class ListIteratorRepr(IteratorRepr): +class ListIteratorRepr(AbstractListIteratorRepr): def __init__(self, r_list): self.r_list = r_list self.lowleveltype = Ptr(GcStruct('listiter', ('list', r_list.lowleveltype), ('index', Signed))) - - def newiter(self, hop): - v_lst, = hop.inputargs(self.r_list) - citerptr = hop.inputconst(Void, self.lowleveltype) - return hop.gendirectcall(ll_listiter, citerptr, v_lst) - - def rtype_next(self, hop): - v_iter, = hop.inputargs(self) - hop.has_implicit_exception(StopIteration) # record that we know about it - hop.exception_is_here() - v_res = hop.gendirectcall(ll_listnext, v_iter) - return self.r_list.recast(hop.llops, v_res) + self.ll_listiter = ll_listiter + self.ll_listnext = ll_listnext def ll_listiter(ITERPTR, lst): iter = malloc(ITERPTR.TO) Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Wed Apr 5 11:04:49 2006 @@ -180,7 +180,7 @@ # to map to their native list implementations. self._METHODS = frozendict({ # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) - "length": Meth([], Unsigned), + "length": Meth([], Signed), "append": Meth([ITEMTYPE], Void), "getitem": Meth([Signed], ITEMTYPE), "setitem": Meth([Signed, ITEMTYPE], Void), Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 11:04:49 2006 @@ -1,6 +1,8 @@ from pypy.annotation.pairtype import pairtype -from pypy.rpython.rlist import AbstractListRepr, rtype_newlist -from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst, externalvsinternal +from pypy.rpython.rlist import AbstractListRepr, AbstractListIteratorRepr, \ + rtype_newlist +from pypy.rpython.rmodel import Repr, IntegerRepr +from pypy.rpython.rmodel import inputconst, externalvsinternal from pypy.rpython.ootypesystem import ootype class BaseListRepr(AbstractListRepr): @@ -13,7 +15,7 @@ assert callable(item_repr) self._item_repr_computer = item_repr else: - self.lowleveltype = ootype.List(item_repr.lowleveltype) + self.lowleveltype = list_type(item_repr) self.external_item_repr, self.item_repr = \ externalvsinternal(rtyper, item_repr) self.listitem = listitem @@ -24,7 +26,7 @@ if 'item_repr' not in self.__dict__: self.external_item_repr, self.item_repr = \ externalvsinternal(self.rtyper, self._item_repr_computer()) - self.lowleveltype = ootype.List(self.item_repr.lowleveltype) + self.lowleveltype = list_type(self.item_repr) def send_message(self, hop, message, can_raise=False): v_args = hop.inputargs(self, *hop.args_r[1:]) @@ -40,9 +42,23 @@ def rtype_method_append(self, hop): return self.send_message(hop, "append") + def make_iterator_repr(self): + return ListIteratorRepr(self) + ListRepr = BaseListRepr FixedSizeListRepr = BaseListRepr +_list_types = {} + +def list_type(item_repr): + key = item_repr.lowleveltype + if _list_types.has_key(key): + return _list_types[key] + else: + LIST = ootype.List(key) + _list_types[key] = LIST + return LIST + class __extend__(pairtype(BaseListRepr, IntegerRepr)): def rtype_getitem((r_list, r_int), hop): @@ -63,3 +79,41 @@ resulttype=ootype.Void) return v_result +# ____________________________________________________________ +# +# Iteration. + +_list_iter_types = {} + +def list_iter_type(r_list): + key = r_list.lowleveltype + if _list_iter_types.has_key(key): + return _list_iter_types[key] + else: + INST = ootype.Instance("ListIter", ootype.ROOT, + {"list": r_list.lowleveltype, "index": ootype.Signed}) + _list_iter_types[key] = INST + return INST + +class ListIteratorRepr(AbstractListIteratorRepr): + + def __init__(self, r_list): + self.r_list = r_list + self.lowleveltype = list_iter_type(r_list) + self.ll_listiter = ll_listiter + self.ll_listnext = ll_listnext + +def ll_listiter(ITER, lst): + iter = ootype.new(ITER) + iter.list = lst + iter.index = 0 + return iter + +def ll_listnext(iter): + l = iter.list + index = iter.index + if index >= l.length(): + raise StopIteration + iter.index = index + 1 + return l.getitem(index) + Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Wed Apr 5 11:04:49 2006 @@ -1,5 +1,7 @@ from pypy import conftest from pypy.rpython.ootypesystem.ootype import * +from pypy.rpython.ootypesystem.rlist import ListRepr +from pypy.rpython.rint import signed_repr from pypy.annotation import model as annmodel from pypy.objspace.flow import FlowObjSpace from pypy.translator.translator import TranslationContext, graphof @@ -106,7 +108,7 @@ g = gengraph(oof, []) rettype = g.getreturnvar().concretetype - assert rettype == Unsigned + assert rettype == Signed def test_list_append(): LT = List(Signed) @@ -118,7 +120,7 @@ g = gengraph(oof, []) rettype = g.getreturnvar().concretetype - assert rettype == Unsigned + assert rettype == Signed def test_list_getitem_setitem(): LT = List(Signed) @@ -146,3 +148,22 @@ res = interpret(oof, [], type_system='ootype') assert res is -1 + +def test_list_lltype_identity(): + t = TranslationContext() + t.buildannotator() + rtyper = t.buildrtyper() + repr1 = ListRepr(rtyper, signed_repr) + repr2 = ListRepr(rtyper, signed_repr) + assert repr1.lowleveltype is repr2.lowleveltype + +def test_list_annotation(): + LIST = List(Signed) + + def oof(lst): + return lst.length() + + lst = new(LIST) + lst.append(1) + res = interpret(oof, [lst], type_system='ootype') + assert res == 1 Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Wed Apr 5 11:04:49 2006 @@ -1,5 +1,6 @@ from pypy.annotation import model as annmodel -from pypy.rpython.rmodel import Repr, inputconst +from pypy.rpython.rmodel import Repr, IteratorRepr, inputconst +from pypy.rpython.lltypesystem import lltype from pypy.rpython import robject @@ -29,7 +30,8 @@ class AbstractListRepr(Repr): - pass + def recast(self, llops, v): + return llops.convertvar(v, self.item_repr, self.external_item_repr) def rtype_newlist(hop): nb_args = hop.nb_args @@ -51,3 +53,21 @@ def dum_checkidx(): pass def dum_nocheck(): pass +# ____________________________________________________________ +# +# Iteration. + +class AbstractListIteratorRepr(IteratorRepr): + + def newiter(self, hop): + v_lst, = hop.inputargs(self.r_list) + citerptr = hop.inputconst(lltype.Void, self.lowleveltype) + return hop.gendirectcall(self.ll_listiter, citerptr, v_lst) + + def rtype_next(self, hop): + v_iter, = hop.inputargs(self) + hop.has_implicit_exception(StopIteration) # record that we know about it + hop.exception_is_here() + v_res = hop.gendirectcall(self.ll_listnext, v_iter) + return self.r_list.recast(hop.llops, v_res) + Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Wed Apr 5 11:04:49 2006 @@ -194,26 +194,26 @@ l = [5] l.append(6) return len(l) - res = interpret(dummyfn, []) + res = interpret(dummyfn, [], type_system=self.ts) assert res == 2 -def test_iterate(): - def dummyfn(): - total = 0 - for x in [1, 3, 5, 7, 9]: - total += x - return total - res = interpret(dummyfn, []) - assert res == 25 - def dummyfn(): - total = 0 - l = [1, 3, 5, 7] - l.append(9) - for x in l: - total += x - return total - res = interpret(dummyfn, []) - assert res == 25 + def test_iterate(self): + def dummyfn(): + total = 0 + for x in [1, 3, 5, 7, 9]: + total += x + return total + res = interpret(dummyfn, [], type_system=self.ts) + assert res == 25 + def dummyfn(): + total = 0 + l = [1, 3, 5, 7] + l.append(9) + for x in l: + total += x + return total + res = interpret(dummyfn, [], type_system=self.ts) + assert res == 25 def test_recursive(): def dummyfn(N): From hpk at codespeak.net Wed Apr 5 11:08:23 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 5 Apr 2006 11:08:23 +0200 (CEST) Subject: [pypy-svn] r25341 - in pypy/dist/pypy: annotation/test bin doc/revreport interpreter interpreter/pyparser/test interpreter/test lib/test2 module/__builtin__/test module/_codecs/test module/_sre/test module/sys/test objspace/flow/test objspace/std objspace/std/test objspace/test tool tool/algo/test tool/pytest tool/test translator translator/asm translator/asm/ppcgen translator/asm/ppcgen/test translator/asm/test translator/c translator/c/test translator/goal translator/java translator/java/test translator/llvm/demo translator/llvm/pyllvm translator/llvm/pythonllvm translator/pyrex translator/pyrex/test translator/test translator/tool translator/tool/pygame Message-ID: <20060405090823.B92A010198@code0.codespeak.net> Author: hpk Date: Wed Apr 5 11:08:21 2006 New Revision: 25341 Modified: pypy/dist/pypy/annotation/test/autopath.py pypy/dist/pypy/bin/autopath.py pypy/dist/pypy/doc/revreport/autopath.py pypy/dist/pypy/interpreter/autopath.py pypy/dist/pypy/interpreter/pyparser/test/autopath.py pypy/dist/pypy/interpreter/test/autopath.py pypy/dist/pypy/lib/test2/autopath.py pypy/dist/pypy/module/__builtin__/test/autopath.py pypy/dist/pypy/module/_codecs/test/autopath.py pypy/dist/pypy/module/_sre/test/autopath.py pypy/dist/pypy/module/sys/test/autopath.py pypy/dist/pypy/objspace/flow/test/autopath.py pypy/dist/pypy/objspace/std/autopath.py pypy/dist/pypy/objspace/std/test/autopath.py pypy/dist/pypy/objspace/test/autopath.py pypy/dist/pypy/tool/algo/test/autopath.py pypy/dist/pypy/tool/autopath.py pypy/dist/pypy/tool/pytest/autopath.py pypy/dist/pypy/tool/test/autopath.py pypy/dist/pypy/translator/asm/autopath.py pypy/dist/pypy/translator/asm/ppcgen/autopath.py pypy/dist/pypy/translator/asm/ppcgen/test/autopath.py pypy/dist/pypy/translator/asm/test/autopath.py pypy/dist/pypy/translator/autopath.py pypy/dist/pypy/translator/c/autopath.py pypy/dist/pypy/translator/c/test/autopath.py pypy/dist/pypy/translator/goal/autopath.py pypy/dist/pypy/translator/java/autopath.py pypy/dist/pypy/translator/java/test/autopath.py pypy/dist/pypy/translator/llvm/demo/autopath.py pypy/dist/pypy/translator/llvm/pyllvm/autopath.py pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py pypy/dist/pypy/translator/pyrex/autopath.py pypy/dist/pypy/translator/pyrex/test/autopath.py pypy/dist/pypy/translator/test/autopath.py pypy/dist/pypy/translator/tool/autopath.py pypy/dist/pypy/translator/tool/pygame/autopath.py Log: don't magically remove pypy subdirectories from sys.path (might lead to problems if you are in a CWD directory that has modules with exact CPython names but those should be fixed or you need to run from somewhere else, autopath should not have to care for this) Modified: pypy/dist/pypy/annotation/test/autopath.py ============================================================================== --- pypy/dist/pypy/annotation/test/autopath.py (original) +++ pypy/dist/pypy/annotation/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/bin/autopath.py ============================================================================== --- pypy/dist/pypy/bin/autopath.py (original) +++ pypy/dist/pypy/bin/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/doc/revreport/autopath.py ============================================================================== --- pypy/dist/pypy/doc/revreport/autopath.py (original) +++ pypy/dist/pypy/doc/revreport/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/interpreter/autopath.py ============================================================================== --- pypy/dist/pypy/interpreter/autopath.py (original) +++ pypy/dist/pypy/interpreter/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/interpreter/pyparser/test/autopath.py ============================================================================== --- pypy/dist/pypy/interpreter/pyparser/test/autopath.py (original) +++ pypy/dist/pypy/interpreter/pyparser/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/interpreter/test/autopath.py ============================================================================== --- pypy/dist/pypy/interpreter/test/autopath.py (original) +++ pypy/dist/pypy/interpreter/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/lib/test2/autopath.py ============================================================================== --- pypy/dist/pypy/lib/test2/autopath.py (original) +++ pypy/dist/pypy/lib/test2/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/module/__builtin__/test/autopath.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/test/autopath.py (original) +++ pypy/dist/pypy/module/__builtin__/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/module/_codecs/test/autopath.py ============================================================================== --- pypy/dist/pypy/module/_codecs/test/autopath.py (original) +++ pypy/dist/pypy/module/_codecs/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/module/_sre/test/autopath.py ============================================================================== --- pypy/dist/pypy/module/_sre/test/autopath.py (original) +++ pypy/dist/pypy/module/_sre/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/module/sys/test/autopath.py ============================================================================== --- pypy/dist/pypy/module/sys/test/autopath.py (original) +++ pypy/dist/pypy/module/sys/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/objspace/flow/test/autopath.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/autopath.py (original) +++ pypy/dist/pypy/objspace/flow/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/objspace/std/autopath.py ============================================================================== --- pypy/dist/pypy/objspace/std/autopath.py (original) +++ pypy/dist/pypy/objspace/std/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/objspace/std/test/autopath.py ============================================================================== --- pypy/dist/pypy/objspace/std/test/autopath.py (original) +++ pypy/dist/pypy/objspace/std/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/objspace/test/autopath.py ============================================================================== --- pypy/dist/pypy/objspace/test/autopath.py (original) +++ pypy/dist/pypy/objspace/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/tool/algo/test/autopath.py ============================================================================== --- pypy/dist/pypy/tool/algo/test/autopath.py (original) +++ pypy/dist/pypy/tool/algo/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/tool/autopath.py ============================================================================== --- pypy/dist/pypy/tool/autopath.py (original) +++ pypy/dist/pypy/tool/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/tool/pytest/autopath.py ============================================================================== --- pypy/dist/pypy/tool/pytest/autopath.py (original) +++ pypy/dist/pypy/tool/pytest/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/tool/test/autopath.py ============================================================================== --- pypy/dist/pypy/tool/test/autopath.py (original) +++ pypy/dist/pypy/tool/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/asm/autopath.py ============================================================================== --- pypy/dist/pypy/translator/asm/autopath.py (original) +++ pypy/dist/pypy/translator/asm/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/asm/ppcgen/autopath.py ============================================================================== --- pypy/dist/pypy/translator/asm/ppcgen/autopath.py (original) +++ pypy/dist/pypy/translator/asm/ppcgen/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/asm/ppcgen/test/autopath.py ============================================================================== --- pypy/dist/pypy/translator/asm/ppcgen/test/autopath.py (original) +++ pypy/dist/pypy/translator/asm/ppcgen/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/asm/test/autopath.py ============================================================================== --- pypy/dist/pypy/translator/asm/test/autopath.py (original) +++ pypy/dist/pypy/translator/asm/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/autopath.py ============================================================================== --- pypy/dist/pypy/translator/autopath.py (original) +++ pypy/dist/pypy/translator/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/c/autopath.py ============================================================================== --- pypy/dist/pypy/translator/c/autopath.py (original) +++ pypy/dist/pypy/translator/c/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/c/test/autopath.py ============================================================================== --- pypy/dist/pypy/translator/c/test/autopath.py (original) +++ pypy/dist/pypy/translator/c/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/goal/autopath.py ============================================================================== --- pypy/dist/pypy/translator/goal/autopath.py (original) +++ pypy/dist/pypy/translator/goal/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/java/autopath.py ============================================================================== --- pypy/dist/pypy/translator/java/autopath.py (original) +++ pypy/dist/pypy/translator/java/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/java/test/autopath.py ============================================================================== --- pypy/dist/pypy/translator/java/test/autopath.py (original) +++ pypy/dist/pypy/translator/java/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/llvm/demo/autopath.py ============================================================================== --- pypy/dist/pypy/translator/llvm/demo/autopath.py (original) +++ pypy/dist/pypy/translator/llvm/demo/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/llvm/pyllvm/autopath.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pyllvm/autopath.py (original) +++ pypy/dist/pypy/translator/llvm/pyllvm/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/pyrex/autopath.py ============================================================================== --- pypy/dist/pypy/translator/pyrex/autopath.py (original) +++ pypy/dist/pypy/translator/pyrex/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/pyrex/test/autopath.py ============================================================================== --- pypy/dist/pypy/translator/pyrex/test/autopath.py (original) +++ pypy/dist/pypy/translator/pyrex/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/test/autopath.py ============================================================================== --- pypy/dist/pypy/translator/test/autopath.py (original) +++ pypy/dist/pypy/translator/test/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/tool/autopath.py ============================================================================== --- pypy/dist/pypy/translator/tool/autopath.py (original) +++ pypy/dist/pypy/translator/tool/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: Modified: pypy/dist/pypy/translator/tool/pygame/autopath.py ============================================================================== --- pypy/dist/pypy/translator/tool/pygame/autopath.py (original) +++ pypy/dist/pypy/translator/tool/pygame/autopath.py Wed Apr 5 11:08:21 2006 @@ -41,15 +41,7 @@ else: raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) - checkpaths = sys.path[:] pypy_root = os.path.join(head, '') - - while checkpaths: - orig = checkpaths.pop() - fullorig = os.path.join(os.path.realpath(orig), '') - if fullorig.startswith(pypy_root): - if os.path.exists(os.path.join(fullorig, '__init__.py')): - sys.path.remove(orig) try: sys.path.remove(head) except ValueError: From ericvrp at codespeak.net Wed Apr 5 11:12:41 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 11:12:41 +0200 (CEST) Subject: [pypy-svn] r25342 - in pypy/dist/pypy/translator/llvm/pythonllvm: . test Message-ID: <20060405091241.7621E1019C@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 11:12:40 2006 New Revision: 25342 Modified: pypy/dist/pypy/translator/llvm/pythonllvm/helper.py pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Log: Added (python) to_llvm_value support. Modified: pypy/dist/pypy/translator/llvm/pythonllvm/helper.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/helper.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/helper.py Wed Apr 5 11:12:40 2006 @@ -26,9 +26,31 @@ def to_llvm_value(pythonvalue, type_): value = GenericValue_() value.LongVal = 0 - - # XXX store in correct value. here - value.IntVal = pythonvalue + id_ = type_.getTypeID() + if id_ == PointerTyID and type_.getContainedType(0).getTypeID() == SByteTyID: + value.PointerVal = pointer(pythonvalue) + elif id_ == VoidTyID: + pass + elif id_ == BoolTyID: + value.BoolVal = pythonvalue + elif id_ == UByteTyID: + value.UByteVal = pythonvalue + elif id_ == SByteTyID: + value.SByteVal = pythonvalue + elif id_ == UShortTyID: + value.UShortVal = pythonvalue + elif id_ == ShortTyID: + value.ShortVal = pythonvalue + elif id_ == UIntTyID: + value.UIntVal = pythonvalue + elif id_ == IntTyID: + value.IntVal = pythonvalue + elif id_ == ULongTyID: + value.ULongVal = pythonvalue + elif id_ == LongTyID: + value.ULongVal = pythonvalue + else: + raise Exception("don't know how to convert pythontype '%s' to llvm" % type_.getDescription()) return value Modified: pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py (original) +++ pypy/dist/pypy/translator/llvm/pythonllvm/test/test_ee.py Wed Apr 5 11:12:40 2006 @@ -130,6 +130,15 @@ ee.parse(ll_snippet.add1) assert f("calc")(100) == 101 +def TODOtest_multiple_executionengines(): + pass + +def TODOtest_returntypes(): + pass + +def TODOtest_paramtypes(): + pass + def TODOtest_add_to_function(): pass From nik at codespeak.net Wed Apr 5 11:33:26 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 11:33:26 +0200 (CEST) Subject: [pypy-svn] r25343 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060405093326.1246C1019D@code0.codespeak.net> Author: nik Date: Wed Apr 5 11:33:25 2006 New Revision: 25343 Added: pypy/dist/pypy/rpython/ootypesystem/riterable.py (contents, props changed) Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rtuple.py Log: generalize iterator type construction for tuples and lists in ootypes. Added: pypy/dist/pypy/rpython/ootypesystem/riterable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/riterable.py Wed Apr 5 11:33:25 2006 @@ -0,0 +1,13 @@ +from pypy.rpython.ootypesystem import ootype + +_iter_types = {} + +def iterator_type(r_iterable): + key = r_iterable.lowleveltype + if _iter_types.has_key(key): + return _iter_types[key] + else: + ITER = ootype.Instance("Iterator", ootype.ROOT, + {"iterable": key, "index": ootype.Signed}) + _iter_types[key] = ITER + return ITER Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 11:33:25 2006 @@ -4,6 +4,7 @@ from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.rmodel import inputconst, externalvsinternal from pypy.rpython.ootypesystem import ootype +from pypy.rpython.ootypesystem.riterable import iterator_type class BaseListRepr(AbstractListRepr): @@ -83,34 +84,22 @@ # # Iteration. -_list_iter_types = {} - -def list_iter_type(r_list): - key = r_list.lowleveltype - if _list_iter_types.has_key(key): - return _list_iter_types[key] - else: - INST = ootype.Instance("ListIter", ootype.ROOT, - {"list": r_list.lowleveltype, "index": ootype.Signed}) - _list_iter_types[key] = INST - return INST - class ListIteratorRepr(AbstractListIteratorRepr): def __init__(self, r_list): self.r_list = r_list - self.lowleveltype = list_iter_type(r_list) + self.lowleveltype = iterator_type(r_list) self.ll_listiter = ll_listiter self.ll_listnext = ll_listnext def ll_listiter(ITER, lst): iter = ootype.new(ITER) - iter.list = lst + iter.iterable = lst iter.index = 0 return iter def ll_listnext(iter): - l = iter.list + l = iter.iterable index = iter.index if index >= l.length(): raise StopIteration Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Wed Apr 5 11:33:25 2006 @@ -1,5 +1,6 @@ from pypy.rpython.rmodel import inputconst from pypy.rpython.rtuple import AbstractTupleRepr, AbstractTupleIteratorRepr +from pypy.rpython.ootypesystem.riterable import iterator_type from pypy.rpython.ootypesystem import ootype @@ -55,23 +56,11 @@ # # Iteration. -_tuple_iter_types = {} - -def tuple_iter_type(r_tuple): - key = r_tuple.lowleveltype - if _tuple_iter_types.has_key(key): - return _tuple_iter_types[key] - else: - INST = ootype.Instance("TupleIter", ootype.ROOT, - {"tuple": r_tuple.lowleveltype}) - _tuple_iter_types[key] = INST - return INST - class Length1TupleIteratorRepr(AbstractTupleIteratorRepr): def __init__(self, r_tuple): self.r_tuple = r_tuple - self.lowleveltype = tuple_iter_type(r_tuple) + self.lowleveltype = iterator_type(r_tuple) self.ll_tupleiter = ll_tupleiter self.ll_tuplenext = ll_tuplenext @@ -79,14 +68,14 @@ def ll_tupleiter(ITERINST, tuple): iter = ootype.new(ITERINST) - iter.tuple = tuple + iter.iterable = tuple return iter def ll_tuplenext(iter): # for iterating over length 1 tuples only! - t = iter.tuple + t = iter.iterable if t: - iter.tuple = ootype.null(ootype.typeOf(t)) + iter.iterable = ootype.null(ootype.typeOf(t)) return t.item0 else: raise StopIteration From mwh at codespeak.net Wed Apr 5 11:40:24 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 11:40:24 +0200 (CEST) Subject: [pypy-svn] r25344 - in pypy/branch/explicit-exceptions: rpython/memory translator/c translator/c/test Message-ID: <20060405094024.393311019F@code0.codespeak.net> Author: mwh Date: Wed Apr 5 11:40:22 2006 New Revision: 25344 Modified: pypy/branch/explicit-exceptions/rpython/memory/gctransform.py pypy/branch/explicit-exceptions/translator/c/funcgen.py pypy/branch/explicit-exceptions/translator/c/test/test_genc.py Log: (hpk, mwh, pedronis) fix the "PyObjPtr mess" on the explicit exceptions branch - don't special case PyObject* in genc at all - special case the few PyObject*-returning operations that need to be followed with an incref in gctransform - a new entry in the 'most insane test of the year' contest Modified: pypy/branch/explicit-exceptions/rpython/memory/gctransform.py ============================================================================== --- pypy/branch/explicit-exceptions/rpython/memory/gctransform.py (original) +++ pypy/branch/explicit-exceptions/rpython/memory/gctransform.py Wed Apr 5 11:40:22 2006 @@ -110,7 +110,12 @@ origname = op.opname op = ops[index] if var_needsgc(op.result): - if op.opname not in ('direct_call', 'indirect_call') and not var_ispyobj(op.result): + if var_ispyobj(op.result): + assert op.opname != 'cast_pointer', 'casting to a PyObject*??' + if op.opname in ('getfield', 'getarrayitem', 'same_as'): + lst = list(self.push_alive(op.result)) + newops.extend(lst) + elif op.opname not in ('direct_call', 'indirect_call'): lst = list(self.push_alive(op.result)) newops.extend(lst) livevars.append(op.result) Modified: pypy/branch/explicit-exceptions/translator/c/funcgen.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/funcgen.py (original) +++ pypy/branch/explicit-exceptions/translator/c/funcgen.py Wed Apr 5 11:40:22 2006 @@ -11,9 +11,6 @@ PyObjPtr = Ptr(PyObject) LOCALVAR = 'l_%s' -# I'm not absolutely sure, so just in case: -NEED_OLD_EXTRA_REFS = True # oupps seems to be - class FunctionCodeGenerator(object): """ Collects information about a function which we have to generate @@ -364,9 +361,6 @@ T = self.lltypemap(op.result) newvalue = self.expr(op.result, special_case_void=False) result = ['%s = %s;' % (newvalue, sourceexpr)] - # need to adjust the refcount of the result only for PyObjects - if NEED_OLD_EXTRA_REFS and T == PyObjPtr: - result.append('Py_XINCREF(%s);' % newvalue) result = '\n'.join(result) if T is Void: result = '/* %s */' % result @@ -506,9 +500,7 @@ result.append('%s = (%s)%s;' % (self.expr(op.result), cdecl(typename, ''), self.expr(op.args[0]))) - - if NEED_OLD_EXTRA_REFS and TYPE == PyObjPtr: - result.append('Py_XINCREF(%s);'%(LOCAL_VAR % op.result.name)) + assert TYPE != PyObjPtr return '\t'.join(result) OP_CAST_PTR_TO_ADR = OP_CAST_POINTER @@ -527,8 +519,6 @@ if TYPE is not Void: result.append('%s = %s;' % (self.expr(op.result), self.expr(op.args[0]))) - if NEED_OLD_EXTRA_REFS and TYPE == PyObjPtr: - result.append('Py_XINCREF(%s);'%(LOCAL_VAR % op.result.name)) return '\t'.join(result) def OP_HINT(self, op): Modified: pypy/branch/explicit-exceptions/translator/c/test/test_genc.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_genc.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_genc.py Wed Apr 5 11:40:22 2006 @@ -298,3 +298,35 @@ f1 = compile(f, []) assert f1() == 1 + +# this test shows if we have a problem with refcounting PyObject +def test_refcount_pyobj(): + def prob_with_pyobj(b): + return 3, b + + f = compile(prob_with_pyobj, [object]) + from sys import getrefcount as g + obj = None + before = g(obj) + f(obj) + after = g(obj) + assert before == after + +def test_refcount_pyobj_setfield(): + import weakref, gc + class S(object): + def __init__(self): + self.p = None + def foo(wref, objfact): + s = S() + b = objfact() + s.p = b + wr = wref(b) + s.p = None + return wr + f = compile(foo, [object, object], backendopt=False) + class C(object): + pass + wref = f(weakref.ref, C) + gc.collect() + assert not wref() From mwh at codespeak.net Wed Apr 5 11:46:32 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 11:46:32 +0200 (CEST) Subject: [pypy-svn] r25345 - pypy/branch/explicit-exceptions/translator Message-ID: <20060405094632.401EF101A2@code0.codespeak.net> Author: mwh Date: Wed Apr 5 11:46:31 2006 New Revision: 25345 Modified: pypy/branch/explicit-exceptions/translator/unsimplify.py Log: don't call checkgraph in split_block, it slows down translation by a factor of 6 or so. Modified: pypy/branch/explicit-exceptions/translator/unsimplify.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/unsimplify.py (original) +++ pypy/branch/explicit-exceptions/translator/unsimplify.py Wed Apr 5 11:46:31 2006 @@ -84,7 +84,7 @@ block.exits = [link] block.exitswitch = None block.exc_handler = False - checkgraph(graph) + #checkgraph(graph) return newblock def remove_direct_loops(translator, graph): From arigo at codespeak.net Wed Apr 5 11:49:07 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 11:49:07 +0200 (CEST) Subject: [pypy-svn] r25346 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060405094907.9228C101A4@code0.codespeak.net> Author: arigo Date: Wed Apr 5 11:49:06 2006 New Revision: 25346 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: (arre, arigo) Switched rarray.py to the new FixedSizeArray lltype. Got rid of the conversion from rctypes box to primitive lltypes, which was both special-case for primitive things only and which may be triggered unexpectedly. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Wed Apr 5 11:49:06 2006 @@ -1,7 +1,7 @@ from ctypes import ARRAY, c_int from pypy.annotation.model import SomeCTypesObject, SomeBuiltin from pypy.rpython import extregistry -from pypy.rpython.rmodel import Repr +from pypy.rpython.rmodel import Repr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr @@ -21,33 +21,54 @@ SomeCTypesObject.MEMORYALIAS)) # Here, self.c_data_type == self.ll_type - c_data_type = lltype.Array(self.r_item.ll_type, - hints={"nolength": True}) - - # Array elements are of the low-level type (Signed, etc) and not - # of the boxed low level type (Ptr(GcStruct(...))) + c_data_type = lltype.FixedSizeArray(self.r_item.c_data_type, + self.length) super(ArrayRepr, self).__init__(rtyper, s_array, c_data_type) + def get_c_data_of_item(self, llops, v_array, v_index): + v_c_array = self.get_c_data(llops, v_array) + return llops.genop('getarraysubstruct', [v_c_array, v_index], + lltype.Ptr(self.r_item.c_data_type)) + class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, - r_array.r_item.ll_type) - v_c_data = r_array.get_c_data(hop.llops, v_array) - hop.genop('setarrayitem', [v_c_data, v_index, v_item]) + r_array.r_item) + v_item_c_data = r_array.r_item.get_c_data(hop.llops, v_item) + v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) + # copy the whole structure's content over + reccopy(hop.llops, v_item_c_data, v_c_data) def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) + v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) + return r_array.r_item.return_c_data(hop.llops, v_c_data) + + +def reccopy(llops, v_source, v_dest): + # helper (XXX move somewhere else) to copy recursively a structure + # or array onto another. + T = v_source.concretetype.TO + assert T == v_dest.concretetype.TO + assert isinstance(T, lltype.Struct) + for name in T._names: + FIELDTYPE = getattr(T, name) + cname = inputconst(lltype.Void, name) + if isinstance(FIELDTYPE, lltype.ContainerType): + RESTYPE = lltype.Ptr(FIELDTYPE) + v_subsrc = llops.genop('getsubstruct', [v_source, cname], RESTYPE) + v_subdst = llops.genop('getsubstruct', [v_dest, cname], RESTYPE) + reccopy(llops, v_subsrc, v_subdst) + else: + v_value = llops.genop('getfield', [v_source, cname], FIELDTYPE) + llops.genop('setfield', [v_dest, cname, v_value]) - v_c_data = r_array.get_c_data(hop.llops, v_array) - return hop.genop('getarrayitem', [v_c_data, v_index], - r_array.r_item.ll_type) def arraytype_specialize_call(hop): r_array = hop.r_result - return hop.genop("malloc_varsize", [ + return hop.genop("malloc", [ hop.inputconst(lltype.Void, r_array.lowleveltype.TO), - hop.inputconst(lltype.Signed, r_array.length), ], resulttype=r_array.lowleveltype, ) Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Wed Apr 5 11:49:06 2006 @@ -7,6 +7,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr +from pypy.rpython.error import TyperError from pypy.rpython.rctypes.rmodel import CTypesValueRepr ctypes_annotation_list = [ @@ -34,16 +35,32 @@ return self.getvalue_from_c_data(llops, v_c_data) def convert_const(self, ctype_value): - assert isinstance(ctype_value, self.ctype) - key = id(ctype_value) + if isinstance(ctype_value, self.ctype): + key = "by_id", id(ctype_value) + value = ctype_value.value + keepalive = ctype_value + else: + if self.ownsmemory: + raise TyperError("convert_const(%r) but repr owns memory" % ( + ctype_value,)) + key = "by_value", ctype_value + value = ctype_value + keepalive = None try: return self.const_cache[key][0] except KeyError: - p = lltype.malloc(self.lowleveltype.TO) - - self.const_cache[key] = p, ctype_value - p.c_data.value = ctype_value.value - return p + p = lltype.malloc(self.owner_lowleveltype.TO) + p.c_data.value = value + if self.ownsmemory: + result = p + else: + # we must return a non-memory-owning box that keeps the + # memory-owning box alive + result = lltype.malloc(self.lowleveltype.TO) + result.c_data_ref = p.c_data + result.c_data_owner_keepalive = p + self.const_cache[key] = result, keepalive + return result def rtype_getattr(self, hop): s_attr = hop.args_s[1] @@ -60,14 +77,6 @@ self.ll_type) self.setvalue(hop.llops, v_primitive, v_value) -# need to extend primitive repr to implement convert_from_to() for various -# conversions, firstly the conversion from c_long() to Signed - -class __extend__(pairtype(PrimitiveRepr, IntegerRepr)): - def convert_from_to((r_from, r_to), v, llops): - assert r_from.ll_type == r_to.lowleveltype - - return r_from.getvalue(llops, v) def primitive_specialize_call(hop): r_primitive = hop.r_result Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Wed Apr 5 11:49:06 2006 @@ -108,10 +108,10 @@ res = interpret(create_array, []) c_data = res.c_data - assert c_data[0] == 0 - assert c_data[9] == 0 + assert c_data[0].value == 0 + assert c_data[9].value == 0 py.test.raises(IndexError, "c_data[10]") - py.test.raises(TypeError, "len(c_data)") + assert len(c_data) == 10 def test_specialize_array_access(self): def access_array(): @@ -126,6 +126,7 @@ class Test_compilation: def test_compile_array_access(self): + py.test.skip("in-progress") def access_array(): my_array = c_int_10() my_array[0] = 1 From nik at codespeak.net Wed Apr 5 12:10:55 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 12:10:55 +0200 (CEST) Subject: [pypy-svn] r25347 - in pypy/dist/pypy/rpython: ootypesystem test Message-ID: <20060405101055.40849101A5@code0.codespeak.net> Author: nik Date: Wed Apr 5 12:10:52 2006 New Revision: 25347 Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rtuple.py pypy/dist/pypy/rpython/test/test_rtuple.py Log: implement rtyping of list(sometuple) for ootypesystem. Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 12:10:52 2006 @@ -80,6 +80,14 @@ resulttype=ootype.Void) return v_result +# These helpers are trivial but help encapsulation + +def ll_newlist(LIST): + return ootype.new(LIST) + +def ll_append(lst, item): + lst.append(item) + # ____________________________________________________________ # # Iteration. Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Wed Apr 5 12:10:52 2006 @@ -35,6 +35,20 @@ vinst, = hop.inputargs(self) return hop.genop('ooidentityhash', [vinst], resulttype=ootype.Signed) + def rtype_bltn_list(self, hop): + from pypy.rpython.ootypesystem import rlist + v_tup = hop.inputarg(self, 0) + LIST = hop.r_result.lowleveltype + c_list = inputconst(ootype.Void, LIST) + v_list = hop.gendirectcall(rlist.ll_newlist, c_list) + for index in range(len(self.items_r)): + name = self.fieldnames[index] + r_item = self.items_r[index] + c_name = hop.inputconst(ootype.Void, name) + v_item = hop.genop("oogetfield", [v_tup, c_name], resulttype=r_item) + v_item = hop.llops.convertvar(v_item, r_item, hop.r_result.item_repr) + hop.gendirectcall(rlist.ll_append, v_list, v_item) + return v_list _tuple_types = {} Modified: pypy/dist/pypy/rpython/test/test_rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rtuple.py (original) +++ pypy/dist/pypy/rpython/test/test_rtuple.py Wed Apr 5 12:10:52 2006 @@ -187,14 +187,14 @@ assert res1 != res2 def test_tuple_to_list(self): - if self.type_system == "ootype": - py.test.skip("XXX fix me if ootypes support lists") - def f(i, j): return list((i, j)) res = self.interpret(f, [2, 3]) - assert res._obj.items == [2, 3] + if self.type_system == "lltype": + assert res._obj.items == [2, 3] + else: + assert res._list == [2, 3] def test_tuple_iterator_length1(self): def f(i): From auc at codespeak.net Wed Apr 5 12:13:46 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 12:13:46 +0200 (CEST) Subject: [pypy-svn] r25348 - pypy/dist/pypy/objspace Message-ID: <20060405101346.D02F7101A7@code0.codespeak.net> Author: auc Date: Wed Apr 5 12:13:46 2006 New Revision: 25348 Modified: pypy/dist/pypy/objspace/logic.py Log: only one return path Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 12:13:46 2006 @@ -78,9 +78,10 @@ def pop_blocked_on(self, w_var): assert isinstance(w_var, W_Var) if w_var not in self.uthreads_blocked_on: - return [] - blocked = self.uthreads_blocked_on[w_var] - del self.uthreads_blocked_on[w_var] + blocked = [] + else: + blocked = self.uthreads_blocked_on[w_var] + del self.uthreads_blocked_on[w_var] return blocked def add_to_blocked_byneed(self, w_var, uthread): @@ -98,9 +99,10 @@ assert isinstance(w_var, W_Var) if w_var not in self.uthreads_blocked_byneed: #print " there was nobody to remove for", w_var - return [] - blocked = self.uthreads_blocked_byneed[w_var] - del self.uthreads_blocked_byneed[w_var] + blocked = [] + else: + blocked = self.uthreads_blocked_byneed[w_var] + del self.uthreads_blocked_byneed[w_var] #print " removing", blocked, "from byneed on", w_var return blocked From nik at codespeak.net Wed Apr 5 12:33:35 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 12:33:35 +0200 (CEST) Subject: [pypy-svn] r25349 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060405103335.42009101AC@code0.codespeak.net> Author: nik Date: Wed Apr 5 12:33:33 2006 New Revision: 25349 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: don't cache list types for the reprs, instead define sensible eq/hash behaviour. suggestion by samuele, also necessary for recursive list types which are about to be introduced. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Wed Apr 5 12:33:33 2006 @@ -189,6 +189,17 @@ def __str__(self): return '%s(%s)' % (self.__class__.__name__, self._ITEMTYPE) + def __eq__(self, other): + if not isinstance(other, List): + return False + return self._ITEMTYPE == other._ITEMTYPE + + def __ne__(self, other): + return not (self == other) + + def __hash__(self): + return hash(self._ITEMTYPE) + def _lookup(self, meth_name): METH = self._METHODS.get(meth_name) meth = None Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 12:33:33 2006 @@ -16,7 +16,7 @@ assert callable(item_repr) self._item_repr_computer = item_repr else: - self.lowleveltype = list_type(item_repr) + self.lowleveltype = ootype.List(item_repr.lowleveltype) self.external_item_repr, self.item_repr = \ externalvsinternal(rtyper, item_repr) self.listitem = listitem @@ -27,7 +27,7 @@ if 'item_repr' not in self.__dict__: self.external_item_repr, self.item_repr = \ externalvsinternal(self.rtyper, self._item_repr_computer()) - self.lowleveltype = list_type(self.item_repr) + self.lowleveltype = ootype.List(self.item_repr.lowleveltype) def send_message(self, hop, message, can_raise=False): v_args = hop.inputargs(self, *hop.args_r[1:]) @@ -49,17 +49,6 @@ ListRepr = BaseListRepr FixedSizeListRepr = BaseListRepr -_list_types = {} - -def list_type(item_repr): - key = item_repr.lowleveltype - if _list_types.has_key(key): - return _list_types[key] - else: - LIST = ootype.List(key) - _list_types[key] = LIST - return LIST - class __extend__(pairtype(BaseListRepr, IntegerRepr)): def rtype_getitem((r_list, r_int), hop): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Wed Apr 5 12:33:33 2006 @@ -37,6 +37,14 @@ n = null(LT) py.test.raises(RuntimeError, "n.append(0)") +def test_eq_hash(): + LT1 = List(Signed) + LT2 = List(Signed) + LT3 = List(Unsigned) + assert LT1 == LT2 + assert LT1 != LT3 + assert hash(LT1) == hash(LT2) + class TestInterpreted: def test_append_length(self): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Wed Apr 5 12:33:33 2006 @@ -155,7 +155,8 @@ rtyper = t.buildrtyper() repr1 = ListRepr(rtyper, signed_repr) repr2 = ListRepr(rtyper, signed_repr) - assert repr1.lowleveltype is repr2.lowleveltype + assert repr1.lowleveltype == repr2.lowleveltype + assert hash(repr1.lowleveltype) == hash(repr2.lowleveltype) def test_list_annotation(): LIST = List(Signed) From auc at codespeak.net Wed Apr 5 12:40:03 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 12:40:03 +0200 (CEST) Subject: [pypy-svn] r25352 - pypy/dist/pypy/objspace Message-ID: <20060405104003.F23CC101AE@code0.codespeak.net> Author: auc Date: Wed Apr 5 12:40:03 2006 New Revision: 25352 Modified: pypy/dist/pypy/objspace/logic.py Log: assert type of a Coroutine (not Greenlet) Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 12:40:03 2006 @@ -67,7 +67,8 @@ def add_to_blocked(self, w_var, uthread): assert isinstance(w_var, W_Var) - assert isinstance(uthread, GreenletCoroutine) + if we_are_translated(): + assert isinstance(uthread, Coroutine) if w_var in self.uthreads_blocked_on: blocked = self.uthreads_blocked_on[w_var] else: @@ -86,7 +87,8 @@ def add_to_blocked_byneed(self, w_var, uthread): assert isinstance(w_var, W_Var) - assert isinstance(uthread, GreenletCoroutine) + if we_are_translated(): + assert isinstance(uthread, Coroutine) #print " adding", uthread, "to byneed on", w_var if w_var in self.uthreads_blocked_byneed: blocked = self.uthreads_blocked_byneed[w_var] From mwh at codespeak.net Wed Apr 5 13:12:57 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:12:57 +0200 (CEST) Subject: [pypy-svn] r25355 - pypy/branch/explicit-exceptions/objspace/flow Message-ID: <20060405111257.61FF6101AF@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:12:56 2006 New Revision: 25355 Modified: pypy/branch/explicit-exceptions/objspace/flow/model.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/objspace/flow/model.py revisions 24785 to 25354: ------------------------------------------------------------------------ r24794 | mwh | 2006-03-22 13:34:35 +0100 (Wed, 22 Mar 2006) | 4 lines don't use id dictionaries in iterlinks(), itergraphs(), Blocks don't define __eq__ or __cmp__ so there's no need. Also, not doing so is ever-so-slightly quicker. ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/objspace/flow/model.py ============================================================================== --- pypy/branch/explicit-exceptions/objspace/flow/model.py (original) +++ pypy/branch/explicit-exceptions/objspace/flow/model.py Wed Apr 5 13:12:56 2006 @@ -77,25 +77,25 @@ def iterblocks(self): block = self.startblock yield block - seen = {id(block): True} + seen = {block: True} stack = list(block.exits[::-1]) while stack: block = stack.pop().target - if id(block) not in seen: + if block not in seen: yield block - seen[id(block)] = True + seen[block] = True stack += block.exits[::-1] def iterlinks(self): block = self.startblock - seen = {id(block): True} + seen = {block: True} stack = list(block.exits[::-1]) while stack: link = stack.pop() yield link block = link.target - if id(block) not in seen: - seen[id(block)] = True + if block not in seen: + seen[block] = True stack += block.exits[::-1] def show(self): From mwh at codespeak.net Wed Apr 5 13:13:01 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:01 +0200 (CEST) Subject: [pypy-svn] r25356 - pypy/branch/explicit-exceptions/rpython/lltypesystem Message-ID: <20060405111301.552D4101B4@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:12:58 2006 New Revision: 25356 Modified: pypy/branch/explicit-exceptions/rpython/lltypesystem/lloperation.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/rpython/lltypesystem/lloperation.py revisions 24785 to 25354: ------------------------------------------------------------------------ r25000 | arigo | 2006-03-25 19:19:57 +0100 (Sat, 25 Mar 2006) | 7 lines Remove some of the division low-level operations. Now we only have xxx_floordiv for integer types, and float_truediv for floats. The high-level operations are mapped to one of these, possibly with conversions (inserted automagically by the rtyper). ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/explicit-exceptions/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/explicit-exceptions/rpython/lltypesystem/lloperation.py Wed Apr 5 13:12:58 2006 @@ -63,7 +63,6 @@ # XXX Some clean-ups are needed: # * many exception-raising operations are being replaced by calls to helpers # * there are still many _ovf operations that cannot really raise OverflowError -# * the div/truediv/floordiv mess needs to be sorted out and reduced # * float_mod vs float_fmod ? # Run test_lloperation after changes. Feel free to clean up LLInterpreter too :-) @@ -96,8 +95,6 @@ 'int_add': LLOp(canfold=True), 'int_sub': LLOp(canfold=True), 'int_mul': LLOp(canfold=True), - 'int_div': LLOp(canfold=True), - 'int_truediv': LLOp(canfold=True), 'int_floordiv': LLOp(canfold=True), 'int_mod': LLOp(canfold=True), 'int_lt': LLOp(canfold=True), @@ -115,8 +112,6 @@ 'int_add_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_sub_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_mul_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_div_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_truediv_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_floordiv_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_mod_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_lt_ovf': LLOp(canfold=True, canraise=(OverflowError,)), @@ -145,10 +140,6 @@ 'uint_add': LLOp(canfold=True), 'uint_sub': LLOp(canfold=True), 'uint_mul': LLOp(canfold=True), - 'uint_div': LLOp(canfold=True), - 'uint_div_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), - 'uint_truediv': LLOp(canfold=True), - 'uint_truediv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'uint_floordiv': LLOp(canfold=True), 'uint_floordiv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'uint_mod': LLOp(canfold=True), @@ -171,9 +162,7 @@ 'float_add': LLOp(canfold=True), 'float_sub': LLOp(canfold=True), 'float_mul': LLOp(canfold=True), - 'float_div': LLOp(canfold=True), 'float_truediv': LLOp(canfold=True), - 'float_floordiv': LLOp(canfold=True), 'float_mod': LLOp(canfold=True), 'float_lt': LLOp(canfold=True), 'float_le': LLOp(canfold=True), @@ -193,8 +182,6 @@ 'llong_add': LLOp(canfold=True), 'llong_sub': LLOp(canfold=True), 'llong_mul': LLOp(canfold=True), - 'llong_div': LLOp(canfold=True), - 'llong_truediv': LLOp(canfold=True), 'llong_floordiv': LLOp(canfold=True), 'llong_mod': LLOp(canfold=True), 'llong_lt': LLOp(canfold=True), @@ -217,8 +204,6 @@ 'ullong_add': LLOp(canfold=True), 'ullong_sub': LLOp(canfold=True), 'ullong_mul': LLOp(canfold=True), - 'ullong_div': LLOp(canfold=True), - 'ullong_truediv': LLOp(canfold=True), 'ullong_floordiv': LLOp(canfold=True), 'ullong_mod': LLOp(canfold=True), 'ullong_lt': LLOp(canfold=True), From mwh at codespeak.net Wed Apr 5 13:13:03 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:03 +0200 (CEST) Subject: [pypy-svn] r25357 - pypy/branch/explicit-exceptions/translator/c Message-ID: <20060405111303.16CD6101B4@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:02 2006 New Revision: 25357 Modified: pypy/branch/explicit-exceptions/translator/c/database.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/c/database.py revisions 24785 to 25354: ------------------------------------------------------------------------ r24990 | tismer | 2006-03-25 08:25:03 +0100 (Sat, 25 Mar 2006) | 7 lines finally after lots of tiny hassles, here is a working implementation of a generated extension module that exposes its classes and can really be used. sigh :=) The next thing to do should be to get rid of PyCObject and use a special descendant of type object, instead. ------------------------------------------------------------------------ r24795 | mwh | 2006-03-22 13:36:26 +0100 (Wed, 22 Mar 2006) | 3 lines add a check that we don't call getcontainernode() on a new container too late. ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/c/database.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/database.py (original) +++ pypy/branch/explicit-exceptions/translator/c/database.py Wed Apr 5 13:13:02 2006 @@ -19,7 +19,8 @@ class LowLevelDatabase(object): - def __init__(self, translator=None, standalone=False, gcpolicy=None, thread_enabled=False): + def __init__(self, translator=None, standalone=False, gcpolicy=None, thread_enabled=False, + instantiators={}): self.translator = translator self.standalone = standalone self.structdefnodes = {} @@ -35,7 +36,7 @@ self.infs = [] self.namespace = CNameManager() if not standalone: - self.pyobjmaker = PyObjMaker(self.namespace, self.get, translator) + self.pyobjmaker = PyObjMaker(self.namespace, self.get, translator, instantiators) if gcpolicy is None: from pypy.translator.c import gc gcpolicy = gc.RefcountingGcPolicy @@ -112,6 +113,7 @@ try: node = self.containernodes[container] except KeyError: + assert not self.completed T = typeOf(container) if isinstance(T, (lltype.Array, lltype.Struct)): if hasattr(self.gctransformer, 'consider_constant'): From mwh at codespeak.net Wed Apr 5 13:13:08 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:08 +0200 (CEST) Subject: [pypy-svn] r25358 - pypy/branch/explicit-exceptions/translator/c Message-ID: <20060405111308.D0101101BC@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:04 2006 New Revision: 25358 Modified: pypy/branch/explicit-exceptions/translator/c/extfunc.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/c/extfunc.py revisions 24785 to 25354: ------------------------------------------------------------------------ r25261 | nik | 2006-04-03 17:24:29 +0200 (Mon, 03 Apr 2006) | 2 lines fix some references to rlist that i forgot in the last checkin. ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/c/extfunc.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/extfunc.py (original) +++ pypy/branch/explicit-exceptions/translator/c/extfunc.py Wed Apr 5 13:13:04 2006 @@ -3,7 +3,8 @@ from pypy.rpython.lltypesystem import lltype from pypy.translator.c.support import cdecl from pypy.rpython.rstr import STR -from pypy.rpython import rlist, rstr +from pypy.rpython import rstr +from pypy.rpython.lltypesystem import rlist from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack from pypy.module.thread.rpython import ll_thread From mwh at codespeak.net Wed Apr 5 13:13:13 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:13 +0200 (CEST) Subject: [pypy-svn] r25359 - pypy/branch/explicit-exceptions/translator/c Message-ID: <20060405111313.5D3D1101C0@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:09 2006 New Revision: 25359 Modified: pypy/branch/explicit-exceptions/translator/c/stackless.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/c/stackless.py revisions 24785 to 25354: ------------------------------------------------------------------------ r25330 | tismer | 2006-04-04 23:33:08 +0200 (Tue, 04 Apr 2006) | 1 line argh. When do I learn that getattr without default is not helpful! ------------------------------------------------------------------------ r25329 | tismer | 2006-04-04 23:23:22 +0200 (Tue, 04 Apr 2006) | 1 line do not rely on graphs always having a 'func' attribute ------------------------------------------------------------------------ r25266 | hpk | 2006-04-03 19:42:08 +0200 (Mon, 03 Apr 2006) | 2 lines clarify code a bit while trying to understand it ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/c/stackless.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/stackless.py (original) +++ pypy/branch/explicit-exceptions/translator/c/stackless.py Wed Apr 5 13:13:09 2006 @@ -69,35 +69,35 @@ for caller, callee in translator.callgraph.values(): # ignore calls issued suggested_primitives -- they are not # compiled in, and they typically contain pseudo-recursions - try: - suggprim = caller.func.suggested_primitive - except AttributeError: - suggprim = False - if not suggprim: + func = getattr(caller, 'func', None) + # there are graphs without funcs + if not getattr(func, 'suggested_primitive', False): callers[caller].append(callee) # check all callees if they can reach unwind - seen = self.can_reach_unwind + can_reach_unwind = self.can_reach_unwind pending = {} - ext = self.database.externalfuncs def check_unwind(graph): if graph in pending: - seen[graph] = True + can_reach_unwind[graph] = True return True pending[graph] = graph for callee in callers[graph]: - if callee in seen: - ret = seen[callee] + if callee in can_reach_unwind: + ret = can_reach_unwind[callee] else: ret = check_unwind(callee) if ret: break else: - ret = graph.func in self.stackless_roots + func = getattr(graph, 'func', None) + ret = func in self.stackless_roots del pending[graph] - seen[graph] = ret + can_reach_unwind[graph] = ret return ret - [check_unwind(caller) for caller in callers if caller not in seen] + for caller in callers: + if caller not in can_reach_unwind: + check_unwind(caller) def registerunwindable(self, functionname, FUNC, resume_points): if resume_points >= 1: From mwh at codespeak.net Wed Apr 5 13:13:19 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:19 +0200 (CEST) Subject: [pypy-svn] r25360 - pypy/branch/explicit-exceptions/translator/backendopt/test Message-ID: <20060405111319.2E8B7101C4@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:15 2006 New Revision: 25360 Modified: pypy/branch/explicit-exceptions/translator/backendopt/test/test_inline.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/backendopt/test/test_inline.py revisions 24785 to 25354: ------------------------------------------------------------------------ r25305 | pedronis | 2006-04-04 17:43:06 +0200 (Tue, 04 Apr 2006) | 3 lines case were the current keepalive insertion by inlining is insufficient. ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/backendopt/test/test_inline.py (original) +++ pypy/branch/explicit-exceptions/translator/backendopt/test/test_inline.py Wed Apr 5 13:13:15 2006 @@ -427,3 +427,24 @@ return 1 assert x4() == 1 py.test.raises(CannotInline, check_inline, x3, x4, []) + +def test_keepalive_hard_case(): + py.test.skip("fix this :(") + from pypy.rpython.lltypesystem import lltype + Y = lltype.Struct('y', ('n', lltype.Signed)) + X = lltype.GcStruct('x', ('y', Y)) + def g(x): + if x: + return 3 + else: + return 4 + def f(): + x = lltype.malloc(X) + x.y.n = 2 + y = x.y + z1 = g(y.n) + z = y.n + return z+z1 + eval_func = check_inline(g, f, []) + res = eval_func([]) + assert res == 5 From mwh at codespeak.net Wed Apr 5 13:13:23 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:23 +0200 (CEST) Subject: [pypy-svn] r25361 - pypy/branch/explicit-exceptions/translator/c/src Message-ID: <20060405111323.9A72F101C8@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:19 2006 New Revision: 25361 Modified: pypy/branch/explicit-exceptions/translator/c/src/float.h Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/c/src/float.h revisions 24785 to 25354: ------------------------------------------------------------------------ r25000 | arigo | 2006-03-25 19:19:57 +0100 (Sat, 25 Mar 2006) | 7 lines Remove some of the division low-level operations. Now we only have xxx_floordiv for integer types, and float_truediv for floats. The high-level operations are mapped to one of these, possibly with conversions (inserted automagically by the rtyper). ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/c/src/float.h ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/src/float.h (original) +++ pypy/branch/explicit-exceptions/translator/c/src/float.h Wed Apr 5 13:13:19 2006 @@ -26,7 +26,6 @@ #define OP_FLOAT_ADD(x,y,r) r = x + y #define OP_FLOAT_SUB(x,y,r) r = x - y #define OP_FLOAT_MUL(x,y,r) r = x * y -#define OP_FLOAT_DIV(x,y,r) r = x / y #define OP_FLOAT_TRUEDIV(x,y,r) OP_FLOAT_DIV(x,y,r) #define OP_FLOAT_POW(x,y,r) r = pow(x, y) From mwh at codespeak.net Wed Apr 5 13:13:26 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:26 +0200 (CEST) Subject: [pypy-svn] r25362 - pypy/branch/explicit-exceptions/translator/c/src Message-ID: <20060405111326.33B99101C8@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:24 2006 New Revision: 25362 Modified: pypy/branch/explicit-exceptions/translator/c/src/int.h Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/c/src/int.h revisions 24785 to 25354: ------------------------------------------------------------------------ r25000 | arigo | 2006-03-25 19:19:57 +0100 (Sat, 25 Mar 2006) | 7 lines Remove some of the division low-level operations. Now we only have xxx_floordiv for integer types, and float_truediv for floats. The high-level operations are mapped to one of these, possibly with conversions (inserted automagically by the rtyper). ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/c/src/int.h ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/src/int.h (original) +++ pypy/branch/explicit-exceptions/translator/c/src/int.h Wed Apr 5 13:13:24 2006 @@ -290,7 +290,7 @@ #define OP_LLONG_ADD OP_INT_ADD #define OP_LLONG_SUB OP_INT_SUB #define OP_LLONG_MUL OP_INT_MUL -#define OP_LLONG_DIV OP_INT_DIV +#define OP_LLONG_FLOORDIV OP_INT_FLOORDIV #define OP_LLONG_EQ OP_INT_EQ #define OP_LLONG_NE OP_INT_NE From mwh at codespeak.net Wed Apr 5 13:13:29 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:13:29 +0200 (CEST) Subject: [pypy-svn] r25363 - pypy/branch/explicit-exceptions/translator/c/test Message-ID: <20060405111329.CAF98101C8@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:13:27 2006 New Revision: 25363 Modified: pypy/branch/explicit-exceptions/translator/c/test/test_typed.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/c/test/test_typed.py revisions 24785 to 25354: ------------------------------------------------------------------------ r25000 | arigo | 2006-03-25 19:19:57 +0100 (Sat, 25 Mar 2006) | 7 lines Remove some of the division low-level operations. Now we only have xxx_floordiv for integer types, and float_truediv for floats. The high-level operations are mapped to one of these, possibly with conversions (inserted automagically by the rtyper). ------------------------------------------------------------------------ Modified: pypy/branch/explicit-exceptions/translator/c/test/test_typed.py ============================================================================== --- pypy/branch/explicit-exceptions/translator/c/test/test_typed.py (original) +++ pypy/branch/explicit-exceptions/translator/c/test/test_typed.py Wed Apr 5 13:13:27 2006 @@ -477,7 +477,7 @@ fn = self.getcompiled(snippet.add_func) raises(OverflowError, fn, sys.maxint) - def test_int_div_ovf_zer(self): + def test_int_floordiv_ovf_zer(self): fn = self.getcompiled(snippet.div_func) raises(OverflowError, fn, -1) raises(ZeroDivisionError, fn, 0) From auc at codespeak.net Wed Apr 5 13:25:31 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 13:25:31 +0200 (CEST) Subject: [pypy-svn] r25365 - pypy/dist/pypy/objspace Message-ID: <20060405112531.B04F1101B1@code0.codespeak.net> Author: auc Date: Wed Apr 5 13:25:31 2006 New Revision: 25365 Modified: pypy/dist/pypy/objspace/logic.py Log: some renaming, more assertions Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 13:25:31 2006 @@ -188,7 +188,6 @@ class W_Var(W_Root, object): def __init__(w_self): - #w_self.typedef = 'Variable' w_self.w_bound_to = w_self w_self.w_needed = False @@ -203,20 +202,20 @@ return W_Var() app_newvar = gateway.interp2app(newvar) -def wait__Root(space, w_self): - return w_self +def wait__Root(space, w_obj): + return w_obj -def wait__Var(space, w_self): +def wait__Var(space, w_var): while 1: - #print " :wait", w_self - if space.is_true(space.is_free(w_self)): + #print " :wait", w_var + if space.is_true(space.is_free(w_var)): if not have_uthreads(): raise OperationError(space.w_RuntimeError, space.wrap("trying to perform an operation on an unbound variable")) else: # notify wait_needed clients, give them a chance to run - w_self.w_needed = True - for w_alias in aliases(space, w_self): + w_var.w_needed = True + for w_alias in aliases(space, w_var): need_waiters = schedule_state.pop_blocked_byneed_on(w_alias) w_alias.w_needed = True for waiter in need_waiters: @@ -224,7 +223,7 @@ schedule_state.add_to_runnable(waiter) # set curr thread to blocked, switch to runnable thread current = get_current_coroutine() - schedule_state.add_to_blocked(w_self, current) + schedule_state.add_to_blocked(w_var, current) while schedule_state.have_runnable_threads(): next_coro = schedule_state.pop_runnable_thread() if next_coro.is_alive(): @@ -237,7 +236,7 @@ raise OperationError(space.w_RuntimeError, space.wrap("blocked on variable, but no uthread that can bind it")) else: - return w_self.w_bound_to + return w_var.w_bound_to def wait(space, w_obj): assert isinstance(w_obj, W_Root) @@ -250,11 +249,11 @@ all_mms['wait'] = wait_mm -def wait_needed__Var(space, w_self): +def wait_needed__Var(space, w_var): while 1: - #print " :needed", w_self - if space.is_true(space.is_free(w_self)): - if w_self.w_needed: + #print " :needed", w_var + if space.is_true(space.is_free(w_var)): + if w_var.w_needed: break # we're done if not have_uthreads(): raise OperationError(space.w_RuntimeError, @@ -262,7 +261,7 @@ else: # add current thread to blocked byneed and switch current = get_current_coroutine() - for w_alias in aliases(space, w_self): + for w_alias in aliases(space, w_var): schedule_state.add_to_blocked_byneed(w_alias, current) while schedule_state.have_runnable_threads(): next_coro = schedule_state.pop_runnable_thread() @@ -451,27 +450,30 @@ space.bind(w_var, w_obj) app_bind = gateway.interp2app(bind) -def bind__Var_Var(space, w_var, w_obj): - if space.is_true(space.is_bound(w_var)): - if space.is_true(space.is_bound(w_obj)): +def bind__Var_Var(space, w_v1, w_v2): + if space.is_true(space.is_bound(w_v1)): + if space.is_true(space.is_bound(w_v2)): return unify(space, #FIXME: we could just raise - deref(space, w_var), - deref(space, w_obj)) + deref(space, w_v1), + deref(space, w_v2)) # 2. a (obj unbound, var bound) - return _assign(space, w_obj, deref(space, w_var)) - elif space.is_true(space.is_bound(w_obj)): + return _assign(space, w_v2, deref(space, w_v1)) + elif space.is_true(space.is_bound(w_v2)): # 2. b (var unbound, obj bound) - return _assign(space, w_var, deref(space, w_obj)) + return _assign(space, w_v1, deref(space, w_v2)) else: # 1. both are unbound - return _alias(space, w_var, w_obj) + return _alias(space, w_v1, w_v2) -def bind__Var_Root(space, w_v1, w_v2): +def bind__Var_Root(space, w_var, w_obj): # 3. var and value - if space.is_true(space.is_free(w_v1)): - return _assign(space, w_v1, w_v2) - #print "uh !" - fail(space, w_v1, w_v2) + if space.is_true(space.is_free(w_var)): + return _assign(space, w_var, w_obj) + # for dataflow behaviour we should allow + # rebinding of unifiable values + raise OperationError(space.w_RuntimeError, + space.wrap("Cannot bind twice")) + bind_mm = StdObjSpaceMultiMethod('bind', 2) bind_mm.register(bind__Var_Root, W_Var, W_Root) @@ -520,6 +522,8 @@ return space.w_None def _add_to_aliases(space, w_v1, w_v2): + assert isinstance(w_v1, W_Var) + assert isinstance(w_v2, W_Var) #print " :add to aliases", w_v1, w_v2 w_tail = w_v1.w_bound_to w_v1.w_bound_to = w_v2 @@ -527,6 +531,8 @@ return space.w_None def _merge_aliases(space, w_v1, w_v2): + assert isinstance(w_v1, W_Var) + assert isinstance(w_v2, W_Var) #print " :merge aliases", w_v1, w_v2 w_tail1 = get_ring_tail(space, w_v1) w_tail2 = get_ring_tail(space, w_v2) From ericvrp at codespeak.net Wed Apr 5 13:28:40 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 13:28:40 +0200 (CEST) Subject: [pypy-svn] r25366 - pypy/dist/pypy/jit/codegen/llvm Message-ID: <20060405112840.3ABA8101B3@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 13:28:38 2006 New Revision: 25366 Modified: pypy/dist/pypy/jit/codegen/llvm/jitcode.py Log: Reflect changes to pyllvm in llvm jitcode.py Modified: pypy/dist/pypy/jit/codegen/llvm/jitcode.py ============================================================================== --- pypy/dist/pypy/jit/codegen/llvm/jitcode.py (original) +++ pypy/dist/pypy/jit/codegen/llvm/jitcode.py Wed Apr 5 13:28:38 2006 @@ -10,7 +10,7 @@ from pypy.translator.llvm.funcnode import FuncNode from pypy.translator.llvm.gc import GcPolicy from pypy.translator.llvm.exception import ExceptionPolicy -from pypy.translator.llvm.pyllvm.build import pyllvm +from pypy.translator.llvm.pyllvm import pyllvm from cStringIO import StringIO from pypy.tool.ansi_print import ansi_log @@ -28,10 +28,10 @@ self.graph = graph # LLVM execution engine (llvm module) that is going to contain the code -ee = pyllvm.get_ee() +ee = pyllvm.ExecutionEngine() -# Create a LLVM 'execution engine' which wrap the LLVM JIT +# Create a LLVM 'execution engine' which wraps the LLVM JIT class JITcode(object): def __init__(self, typer): @@ -73,4 +73,5 @@ log('eval_graph: %s(%s)' % (graph.name, args)) if graph not in self.graph_ref: self.graph_ref[graph] = self.codegen(graph) - return ee.call(self.graph_ref[graph], *args) + llvmfunc = ee.getModule().getNamedFunction(self.graph_ref[graph]) + return llvmfunc(*args) From pedronis at codespeak.net Wed Apr 5 13:32:13 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 5 Apr 2006 13:32:13 +0200 (CEST) Subject: [pypy-svn] r25368 - in pypy/dist/pypy: annotation annotation/test rpython rpython/test Message-ID: <20060405113213.B26AD101BA@code0.codespeak.net> Author: pedronis Date: Wed Apr 5 13:32:12 2006 New Revision: 25368 Modified: pypy/dist/pypy/annotation/test/test_annrpython.py pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: implemented setdefault in RPython Modified: pypy/dist/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/test/test_annrpython.py (original) +++ pypy/dist/pypy/annotation/test/test_annrpython.py Wed Apr 5 13:32:12 2006 @@ -495,6 +495,19 @@ assert isinstance(s_key, annmodel.SomeString) assert isinstance(s_value, annmodel.SomeInteger) + def test_dict_setdefault(self): + a = self.RPythonAnnotator() + def f(): + d = {} + d.setdefault('a', 2) + d.setdefault('a', -3) + return d + s = a.build_types(f, []) + assert isinstance(s, annmodel.SomeDict) + assert isinstance(dictkey(s), annmodel.SomeString) + assert isinstance(dictvalue(s), annmodel.SomeInteger) + assert not dictvalue(s).nonneg + def test_exception_deduction(self): a = self.RPythonAnnotator() s = a.build_types(snippet.exception_deduction, []) Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Wed Apr 5 13:32:12 2006 @@ -327,6 +327,8 @@ dct.dictdef.generalize_value(dfl) return dct.dictdef.read_value() + method_setdefault = method_get + def method_copy(dct): return SomeDict(dct.dictdef) Modified: pypy/dist/pypy/rpython/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/rdict.py (original) +++ pypy/dist/pypy/rpython/rdict.py Wed Apr 5 13:32:12 2006 @@ -274,6 +274,13 @@ v_res = hop.gendirectcall(ll_get, v_dict, v_key, v_default) return self.recast_value(hop.llops, v_res) + def rtype_method_setdefault(self, hop): + v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, + self.value_repr) + hop.exception_cannot_occur() + v_res = hop.gendirectcall(ll_setdefault, v_dict, v_key, v_default) + return self.recast_value(hop.llops, v_res) + def rtype_method_copy(self, hop): v_dict, = hop.inputargs(self) hop.exception_cannot_occur() @@ -706,6 +713,14 @@ else: return default +def ll_setdefault(dict, key, default): + entry = ll_dict_lookup(dict, key, dict.keyhash(key)) + if entry.valid(): + return entry.value + else: + ll_dict_setitem(dict, key, default) + return default + def ll_copy(dict): DICT = lltype.typeOf(dict).TO dictsize = len(dict.entries) Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Wed Apr 5 13:32:12 2006 @@ -234,6 +234,22 @@ res = interpret(func, ()) assert res == 422 +def test_dict_setdefault(): + def f(): + d = {} + d.setdefault('a', 2) + return d['a'] + res = interpret(f, ()) + assert res == 2 + + def f(): + d = {} + d.setdefault('a', 2) + x = d.setdefault('a', -3) + return x + res = interpret(f, ()) + assert res == 2 + def test_dict_copy(): def func(): # XXX this does not work if we use chars, only! From ericvrp at codespeak.net Wed Apr 5 13:34:17 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 13:34:17 +0200 (CEST) Subject: [pypy-svn] r25369 - pypy/dist/pypy/translator/llvm/pythonllvm Message-ID: <20060405113417.E85AC101BB@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 13:34:17 2006 New Revision: 25369 Removed: pypy/dist/pypy/translator/llvm/pythonllvm/ Log: Remove 'old' CPython extions From mwh at codespeak.net Wed Apr 5 13:37:29 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:37:29 +0200 (CEST) Subject: [pypy-svn] r25371 - in pypy/dist/pypy: objspace/flow rpython/lltypesystem rpython/memory rpython/memory/test translator translator/backendopt translator/backendopt/test translator/c translator/c/src translator/c/test Message-ID: <20060405113729.1539A101BF@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:37:28 2006 New Revision: 25371 Added: pypy/dist/pypy/objspace/flow/model.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/objspace/flow/model.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/memory/gctransform.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/rpython/memory/test/test_gctransform.py pypy/dist/pypy/translator/backendopt/inline.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/test/test_inline.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/backendopt/test/test_inline.py pypy/dist/pypy/translator/c/database.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/database.py pypy/dist/pypy/translator/c/exceptiontransform.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/exceptiontransform.py pypy/dist/pypy/translator/c/extfunc.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/extfunc.py pypy/dist/pypy/translator/c/funcgen.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/funcgen.py pypy/dist/pypy/translator/c/gc.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/gc.py pypy/dist/pypy/translator/c/src/address.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/address.h pypy/dist/pypy/translator/c/src/char.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/char.h pypy/dist/pypy/translator/c/src/exception.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/exception.h pypy/dist/pypy/translator/c/src/float.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/float.h pypy/dist/pypy/translator/c/src/int.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/int.h pypy/dist/pypy/translator/c/src/ll__socket.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/ll__socket.h pypy/dist/pypy/translator/c/src/ll_stackless.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/ll_stackless.h pypy/dist/pypy/translator/c/src/main.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/main.h pypy/dist/pypy/translator/c/src/mem.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/mem.h pypy/dist/pypy/translator/c/src/pyobj.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/pyobj.h pypy/dist/pypy/translator/c/src/support.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/support.h pypy/dist/pypy/translator/c/src/trace.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/trace.h pypy/dist/pypy/translator/c/src/unichar.h - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/src/unichar.h pypy/dist/pypy/translator/c/stackless.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/stackless.py pypy/dist/pypy/translator/c/test/test_exceptiontransform.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/test/test_exceptiontransform.py pypy/dist/pypy/translator/c/test/test_genc.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/test/test_genc.py pypy/dist/pypy/translator/c/test/test_newgc.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/test/test_newgc.py pypy/dist/pypy/translator/c/test/test_typed.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/test/test_typed.py pypy/dist/pypy/translator/c/wrapper.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/c/wrapper.py pypy/dist/pypy/translator/simplify.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/simplify.py pypy/dist/pypy/translator/unsimplify.py - copied unchanged from r25365, pypy/branch/explicit-exceptions/translator/unsimplify.py Removed: pypy/dist/pypy/translator/c/test/test_annotated.py pypy/dist/pypy/translator/c/test/test_notype.py pypy/dist/pypy/translator/c/test/test_operation.py Log: merge the explicit-exceptions branch, perhaps From mwh at codespeak.net Wed Apr 5 13:40:18 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 13:40:18 +0200 (CEST) Subject: [pypy-svn] r25372 - pypy/branch/explicit-exceptions Message-ID: <20060405114018.181D5101C3@code0.codespeak.net> Author: mwh Date: Wed Apr 5 13:40:17 2006 New Revision: 25372 Removed: pypy/branch/explicit-exceptions/ Log: delete the merged explicit-exceptions branch From hpk at codespeak.net Wed Apr 5 13:41:39 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 5 Apr 2006 13:41:39 +0200 (CEST) Subject: [pypy-svn] r25373 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060405114139.58975101C7@code0.codespeak.net> Author: hpk Date: Wed Apr 5 13:41:39 2006 New Revision: 25373 Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py Log: add a missing gc operation Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Wed Apr 5 13:41:39 2006 @@ -284,6 +284,7 @@ 'gc_pop_alive_pyobj': LLOp(), 'gc_protect': LLOp(), 'gc_unprotect': LLOp(), + 'gc_reload_possibly_moved': LLOp(), # __________ stackless operation(s) __________ From ericvrp at codespeak.net Wed Apr 5 13:48:41 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 13:48:41 +0200 (CEST) Subject: [pypy-svn] r25374 - pypy/dist/pypy/translator/llvm/pyllvm Message-ID: <20060405114841.58932101CC@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 13:48:40 2006 New Revision: 25374 Removed: pypy/dist/pypy/translator/llvm/pyllvm/ Log: Remove 'old' CPython extension From ericvrp at codespeak.net Wed Apr 5 13:58:08 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 13:58:08 +0200 (CEST) Subject: [pypy-svn] r25375 - in pypy/dist/pypy/translator/llvm/pyllvm: . test Message-ID: <20060405115808.E19C8101D8@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 13:57:58 2006 New Revision: 25375 Added: pypy/dist/pypy/translator/llvm/pyllvm/ pypy/dist/pypy/translator/llvm/pyllvm/__init__.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/autopath.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/cc.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/create_llvmcapi.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/helper.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/pyllvm.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/test/ pypy/dist/pypy/translator/llvm/pyllvm/test/__init__.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/test/addnumbers.c pypy/dist/pypy/translator/llvm/pyllvm/test/addnumbers.s pypy/dist/pypy/translator/llvm/pyllvm/test/hello.c pypy/dist/pypy/translator/llvm/pyllvm/test/hello.s pypy/dist/pypy/translator/llvm/pyllvm/test/ll_snippet.py (contents, props changed) pypy/dist/pypy/translator/llvm/pyllvm/test/test_ee.py (contents, props changed) Log: Add the new ctypes based pyllvm again. Using svn --force is not adviseable as I find out the hard way. Added: pypy/dist/pypy/translator/llvm/pyllvm/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/llvm/pyllvm/autopath.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/autopath.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,120 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories don't have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.realpath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.realpath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + break + else: + raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) + + checkpaths = sys.path[:] + pypy_root = os.path.join(head, '') + + while checkpaths: + orig = checkpaths.pop() + fullorig = os.path.join(os.path.realpath(orig), '') + if fullorig.startswith(pypy_root): + if os.path.exists(os.path.join(fullorig, '__init__.py')): + sys.path.remove(orig) + try: + sys.path.remove(head) + except ValueError: + pass + sys.path.insert(0, head) + + munged = {} + for name, mod in sys.modules.items(): + fn = getattr(mod, '__file__', None) + if '.' in name or not isinstance(fn, str): + continue + newname = os.path.splitext(os.path.basename(fn))[0] + if not newname.startswith(part + '.'): + continue + path = os.path.join(os.path.dirname(os.path.realpath(fn)), '') + if path.startswith(pypy_root) and newname != part: + modpaths = os.path.normpath(path[len(pypy_root):]).split(os.sep) + if newname != '__init__': + modpaths.append(newname) + modpath = '.'.join(modpaths) + if modpath not in sys.modules: + munged[modpath] = mod + + for name, mod in munged.iteritems(): + if name not in sys.modules: + sys.modules[name] = mod + if '.' in name: + prename = name[:name.rfind('.')] + postname = name[len(prename)+1:] + if prename not in sys.modules: + __import__(prename) + if not hasattr(sys.modules[prename], postname): + setattr(sys.modules[prename], postname, mod) + + return partdir, this_dir + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Added: pypy/dist/pypy/translator/llvm/pyllvm/cc.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/cc.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,21 @@ +''' +Added this because ctypes on my computer was missing cdecl. +''' +from ctypes import * + +class cdecl(object): + def __init__(self, restype, libname, argtypes): + d = __file__[:__file__.find("pyllvm")] + "llvmcapi/" + #XXX does this load once or every time? + try: + self.library = cdll.load(d + libname + ".so") + except: + raise Exception("llvmcapi not found: run 'python setup.py build_ext -i' in " + d) + self.restype = restype + self.argtypes = argtypes + + def __call__(self, func): + func._api_ = getattr(self.library, func.__name__) + func._api_.restype = self.restype + func._api_.argtypes = self.argtypes + return func Added: pypy/dist/pypy/translator/llvm/pyllvm/create_llvmcapi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/create_llvmcapi.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,18 @@ +#!/usr/bin/env python +import os + +WRAPDIR = "/Users/eric/projects/ctypes/ctypes/wrap" #XXX get rid of this hardcoded path + +if os.path.exists(WRAPDIR + "/h2xml.py"): + print "h2xml.py" + os.system("python " + WRAPDIR + "/h2xml.py ../llvmcapi/include/llvmcapi.h -q -I. -I/opt/local/include -D_GNU_SOURCE -D__STDC_LIMIT_MACROS -o llvmcapi.xml") + print "xml2py.py" + os.system("python " + WRAPDIR + "/xml2py.py llvmcapi.xml -l ../llvmcapi/llvmcapi.so -o llvmcapi.tmp1") + print "massage output" + os.system("sed -e s/from\ ctypes\ import/from\ cc\ import/ llvmcapi.tmp1 > llvmcapi.tmp2") + os.system("sed -e s/assert\ alignment.*// llvmcapi.tmp2 > llvmcapi.py") + print "cleanup some files" + os.system("rm -f llvmcapi.tmp* llvmcapi.xml") + print "created llvmcapi.py" +else: + print "skipping llvmcapi.py creation" Added: pypy/dist/pypy/translator/llvm/pyllvm/helper.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/helper.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,83 @@ +''' +Added this because ctypes on my computer was missing cdecl. +''' +from llvmcapi import * + +class Method(object): + def __init__(self, instance, method): + self.instance = instance + self.method = method + + def __call__(self, *args): + a = [self.instance] + for arg in args: + if isinstance(arg, Wrapper): #pass on value to actual C (not Python) object + a.append(arg.instance) + else: + a.append(arg) + return apply(self.method, a) + +class Wrapper(object): + def __getattr__(self, name): + global_funcname = self.__class__.__name__ + "_" + name + return Method(self.instance, globals()[global_funcname]) + + +def to_llvm_value(pythonvalue, type_): + value = GenericValue_() + value.LongVal = 0 + id_ = type_.getTypeID() + if id_ == PointerTyID and type_.getContainedType(0).getTypeID() == SByteTyID: + value.PointerVal = pointer(pythonvalue) + elif id_ == VoidTyID: + pass + elif id_ == BoolTyID: + value.BoolVal = pythonvalue + elif id_ == UByteTyID: + value.UByteVal = pythonvalue + elif id_ == SByteTyID: + value.SByteVal = pythonvalue + elif id_ == UShortTyID: + value.UShortVal = pythonvalue + elif id_ == ShortTyID: + value.ShortVal = pythonvalue + elif id_ == UIntTyID: + value.UIntVal = pythonvalue + elif id_ == IntTyID: + value.IntVal = pythonvalue + elif id_ == ULongTyID: + value.ULongVal = pythonvalue + elif id_ == LongTyID: + value.ULongVal = pythonvalue + else: + raise Exception("don't know how to convert pythontype '%s' to llvm" % type_.getDescription()) + return value + + +def to_python_value(llvmvalue, type_): + value = GenericValue_() + value.LongVal = llvmvalue # XXX convert llvmvalue from long long + id_ = type_.getTypeID() + if id_ == PointerTyID and type_.getContainedType(0).getTypeID() == SByteTyID: + return STRING(value.PointerVal).value + elif id_ == VoidTyID: + return None + elif id_ == BoolTyID: + return value.BoolVal + elif id_ == UByteTyID: + return value.UByteVal + elif id_ == SByteTyID: + return value.SByteVal + elif id_ == UShortTyID: + return value.UShortVal + elif id_ == ShortTyID: + return value.ShortVal + elif id_ == UIntTyID: + return value.UIntVal + elif id_ == IntTyID: + return value.IntVal + elif id_ == ULongTyID: + return value.ULongVal + elif id_ == LongTyID: + return value.ULongVal + raise Exception("don't know how to convert llvmtype '%s' to python" % type_.getDescription()) Added: pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/llvmcapi.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,225 @@ +# generated by 'xml2py' +# flags 'llvmcapi.xml -l ../llvmcapi/llvmcapi.so -o llvmcapi.tmp1' +from cc import * +STRING = c_char_p + + +UShortTyID = 4 +FirstDerivedTyID = 13 +FunctionTyID = 13 +ShortTyID = 5 +LabelTyID = 12 +UIntTyID = 6 +ArrayTyID = 15 +IntTyID = 7 +PointerTyID = 16 +ULongTyID = 8 +LongTyID = 9 +BoolTyID = 1 +StructTyID = 14 +PackedTyID = 18 +FloatTyID = 10 +NumTypeIDs = 19 +UByteTyID = 2 +VoidTyID = 0 +SByteTyID = 3 +OpaqueTyID = 17 +LastPrimitiveTyID = 12 +DoubleTyID = 11 + +def ExecutionEngine__create__(MP, ForceInterpreter): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 12 + return ExecutionEngine__create__._api_(MP, ForceInterpreter) +ExecutionEngine__create__ = cdecl(c_void_p, 'llvmcapi', [c_void_p, c_int]) (ExecutionEngine__create__) + + +def ExecutionEngine_getModule(EE): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 13 + return ExecutionEngine_getModule._api_(EE) +ExecutionEngine_getModule = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (ExecutionEngine_getModule) + + +def ExecutionEngine_freeMachineCodeForFunction(EE, F): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 14 + return ExecutionEngine_freeMachineCodeForFunction._api_(EE, F) +ExecutionEngine_freeMachineCodeForFunction = cdecl(None, 'llvmcapi', [c_void_p, c_void_p]) (ExecutionEngine_freeMachineCodeForFunction) + + +def ExecutionEngine_runFunction(EE, F, A): + # ../llvmcapi/include/ExecutionEngine/ExecutionEngine.h 17 + return ExecutionEngine_runFunction._api_(EE, F, A) +ExecutionEngine_runFunction = cdecl(c_longlong, 'llvmcapi', [c_void_p, c_void_p, c_void_p]) (ExecutionEngine_runFunction) + +# ../llvmcapi/include/ExecutionEngine/GenericValue.h 20 +class GenericValue_(Union): + pass +# ../llvmcapi/include/ExecutionEngine/GenericValue.h 32 +class N13GenericValue_3DOLLAR_0E(Structure): + pass +N13GenericValue_3DOLLAR_0E._fields_ = [ + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 32 + ('first', c_uint), + ('second', c_uint), +] +assert sizeof(N13GenericValue_3DOLLAR_0E) == 8, sizeof(N13GenericValue_3DOLLAR_0E) + +GenericValue_._fields_ = [ + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 20 + ('BoolVal', c_byte), + ('UByteVal', c_ubyte), + ('SByteVal', c_byte), + ('UShortVal', c_ushort), + ('ShortVal', c_short), + ('UIntVal', c_uint), + ('IntVal', c_int), + ('ULongVal', c_ulonglong), + ('LongVal', c_longlong), + ('DoubleVal', c_double), + ('FloatVal', c_float), + ('UIntPairVal', N13GenericValue_3DOLLAR_0E), + ('PointerVal', c_void_p), + ('Untyped', c_ubyte * 8), +] +assert sizeof(GenericValue_) == 8, sizeof(GenericValue_) + + +def GenericValue__init__(): + # ../llvmcapi/include/ExecutionEngine/GenericValue.h 37 + return GenericValue__init__._api_() +GenericValue__init__ = cdecl(c_void_p, 'llvmcapi', []) (GenericValue__init__) + + +def FunctionType_getNumParams(FT): + # ../llvmcapi/include/VMCore/DerivedTypes.h 8 + return FunctionType_getNumParams._api_(FT) +FunctionType_getNumParams = cdecl(c_int, 'llvmcapi', [c_void_p]) (FunctionType_getNumParams) + + +def FunctionType_getParamType(FT, i): + # ../llvmcapi/include/VMCore/DerivedTypes.h 9 + return FunctionType_getParamType._api_(FT, i) +FunctionType_getParamType = cdecl(c_void_p, 'llvmcapi', [c_void_p, c_int]) (FunctionType_getParamType) + + +def FunctionType_getReturnType(FT): + # ../llvmcapi/include/VMCore/DerivedTypes.h 10 + return FunctionType_getReturnType._api_(FT) +FunctionType_getReturnType = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (FunctionType_getReturnType) + + +def Function_eraseFromParent(F): + # ../llvmcapi/include/VMCore/Function.h 8 + return Function_eraseFromParent._api_(F) +Function_eraseFromParent = cdecl(None, 'llvmcapi', [c_void_p]) (Function_eraseFromParent) + + +def Function_getFunctionType(F): + # ../llvmcapi/include/VMCore/Function.h 9 + return Function_getFunctionType._api_(F) +Function_getFunctionType = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (Function_getFunctionType) + + +def Module__init__(ModuleID): + # ../llvmcapi/include/VMCore/Module.h 8 + return Module__init__._api_(ModuleID) +Module__init__ = cdecl(c_void_p, 'llvmcapi', [STRING]) (Module__init__) + + +def Module_getModuleIdentifier(M): + # ../llvmcapi/include/VMCore/Module.h 9 + return Module_getModuleIdentifier._api_(M) +Module_getModuleIdentifier = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getModuleIdentifier) + + +def Module_setModuleIdentifier(M, ID): + # ../llvmcapi/include/VMCore/Module.h 10 + return Module_setModuleIdentifier._api_(M, ID) +Module_setModuleIdentifier = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setModuleIdentifier) + + +def Module_getTargetTriple(M): + # ../llvmcapi/include/VMCore/Module.h 11 + return Module_getTargetTriple._api_(M) +Module_getTargetTriple = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getTargetTriple) + + +def Module_setTargetTriple(M, T): + # ../llvmcapi/include/VMCore/Module.h 12 + return Module_setTargetTriple._api_(M, T) +Module_setTargetTriple = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setTargetTriple) + + +def Module_getModuleInlineAsm(M): + # ../llvmcapi/include/VMCore/Module.h 13 + return Module_getModuleInlineAsm._api_(M) +Module_getModuleInlineAsm = cdecl(STRING, 'llvmcapi', [c_void_p]) (Module_getModuleInlineAsm) + + +def Module_setModuleInlineAsm(M, Asm): + # ../llvmcapi/include/VMCore/Module.h 14 + return Module_setModuleInlineAsm._api_(M, Asm) +Module_setModuleInlineAsm = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_setModuleInlineAsm) + + +def Module_getNamedFunction(M, fnname): + # ../llvmcapi/include/VMCore/Module.h 15 + return Module_getNamedFunction._api_(M, fnname) +Module_getNamedFunction = cdecl(c_void_p, 'llvmcapi', [c_void_p, STRING]) (Module_getNamedFunction) + + +def Module_ParseAssemblyString(M, AsmString): + # ../llvmcapi/include/VMCore/Module.h 18 + return Module_ParseAssemblyString._api_(M, AsmString) +Module_ParseAssemblyString = cdecl(None, 'llvmcapi', [c_void_p, STRING]) (Module_ParseAssemblyString) + + +def Module_verifyModule(M): + # ../llvmcapi/include/VMCore/Module.h 19 + return Module_verifyModule._api_(M) +Module_verifyModule = cdecl(c_int, 'llvmcapi', [c_void_p]) (Module_verifyModule) + + +def Module_n_functions(M): + # ../llvmcapi/include/VMCore/Module.h 22 + return Module_n_functions._api_(M) +Module_n_functions = cdecl(c_int, 'llvmcapi', [c_void_p]) (Module_n_functions) + + +def Module_function_exists(M, fnname): + # ../llvmcapi/include/VMCore/Module.h 23 + return Module_function_exists._api_(M, fnname) +Module_function_exists = cdecl(c_int, 'llvmcapi', [c_void_p, STRING]) (Module_function_exists) + + +def ExistingModuleProvider__init__(M): + # ../llvmcapi/include/VMCore/ModuleProvider.h 8 + return ExistingModuleProvider__init__._api_(M) +ExistingModuleProvider__init__ = cdecl(c_void_p, 'llvmcapi', [c_void_p]) (ExistingModuleProvider__init__) + + +# values for enumeration 'TypeID' +TypeID = c_int # enum + +def Type_getTypeID(T): + # ../llvmcapi/include/VMCore/Type.h 32 + return Type_getTypeID._api_(T) +Type_getTypeID = cdecl(TypeID, 'llvmcapi', [c_void_p]) (Type_getTypeID) + + +def Type_getContainedType(T, n): + # ../llvmcapi/include/VMCore/Type.h 33 + return Type_getContainedType._api_(T, n) +Type_getContainedType = cdecl(c_void_p, 'llvmcapi', [c_void_p, c_int]) (Type_getContainedType) + + +def Type_getDescription(T): + # ../llvmcapi/include/VMCore/Type.h 34 + return Type_getDescription._api_(T) +Type_getDescription = cdecl(STRING, 'llvmcapi', [c_void_p]) (Type_getDescription) + + +def toggle_print_machineinstrs(): + # ../llvmcapi/include/llvmcapi.h 16 + return toggle_print_machineinstrs._api_() +toggle_print_machineinstrs = cdecl(None, 'llvmcapi', []) (toggle_print_machineinstrs) + Added: pypy/dist/pypy/translator/llvm/pyllvm/pyllvm.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/pyllvm.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,101 @@ +from helper import * + + +class Module(Wrapper): + def __init__(self, ModuleID='mymodule'): + self.instance = Module__init__(ModuleID) + + def getNamedFunction(self, fnname): + f = object.__new__(Function) + f.instance = Module_getNamedFunction(self.instance, fnname) + return f + + +class ExistingModuleProvider(Wrapper): + def __init__(self, M=None): + if not M: + M = Module() + self.instance = ExistingModuleProvider__init__(M.instance) + +global ee_hack + +class ExecutionEngine(Wrapper): + def __init__(self, MP=None, ForceInterpreter=False): + if not MP: + MP = ExistingModuleProvider(); + self.instance = ExecutionEngine__create__(MP.instance, ForceInterpreter) + global ee_hack + ee_hack = self.instance #XXX how to get to the executionengine from a function? + + # XXX cast to actual Python Module (can't we automate this?) + def getModule(self): + m = object.__new__(Module) + m.instance = ExecutionEngine_getModule(self.instance) + return m + + # helpers + def parse(self, llcode, fnname=None): + mod = self.getModule() + mod.ParseAssemblyString(llcode) + mod.verifyModule() + + def delete(self, fnname): + mod = self.getModule() + f = mod.getNamedFunction(fnname) # XXX handle fnname not found? + self.freeMachineCodeForFunction(f) # still no-op on march 27th 2006 + f.eraseFromParent() + + +class Function(Wrapper): + def __init__(self): + self.instance = Function__init__() #XXX this get annoying quickly + + def __call__(self, *args): + #print 'calling %s(%s)' % ('Function', ','.join([str(arg) for arg in args])) + ft = self.getFunctionType() + argcount = ft.getNumParams() + #print 'argcount = ',argcounT + if argcount != len(args): + raise Exception("incorrect number of parameters") + ParamType = c_longlong * argcount + llvmvalues = ParamType() + for i, arg in enumerate(args): + llvmvalues[i] = to_llvm_value(arg, ft.getParamType(i)).LongVal + pLlvmValues = cast(llvmvalues, c_void_p) #would like to cast to c_longlong_p + llvmreturnvalue = ExecutionEngine_runFunction(ee_hack, self.instance, pLlvmValues) + return to_python_value(llvmreturnvalue, ft.getReturnType()) + + def getFunctionType(self): + ft = object.__new__(FunctionType) + ft.instance = Function_getFunctionType(self.instance) + return ft + + +class GenericValue(Wrapper): + def __init(self): + self.instance = GenericValue__init__() + + +class Type(Wrapper): + def __init(self): + self.instance = Type__init__() + + def getContainedType(self, n): + t = object.__new__(Type) + t.instance = Type_getContainedType(self.instance, n) + return t + + +class FunctionType(Wrapper): + def __init(self): + self.instance = FunctionType__init__() + + def getReturnType(self): + t = object.__new__(Type) + t.instance = FunctionType_getReturnType(self.instance) + return t + + def getParamType(self, i): + t = object.__new__(Type) + t.instance = FunctionType_getParamType(self.instance, i) + return t Added: pypy/dist/pypy/translator/llvm/pyllvm/test/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/llvm/pyllvm/test/addnumbers.c ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/test/addnumbers.c Wed Apr 5 13:57:58 2006 @@ -0,0 +1,4 @@ +int add(int n, int y) { + return n + y; +} + Added: pypy/dist/pypy/translator/llvm/pyllvm/test/addnumbers.s ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/test/addnumbers.s Wed Apr 5 13:57:58 2006 @@ -0,0 +1,441 @@ +; GNU C version 3.4-llvm 20051104 (LLVM 1.6) (i686-pc-linux-gnu) +; compiled by GNU C version 3.4.0. +; GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096 +; options passed: -iprefix -mtune=pentiumpro -auxbase +; options enabled: -feliminate-unused-debug-types -fpeephole +; -ffunction-cse -fkeep-static-consts -fpcc-struct-return -fgcse-lm +; -fgcse-sm -fsched-interblock -fsched-spec -fbranch-count-reg -fcommon +; -fgnu-linker -fargument-alias -fzero-initialized-in-bss -fident +; -fmath-errno -ftrapping-math -m80387 -mhard-float -mno-soft-float +; -mieee-fp -mfp-ret-in-387 -maccumulate-outgoing-args -mno-red-zone +; -mtls-direct-seg-refs -mtune=pentiumpro -march=i386 + +target triple = "i686-pc-linux-gnu" +target pointersize = 32 +target endian = little +deplibs = ["c", "crtend"] + +"complex double" = type { double, double } +"complex float" = type { float, float } +"complex long double" = type { double, double } + +implementation +declare double %acos(double) ;; __builtin_acos +declare float %acosf(float) ;; __builtin_acosf +declare double %acosh(double) ;; __builtin_acosh +declare float %acoshf(float) ;; __builtin_acoshf +declare double %acoshl(double) ;; __builtin_acoshl +declare double %acosl(double) ;; __builtin_acosl +declare double %asin(double) ;; __builtin_asin +declare float %asinf(float) ;; __builtin_asinf +declare double %asinh(double) ;; __builtin_asinh +declare float %asinhf(float) ;; __builtin_asinhf +declare double %asinhl(double) ;; __builtin_asinhl +declare double %asinl(double) ;; __builtin_asinl +declare double %atan(double) ;; __builtin_atan +declare double %atan2(double,double) ;; __builtin_atan2 +declare float %atan2f(float,float) ;; __builtin_atan2f +declare double %atan2l(double,double) ;; __builtin_atan2l +declare float %atanf(float) ;; __builtin_atanf +declare double %atanh(double) ;; __builtin_atanh +declare float %atanhf(float) ;; __builtin_atanhf +declare double %atanhl(double) ;; __builtin_atanhl +declare double %atanl(double) ;; __builtin_atanl +declare double %cbrt(double) ;; __builtin_cbrt +declare float %cbrtf(float) ;; __builtin_cbrtf +declare double %cbrtl(double) ;; __builtin_cbrtl +declare double %ceil(double) ;; __builtin_ceil +declare float %ceilf(float) ;; __builtin_ceilf +declare double %ceill(double) ;; __builtin_ceill +declare double %copysign(double,double) ;; __builtin_copysign +declare float %copysignf(float,float) ;; __builtin_copysignf +declare double %copysignl(double,double) ;; __builtin_copysignl +declare double %cos(double) ;; __builtin_cos +declare float %cosf(float) ;; __builtin_cosf +declare double %cosh(double) ;; __builtin_cosh +declare float %coshf(float) ;; __builtin_coshf +declare double %coshl(double) ;; __builtin_coshl +declare double %cosl(double) ;; __builtin_cosl +declare double %drem(double,double) ;; __builtin_drem +declare float %dremf(float,float) ;; __builtin_dremf +declare double %dreml(double,double) ;; __builtin_dreml +declare double %erf(double) ;; __builtin_erf +declare double %erfc(double) ;; __builtin_erfc +declare float %erfcf(float) ;; __builtin_erfcf +declare double %erfcl(double) ;; __builtin_erfcl +declare float %erff(float) ;; __builtin_erff +declare double %erfl(double) ;; __builtin_erfl +declare double %exp(double) ;; __builtin_exp +declare double %exp10(double) ;; __builtin_exp10 +declare float %exp10f(float) ;; __builtin_exp10f +declare double %exp10l(double) ;; __builtin_exp10l +declare double %exp2(double) ;; __builtin_exp2 +declare float %exp2f(float) ;; __builtin_exp2f +declare double %exp2l(double) ;; __builtin_exp2l +declare float %expf(float) ;; __builtin_expf +declare double %expl(double) ;; __builtin_expl +declare double %expm1(double) ;; __builtin_expm1 +declare float %expm1f(float) ;; __builtin_expm1f +declare double %expm1l(double) ;; __builtin_expm1l +declare double %fabs(double) ;; __builtin_fabs +declare float %fabsf(float) ;; __builtin_fabsf +declare double %fabsl(double) ;; __builtin_fabsl +declare double %fdim(double,double) ;; __builtin_fdim +declare float %fdimf(float,float) ;; __builtin_fdimf +declare double %fdiml(double,double) ;; __builtin_fdiml +declare double %floor(double) ;; __builtin_floor +declare float %floorf(float) ;; __builtin_floorf +declare double %floorl(double) ;; __builtin_floorl +declare double %fma(double,double,double) ;; __builtin_fma +declare float %fmaf(float,float,float) ;; __builtin_fmaf +declare double %fmal(double,double,double) ;; __builtin_fmal +declare double %fmax(double,double) ;; __builtin_fmax +declare float %fmaxf(float,float) ;; __builtin_fmaxf +declare double %fmaxl(double,double) ;; __builtin_fmaxl +declare double %fmin(double,double) ;; __builtin_fmin +declare float %fminf(float,float) ;; __builtin_fminf +declare double %fminl(double,double) ;; __builtin_fminl +declare double %fmod(double,double) ;; __builtin_fmod +declare float %fmodf(float,float) ;; __builtin_fmodf +declare double %fmodl(double,double) ;; __builtin_fmodl +declare double %frexp(double,int*) ;; __builtin_frexp +declare float %frexpf(float,int*) ;; __builtin_frexpf +declare double %frexpl(double,int*) ;; __builtin_frexpl +declare double %gamma(double) ;; __builtin_gamma +declare float %gammaf(float) ;; __builtin_gammaf +declare double %gammal(double) ;; __builtin_gammal +declare double %__builtin_huge_val() +declare float %__builtin_huge_valf() +declare double %__builtin_huge_vall() +declare double %hypot(double,double) ;; __builtin_hypot +declare float %hypotf(float,float) ;; __builtin_hypotf +declare double %hypotl(double,double) ;; __builtin_hypotl +declare int %ilogb(double) ;; __builtin_ilogb +declare int %ilogbf(float) ;; __builtin_ilogbf +declare int %ilogbl(double) ;; __builtin_ilogbl +declare double %__builtin_inf() +declare float %__builtin_inff() +declare double %__builtin_infl() +declare double %j0(double) ;; __builtin_j0 +declare float %j0f(float) ;; __builtin_j0f +declare double %j0l(double) ;; __builtin_j0l +declare double %j1(double) ;; __builtin_j1 +declare float %j1f(float) ;; __builtin_j1f +declare double %j1l(double) ;; __builtin_j1l +declare double %jn(int,double) ;; __builtin_jn +declare float %jnf(int,float) ;; __builtin_jnf +declare double %jnl(int,double) ;; __builtin_jnl +declare double %ldexp(double,int) ;; __builtin_ldexp +declare float %ldexpf(float,int) ;; __builtin_ldexpf +declare double %ldexpl(double,int) ;; __builtin_ldexpl +declare double %lgamma(double) ;; __builtin_lgamma +declare float %lgammaf(float) ;; __builtin_lgammaf +declare double %lgammal(double) ;; __builtin_lgammal +declare long %llrint(double) ;; __builtin_llrint +declare long %llrintf(float) ;; __builtin_llrintf +declare long %llrintl(double) ;; __builtin_llrintl +declare long %llround(double) ;; __builtin_llround +declare long %llroundf(float) ;; __builtin_llroundf +declare long %llroundl(double) ;; __builtin_llroundl +declare double %log(double) ;; __builtin_log +declare double %log10(double) ;; __builtin_log10 +declare float %log10f(float) ;; __builtin_log10f +declare double %log10l(double) ;; __builtin_log10l +declare double %log1p(double) ;; __builtin_log1p +declare float %log1pf(float) ;; __builtin_log1pf +declare double %log1pl(double) ;; __builtin_log1pl +declare double %log2(double) ;; __builtin_log2 +declare float %log2f(float) ;; __builtin_log2f +declare double %log2l(double) ;; __builtin_log2l +declare double %logb(double) ;; __builtin_logb +declare float %logbf(float) ;; __builtin_logbf +declare double %logbl(double) ;; __builtin_logbl +declare float %logf(float) ;; __builtin_logf +declare double %logl(double) ;; __builtin_logl +declare int %lrint(double) ;; __builtin_lrint +declare int %lrintf(float) ;; __builtin_lrintf +declare int %lrintl(double) ;; __builtin_lrintl +declare int %lround(double) ;; __builtin_lround +declare int %lroundf(float) ;; __builtin_lroundf +declare int %lroundl(double) ;; __builtin_lroundl +declare double %modf(double,double*) ;; __builtin_modf +declare float %modff(float,float*) ;; __builtin_modff +declare double %modfl(double,double*) ;; __builtin_modfl +declare double %nan(sbyte*) ;; __builtin_nan +declare float %nanf(sbyte*) ;; __builtin_nanf +declare double %nanl(sbyte*) ;; __builtin_nanl +declare double %nans(sbyte*) ;; __builtin_nans +declare float %nansf(sbyte*) ;; __builtin_nansf +declare double %nansl(sbyte*) ;; __builtin_nansl +declare double %nearbyint(double) ;; __builtin_nearbyint +declare float %nearbyintf(float) ;; __builtin_nearbyintf +declare double %nearbyintl(double) ;; __builtin_nearbyintl +declare double %nextafter(double,double) ;; __builtin_nextafter +declare float %nextafterf(float,float) ;; __builtin_nextafterf +declare double %nextafterl(double,double) ;; __builtin_nextafterl +declare double %nexttoward(double,double) ;; __builtin_nexttoward +declare float %nexttowardf(float,double) ;; __builtin_nexttowardf +declare double %nexttowardl(double,double) ;; __builtin_nexttowardl +declare double %pow(double,double) ;; __builtin_pow +declare double %pow10(double) ;; __builtin_pow10 +declare float %pow10f(float) ;; __builtin_pow10f +declare double %pow10l(double) ;; __builtin_pow10l +declare float %powf(float,float) ;; __builtin_powf +declare double %powl(double,double) ;; __builtin_powl +declare double %remainder(double,double) ;; __builtin_remainder +declare float %remainderf(float,float) ;; __builtin_remainderf +declare double %remainderl(double,double) ;; __builtin_remainderl +declare double %remquo(double,double,int*) ;; __builtin_remquo +declare float %remquof(float,float,int*) ;; __builtin_remquof +declare double %remquol(double,double,int*) ;; __builtin_remquol +declare double %rint(double) ;; __builtin_rint +declare float %rintf(float) ;; __builtin_rintf +declare double %rintl(double) ;; __builtin_rintl +declare double %round(double) ;; __builtin_round +declare float %roundf(float) ;; __builtin_roundf +declare double %roundl(double) ;; __builtin_roundl +declare double %scalb(double,double) ;; __builtin_scalb +declare float %scalbf(float,float) ;; __builtin_scalbf +declare double %scalbl(double,double) ;; __builtin_scalbl +declare double %scalbln(double,int) ;; __builtin_scalbln +declare float %scalblnf(float,int) ;; __builtin_scalblnf +declare double %scalblnl(double,int) ;; __builtin_scalblnl +declare double %scalbn(double,int) ;; __builtin_scalbn +declare float %scalbnf(float,int) ;; __builtin_scalbnf +declare double %scalbnl(double,int) ;; __builtin_scalbnl +declare double %significand(double) ;; __builtin_significand +declare float %significandf(float) ;; __builtin_significandf +declare double %significandl(double) ;; __builtin_significandl +declare double %sin(double) ;; __builtin_sin +declare void %sincos(double,double*,double*) ;; __builtin_sincos +declare void %sincosf(float,float*,float*) ;; __builtin_sincosf +declare void %sincosl(double,double*,double*) ;; __builtin_sincosl +declare float %sinf(float) ;; __builtin_sinf +declare double %sinh(double) ;; __builtin_sinh +declare float %sinhf(float) ;; __builtin_sinhf +declare double %sinhl(double) ;; __builtin_sinhl +declare double %sinl(double) ;; __builtin_sinl +declare double %sqrt(double) ;; __builtin_sqrt +declare float %sqrtf(float) ;; __builtin_sqrtf +declare double %sqrtl(double) ;; __builtin_sqrtl +declare double %tan(double) ;; __builtin_tan +declare float %tanf(float) ;; __builtin_tanf +declare double %tanh(double) ;; __builtin_tanh +declare float %tanhf(float) ;; __builtin_tanhf +declare double %tanhl(double) ;; __builtin_tanhl +declare double %tanl(double) ;; __builtin_tanl +declare double %tgamma(double) ;; __builtin_tgamma +declare float %tgammaf(float) ;; __builtin_tgammaf +declare double %tgammal(double) ;; __builtin_tgammal +declare double %trunc(double) ;; __builtin_trunc +declare float %truncf(float) ;; __builtin_truncf +declare double %truncl(double) ;; __builtin_truncl +declare double %y0(double) ;; __builtin_y0 +declare float %y0f(float) ;; __builtin_y0f +declare double %y0l(double) ;; __builtin_y0l +declare double %y1(double) ;; __builtin_y1 +declare float %y1f(float) ;; __builtin_y1f +declare double %y1l(double) ;; __builtin_y1l +declare double %yn(int,double) ;; __builtin_yn +declare float %ynf(int,float) ;; __builtin_ynf +declare double %ynl(int,double) ;; __builtin_ynl +declare double %cabs(double,double) ;; __builtin_cabs +declare float %cabsf(float,float) ;; __builtin_cabsf +declare double %cabsl(double,double) ;; __builtin_cabsl +declare void %cacos("complex double"*,double,double) ;; __builtin_cacos +declare void %cacosf("complex float"*,float,float) ;; __builtin_cacosf +declare void %cacosh("complex double"*,double,double) ;; __builtin_cacosh +declare void %cacoshf("complex float"*,float,float) ;; __builtin_cacoshf +declare void %cacoshl("complex long double"*,double,double) ;; __builtin_cacoshl +declare void %cacosl("complex long double"*,double,double) ;; __builtin_cacosl +declare double %carg(double,double) ;; __builtin_carg +declare float %cargf(float,float) ;; __builtin_cargf +declare double %cargl(double,double) ;; __builtin_cargl +declare void %casin("complex double"*,double,double) ;; __builtin_casin +declare void %casinf("complex float"*,float,float) ;; __builtin_casinf +declare void %casinh("complex double"*,double,double) ;; __builtin_casinh +declare void %casinhf("complex float"*,float,float) ;; __builtin_casinhf +declare void %casinhl("complex long double"*,double,double) ;; __builtin_casinhl +declare void %casinl("complex long double"*,double,double) ;; __builtin_casinl +declare void %catan("complex double"*,double,double) ;; __builtin_catan +declare void %catanf("complex float"*,float,float) ;; __builtin_catanf +declare void %catanh("complex double"*,double,double) ;; __builtin_catanh +declare void %catanhf("complex float"*,float,float) ;; __builtin_catanhf +declare void %catanhl("complex long double"*,double,double) ;; __builtin_catanhl +declare void %catanl("complex long double"*,double,double) ;; __builtin_catanl +declare void %ccos("complex double"*,double,double) ;; __builtin_ccos +declare void %ccosf("complex float"*,float,float) ;; __builtin_ccosf +declare void %ccosh("complex double"*,double,double) ;; __builtin_ccosh +declare void %ccoshf("complex float"*,float,float) ;; __builtin_ccoshf +declare void %ccoshl("complex long double"*,double,double) ;; __builtin_ccoshl +declare void %ccosl("complex long double"*,double,double) ;; __builtin_ccosl +declare void %cexp("complex double"*,double,double) ;; __builtin_cexp +declare void %cexpf("complex float"*,float,float) ;; __builtin_cexpf +declare void %cexpl("complex long double"*,double,double) ;; __builtin_cexpl +declare double %cimag(double,double) ;; __builtin_cimag +declare float %cimagf(float,float) ;; __builtin_cimagf +declare double %cimagl(double,double) ;; __builtin_cimagl +declare void %conj("complex double"*,double,double) ;; __builtin_conj +declare void %conjf("complex float"*,float,float) ;; __builtin_conjf +declare void %conjl("complex long double"*,double,double) ;; __builtin_conjl +declare void %cpow("complex double"*,double,double,double,double) ;; __builtin_cpow +declare void %cpowf("complex float"*,float,float,float,float) ;; __builtin_cpowf +declare void %cpowl("complex long double"*,double,double,double,double) ;; __builtin_cpowl +declare void %cproj("complex double"*,double,double) ;; __builtin_cproj +declare void %cprojf("complex float"*,float,float) ;; __builtin_cprojf +declare void %cprojl("complex long double"*,double,double) ;; __builtin_cprojl +declare double %creal(double,double) ;; __builtin_creal +declare float %crealf(float,float) ;; __builtin_crealf +declare double %creall(double,double) ;; __builtin_creall +declare void %csin("complex double"*,double,double) ;; __builtin_csin +declare void %csinf("complex float"*,float,float) ;; __builtin_csinf +declare void %csinh("complex double"*,double,double) ;; __builtin_csinh +declare void %csinhf("complex float"*,float,float) ;; __builtin_csinhf +declare void %csinhl("complex long double"*,double,double) ;; __builtin_csinhl +declare void %csinl("complex long double"*,double,double) ;; __builtin_csinl +declare void %csqrt("complex double"*,double,double) ;; __builtin_csqrt +declare void %csqrtf("complex float"*,float,float) ;; __builtin_csqrtf +declare void %csqrtl("complex long double"*,double,double) ;; __builtin_csqrtl +declare void %ctan("complex double"*,double,double) ;; __builtin_ctan +declare void %ctanf("complex float"*,float,float) ;; __builtin_ctanf +declare void %ctanh("complex double"*,double,double) ;; __builtin_ctanh +declare void %ctanhf("complex float"*,float,float) ;; __builtin_ctanhf +declare void %ctanhl("complex long double"*,double,double) ;; __builtin_ctanhl +declare void %ctanl("complex long double"*,double,double) ;; __builtin_ctanl +declare int %bcmp(sbyte*,sbyte*,uint) ;; __builtin_bcmp +declare void %bcopy(sbyte*,sbyte*,uint) ;; __builtin_bcopy +declare void %bzero(sbyte*,uint) ;; __builtin_bzero +declare int %ffs(int) ;; __builtin_ffs +declare int %ffsl(int) ;; __builtin_ffsl +declare int %ffsll(long) ;; __builtin_ffsll +declare sbyte* %index(sbyte*,int) ;; __builtin_index +declare int %memcmp(sbyte*,sbyte*,uint) ;; __builtin_memcmp +declare sbyte* %memcpy(sbyte*,sbyte*,uint) ;; __builtin_memcpy +declare sbyte* %memmove(sbyte*,sbyte*,uint) ;; __builtin_memmove +declare sbyte* %mempcpy(sbyte*,sbyte*,uint) ;; __builtin_mempcpy +declare sbyte* %memset(sbyte*,int,uint) ;; __builtin_memset +declare sbyte* %rindex(sbyte*,int) ;; __builtin_rindex +declare sbyte* %stpcpy(sbyte*,sbyte*) ;; __builtin_stpcpy +declare sbyte* %strcat(sbyte*,sbyte*) ;; __builtin_strcat +declare sbyte* %strchr(sbyte*,int) ;; __builtin_strchr +declare int %strcmp(sbyte*,sbyte*) ;; __builtin_strcmp +declare sbyte* %strcpy(sbyte*,sbyte*) ;; __builtin_strcpy +declare uint %strcspn(sbyte*,sbyte*) ;; __builtin_strcspn +declare sbyte* %strdup(sbyte*) ;; __builtin_strdup +declare uint %strlen(sbyte*) ;; __builtin_strlen +declare sbyte* %strncat(sbyte*,sbyte*,uint) ;; __builtin_strncat +declare int %strncmp(sbyte*,sbyte*,uint) ;; __builtin_strncmp +declare sbyte* %strncpy(sbyte*,sbyte*,uint) ;; __builtin_strncpy +declare sbyte* %strpbrk(sbyte*,sbyte*) ;; __builtin_strpbrk +declare sbyte* %strrchr(sbyte*,int) ;; __builtin_strrchr +declare uint %strspn(sbyte*,sbyte*) ;; __builtin_strspn +declare sbyte* %strstr(sbyte*,sbyte*) ;; __builtin_strstr +declare int %fprintf(sbyte*,sbyte*, ...) ;; __builtin_fprintf +declare int %fprintf_unlocked(sbyte*,sbyte*, ...) ;; __builtin_fprintf_unlocked +declare int %fputc(int,sbyte*) ;; __builtin_fputc +declare int %fputc_unlocked(int,sbyte*) ;; __builtin_fputc_unlocked +declare int %fputs(sbyte*,sbyte*) ;; __builtin_fputs +declare int %fputs_unlocked(sbyte*,sbyte*) ;; __builtin_fputs_unlocked +declare int %fscanf(sbyte*,sbyte*, ...) ;; __builtin_fscanf +declare uint %fwrite(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite +declare uint %fwrite_unlocked(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite_unlocked +declare int %printf(sbyte*, ...) ;; __builtin_printf +declare int %printf_unlocked(sbyte*, ...) ;; __builtin_printf_unlocked +declare int %putchar(int) ;; __builtin_putchar +declare int %putchar_unlocked(int) ;; __builtin_putchar_unlocked +declare int %puts(sbyte*) ;; __builtin_puts +declare int %puts_unlocked(sbyte*) ;; __builtin_puts_unlocked +declare int %scanf(sbyte*, ...) ;; __builtin_scanf +declare int %snprintf(sbyte*,uint,sbyte*, ...) ;; __builtin_snprintf +declare int %sprintf(sbyte*,sbyte*, ...) ;; __builtin_sprintf +declare int %sscanf(sbyte*,sbyte*, ...) ;; __builtin_sscanf +declare int %vfprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfprintf +declare int %vfscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfscanf +declare int %vprintf(sbyte*,sbyte*) ;; __builtin_vprintf +declare int %vscanf(sbyte*,sbyte*) ;; __builtin_vscanf +declare int %vsnprintf(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_vsnprintf +declare int %vsprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsprintf +declare int %vsscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsscanf +declare void %abort() ;; __builtin_abort +declare int %abs(int) ;; __builtin_abs +declare sbyte* %__builtin_aggregate_incoming_address(...) +declare sbyte* %alloca(uint) ;; __builtin_alloca +declare sbyte* %__builtin_apply(void (...)*,sbyte*,uint) +declare sbyte* %__builtin_apply_args(...) +declare int %__builtin_args_info(int) +declare sbyte* %calloc(uint,uint) ;; __builtin_calloc +declare int %__builtin_classify_type(...) +declare int %__builtin_clz(int) +declare int %__builtin_clzl(int) +declare int %__builtin_clzll(long) +declare int %__builtin_constant_p(...) +declare int %__builtin_ctz(int) +declare int %__builtin_ctzl(int) +declare int %__builtin_ctzll(long) +declare sbyte* %dcgettext(sbyte*,sbyte*,int) ;; __builtin_dcgettext +declare sbyte* %dgettext(sbyte*,sbyte*) ;; __builtin_dgettext +declare sbyte* %__builtin_dwarf_cfa() +declare uint %__builtin_dwarf_sp_column() +declare void %__builtin_eh_return(int,sbyte*) +declare int %__builtin_eh_return_data_regno(int) +declare void %exit(int) ;; __builtin_exit +declare int %__builtin_expect(int,int) +declare sbyte* %__builtin_extract_return_addr(sbyte*) +declare sbyte* %__builtin_frame_address(uint) +declare sbyte* %__builtin_frob_return_addr(sbyte*) +declare sbyte* %gettext(sbyte*) ;; __builtin_gettext +declare long %imaxabs(long) ;; __builtin_imaxabs +declare void %__builtin_init_dwarf_reg_size_table(sbyte*) +declare int %__builtin_isgreater(...) +declare int %__builtin_isgreaterequal(...) +declare int %__builtin_isless(...) +declare int %__builtin_islessequal(...) +declare int %__builtin_islessgreater(...) +declare int %__builtin_isunordered(...) +declare int %labs(int) ;; __builtin_labs +declare long %llabs(long) ;; __builtin_llabs +declare void %__builtin_longjmp(sbyte*,int) +declare sbyte* %malloc(uint) ;; __builtin_malloc +declare sbyte* %__builtin_next_arg(...) +declare int %__builtin_parity(int) +declare int %__builtin_parityl(int) +declare int %__builtin_parityll(long) +declare int %__builtin_popcount(int) +declare int %__builtin_popcountl(int) +declare int %__builtin_popcountll(long) +declare void %__builtin_prefetch(sbyte*, ...) +declare void %__builtin_return(sbyte*) +declare sbyte* %__builtin_return_address(uint) +declare sbyte* %__builtin_saveregs(...) +declare int %__builtin_setjmp(sbyte*) +declare void %__builtin_stdarg_start(sbyte**, ...) +declare int %strfmon(sbyte*,uint,sbyte*, ...) ;; __builtin_strfmon +declare uint %strftime(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_strftime +declare void %__builtin_trap() +declare void %__builtin_unwind_init() +declare void %__builtin_va_copy(sbyte**,sbyte*) +declare void %__builtin_va_end(sbyte**) +declare void %__builtin_va_start(sbyte**, ...) +declare void %_exit(int) ;; __builtin__exit +declare void %_Exit(int) ;; __builtin__Exit + +int %add(int %n, int %y) { +entry: + %n_addr = alloca int ; ty=int* + %y_addr = alloca int ; ty=int* + %result = alloca int ; ty=int* + store int %n, int* %n_addr + store int %y, int* %y_addr + %tmp.0 = load int* %n_addr ; ty=int + %tmp.1 = load int* %y_addr ; ty=int + %tmp.2 = add int %tmp.0, %tmp.1 ; ty=int + store int %tmp.2, int* %result + br label %return +after_ret: + br label %return +return: + %tmp.3 = load int* %result ; ty=int + ret int %tmp.3 +} + +;; Created by "GCC: (GNU) 3.4-llvm 20051104 (LLVM 1.6)" Added: pypy/dist/pypy/translator/llvm/pyllvm/test/hello.c ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/test/hello.c Wed Apr 5 13:57:58 2006 @@ -0,0 +1,9 @@ +char *gethellostr() { + return "hello world\n"; +} + +int hello() { + printf(gethellostr()); + return 0; +} + Added: pypy/dist/pypy/translator/llvm/pyllvm/test/hello.s ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/test/hello.s Wed Apr 5 13:57:58 2006 @@ -0,0 +1,451 @@ +; GNU C version 3.4-llvm 20051104 (LLVM 1.6) (i686-pc-linux-gnu) +; compiled by GNU C version 3.4.0. +; GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096 +; options passed: -iprefix -mtune=pentiumpro -auxbase +; options enabled: -feliminate-unused-debug-types -fpeephole +; -ffunction-cse -fkeep-static-consts -fpcc-struct-return -fgcse-lm +; -fgcse-sm -fsched-interblock -fsched-spec -fbranch-count-reg -fcommon +; -fgnu-linker -fargument-alias -fzero-initialized-in-bss -fident +; -fmath-errno -ftrapping-math -m80387 -mhard-float -mno-soft-float +; -mieee-fp -mfp-ret-in-387 -maccumulate-outgoing-args -mno-red-zone +; -mtls-direct-seg-refs -mtune=pentiumpro -march=i386 + +target triple = "i686-pc-linux-gnu" +target pointersize = 32 +target endian = little +deplibs = ["c", "crtend"] + +"complex double" = type { double, double } +"complex float" = type { float, float } +"complex long double" = type { double, double } +%.str_1 = internal constant [13 x sbyte] c"hello world\0A\00" + + +implementation +declare double %acos(double) ;; __builtin_acos +declare float %acosf(float) ;; __builtin_acosf +declare double %acosh(double) ;; __builtin_acosh +declare float %acoshf(float) ;; __builtin_acoshf +declare double %acoshl(double) ;; __builtin_acoshl +declare double %acosl(double) ;; __builtin_acosl +declare double %asin(double) ;; __builtin_asin +declare float %asinf(float) ;; __builtin_asinf +declare double %asinh(double) ;; __builtin_asinh +declare float %asinhf(float) ;; __builtin_asinhf +declare double %asinhl(double) ;; __builtin_asinhl +declare double %asinl(double) ;; __builtin_asinl +declare double %atan(double) ;; __builtin_atan +declare double %atan2(double,double) ;; __builtin_atan2 +declare float %atan2f(float,float) ;; __builtin_atan2f +declare double %atan2l(double,double) ;; __builtin_atan2l +declare float %atanf(float) ;; __builtin_atanf +declare double %atanh(double) ;; __builtin_atanh +declare float %atanhf(float) ;; __builtin_atanhf +declare double %atanhl(double) ;; __builtin_atanhl +declare double %atanl(double) ;; __builtin_atanl +declare double %cbrt(double) ;; __builtin_cbrt +declare float %cbrtf(float) ;; __builtin_cbrtf +declare double %cbrtl(double) ;; __builtin_cbrtl +declare double %ceil(double) ;; __builtin_ceil +declare float %ceilf(float) ;; __builtin_ceilf +declare double %ceill(double) ;; __builtin_ceill +declare double %copysign(double,double) ;; __builtin_copysign +declare float %copysignf(float,float) ;; __builtin_copysignf +declare double %copysignl(double,double) ;; __builtin_copysignl +declare double %cos(double) ;; __builtin_cos +declare float %cosf(float) ;; __builtin_cosf +declare double %cosh(double) ;; __builtin_cosh +declare float %coshf(float) ;; __builtin_coshf +declare double %coshl(double) ;; __builtin_coshl +declare double %cosl(double) ;; __builtin_cosl +declare double %drem(double,double) ;; __builtin_drem +declare float %dremf(float,float) ;; __builtin_dremf +declare double %dreml(double,double) ;; __builtin_dreml +declare double %erf(double) ;; __builtin_erf +declare double %erfc(double) ;; __builtin_erfc +declare float %erfcf(float) ;; __builtin_erfcf +declare double %erfcl(double) ;; __builtin_erfcl +declare float %erff(float) ;; __builtin_erff +declare double %erfl(double) ;; __builtin_erfl +declare double %exp(double) ;; __builtin_exp +declare double %exp10(double) ;; __builtin_exp10 +declare float %exp10f(float) ;; __builtin_exp10f +declare double %exp10l(double) ;; __builtin_exp10l +declare double %exp2(double) ;; __builtin_exp2 +declare float %exp2f(float) ;; __builtin_exp2f +declare double %exp2l(double) ;; __builtin_exp2l +declare float %expf(float) ;; __builtin_expf +declare double %expl(double) ;; __builtin_expl +declare double %expm1(double) ;; __builtin_expm1 +declare float %expm1f(float) ;; __builtin_expm1f +declare double %expm1l(double) ;; __builtin_expm1l +declare double %fabs(double) ;; __builtin_fabs +declare float %fabsf(float) ;; __builtin_fabsf +declare double %fabsl(double) ;; __builtin_fabsl +declare double %fdim(double,double) ;; __builtin_fdim +declare float %fdimf(float,float) ;; __builtin_fdimf +declare double %fdiml(double,double) ;; __builtin_fdiml +declare double %floor(double) ;; __builtin_floor +declare float %floorf(float) ;; __builtin_floorf +declare double %floorl(double) ;; __builtin_floorl +declare double %fma(double,double,double) ;; __builtin_fma +declare float %fmaf(float,float,float) ;; __builtin_fmaf +declare double %fmal(double,double,double) ;; __builtin_fmal +declare double %fmax(double,double) ;; __builtin_fmax +declare float %fmaxf(float,float) ;; __builtin_fmaxf +declare double %fmaxl(double,double) ;; __builtin_fmaxl +declare double %fmin(double,double) ;; __builtin_fmin +declare float %fminf(float,float) ;; __builtin_fminf +declare double %fminl(double,double) ;; __builtin_fminl +declare double %fmod(double,double) ;; __builtin_fmod +declare float %fmodf(float,float) ;; __builtin_fmodf +declare double %fmodl(double,double) ;; __builtin_fmodl +declare double %frexp(double,int*) ;; __builtin_frexp +declare float %frexpf(float,int*) ;; __builtin_frexpf +declare double %frexpl(double,int*) ;; __builtin_frexpl +declare double %gamma(double) ;; __builtin_gamma +declare float %gammaf(float) ;; __builtin_gammaf +declare double %gammal(double) ;; __builtin_gammal +declare double %__builtin_huge_val() +declare float %__builtin_huge_valf() +declare double %__builtin_huge_vall() +declare double %hypot(double,double) ;; __builtin_hypot +declare float %hypotf(float,float) ;; __builtin_hypotf +declare double %hypotl(double,double) ;; __builtin_hypotl +declare int %ilogb(double) ;; __builtin_ilogb +declare int %ilogbf(float) ;; __builtin_ilogbf +declare int %ilogbl(double) ;; __builtin_ilogbl +declare double %__builtin_inf() +declare float %__builtin_inff() +declare double %__builtin_infl() +declare double %j0(double) ;; __builtin_j0 +declare float %j0f(float) ;; __builtin_j0f +declare double %j0l(double) ;; __builtin_j0l +declare double %j1(double) ;; __builtin_j1 +declare float %j1f(float) ;; __builtin_j1f +declare double %j1l(double) ;; __builtin_j1l +declare double %jn(int,double) ;; __builtin_jn +declare float %jnf(int,float) ;; __builtin_jnf +declare double %jnl(int,double) ;; __builtin_jnl +declare double %ldexp(double,int) ;; __builtin_ldexp +declare float %ldexpf(float,int) ;; __builtin_ldexpf +declare double %ldexpl(double,int) ;; __builtin_ldexpl +declare double %lgamma(double) ;; __builtin_lgamma +declare float %lgammaf(float) ;; __builtin_lgammaf +declare double %lgammal(double) ;; __builtin_lgammal +declare long %llrint(double) ;; __builtin_llrint +declare long %llrintf(float) ;; __builtin_llrintf +declare long %llrintl(double) ;; __builtin_llrintl +declare long %llround(double) ;; __builtin_llround +declare long %llroundf(float) ;; __builtin_llroundf +declare long %llroundl(double) ;; __builtin_llroundl +declare double %log(double) ;; __builtin_log +declare double %log10(double) ;; __builtin_log10 +declare float %log10f(float) ;; __builtin_log10f +declare double %log10l(double) ;; __builtin_log10l +declare double %log1p(double) ;; __builtin_log1p +declare float %log1pf(float) ;; __builtin_log1pf +declare double %log1pl(double) ;; __builtin_log1pl +declare double %log2(double) ;; __builtin_log2 +declare float %log2f(float) ;; __builtin_log2f +declare double %log2l(double) ;; __builtin_log2l +declare double %logb(double) ;; __builtin_logb +declare float %logbf(float) ;; __builtin_logbf +declare double %logbl(double) ;; __builtin_logbl +declare float %logf(float) ;; __builtin_logf +declare double %logl(double) ;; __builtin_logl +declare int %lrint(double) ;; __builtin_lrint +declare int %lrintf(float) ;; __builtin_lrintf +declare int %lrintl(double) ;; __builtin_lrintl +declare int %lround(double) ;; __builtin_lround +declare int %lroundf(float) ;; __builtin_lroundf +declare int %lroundl(double) ;; __builtin_lroundl +declare double %modf(double,double*) ;; __builtin_modf +declare float %modff(float,float*) ;; __builtin_modff +declare double %modfl(double,double*) ;; __builtin_modfl +declare double %nan(sbyte*) ;; __builtin_nan +declare float %nanf(sbyte*) ;; __builtin_nanf +declare double %nanl(sbyte*) ;; __builtin_nanl +declare double %nans(sbyte*) ;; __builtin_nans +declare float %nansf(sbyte*) ;; __builtin_nansf +declare double %nansl(sbyte*) ;; __builtin_nansl +declare double %nearbyint(double) ;; __builtin_nearbyint +declare float %nearbyintf(float) ;; __builtin_nearbyintf +declare double %nearbyintl(double) ;; __builtin_nearbyintl +declare double %nextafter(double,double) ;; __builtin_nextafter +declare float %nextafterf(float,float) ;; __builtin_nextafterf +declare double %nextafterl(double,double) ;; __builtin_nextafterl +declare double %nexttoward(double,double) ;; __builtin_nexttoward +declare float %nexttowardf(float,double) ;; __builtin_nexttowardf +declare double %nexttowardl(double,double) ;; __builtin_nexttowardl +declare double %pow(double,double) ;; __builtin_pow +declare double %pow10(double) ;; __builtin_pow10 +declare float %pow10f(float) ;; __builtin_pow10f +declare double %pow10l(double) ;; __builtin_pow10l +declare float %powf(float,float) ;; __builtin_powf +declare double %powl(double,double) ;; __builtin_powl +declare double %remainder(double,double) ;; __builtin_remainder +declare float %remainderf(float,float) ;; __builtin_remainderf +declare double %remainderl(double,double) ;; __builtin_remainderl +declare double %remquo(double,double,int*) ;; __builtin_remquo +declare float %remquof(float,float,int*) ;; __builtin_remquof +declare double %remquol(double,double,int*) ;; __builtin_remquol +declare double %rint(double) ;; __builtin_rint +declare float %rintf(float) ;; __builtin_rintf +declare double %rintl(double) ;; __builtin_rintl +declare double %round(double) ;; __builtin_round +declare float %roundf(float) ;; __builtin_roundf +declare double %roundl(double) ;; __builtin_roundl +declare double %scalb(double,double) ;; __builtin_scalb +declare float %scalbf(float,float) ;; __builtin_scalbf +declare double %scalbl(double,double) ;; __builtin_scalbl +declare double %scalbln(double,int) ;; __builtin_scalbln +declare float %scalblnf(float,int) ;; __builtin_scalblnf +declare double %scalblnl(double,int) ;; __builtin_scalblnl +declare double %scalbn(double,int) ;; __builtin_scalbn +declare float %scalbnf(float,int) ;; __builtin_scalbnf +declare double %scalbnl(double,int) ;; __builtin_scalbnl +declare double %significand(double) ;; __builtin_significand +declare float %significandf(float) ;; __builtin_significandf +declare double %significandl(double) ;; __builtin_significandl +declare double %sin(double) ;; __builtin_sin +declare void %sincos(double,double*,double*) ;; __builtin_sincos +declare void %sincosf(float,float*,float*) ;; __builtin_sincosf +declare void %sincosl(double,double*,double*) ;; __builtin_sincosl +declare float %sinf(float) ;; __builtin_sinf +declare double %sinh(double) ;; __builtin_sinh +declare float %sinhf(float) ;; __builtin_sinhf +declare double %sinhl(double) ;; __builtin_sinhl +declare double %sinl(double) ;; __builtin_sinl +declare double %sqrt(double) ;; __builtin_sqrt +declare float %sqrtf(float) ;; __builtin_sqrtf +declare double %sqrtl(double) ;; __builtin_sqrtl +declare double %tan(double) ;; __builtin_tan +declare float %tanf(float) ;; __builtin_tanf +declare double %tanh(double) ;; __builtin_tanh +declare float %tanhf(float) ;; __builtin_tanhf +declare double %tanhl(double) ;; __builtin_tanhl +declare double %tanl(double) ;; __builtin_tanl +declare double %tgamma(double) ;; __builtin_tgamma +declare float %tgammaf(float) ;; __builtin_tgammaf +declare double %tgammal(double) ;; __builtin_tgammal +declare double %trunc(double) ;; __builtin_trunc +declare float %truncf(float) ;; __builtin_truncf +declare double %truncl(double) ;; __builtin_truncl +declare double %y0(double) ;; __builtin_y0 +declare float %y0f(float) ;; __builtin_y0f +declare double %y0l(double) ;; __builtin_y0l +declare double %y1(double) ;; __builtin_y1 +declare float %y1f(float) ;; __builtin_y1f +declare double %y1l(double) ;; __builtin_y1l +declare double %yn(int,double) ;; __builtin_yn +declare float %ynf(int,float) ;; __builtin_ynf +declare double %ynl(int,double) ;; __builtin_ynl +declare double %cabs(double,double) ;; __builtin_cabs +declare float %cabsf(float,float) ;; __builtin_cabsf +declare double %cabsl(double,double) ;; __builtin_cabsl +declare void %cacos("complex double"*,double,double) ;; __builtin_cacos +declare void %cacosf("complex float"*,float,float) ;; __builtin_cacosf +declare void %cacosh("complex double"*,double,double) ;; __builtin_cacosh +declare void %cacoshf("complex float"*,float,float) ;; __builtin_cacoshf +declare void %cacoshl("complex long double"*,double,double) ;; __builtin_cacoshl +declare void %cacosl("complex long double"*,double,double) ;; __builtin_cacosl +declare double %carg(double,double) ;; __builtin_carg +declare float %cargf(float,float) ;; __builtin_cargf +declare double %cargl(double,double) ;; __builtin_cargl +declare void %casin("complex double"*,double,double) ;; __builtin_casin +declare void %casinf("complex float"*,float,float) ;; __builtin_casinf +declare void %casinh("complex double"*,double,double) ;; __builtin_casinh +declare void %casinhf("complex float"*,float,float) ;; __builtin_casinhf +declare void %casinhl("complex long double"*,double,double) ;; __builtin_casinhl +declare void %casinl("complex long double"*,double,double) ;; __builtin_casinl +declare void %catan("complex double"*,double,double) ;; __builtin_catan +declare void %catanf("complex float"*,float,float) ;; __builtin_catanf +declare void %catanh("complex double"*,double,double) ;; __builtin_catanh +declare void %catanhf("complex float"*,float,float) ;; __builtin_catanhf +declare void %catanhl("complex long double"*,double,double) ;; __builtin_catanhl +declare void %catanl("complex long double"*,double,double) ;; __builtin_catanl +declare void %ccos("complex double"*,double,double) ;; __builtin_ccos +declare void %ccosf("complex float"*,float,float) ;; __builtin_ccosf +declare void %ccosh("complex double"*,double,double) ;; __builtin_ccosh +declare void %ccoshf("complex float"*,float,float) ;; __builtin_ccoshf +declare void %ccoshl("complex long double"*,double,double) ;; __builtin_ccoshl +declare void %ccosl("complex long double"*,double,double) ;; __builtin_ccosl +declare void %cexp("complex double"*,double,double) ;; __builtin_cexp +declare void %cexpf("complex float"*,float,float) ;; __builtin_cexpf +declare void %cexpl("complex long double"*,double,double) ;; __builtin_cexpl +declare double %cimag(double,double) ;; __builtin_cimag +declare float %cimagf(float,float) ;; __builtin_cimagf +declare double %cimagl(double,double) ;; __builtin_cimagl +declare void %conj("complex double"*,double,double) ;; __builtin_conj +declare void %conjf("complex float"*,float,float) ;; __builtin_conjf +declare void %conjl("complex long double"*,double,double) ;; __builtin_conjl +declare void %cpow("complex double"*,double,double,double,double) ;; __builtin_cpow +declare void %cpowf("complex float"*,float,float,float,float) ;; __builtin_cpowf +declare void %cpowl("complex long double"*,double,double,double,double) ;; __builtin_cpowl +declare void %cproj("complex double"*,double,double) ;; __builtin_cproj +declare void %cprojf("complex float"*,float,float) ;; __builtin_cprojf +declare void %cprojl("complex long double"*,double,double) ;; __builtin_cprojl +declare double %creal(double,double) ;; __builtin_creal +declare float %crealf(float,float) ;; __builtin_crealf +declare double %creall(double,double) ;; __builtin_creall +declare void %csin("complex double"*,double,double) ;; __builtin_csin +declare void %csinf("complex float"*,float,float) ;; __builtin_csinf +declare void %csinh("complex double"*,double,double) ;; __builtin_csinh +declare void %csinhf("complex float"*,float,float) ;; __builtin_csinhf +declare void %csinhl("complex long double"*,double,double) ;; __builtin_csinhl +declare void %csinl("complex long double"*,double,double) ;; __builtin_csinl +declare void %csqrt("complex double"*,double,double) ;; __builtin_csqrt +declare void %csqrtf("complex float"*,float,float) ;; __builtin_csqrtf +declare void %csqrtl("complex long double"*,double,double) ;; __builtin_csqrtl +declare void %ctan("complex double"*,double,double) ;; __builtin_ctan +declare void %ctanf("complex float"*,float,float) ;; __builtin_ctanf +declare void %ctanh("complex double"*,double,double) ;; __builtin_ctanh +declare void %ctanhf("complex float"*,float,float) ;; __builtin_ctanhf +declare void %ctanhl("complex long double"*,double,double) ;; __builtin_ctanhl +declare void %ctanl("complex long double"*,double,double) ;; __builtin_ctanl +declare int %bcmp(sbyte*,sbyte*,uint) ;; __builtin_bcmp +declare void %bcopy(sbyte*,sbyte*,uint) ;; __builtin_bcopy +declare void %bzero(sbyte*,uint) ;; __builtin_bzero +declare int %ffs(int) ;; __builtin_ffs +declare int %ffsl(int) ;; __builtin_ffsl +declare int %ffsll(long) ;; __builtin_ffsll +declare sbyte* %index(sbyte*,int) ;; __builtin_index +declare int %memcmp(sbyte*,sbyte*,uint) ;; __builtin_memcmp +declare sbyte* %memcpy(sbyte*,sbyte*,uint) ;; __builtin_memcpy +declare sbyte* %memmove(sbyte*,sbyte*,uint) ;; __builtin_memmove +declare sbyte* %mempcpy(sbyte*,sbyte*,uint) ;; __builtin_mempcpy +declare sbyte* %memset(sbyte*,int,uint) ;; __builtin_memset +declare sbyte* %rindex(sbyte*,int) ;; __builtin_rindex +declare sbyte* %stpcpy(sbyte*,sbyte*) ;; __builtin_stpcpy +declare sbyte* %strcat(sbyte*,sbyte*) ;; __builtin_strcat +declare sbyte* %strchr(sbyte*,int) ;; __builtin_strchr +declare int %strcmp(sbyte*,sbyte*) ;; __builtin_strcmp +declare sbyte* %strcpy(sbyte*,sbyte*) ;; __builtin_strcpy +declare uint %strcspn(sbyte*,sbyte*) ;; __builtin_strcspn +declare sbyte* %strdup(sbyte*) ;; __builtin_strdup +declare uint %strlen(sbyte*) ;; __builtin_strlen +declare sbyte* %strncat(sbyte*,sbyte*,uint) ;; __builtin_strncat +declare int %strncmp(sbyte*,sbyte*,uint) ;; __builtin_strncmp +declare sbyte* %strncpy(sbyte*,sbyte*,uint) ;; __builtin_strncpy +declare sbyte* %strpbrk(sbyte*,sbyte*) ;; __builtin_strpbrk +declare sbyte* %strrchr(sbyte*,int) ;; __builtin_strrchr +declare uint %strspn(sbyte*,sbyte*) ;; __builtin_strspn +declare sbyte* %strstr(sbyte*,sbyte*) ;; __builtin_strstr +declare int %fprintf(sbyte*,sbyte*, ...) ;; __builtin_fprintf +declare int %fprintf_unlocked(sbyte*,sbyte*, ...) ;; __builtin_fprintf_unlocked +declare int %fputc(int,sbyte*) ;; __builtin_fputc +declare int %fputc_unlocked(int,sbyte*) ;; __builtin_fputc_unlocked +declare int %fputs(sbyte*,sbyte*) ;; __builtin_fputs +declare int %fputs_unlocked(sbyte*,sbyte*) ;; __builtin_fputs_unlocked +declare int %fscanf(sbyte*,sbyte*, ...) ;; __builtin_fscanf +declare uint %fwrite(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite +declare uint %fwrite_unlocked(sbyte*,uint,uint,sbyte*) ;; __builtin_fwrite_unlocked +declare int %printf(sbyte*, ...) ;; __builtin_printf +declare int %printf_unlocked(sbyte*, ...) ;; __builtin_printf_unlocked +declare int %putchar(int) ;; __builtin_putchar +declare int %putchar_unlocked(int) ;; __builtin_putchar_unlocked +declare int %puts(sbyte*) ;; __builtin_puts +declare int %puts_unlocked(sbyte*) ;; __builtin_puts_unlocked +declare int %scanf(sbyte*, ...) ;; __builtin_scanf +declare int %snprintf(sbyte*,uint,sbyte*, ...) ;; __builtin_snprintf +declare int %sprintf(sbyte*,sbyte*, ...) ;; __builtin_sprintf +declare int %sscanf(sbyte*,sbyte*, ...) ;; __builtin_sscanf +declare int %vfprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfprintf +declare int %vfscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vfscanf +declare int %vprintf(sbyte*,sbyte*) ;; __builtin_vprintf +declare int %vscanf(sbyte*,sbyte*) ;; __builtin_vscanf +declare int %vsnprintf(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_vsnprintf +declare int %vsprintf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsprintf +declare int %vsscanf(sbyte*,sbyte*,sbyte*) ;; __builtin_vsscanf +declare void %abort() ;; __builtin_abort +declare int %abs(int) ;; __builtin_abs +declare sbyte* %__builtin_aggregate_incoming_address(...) +declare sbyte* %alloca(uint) ;; __builtin_alloca +declare sbyte* %__builtin_apply(void (...)*,sbyte*,uint) +declare sbyte* %__builtin_apply_args(...) +declare int %__builtin_args_info(int) +declare sbyte* %calloc(uint,uint) ;; __builtin_calloc +declare int %__builtin_classify_type(...) +declare int %__builtin_clz(int) +declare int %__builtin_clzl(int) +declare int %__builtin_clzll(long) +declare int %__builtin_constant_p(...) +declare int %__builtin_ctz(int) +declare int %__builtin_ctzl(int) +declare int %__builtin_ctzll(long) +declare sbyte* %dcgettext(sbyte*,sbyte*,int) ;; __builtin_dcgettext +declare sbyte* %dgettext(sbyte*,sbyte*) ;; __builtin_dgettext +declare sbyte* %__builtin_dwarf_cfa() +declare uint %__builtin_dwarf_sp_column() +declare void %__builtin_eh_return(int,sbyte*) +declare int %__builtin_eh_return_data_regno(int) +declare void %exit(int) ;; __builtin_exit +declare int %__builtin_expect(int,int) +declare sbyte* %__builtin_extract_return_addr(sbyte*) +declare sbyte* %__builtin_frame_address(uint) +declare sbyte* %__builtin_frob_return_addr(sbyte*) +declare sbyte* %gettext(sbyte*) ;; __builtin_gettext +declare long %imaxabs(long) ;; __builtin_imaxabs +declare void %__builtin_init_dwarf_reg_size_table(sbyte*) +declare int %__builtin_isgreater(...) +declare int %__builtin_isgreaterequal(...) +declare int %__builtin_isless(...) +declare int %__builtin_islessequal(...) +declare int %__builtin_islessgreater(...) +declare int %__builtin_isunordered(...) +declare int %labs(int) ;; __builtin_labs +declare long %llabs(long) ;; __builtin_llabs +declare void %__builtin_longjmp(sbyte*,int) +declare sbyte* %malloc(uint) ;; __builtin_malloc +declare sbyte* %__builtin_next_arg(...) +declare int %__builtin_parity(int) +declare int %__builtin_parityl(int) +declare int %__builtin_parityll(long) +declare int %__builtin_popcount(int) +declare int %__builtin_popcountl(int) +declare int %__builtin_popcountll(long) +declare void %__builtin_prefetch(sbyte*, ...) +declare void %__builtin_return(sbyte*) +declare sbyte* %__builtin_return_address(uint) +declare sbyte* %__builtin_saveregs(...) +declare int %__builtin_setjmp(sbyte*) +declare void %__builtin_stdarg_start(sbyte**, ...) +declare int %strfmon(sbyte*,uint,sbyte*, ...) ;; __builtin_strfmon +declare uint %strftime(sbyte*,uint,sbyte*,sbyte*) ;; __builtin_strftime +declare void %__builtin_trap() +declare void %__builtin_unwind_init() +declare void %__builtin_va_copy(sbyte**,sbyte*) +declare void %__builtin_va_end(sbyte**) +declare void %__builtin_va_start(sbyte**, ...) +declare void %_exit(int) ;; __builtin__exit +declare void %_Exit(int) ;; __builtin__Exit + +sbyte* %gethellostr() { +entry: + %result = alloca sbyte* ; ty=sbyte** + store sbyte* getelementptr ([13 x sbyte]* %.str_1, int 0, int 0), sbyte** %result + br label %return +after_ret: + br label %return +return: + %tmp.1 = load sbyte** %result ; ty=sbyte* + ret sbyte* %tmp.1 +} + + +int %hello() { +entry: + %result = alloca int ; ty=int* + %tmp.2 = call sbyte* (...)* cast (sbyte* ()* %gethellostr to sbyte* (...)*)() ; ty=sbyte* + %tmp.0 = call int (sbyte*, ...)* %printf(sbyte* %tmp.2) ; ty=int + store int 0, int* %result + br label %return +after_ret: + br label %return +return: + %tmp.3 = load int* %result ; ty=int + ret int %tmp.3 +} + +;; Created by "GCC: (GNU) 3.4-llvm 20051104 (LLVM 1.6)" Added: pypy/dist/pypy/translator/llvm/pyllvm/test/ll_snippet.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/test/ll_snippet.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,32 @@ +calc = """int %calc(int %n) { + %tmp.0 = call int %add1(int %n) + ret int %tmp.0 +} +declare int %add1(int)""" + +add1 = """int %add1(int %n) { + %tmp.0 = add int %n, 1 + ret int %tmp.0 +}""" + +add1_version2 = """int %add1(int %n) { + %tmp.0 = add int %n, 100 ;used for testing function replacement + ret int %tmp.0 +}""" + +global_int_a_is_100 = """%a = global int 100""" + +add1_to_global_int_a = """ +int %add1_to_global_int_a() { + %tmp.0 = load int* %a + %tmp.1 = add int %tmp.0, 1 + store int %tmp.1, int* %a + ret int %tmp.1 +}""" + +sub10_from_global_int_a = """int %sub10_from_global_int_a() { + %tmp.0 = load int* %a + %tmp.1 = sub int %tmp.0, 10 + store int %tmp.1, int* %a + ret int %tmp.1 +}""" Added: pypy/dist/pypy/translator/llvm/pyllvm/test/test_ee.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/pyllvm/test/test_ee.py Wed Apr 5 13:57:58 2006 @@ -0,0 +1,146 @@ +import py +from pypy.translator.llvm.buildllvm import llvm_is_on_path +if not llvm_is_on_path(): + py.test.skip("llvm not found") + +from pypy.translator.llvm.pyllvm import pyllvm +from pypy.translator.llvm.pyllvm.test import ll_snippet + + +def test_execution_engine(): + ee = pyllvm.ExecutionEngine() + ee = pyllvm.ExecutionEngine() + ee = pyllvm.ExecutionEngine() + del ee #XXX not actualy deleted at the moment!!! + ee2 = pyllvm.ExecutionEngine() + ee2 = pyllvm.ExecutionEngine() + ee2 = pyllvm.ExecutionEngine() + +codepath = py.path.local(__file__).dirpath() + +def test_load(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + ee.parse(codepath.join("addnumbers.s").read()) + +def test_functions(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + mod = ee.getModule() + assert mod.n_functions() == 2 + #TODO + #for function in functions: + # returnId, name, args = function + # assert len(function) == 3 + # assert returnId > 0 + # assert name in ('gethellostr', 'hello') + # assert len(args) == 0 + py.test.raises(Exception, mod.n_functions, 1) + py.test.raises(Exception, mod.n_functions, "string") + +def test_call_parse_once(): + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + f = ee.getModule().getNamedFunction + hello = f("hello") + gethellostr = f("gethellostr") + assert hello() == 0 + assert gethellostr() == "hello world\n" + +def test_call_parse_twice(): + py.test.skip("WIP") + ee = pyllvm.ExecutionEngine() + ee.parse(codepath.join("hello.s").read()) + f = ee.getModule().getNamedFunction + f1 = f("gethellostr") + assert f1() == "hello world\n" + ee.parse(codepath.join("addnumbers.s").read()) + f2 = f("add") + assert f2(10, 32) == 42 + assert f1() == "hello world\n" + py.test.raises(Exception, ee.parse) + py.test.raises(Exception, ee.parse, 1) + py.test.raises(Exception, ee.parse, "abc") + +def test_call_between_parsed_code(): + """we parse add1 last on purpose to see if the JIT resolves + the function at execution time. Not sure if we really need this + particular feature. It appears that 'calc' requires a forward + declaration to add1 otherwise a segfault will occur!""" + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + f = ee.getModule().getNamedFunction + assert f("add1")(41) == 42 + assert f("calc")(122) == 123 + +def test_replace_function(): + py.test.skip("WIP") + """similar to test_call_between_parsed_code with additional complexity + because we rebind the add1 function to another version after it the + first version already has been used.""" + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + f = ee.getModule().getNamedFunction + assert f("add1")(41) == 42 + assert f("calc")(122) == 123 #XXX need recompileAndRelinkFunction somewhere + ee.parse(ll_snippet.add1_version2, "add1") + assert f("add1")(42) == 142 + assert f("calc")(142) == 242 + +def test_share_data_between_parsed_code(): + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.global_int_a_is_100) + ee.parse(ll_snippet.add1_to_global_int_a) + ee.parse(ll_snippet.sub10_from_global_int_a) + f = ee.getModule().getNamedFunction + assert f("add1_to_global_int_a")() == 101 + assert f("sub10_from_global_int_a")() == 91 + assert f("add1_to_global_int_a")() == 92 + assert f("sub10_from_global_int_a")() == 82 + +def test_native_code(): #examine JIT generate native (assembly) code + py.test.skip("WIP") + pyllvm.toggle_print_machineinstrs() + ee = pyllvm.ExecutionEngine() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + f = ee.getModule().getNamedFunction + assert f("calc")(41) == 42 + pyllvm.toggle_print_machineinstrs() + +def test_delete_function(): #this will only work if nothing uses Fn of course! + py.test.skip("WIP") + ee = pyllvm.ExecutionEngine() + mod = ee.getModule() + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert mod.n_functions() == 2 + + ee.delete("calc") + assert mod.n_functions() == 1 + f = ee.getModule().getNamedFunction + assert f("add1")(41) == 42 + + ee.delete("add1") + assert mod.n_functions() == 0 + + ee.parse(ll_snippet.calc) + ee.parse(ll_snippet.add1) + assert f("calc")(100) == 101 + +def TODOtest_multiple_executionengines(): + pass + +def TODOtest_returntypes(): + pass + +def TODOtest_paramtypes(): + pass + +def TODOtest_add_to_function(): + pass + +def TODOtest_optimize_functions(): #add/del/list llvm transformation passes + pass From mwh at codespeak.net Wed Apr 5 15:00:09 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 15:00:09 +0200 (CEST) Subject: [pypy-svn] r25378 - pypy/dist/pypy/translator/c/src Message-ID: <20060405130009.6EAF3101E0@code0.codespeak.net> Author: mwh Date: Wed Apr 5 15:00:08 2006 New Revision: 25378 Modified: pypy/dist/pypy/translator/c/src/float.h Log: duh, brain-free conflict resolution thinko. Modified: pypy/dist/pypy/translator/c/src/float.h ============================================================================== --- pypy/dist/pypy/translator/c/src/float.h (original) +++ pypy/dist/pypy/translator/c/src/float.h Wed Apr 5 15:00:08 2006 @@ -26,7 +26,7 @@ #define OP_FLOAT_ADD(x,y,r) r = x + y #define OP_FLOAT_SUB(x,y,r) r = x - y #define OP_FLOAT_MUL(x,y,r) r = x * y -#define OP_FLOAT_TRUEDIV(x,y,r) OP_FLOAT_DIV(x,y,r) +#define OP_FLOAT_TRUEDIV(x,y,r) r = x / y #define OP_FLOAT_POW(x,y,r) r = pow(x, y) /*** conversions ***/ From nik at codespeak.net Wed Apr 5 15:06:39 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 15:06:39 +0200 (CEST) Subject: [pypy-svn] r25379 - in pypy/dist/pypy/rpython: ootypesystem ootypesystem/test test Message-ID: <20060405130639.AD52A101E1@code0.codespeak.net> Author: nik Date: Wed Apr 5 15:06:38 2006 New Revision: 25379 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/test/test_rlist.py Log: make recursive data structures involving lists work in ootypesystem. this involved letting List types fall back to LowLevelType for __eq__ and __hash__. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Wed Apr 5 15:06:38 2006 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem.lltype import LowLevelType, Signed, Unsigned, Float, Char from pypy.rpython.lltypesystem.lltype import Bool, Void, UniChar, typeOf, \ - Primitive, isCompatibleType, enforce + Primitive, isCompatibleType, enforce, saferecursive from pypy.rpython.lltypesystem.lltype import frozendict, isCompatibleType from pypy.tool.uid import uid @@ -186,19 +186,15 @@ "setitem": Meth([Signed, ITEMTYPE], Void), }) - def __str__(self): - return '%s(%s)' % (self.__class__.__name__, self._ITEMTYPE) - - def __eq__(self, other): - if not isinstance(other, List): - return False - return self._ITEMTYPE == other._ITEMTYPE - - def __ne__(self, other): - return not (self == other) + # NB: We are expecting Lists of the same ITEMTYPE to compare/hash + # equal. We don't redefine __eq__/__hash__ since the implementations + # from LowLevelType work fine, especially in the face of recursive + # data structures. But it is important to make sure that attributes + # of supposedly equal Lists compare/hash equal. - def __hash__(self): - return hash(self._ITEMTYPE) + def __str__(self): + return '%s(%s)' % (self.__class__.__name__, + saferecursive(str, "...")(self._ITEMTYPE)) def _lookup(self, meth_name): METH = self._METHODS.get(meth_name) @@ -214,6 +210,18 @@ def _defl(self): return self._null + +class ForwardReference(OOType): + def become(self, real_instance): + if not isinstance(real_instance, (Instance, List)): + raise TypeError("ForwardReference can only be to an instance, " + "not %r" % (real_instance,)) + self.__class__ = real_instance.__class__ + self.__dict__ = real_instance.__dict__ + + def __hash__(self): + raise TypeError("%r object is not hashable" % self.__class__.__name__) + # ____________________________________________________________ class _class(object): @@ -316,6 +324,9 @@ if not isinstance(other, klass): raise TypeError("comparing an %s with %r" % (klass.__name__, other)) return not other + + def __hash__(self): + return hash(self._TYPE) return mixin class _null_instance(_null_mixin(_instance), _instance): Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 15:06:38 2006 @@ -15,6 +15,7 @@ if not isinstance(item_repr, Repr): # not computed yet, done by setup() assert callable(item_repr) self._item_repr_computer = item_repr + self.lowleveltype = ootype.ForwardReference() else: self.lowleveltype = ootype.List(item_repr.lowleveltype) self.external_item_repr, self.item_repr = \ @@ -27,7 +28,8 @@ if 'item_repr' not in self.__dict__: self.external_item_repr, self.item_repr = \ externalvsinternal(self.rtyper, self._item_repr_computer()) - self.lowleveltype = ootype.List(self.item_repr.lowleveltype) + if isinstance(self.lowleveltype, ootype.ForwardReference): + self.lowleveltype.become(ootype.List(self.item_repr.lowleveltype)) def send_message(self, hop, message, can_raise=False): v_args = hop.inputargs(self, *hop.args_r[1:]) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Wed Apr 5 15:06:38 2006 @@ -45,6 +45,20 @@ assert LT1 != LT3 assert hash(LT1) == hash(LT2) +def test_recursive(): + FORWARD = ForwardReference() + LT = List(FORWARD) + FORWARD.become(LT) + assert LT == LT + assert hash(LT) == hash(LT) + str(LT) # make sure this doesn't recurse infinitely + + FORWARD2 = ForwardReference() + LT2 = List(FORWARD2) + FORWARD2.become(LT2) + assert LT == LT2 + assert hash(LT) == hash(LT2) + class TestInterpreted: def test_append_length(self): Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Wed Apr 5 15:06:38 2006 @@ -215,24 +215,24 @@ res = interpret(dummyfn, [], type_system=self.ts) assert res == 25 -def test_recursive(): - def dummyfn(N): - l = [] - while N > 0: - l = [l] - N -= 1 - return len(l) - res = interpret(dummyfn, [5]) - assert res == 1 - - def dummyfn(N): - l = [] - while N > 0: - l.append(l) - N -= 1 - return len(l) - res = interpret(dummyfn, [5]) - assert res == 5 + def test_recursive(self): + def dummyfn(N): + l = [] + while N > 0: + l = [l] + N -= 1 + return len(l) + res = interpret(dummyfn, [5], type_system=self.ts) + assert res == 1 + + def dummyfn(N): + l = [] + while N > 0: + l.append(l) + N -= 1 + return len(l) + res = interpret(dummyfn, [5]) + assert res == 5 def tolst(l): return map(None, l.ll_items())[:l.ll_length()] From arigo at codespeak.net Wed Apr 5 15:13:17 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 15:13:17 +0200 (CEST) Subject: [pypy-svn] r25381 - pypy/dist/pypy/rpython/test Message-ID: <20060405131317.1735A101E5@code0.codespeak.net> Author: arigo Date: Wed Apr 5 15:13:16 2006 New Revision: 25381 Modified: pypy/dist/pypy/rpython/test/test_llinterp.py Log: These tests are no long valid after the explicit-exceptions branch merge. Modified: pypy/dist/pypy/rpython/test/test_llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_llinterp.py (original) +++ pypy/dist/pypy/rpython/test/test_llinterp.py Wed Apr 5 15:13:16 2006 @@ -418,57 +418,6 @@ fgraph.startblock.operations[0].args.insert(0, inputconst(Void, "stack")) py.test.raises(AttributeError, "interp.eval_graph(graph, [])") - -def test_cleanup_finally(): - interp, graph = get_interpreter(cleanup_f, [-1]) - clear_tcache() # because we hack the graph in place - operations = graph.startblock.operations - assert operations[0].opname == "direct_call" - assert operations[1].opname == "direct_call" - assert getattr(operations[0], 'cleanup', None) is None - assert getattr(operations[1], 'cleanup', None) is None - cleanup_finally = (operations.pop(1),) - cleanup_except = () - operations[0].cleanup = cleanup_finally, cleanup_except - - # state.current == 1 - res = interp.eval_graph(graph, [1]) - assert res == 102 - # state.current == 2 - res = interp.eval_graph(graph, [1]) - assert res == 203 - # state.current == 3 - py.test.raises(LLException, "interp.eval_graph(graph, [-1])") - # state.current == 4 - res = interp.eval_graph(graph, [1]) - assert res == 405 - # state.current == 5 - -def test_cleanup_except(): - interp, graph = get_interpreter(cleanup_f, [-1]) - clear_tcache() # because we hack the graph in place - operations = graph.startblock.operations - assert operations[0].opname == "direct_call" - assert operations[1].opname == "direct_call" - assert getattr(operations[0], 'cleanup', None) is None - assert getattr(operations[1], 'cleanup', None) is None - cleanup_finally = () - cleanup_except = (operations.pop(1),) - operations[0].cleanup = cleanup_finally, cleanup_except - - # state.current == 1 - res = interp.eval_graph(graph, [1]) - assert res == 101 - # state.current == 1 - res = interp.eval_graph(graph, [1]) - assert res == 101 - # state.current == 1 - py.test.raises(LLException, "interp.eval_graph(graph, [-1])") - # state.current == 2 - res = interp.eval_graph(graph, [1]) - assert res == 202 - # state.current == 2 - #__________________________________________________________________ # example functions for testing the LLInterpreter _snap = globals().copy() @@ -528,20 +477,3 @@ return i except ValueError: raise TypeError - -# test for the 'cleanup' attribute of SpaceOperations -class CleanupState(object): - pass -cleanup_state = CleanupState() -cleanup_state.current = 1 -def cleanup_g(n): - cleanup_state.saved = cleanup_state.current - if n < 0: - raise ZeroDivisionError -def cleanup_h(): - cleanup_state.current += 1 -def cleanup_f(n): - cleanup_g(n) - cleanup_h() # the test hacks the graph to put this h() in the - # cleanup clause of the previous direct_call(g) - return cleanup_state.saved * 100 + cleanup_state.current From arigo at codespeak.net Wed Apr 5 15:14:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 15:14:18 +0200 (CEST) Subject: [pypy-svn] r25382 - in pypy/dist/pypy: objspace/flow rpython rpython/lltypesystem rpython/test Message-ID: <20060405131418.E1B5E101E6@code0.codespeak.net> Author: arigo Date: Wed Apr 5 15:14:17 2006 New Revision: 25382 Modified: pypy/dist/pypy/objspace/flow/operation.py pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/rint.py pypy/dist/pypy/rpython/test/snippet.py Log: Cleaned up the list of overflow-raising low-level operations. Made the table more regular. Removed int_pow (not implemented in the C back-end anyway). Modified: pypy/dist/pypy/objspace/flow/operation.py ============================================================================== --- pypy/dist/pypy/objspace/flow/operation.py (original) +++ pypy/dist/pypy/objspace/flow/operation.py Wed Apr 5 15:14:17 2006 @@ -125,8 +125,8 @@ def mod_ovf(x, y): return ovfcheck(x % y) -def pow_ovf(*two_or_three_args): - return ovfcheck(pow(*two_or_three_args)) +##def pow_ovf(*two_or_three_args): +## return ovfcheck(pow(*two_or_three_args)) def lshift_ovf(x, y): return ovfcheck_lshift(x, y) @@ -194,7 +194,6 @@ ('floordiv_ovf', floordiv_ovf), ('div_ovf', div_ovf), ('mod_ovf', mod_ovf), - ('pow_ovf', pow_ovf), ('lshift_ovf', lshift_ovf), ] Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Wed Apr 5 15:14:17 2006 @@ -370,6 +370,15 @@ def op_hint(self, x, hints): return x + def op_decode_arg(self, fname, i, name, vargs, vkwds): + raise NotImplementedError("decode_arg") + + def op_decode_arg_def(fname, i, name, vargs, vkwds, default): + raise NotImplementedError("decode_arg_def") + + def op_check_no_more_arg(fname, n, vargs): + raise NotImplementedError("check_no_more_arg") + def op_setfield(self, obj, fieldname, fieldvalue): # obj should be pointer FIELDTYPE = getattr(lltype.typeOf(obj).TO, fieldname) @@ -583,20 +592,6 @@ assert -sys.maxint-1 <= b <= sys.maxint return int(b) - def op_int_floordiv_ovf_zer(self, a, b): - assert type(a) is int - assert type(b) is int - if b == 0: - self.make_llexception(ZeroDivisionError()) - return self.op_int_floordiv_ovf(a, b) - - def op_int_mod_ovf_zer(self, a, b): - assert type(a) is int - assert type(b) is int - if b == 0: - self.make_llexception(ZeroDivisionError()) - return self.op_int_mod_ovf(a, b) - def op_float_floor(self, b): assert type(b) is float return math.floor(b) @@ -606,6 +601,11 @@ assert type(c) is float return math.fmod(b,c) + def op_float_pow(self, b,c): + assert type(b) is float + assert type(c) is float + return math.pow(b,c) + def op_gc__collect(self): import gc gc.collect() @@ -622,6 +622,24 @@ def op_gc_call_rtti_destructor(self, rtti, addr): raise NotImplementedError("gc_call_rtti_destructor") + def op_gc_push_alive_pyobj(self, pyobj): + raise NotImplementedError("gc_push_alive_pyobj") + + def op_gc_pop_alive_pyobj(self, pyobj): + raise NotImplementedError("gc_pop_alive_pyobj") + + def op_gc_protect(self, obj): + raise NotImplementedError("gc_protect") + + def op_gc_unprotect(self, obj): + raise NotImplementedError("gc_unprotect") + + def op_gc_reload_possibly_moved(self, newaddr, ptr): + raise NotImplementedError("gc_reload_possibly_moved") + + def op_yield_current_frame_to_caller(self): + raise NotImplementedError("yield_current_frame_to_caller") + # operations on pyobjects! for opname in opimpls.keys(): exec py.code.Source(""" @@ -697,86 +715,112 @@ # __________________________________________________________ # primitive operations - for typ in (float, int, r_uint, r_longlong, r_ulonglong): - typname = typ.__name__ - optup = ('add', 'sub', 'mul', 'truediv', 'floordiv', - 'mod', 'gt', 'lt', 'ge', 'ne', 'le', 'eq',) - if typ is r_uint: - opnameprefix = 'uint' - elif typ is r_longlong: - opnameprefix = 'llong' - elif typ is r_ulonglong: - opnameprefix = 'ullong' - else: - opnameprefix = typname - if typ in (int, r_uint): - optup += 'and_', 'or_', 'lshift', 'rshift', 'xor' - for opname in optup: - assert opname in opimpls - if typ is float and opname == 'floordiv': - continue # 'floordiv' is for integer types - if typ is not float and opname == 'truediv': - continue # 'truediv' is for floats only - if typ is int and opname not in ops_returning_a_bool: - adjust_result = 'intmask' + def setup_primitive_operations(): + for typ in (float, int, r_uint, r_longlong, r_ulonglong): + typname = typ.__name__ + optup = ('add', 'sub', 'mul', 'truediv', 'floordiv', + 'mod', 'gt', 'lt', 'ge', 'ne', 'le', 'eq',) + overflowing_operations = ('add', 'sub', 'mul', 'floordiv', + 'mod', 'lshift') + if typ is r_uint: + opnameprefix = 'uint' + elif typ is r_longlong: + opnameprefix = 'llong' + elif typ is r_ulonglong: + opnameprefix = 'ullong' else: - adjust_result = '' - pureopname = opname.rstrip('_') - exec py.code.Source(""" - def op_%(opnameprefix)s_%(pureopname)s(self, x, y): - assert isinstance(x, %(typname)s) - assert isinstance(y, %(typname)s) - func = opimpls[%(opname)r] - return %(adjust_result)s(func(x, y)) - """ % locals()).compile() - if typ is int: - opname += '_ovf' - exec py.code.Source(""" - def op_%(opnameprefix)s_%(pureopname)s_ovf(self, x, y): + opnameprefix = typname + if typ is not float: + optup += 'and_', 'or_', 'lshift', 'rshift', 'xor' + for opname in optup: + assert opname in opimpls + if typ is float and opname == 'floordiv': + continue # 'floordiv' is for integer types + if typ is not float and opname == 'truediv': + continue # 'truediv' is for floats only + if typ is int and opname not in ops_returning_a_bool: + adjust_result = 'intmask' + else: + adjust_result = '' + pureopname = opname.rstrip('_') + yield """ + def op_%(opnameprefix)s_%(pureopname)s(self, x, y): assert isinstance(x, %(typname)s) assert isinstance(y, %(typname)s) func = opimpls[%(opname)r] - try: - return %(adjust_result)s(func(x, y)) - except OverflowError: - self.make_llexception() - """ % locals()).compile() - for opname in 'is_true', 'neg', 'abs', 'invert': - assert opname in opimpls - if typ is float and opname == 'invert': - continue - if typ is int and opname not in ops_returning_a_bool: - adjust_result = 'intmask' - else: - adjust_result = '' - exec py.code.Source(""" - def op_%(opnameprefix)s_%(opname)s(self, x): - assert isinstance(x, %(typname)s) - func = opimpls[%(opname)r] - return %(adjust_result)s(func(x)) - """ % locals()).compile() - if typ is int and opname in ('neg', 'abs'): - opname += '_ovf' - exec py.code.Source(""" + return %(adjust_result)s(func(x, y)) + """ % locals() + + suffixes = [] + if typ is not float: + if opname in ('lshift', 'rshift'): + suffixes.append(('_val', 'ValueError')) + if opname in ('floordiv', 'mod'): + suffixes.append(('_zer', 'ZeroDivisionError')) + if typ is int and opname in overflowing_operations: + for suffix1, exccls1 in suffixes[:]: + suffixes.append(('_ovf'+suffix1, + '(OverflowError, %s)' % exccls1)) + suffixes.append(('_ovf', 'OverflowError')) + + for suffix, exceptionclasses in suffixes: + if '_ovf' in suffix: + opname_ex = opname + '_ovf' + else: + opname_ex = opname + yield """ + def op_%(opnameprefix)s_%(pureopname)s%(suffix)s(self, x, y): + assert isinstance(x, %(typname)s) + assert isinstance(y, %(typname)s) + func = opimpls[%(opname_ex)r] + try: + return %(adjust_result)s(func(x, y)) + except %(exceptionclasses)s: + self.make_llexception() + """ % locals() + for opname in 'is_true', 'neg', 'abs', 'invert': + assert opname in opimpls + if typ is float and opname == 'invert': + continue + if typ is int and opname not in ops_returning_a_bool: + adjust_result = 'intmask' + else: + adjust_result = '' + yield """ def op_%(opnameprefix)s_%(opname)s(self, x): assert isinstance(x, %(typname)s) func = opimpls[%(opname)r] - try: - return %(adjust_result)s(func(x)) - except OverflowError: - self.make_llexception() - """ % locals()).compile() - - for opname in ('gt', 'lt', 'ge', 'ne', 'le', 'eq'): - assert opname in opimpls - exec py.code.Source(""" - def op_char_%(opname)s(self, x, y): - assert isinstance(x, str) and len(x) == 1 - assert isinstance(y, str) and len(y) == 1 - func = opimpls[%(opname)r] - return func(x, y) - """ % locals()).compile() - + return %(adjust_result)s(func(x)) + """ % locals() + if typ is int and opname in ('neg', 'abs'): + opname += '_ovf' + yield """ + def op_%(opnameprefix)s_%(opname)s(self, x): + assert isinstance(x, %(typname)s) + func = opimpls[%(opname)r] + try: + return %(adjust_result)s(func(x)) + except OverflowError: + self.make_llexception() + """ % locals() + + for opname in ('gt', 'lt', 'ge', 'ne', 'le', 'eq'): + assert opname in opimpls + yield """ + def op_char_%(opname)s(self, x, y): + assert isinstance(x, str) and len(x) == 1 + assert isinstance(y, str) and len(y) == 1 + func = opimpls[%(opname)r] + return func(x, y) + """ % locals() + + for _src in setup_primitive_operations(): + exec py.code.Source(_src).compile() + del _src + del setup_primitive_operations + + # ____________________________________________________________ + original_int_add = op_int_add def op_int_add(self, x, y): @@ -796,14 +840,12 @@ def op_unichar_eq(self, x, y): assert isinstance(x, unicode) and len(x) == 1 assert isinstance(y, unicode) and len(y) == 1 - func = opimpls['eq'] - return func(x, y) + return x == y def op_unichar_ne(self, x, y): assert isinstance(x, unicode) and len(x) == 1 assert isinstance(y, unicode) and len(y) == 1 - func = opimpls['ne'] - return func(x, y) + return x != y #Operation of ootype Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Wed Apr 5 15:14:17 2006 @@ -20,6 +20,7 @@ # Exceptions that can be raised self.canraise = canraise + assert isinstance(canraise, tuple) # The operation manipulates PyObjects self.pyobj = pyobj @@ -62,7 +63,6 @@ # This list corresponds to the operations implemented by the LLInterpreter. # XXX Some clean-ups are needed: # * many exception-raising operations are being replaced by calls to helpers -# * there are still many _ovf operations that cannot really raise OverflowError # * float_mod vs float_fmod ? # Run test_lloperation after changes. Feel free to clean up LLInterpreter too :-) @@ -96,7 +96,9 @@ 'int_sub': LLOp(canfold=True), 'int_mul': LLOp(canfold=True), 'int_floordiv': LLOp(canfold=True), + 'int_floordiv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'int_mod': LLOp(canfold=True), + 'int_mod_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'int_lt': LLOp(canfold=True), 'int_le': LLOp(canfold=True), 'int_eq': LLOp(canfold=True), @@ -106,31 +108,20 @@ 'int_and': LLOp(canfold=True), 'int_or': LLOp(canfold=True), 'int_lshift': LLOp(canfold=True), + 'int_lshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'int_rshift': LLOp(canfold=True), + 'int_rshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'int_xor': LLOp(canfold=True), - 'int_floordiv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), + 'int_add_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_sub_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_mul_ovf': LLOp(canfold=True, canraise=(OverflowError,)), 'int_floordiv_ovf': LLOp(canfold=True, canraise=(OverflowError,)), + 'int_floordiv_ovf_zer': LLOp(canfold=True, canraise=(OverflowError, ZeroDivisionError)), 'int_mod_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_lt_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_le_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_eq_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_ne_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_gt_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_ge_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_and_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_or_ovf': LLOp(canfold=True, canraise=(OverflowError,)), + 'int_mod_ovf_zer': LLOp(canfold=True, canraise=(OverflowError, ZeroDivisionError)), 'int_lshift_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_lshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'int_lshift_ovf_val': LLOp(canfold=True, canraise=(OverflowError, ValueError,)), - 'int_rshift_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_rshift_val': LLOp(canfold=True, canraise=(ValueError,)), - 'int_rshift_ovf_val': LLOp(canfold=True, canraise=(OverflowError, ValueError,)), - 'int_xor_ovf': LLOp(canfold=True, canraise=(OverflowError,)), - 'int_floordiv_ovf_zer': LLOp(canfold=True, canraise=(OverflowError, ZeroDivisionError)), - 'int_mod_ovf_zer': LLOp(canfold=True, canraise=(OverflowError, ZeroDivisionError)), 'uint_is_true': LLOp(canfold=True), 'uint_neg': LLOp(canfold=True), @@ -143,6 +134,7 @@ 'uint_floordiv': LLOp(canfold=True), 'uint_floordiv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'uint_mod': LLOp(canfold=True), + 'uint_mod_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'uint_lt': LLOp(canfold=True), 'uint_le': LLOp(canfold=True), 'uint_eq': LLOp(canfold=True), @@ -152,7 +144,9 @@ 'uint_and': LLOp(canfold=True), 'uint_or': LLOp(canfold=True), 'uint_lshift': LLOp(canfold=True), + 'uint_lshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'uint_rshift': LLOp(canfold=True), + 'uint_rshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'uint_xor': LLOp(canfold=True), 'float_is_true': LLOp(canfold=True), @@ -183,7 +177,9 @@ 'llong_sub': LLOp(canfold=True), 'llong_mul': LLOp(canfold=True), 'llong_floordiv': LLOp(canfold=True), + 'llong_floordiv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'llong_mod': LLOp(canfold=True), + 'llong_mod_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'llong_lt': LLOp(canfold=True), 'llong_le': LLOp(canfold=True), 'llong_eq': LLOp(canfold=True), @@ -193,7 +189,9 @@ 'llong_and': LLOp(canfold=True), 'llong_or': LLOp(canfold=True), 'llong_lshift': LLOp(canfold=True), + 'llong_lshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'llong_rshift': LLOp(canfold=True), + 'llong_rshift_val': LLOp(canfold=True, canraise=(ValueError,)), 'llong_xor': LLOp(canfold=True), 'ullong_is_true': LLOp(canfold=True), @@ -205,18 +203,22 @@ 'ullong_sub': LLOp(canfold=True), 'ullong_mul': LLOp(canfold=True), 'ullong_floordiv': LLOp(canfold=True), + 'ullong_floordiv_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'ullong_mod': LLOp(canfold=True), + 'ullong_mod_zer': LLOp(canfold=True, canraise=(ZeroDivisionError,)), 'ullong_lt': LLOp(canfold=True), 'ullong_le': LLOp(canfold=True), 'ullong_eq': LLOp(canfold=True), 'ullong_ne': LLOp(canfold=True), 'ullong_gt': LLOp(canfold=True), 'ullong_ge': LLOp(canfold=True), - 'ulong_and': LLOp(canfold=True), - 'ulong_or': LLOp(canfold=True), - 'ulong_lshift': LLOp(canfold=True), - 'ulong_rshift': LLOp(canfold=True), - 'ulong_xor': LLOp(canfold=True), + 'ullong_and': LLOp(canfold=True), + 'ullong_or': LLOp(canfold=True), + 'ullong_lshift': LLOp(canfold=True), + 'ullong_lshift_val': LLOp(canfold=True, canraise=(ValueError,)), + 'ullong_rshift': LLOp(canfold=True), + 'ullong_rshift_val': LLOp(canfold=True, canraise=(ValueError,)), + 'ullong_xor': LLOp(canfold=True), 'cast_bool_to_int': LLOp(canfold=True), 'cast_bool_to_uint': LLOp(canfold=True), Modified: pypy/dist/pypy/rpython/rint.py ============================================================================== --- pypy/dist/pypy/rpython/rint.py (original) +++ pypy/dist/pypy/rpython/rint.py Wed Apr 5 15:14:17 2006 @@ -118,26 +118,26 @@ return _rtype_template(hop, 'rshift', [ValueError]) rtype_inplace_rshift = rtype_rshift - def rtype_pow(_, hop, suffix=''): - if hop.has_implicit_exception(ZeroDivisionError): - suffix += '_zer' - s_int3 = hop.args_s[2] - rresult = hop.rtyper.makerepr(hop.s_result) - if s_int3.is_constant() and s_int3.const is None: - vlist = hop.inputargs(rresult, rresult, Void)[:2] - else: - vlist = hop.inputargs(rresult, rresult, rresult) - hop.exception_is_here() - return hop.genop(rresult.opprefix + 'pow' + suffix, vlist, resulttype=rresult) - - def rtype_pow_ovf(_, hop): - if hop.s_result.unsigned: - raise TyperError("forbidden uint_pow_ovf") - hop.has_implicit_exception(OverflowError) # record that we know about it - return self.rtype_pow(_, hop, suffix='_ovf') +## def rtype_pow(_, hop, suffix=''): +## if hop.has_implicit_exception(ZeroDivisionError): +## suffix += '_zer' +## s_int3 = hop.args_s[2] +## rresult = hop.rtyper.makerepr(hop.s_result) +## if s_int3.is_constant() and s_int3.const is None: +## vlist = hop.inputargs(rresult, rresult, Void)[:2] +## else: +## vlist = hop.inputargs(rresult, rresult, rresult) +## hop.exception_is_here() +## return hop.genop(rresult.opprefix + 'pow' + suffix, vlist, resulttype=rresult) + +## def rtype_pow_ovf(_, hop): +## if hop.s_result.unsigned: +## raise TyperError("forbidden uint_pow_ovf") +## hop.has_implicit_exception(OverflowError) # record that we know about it +## return self.rtype_pow(_, hop, suffix='_ovf') - def rtype_inplace_pow(_, hop): - return _rtype_template(hop, 'pow', [ZeroDivisionError]) +## def rtype_inplace_pow(_, hop): +## return _rtype_template(hop, 'pow', [ZeroDivisionError]) #comparisons: eq is_ ne lt le gt ge Modified: pypy/dist/pypy/rpython/test/snippet.py ============================================================================== --- pypy/dist/pypy/rpython/test/snippet.py (original) +++ pypy/dist/pypy/rpython/test/snippet.py Wed Apr 5 15:14:17 2006 @@ -74,8 +74,8 @@ i += abs(i) i &= 255 - i **= n - i += n**3 + #i **= n + #i += n**3 i += -n i += +n From auc at codespeak.net Wed Apr 5 15:23:14 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 15:23:14 +0200 (CEST) Subject: [pypy-svn] r25383 - pypy/dist/pypy/objspace Message-ID: <20060405132314.9FF98101EE@code0.codespeak.net> Author: auc Date: Wed Apr 5 15:23:13 2006 New Revision: 25383 Modified: pypy/dist/pypy/objspace/logic.py Log: Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 15:23:13 2006 @@ -579,16 +579,21 @@ def unify__Root_Var(space, w_x, w_y): return space.unify(w_y, w_x) -def unify__Tuple_Tuple(space, w_x, w_y): - return _unify_iterables(space, w_x, w_y) +def unify__Tuple_Tuple(space, w_i1, w_i2): + if len(w_i1.wrappeditems) != len(w_i2.wrappeditems): + fail(space, w_i1, w_i2) + idx, top = (-1, space.int_w(space.len(w_i1))-1) + while idx < top: + idx += 1 + w_xi = space.getitem(w_i1, space.newint(idx)) + w_yi = space.getitem(w_i2, space.newint(idx)) + if space.is_true(space.is_nb_(w_xi, w_yi)): + continue + unify(space, w_xi, w_yi) + return space.w_None -def unify__List_List(space, w_x, w_y): - return _unify_iterables(space, w_x, w_y) - -def _unify_iterables(space, w_i1, w_i2): - assert isinstance(w_i1, W_TupleObject) or isinstance(w_i1, W_ListObject) - assert isinstance(w_i2, W_TupleObject) or isinstance(w_i2, W_ListObject) - #print " :unify iterables", w_i1, w_i2 + +def unify__List_List(space, w_i1, w_i2): if len(w_i1.wrappeditems) != len(w_i2.wrappeditems): fail(space, w_i1, w_i2) idx, top = (-1, space.int_w(space.len(w_i1))-1) @@ -599,6 +604,12 @@ if space.is_true(space.is_nb_(w_xi, w_yi)): continue unify(space, w_xi, w_yi) + return space.w_None + +## def _unify_iterables(space, w_i1, w_i2): +## assert isinstance(w_i1, W_TupleObject) or isinstance(w_i1, W_ListObject) +## assert isinstance(w_i2, W_TupleObject) or isinstance(w_i2, W_ListObject) +## #print " :unify iterables", w_i1, w_i2 def unify__Dict_Dict(space, w_m1, w_m2): assert isinstance(w_m1, W_DictObject) From nik at codespeak.net Wed Apr 5 15:24:41 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 15:24:41 +0200 (CEST) Subject: [pypy-svn] r25384 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060405132441.90908101EF@code0.codespeak.net> Author: nik Date: Wed Apr 5 15:24:40 2006 New Revision: 25384 Modified: pypy/dist/pypy/rpython/ootypesystem/riterable.py pypy/dist/pypy/rpython/ootypesystem/rlist.py Log: some subtle reshuffling of code to make sure setup for ListReprs is called before IteratorReprs are created. this fixes a failure that i missed at the last checkin. Modified: pypy/dist/pypy/rpython/ootypesystem/riterable.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/riterable.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/riterable.py Wed Apr 5 15:24:40 2006 @@ -2,12 +2,15 @@ _iter_types = {} -def iterator_type(r_iterable): - key = r_iterable.lowleveltype +def iterator_type(r_iterable, key=None): + if key is None: + key = r_iterable.lowleveltype + else: + key = key if _iter_types.has_key(key): return _iter_types[key] else: ITER = ootype.Instance("Iterator", ootype.ROOT, - {"iterable": key, "index": ootype.Signed}) + {"iterable": r_iterable.lowleveltype, "index": ootype.Signed}) _iter_types[key] = ITER return ITER Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 15:24:40 2006 @@ -87,7 +87,7 @@ def __init__(self, r_list): self.r_list = r_list - self.lowleveltype = iterator_type(r_list) + self.lowleveltype = iterator_type(r_list, r_list.item_repr) self.ll_listiter = ll_listiter self.ll_listnext = ll_listnext From arigo at codespeak.net Wed Apr 5 15:32:51 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 15:32:51 +0200 (CEST) Subject: [pypy-svn] r25386 - in pypy: branch/jit-timeshifter-refactoring dist/pypy/jit/timeshifter dist/pypy/jit/timeshifter/test Message-ID: <20060405133251.06105101F3@code0.codespeak.net> Author: arigo Date: Wed Apr 5 15:32:51 2006 New Revision: 25386 Added: pypy/dist/pypy/jit/timeshifter/rcontainer.py - copied unchanged from r25384, pypy/branch/jit-timeshifter-refactoring/timeshifter/rcontainer.py pypy/dist/pypy/jit/timeshifter/rtimeshift.py - copied unchanged from r25384, pypy/branch/jit-timeshifter-refactoring/timeshifter/rtimeshift.py pypy/dist/pypy/jit/timeshifter/rtyper.py - copied unchanged from r25384, pypy/branch/jit-timeshifter-refactoring/timeshifter/rtyper.py pypy/dist/pypy/jit/timeshifter/rvalue.py - copied unchanged from r25384, pypy/branch/jit-timeshifter-refactoring/timeshifter/rvalue.py pypy/dist/pypy/jit/timeshifter/test/ - copied from r25384, pypy/branch/jit-timeshifter-refactoring/timeshifter/test/ pypy/dist/pypy/jit/timeshifter/timeshift.py - copied unchanged from r25384, pypy/branch/jit-timeshifter-refactoring/timeshifter/timeshift.py Removed: pypy/branch/jit-timeshifter-refactoring/ Log: merge of the jit-timeshifter-refactoring branch. From mwh at codespeak.net Wed Apr 5 16:07:38 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 16:07:38 +0200 (CEST) Subject: [pypy-svn] r25387 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060405140738.8745C101E4@code0.codespeak.net> Author: mwh Date: Wed Apr 5 16:07:37 2006 New Revision: 25387 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (mwh, pedronis) test_gctransform had bitrotted slightly. make some stuff more explicit, delete some old tests and write some new ones. Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Wed Apr 5 16:07:37 2006 @@ -21,28 +21,32 @@ else: refs_in = len([v for v in block.inputargs if isinstance(v, Variable) and var_needsgc(v)]) push_alives = len([op for op in block.operations - if op.opname.startswith('gc_push_alive')]) + \ - len([op for op in block.operations - if var_ispyobj(op.result) and 'direct_call' not in op.opname]) + if op.opname == 'gc_push_alive']) + pyobj_push_alives = len([op for op in block.operations + if op.opname == 'gc_push_alive_pyobj']) + + # implicit_pyobj_pushalives included calls to things that return pyobject* + implicit_pyobj_pushalives = len([op for op in block.operations + if var_ispyobj(op.result) + and op.opname not in ('getfield', 'getarrayitem', 'same_as')]) + nonpyobj_gc_returning_calls = len([op for op in block.operations + if op.opname in ('direct_call', 'indirect_call') + and var_needsgc(op.result) + and not var_ispyobj(op.result)]) pop_alives = len([op for op in block.operations - if op.opname.startswith('gc_pop_alive')]) - calls = len([op for op in block.operations - if 'direct_call' in op.opname and var_needsgc(op.result)]) + if op.opname == 'gc_pop_alive']) + pyobj_pop_alives = len([op for op in block.operations + if op.opname == 'gc_pop_alive_pyobj']) if pop_alives == len(block.operations): # it's a block we inserted return for link in block.exits: - fudge = 0 - if (block.exitswitch is c_last_exception and link.exitcase is not None): - fudge -= 1 - if var_needsgc(block.operations[-1].result): - fudge += 1 + assert block.exitswitch is not c_last_exception refs_out = len([v for v in link.args if var_needsgc(v)]) - assert refs_in + push_alives + calls - fudge == pop_alives + refs_out - - if block.exitswitch is c_last_exception and link.exitcase is not None: - assert link.last_exc_value in link.args + pyobj_pushes = pyobj_push_alives + implicit_pyobj_pushalives + nonpyobj_pushes = push_alives + nonpyobj_gc_returning_calls + assert refs_in + pyobj_pushes + nonpyobj_pushes == pop_alives + pyobj_pop_alives + refs_out def getops(graph): ops = {} @@ -150,22 +154,6 @@ passedname = link.target.exits[0].args[0].name assert dyingname != passedname -def DONOTtest_cleanup_vars_on_call(): - S = lltype.GcStruct("S", ('x', lltype.Signed)) - def f(): - return lltype.malloc(S) - def g(): - s1 = f() - s2 = f() - s3 = f() - return s1 - t, transformer = rtype_and_transform(g, [], gctransform.GCTransformer) - ggraph = graphof(t, g) - direct_calls = [op for op in ggraph.startblock.operations if op.opname == "direct_call"] - assert len(direct_calls) == 3 - assert direct_calls[1].cleanup[1][0].args[0] == direct_calls[0].result - assert [op.args[0] for op in direct_calls[2].cleanup[1]] == \ - [direct_calls[0].result, direct_calls[1].result] def test_multiply_passed_var(): S = lltype.GcStruct("S", ('x', lltype.Signed)) @@ -196,6 +184,29 @@ for op in gcops: assert op.opname.endswith("_pyobj") +def test_call_return_pyobj(): + def g(factory): + return factory() + def f(factory): + g(factory) + t, transformer = rtype_and_transform(f, [object], gctransform.GCTransformer) + fgraph = graphof(t, f) + ops = getops(fgraph) + calls = ops['direct_call'] + for call in calls: + if call.result.concretetype is not lltype.Bool: #RPyExceptionOccurred() + assert var_ispyobj(call.result) + +def test_getfield_pyobj(): + class S: + pass + def f(thing): + s = S() + s.x = thing + return s.x + t, transformer = rtype_and_transform(f, [object], gctransform.GCTransformer) + + def test_pass_gc_pointer(): S = lltype.GcStruct("S", ('x', lltype.Signed)) def f(s): From mwh at codespeak.net Wed Apr 5 16:10:45 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 16:10:45 +0200 (CEST) Subject: [pypy-svn] r25388 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060405141045.1E959101F5@code0.codespeak.net> Author: mwh Date: Wed Apr 5 16:10:44 2006 New Revision: 25388 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: assert that a test is actually testing something Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Wed Apr 5 16:10:44 2006 @@ -205,6 +205,19 @@ s.x = thing return s.x t, transformer = rtype_and_transform(f, [object], gctransform.GCTransformer) + fgraph = graphof(t, f) + pyobj_getfields = 0 + pyobj_setfields = 0 + for b in fgraph.iterblocks(): + for op in b.operations: + if op.opname == 'getfield' and var_ispyobj(op.result): + pyobj_getfields += 1 + elif op.opname == 'setfield' and var_ispyobj(op.args[2]): + pyobj_setfields += 1 + assert pyobj_getfields > 0 + assert pyobj_setfields > 0 + + def test_pass_gc_pointer(): From mwh at codespeak.net Wed Apr 5 16:11:07 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 16:11:07 +0200 (CEST) Subject: [pypy-svn] r25389 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060405141107.52EF6101F9@code0.codespeak.net> Author: mwh Date: Wed Apr 5 16:11:06 2006 New Revision: 25389 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: whitespace silliness Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Wed Apr 5 16:11:06 2006 @@ -216,9 +216,6 @@ pyobj_setfields += 1 assert pyobj_getfields > 0 assert pyobj_setfields > 0 - - - def test_pass_gc_pointer(): S = lltype.GcStruct("S", ('x', lltype.Signed)) From auc at codespeak.net Wed Apr 5 16:30:54 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 16:30:54 +0200 (CEST) Subject: [pypy-svn] r25390 - pypy/dist/pypy/objspace/logic Message-ID: <20060405143054.17842101F7@code0.codespeak.net> Author: auc Date: Wed Apr 5 16:30:53 2006 New Revision: 25390 Added: pypy/dist/pypy/objspace/logic/constraint.py pypy/dist/pypy/objspace/logic/distributor.py pypy/dist/pypy/objspace/logic/domain.py Log: to be made interplevel friendly Added: pypy/dist/pypy/objspace/logic/constraint.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/logic/constraint.py Wed Apr 5 16:30:53 2006 @@ -0,0 +1,387 @@ +from variable import NoDom +import operator + +#-- Exceptions --------------------------------------- + +class ConsistencyFailure(Exception): + """The repository is not in a consistent state""" + pass + +class DomainlessVariables(Exception): + """A constraint can't be defined on variables + without a domain""" + pass + +#-- Constraints ------------------------------------------ + +class AbstractConstraint(object): + + def __init__(self, c_space, variables): + """variables is a list of variables which appear in the formula""" + self.cs = c_space + self._names_to_vars = {} + for var in variables: + if self.cs.dom(var) == NoDom: + raise DomainlessVariables + self._names_to_vars[var.name] = var + self._variables = variables + + def affected_variables(self): + """ Return a list of all variables affected by this constraint """ + return self._variables + + def isVariableRelevant(self, variable): + return variable in self._variables + + def estimate_cost(self): + """Return an estimate of the cost of the narrowing of the constraint""" + return reduce(operator.mul, + [self.cs.dom(var).size() for var in self._variables]) + + def copy_to(self, space): + return self.__class__(space, self._variables) + + def __eq__(self, other): #FIXME and parent + if not isinstance(other, self.__class__): return False + return self._variables == other._variables + +class BasicConstraint(object): + """A BasicConstraint, which is never queued by the Repository + A BasicConstraint affects only one variable, and will be entailed + on the first call to narrow()""" + + def __init__(self, variable, reference, operator): + """variables is a list of variables on which + the constraint is applied""" + self._variable = variable + self._reference = reference + self._operator = operator + + def __repr__(self): + return '<%s %s %s>'% (self.__class__, self._variable, self._reference) + + def copy_to(self, space): + raise NotImplementedError + + def isVariableRelevant(self, variable): + return variable == self._variable + + def estimateCost(self, domains): + return 0 # get in the first place in the queue + + def affectedVariables(self): + return [self._variable] + + def getVariable(self): + return self._variable + + def revise(self, domains): + domain = domains[self._variable] + operator = self._operator + ref = self._reference + try: + for val in domain.get_values() : + if not operator(val, ref) : + domain.remove_value(val) + except ConsistencyFailure: + raise ConsistencyFailure('inconsistency while applying %s' % \ + repr(self)) + return 1 + + def __eq__(self, other): + raise NotImplementedError + +def make_lambda_head(vars): + var_ids = ','.join([var.name for var in vars]) + return 'lambda ' + var_ids + ':' + +def expand_expr_template(expr, vars): + for var in vars: + expr.replace(var.name, var.name + '.val') + return expr + + +class AllDistinct(AbstractConstraint): + """Contraint: all values must be distinct""" + + def __init__(self, c_space, variables): + assert len(variables)>1 + AbstractConstraint.__init__(self, c_space, variables) + # worst case complexity + self.__cost = len(variables) * (len(variables) - 1) / 2 + + def __repr__(self): + return '' % str(self._variables) + + def copy_to(self, space): + return self.__class__(space, self._variables) + + def estimateCost(self, domains): + return self.__cost + + def test_solution(self, sol): + """test a solution against this constraint + accept a mapping of variable names to value""" + values = sol.items() + value_set = set(values) + return len(value_set) == len(sol) + + def revise(self): + variables = [(self.cs.dom(variable).size(), + variable, self.cs.dom(variable)) + for variable in self._variables] + + variables.sort() + # if a domain has a size of 1, + # then the value must be removed from the other domains + for size, var, dom in variables: + if dom.size() == 1: + print "AllDistinct removes values" + for _siz, _var, _dom in variables: + if _var != var: + try: + _dom.remove_value(dom.get_values()[0]) + except KeyError: + # we ignore errors caused by the removal of + # non existing values + pass + + # if there are less values than variables, the constraint fails + values = {} + for size, var, dom in variables: + for val in dom: + values[val] = 0 + if len(values) < len(variables): + print "AllDistinct failed" + raise ConsistencyFailure() + + # the constraint is entailed if all domains have a size of 1 + for variable in variables: + if variable[2].size() != 1: + return 0 + + # Question : did we *really* completely check + # our own alldistinctness predicate ? + + return 1 + + +class Expression(AbstractConstraint): + """A constraint represented as a python expression.""" + _FILTER_CACHE = {} + + def __init__(self, c_space, variables, formula, typ='fd.Expression'): + """variables is a list of variables which appear in the formula + formula is a python expression that will be evaluated as a boolean""" + self.formula = formula + self.type = typ + AbstractConstraint.__init__(self, c_space, variables) + try: + self.filterFunc = Expression._FILTER_CACHE[formula] + except KeyError: + self.filterFunc = eval(make_lambda_head(variables) \ + + expand_expr_template(formula, variables), {}, {}) + Expression._FILTER_CACHE[formula] = self.filterFunc + + def test_solution(self, sol ): + """test a solution against this constraint + accept a mapping of variable names to value""" + args = [] + for var in self._variables: + args.append( sol[var.name] ) + return self.filterFunc( *args ) + + + def copy_to(self, space): + return self.__class__(space, self._variables, + self.formula, self.type) + + def _init_result_cache(self): + """key = (variable,value), value = [has_success,has_failure]""" + result_cache = {} + for var_name in self._variables: + result_cache[var_name.name] = {} + return result_cache + + + def _assign_values(self): + variables = [] + kwargs = {} + for variable in self._variables: + domain = self.cs.dom(variable) + values = domain.get_values() + variables.append((domain.size(), [variable, values, 0, len(values)])) + kwargs[variable.name] = values[0] + # sort variables to instanciate those with fewer possible values first + variables.sort() + + go_on = 1 + while go_on: + yield kwargs + # try to instanciate the next variable + for size, curr in variables: + if (curr[2] + 1) < curr[-1]: + curr[2] += 1 + kwargs[curr[0].name] = curr[1][curr[2]] + break + else: + curr[2] = 0 + kwargs[curr[0].name] = curr[1][0] + else: + # it's over + go_on = 0 + + def revise(self): + # removed domain arg. (auc, ale) + """generic propagation algorithm for n-ary expressions""" + maybe_entailed = 1 + ffunc = self.filterFunc + result_cache = self._init_result_cache() + for kwargs in self._assign_values(): + if maybe_entailed: + for var, val in kwargs.iteritems(): + if val not in result_cache[var]: + break + else: + continue + if ffunc(**kwargs): + for var, val in kwargs.items(): + result_cache[var][val] = 1 + else: + maybe_entailed = 0 + + try: + for var, keep in result_cache.iteritems(): + domain = self.cs.dom(self._names_to_vars[var]) + domain.remove_values([val for val in domain if val not in keep]) + + except ConsistencyFailure: + raise ConsistencyFailure('Inconsistency while applying %s' % \ + repr(self)) + except KeyError: + # There are no more value in result_cache + pass + + return maybe_entailed + + def __eq__(self, other): + if not super(Expression, self).__eq__(other): return False + r1 = self.formula == other.formula + r2 = self.type == other.type + return r1 and r2 + + + def __repr__(self): + return '<%s>' % self.formula + +class BinaryExpression(Expression): + """A binary constraint represented as a python expression + + This implementation uses a narrowing algorithm optimized for + binary constraints.""" + + def __init__(self, variables, formula, type = 'fd.BinaryExpression'): + assert len(variables) == 2 + Expression.__init__(self, variables, formula, type) + + def copy_to(self, space): + raise NotImplementedError + + def revise(self, domains): + """specialized narrowing algorithm for binary expressions + Runs much faster than the generic version""" + maybe_entailed = 1 + var1 = self._variables[0] + dom1 = domains[var1] + values1 = dom1.get_values() + var2 = self._variables[1] + dom2 = domains[var2] + values2 = dom2.get_values() + ffunc = self.filterFunc + if dom2.size() < dom1.size(): + var1, var2 = var2, var1 + dom1, dom2 = dom2, dom1 + values1, values2 = values2, values1 + + kwargs = {} + keep1 = {} + keep2 = {} + maybe_entailed = 1 + try: + # iterate for all values + for val1 in values1: + kwargs[var1] = val1 + for val2 in values2: + kwargs[var2] = val2 + if val1 in keep1 and val2 in keep2 and maybe_entailed == 0: + continue + if ffunc(**kwargs): + keep1[val1] = 1 + keep2[val2] = 1 + else: + maybe_entailed = 0 + + dom1.remove_values([val for val in values1 if val not in keep1]) + dom2.remove_values([val for val in values2 if val not in keep2]) + + except ConsistencyFailure: + raise ConsistencyFailure('Inconsistency while applying %s' % \ + repr(self)) + except Exception: + print self, kwargs + raise + return maybe_entailed + + +def make_expression(variables, formula, constraint_type=None): + """create a new constraint of type Expression or BinaryExpression + The chosen class depends on the number of variables in the constraint""" + # encode unicode + vars = [] + for var in variables: + if type(var) == type(u''): + vars.append(var.encode()) + else: + vars.append(var) + if len(vars) == 2: + if constraint_type is not None: + return BinaryExpression(vars, formula, constraint_type) + else: + return BinaryExpression(vars, formula) + + else: + if constraint_type is not None: + return Expression(vars, formula, constraint_type) + else: + return Expression(vars, formula) + + +class Equals(BasicConstraint): + """A basic constraint variable == constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.eq) + +class NotEquals(BasicConstraint): + """A basic constraint variable != constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.ne) + +class LesserThan(BasicConstraint): + """A basic constraint variable < constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.lt) + +class LesserOrEqual(BasicConstraint): + """A basic constraint variable <= constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.le) + +class GreaterThan(BasicConstraint): + """A basic constraint variable > constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.gt) + +class GreaterOrEqual(BasicConstraint): + """A basic constraint variable >= constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.ge) Added: pypy/dist/pypy/objspace/logic/distributor.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/logic/distributor.py Wed Apr 5 16:30:53 2006 @@ -0,0 +1,123 @@ +import math + +def arrange_domains(cs, variables): + """build a data structure from var to dom + that satisfies distribute & friends""" + new_doms = {} + for var in variables: + new_doms[var] = cs.dom(var).copy() + return new_doms + +class AbstractDistributor(object): + """_distribute is left unimplemented.""" + + def __init__(self, c_space, nb_subspaces=2): + self.nb_subspaces = nb_subspaces + self.cs = c_space + self.verbose = 0 + + def set_space(self, space): + self.cs = space + + def findSmallestDomain(self): + """returns the variable having the smallest domain. + (or one of such varibles if there is a tie) + """ + vars_ = [var for var in self.cs.get_variables_with_a_domain() + if self.cs.dom(var).size() > 1] + + best = vars_[0] + for var in vars_: + if self.cs.dom(var).size() < self.cs.dom(best).size(): + best = var + + return best + + def nb_subdomains(self): + """return number of sub domains to explore""" + return self.nb_subspaces + + def distribute(self, choice): + raise NotImplementedError("Use a concrete implementation of " + "the Distributor interface") + +## def distribute(self, verbose=0): +## """do the minimal job and let concrete class distribute variables +## """ +## self.verbose = verbose +## variables = self.cs.get_variables_with_a_domain() +## replicas = [] +## for i in range(self.nb_subdomains()): +## replicas.append(arrange_domains(self.cs, variables)) +## modified_domains = self._distribute(*replicas) +## for domain in modified_domains: +## domain.reset_flags() +## return replicas + + +class NaiveDistributor(AbstractDistributor): + """distributes domains by splitting the smallest domain in 2 new domains + The first new domain has a size of one, + and the second has all the other values""" + + def distribute(self, dom1, dom2): + """See AbstractDistributor""" + raise NotImplementedError + variable = self.findSmallestDomain(dom1) + values = dom1[variable].get_values() + if self.verbose: + print 'Distributing domain for variable', variable, \ + 'at value', values[0] + dom1[variable].remove_values(values[1:]) + dom2[variable].remove_value(values[0]) + return (dom1[variable], dom2[variable]) + + +class SplitDistributor(AbstractDistributor): + """distributes domains by splitting the smallest domain in + nb_subspaces equal parts or as equal as possible. + If nb_subspaces is 0, then the smallest domain is split in + domains of size 1""" + + def __init__(self, c_space, nb_subspaces=3): + AbstractDistributor.__init__(self, c_space, nb_subspaces) + self.__to_split = None + + def nb_subdomains(self): + """See AbstractDistributor""" + self.__to_split = self.findSmallestDomain() + if self.nb_subspaces: + return min(self.nb_subspaces, + self.cs.dom(self.__to_split).size()) + else: + return self.cs.dom(self.__to_split).size() + + + def distribute(self, choice): + variable = self.findSmallestDomain() + nb_subspaces = self.nb_subdomains() + values = self.cs.dom(variable).get_values() + nb_elts = max(1, len(values)*1./nb_subspaces) + start, end = (int(math.floor(choice * nb_elts)), + int(math.floor((choice + 1) * nb_elts))) + self.cs.dom(variable).remove_values(values[:start]) + self.cs.dom(variable).remove_values(values[end:]) + + for const in self.cs.dependant_constraints(variable): + self.cs.event_set.add(const) + + +class DichotomyDistributor(SplitDistributor): + """distributes domains by splitting the smallest domain in + two equal parts or as equal as possible""" + def __init__(self, c_space): + SplitDistributor.__init__(self, c_space, 2) + + +class EnumeratorDistributor(SplitDistributor): + """distributes domains by splitting the smallest domain + in domains of size 1.""" + def __init__(self, c_space): + SplitDistributor.__init__(self, c_space, 0) + +DefaultDistributor = DichotomyDistributor Added: pypy/dist/pypy/objspace/logic/domain.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/logic/domain.py Wed Apr 5 16:30:53 2006 @@ -0,0 +1,101 @@ +from pypy.interpreter.error import OperationError + +from pypy.interpreter import baseobjspace, gateway +from pypy.interpreter.baseobjspace import Wrappable + +from pypy.objspace.std.objspace import W_Object + +# ? +from pypy.objspace.std.listobject import W_ListObject, W_TupleObject + + +class ConsistencyFailure(Exception): + """The repository is not in a consistent state""" + pass + + +class W_AbstractDomain(Wrappable): + """Implements the functionnality related to the changed flag. + Can be used as a starting point for concrete domains""" + + def __init__(self, space): + self._space = space + self.__changed = 0 + + def w_reset_flags(self): + self.__changed = 0 + + def w_has_changed(self): + return self.__changed + + def _value_removed(self): + """The implementation of remove_value should call this method""" + self.__changed = 1 + if self.size() == 0: + raise ConsistencyFailure() + +W_AbstractDomain.typedef = TypeDef("W_AbstractDomain", + reset_flags = interp2app(W_AbstractDomain.w_reset_flags) + has_changed = interp2app(W_AbstractDomain.w_has_changed)) + +class W_FiniteDomain(AbstractDomain): + """ + Variable Domain with a finite set of possible values + """ + + def __init__(self, values): + """values is a list of values in the domain + This class uses a dictionnary to make sure that there are + no duplicate values""" + AbstractDomain.__init__(self) + self.set_values(values) + + def set_values(self, values): + self._values = set(values) + + def w_remove_value(self, value): + """Remove value of domain and check for consistency""" + self._values.remove(value) + self._value_removed() + + def w_remove_values(self, values): + """Remove values of domain and check for consistency""" + if values: + for val in values : + self._values.remove(val) + self._value_removed() + __delitem__ = remove_value + + def w_size(self): + """computes the size of a finite domain""" + return len(self._values) + __len__ = size + + def w_get_values(self): + """return all the values in the domain + in an indexable sequence""" + return list(self._values) + + def __iter__(self): + return iter(self._values) + + def w_copy(self): + """clone the domain""" + return FiniteDomain(self) + + def __repr__(self): + return '' % str(self.get_values()) + + def __eq__(self, other): + if other is NoDom: return False + return self._values == other._values + + def __ne__(self, other): + return not self == other + + def intersection(self, other): + if other is None: return self.get_values() + return self._values & other._values + +W_FiniteDomain.typedef = TypeDef("W_FiniteDomain", + From hpk at codespeak.net Wed Apr 5 16:35:47 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 5 Apr 2006 16:35:47 +0200 (CEST) Subject: [pypy-svn] r25391 - pypy/dist/pypy/translator/c Message-ID: <20060405143547.4E3C9101FA@code0.codespeak.net> Author: hpk Date: Wed Apr 5 16:35:46 2006 New Revision: 25391 Modified: pypy/dist/pypy/translator/c/exceptiontransform.py Log: only insert keepalives of a variable along a link if the variable isn't there already Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Wed Apr 5 16:35:46 2006 @@ -30,6 +30,7 @@ assert 0, "not implemented yet" def insert_keepalives_along(translator, link, vars): + vars = [v for v in vars if v not in link.args] link.args.extend(vars) newvars = [copyvar(translator, v) for v in vars] block = link.target From mwh at codespeak.net Wed Apr 5 16:52:55 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 16:52:55 +0200 (CEST) Subject: [pypy-svn] r25393 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060405145255.774B410202@code0.codespeak.net> Author: mwh Date: Wed Apr 5 16:52:53 2006 New Revision: 25393 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (mwh, pedronis) implement replace_set{arrayitem,field} for pyobject*s in the base gctransformer, as these always need write-barrier style protection. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Wed Apr 5 16:52:53 2006 @@ -190,6 +190,30 @@ needs to overwrite this""" raise NotImplementedError("gc_protect does not make sense for this gc") + def replace_setfield(self, op, livevars, block): + if not var_ispyobj(op.args[2]): + return [op] + oldval = varoftype(op.args[2].concretetype) + getoldvalop = SpaceOperation("getfield", + [op.args[0], op.args[1]], oldval) + result = [getoldvalop] + result.extend(self.push_alive(op.args[2])) + result.append(op) + result.extend(self.pop_alive(oldval)) + return result + + def replace_setarrayitem(self, op, livevars, block): + if not var_ispyobj(op.args[2]): + return [op] + oldval = varoftype(op.args[2].concretetype) + getoldvalop = SpaceOperation("getarrayitem", + [op.args[0], op.args[1]], oldval) + result = [getoldvalop] + result.extend(self.push_alive(op.args[2])) + result.append(op) + result.extend(self.pop_alive(oldval)) + return result + def annotate_helper(self, ll_helper, ll_args, ll_result): assert not self.finished args_s = map(annmodel.lltype_to_annotation, ll_args) Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Wed Apr 5 16:52:53 2006 @@ -214,8 +214,10 @@ pyobj_getfields += 1 elif op.opname == 'setfield' and var_ispyobj(op.args[2]): pyobj_setfields += 1 - assert pyobj_getfields > 0 - assert pyobj_setfields > 0 + # although there's only one explicit getfield in the code, a + # setfield on a pyobj must get the old value out and decref it + assert pyobj_getfields >= 2 + assert pyobj_setfields >= 1 def test_pass_gc_pointer(): S = lltype.GcStruct("S", ('x', lltype.Signed)) From nik at codespeak.net Wed Apr 5 17:05:25 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Wed, 5 Apr 2006 17:05:25 +0200 (CEST) Subject: [pypy-svn] r25394 - in pypy/dist/pypy/rpython: . lltypesystem ootypesystem test Message-ID: <20060405150525.42C6F10204@code0.codespeak.net> Author: nik Date: Wed Apr 5 17:05:23 2006 New Revision: 25394 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py Log: implement rtyping of list concatenation and extenion for ootypes. this introduces a slightly convoluted multiple inheritance setup for ListReprs to allow for maximum code sharing between lltype and ootype. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Wed Apr 5 17:05:23 2006 @@ -4,8 +4,8 @@ from pypy.rpython.error import TyperError from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.rmodel import externalvsinternal -from pypy.rpython.rlist import AbstractListRepr, AbstractListIteratorRepr, \ - rtype_newlist +from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ + AbstractListIteratorRepr, rtype_newlist from pypy.rpython.rlist import dum_nocheck, dum_checkidx from pypy.rpython.rslice import SliceRepr from pypy.rpython.rslice import startstop_slice_repr, startonly_slice_repr @@ -35,7 +35,7 @@ # item_t list_items[] # -class BaseListRepr(AbstractListRepr): +class BaseListRepr(AbstractBaseListRepr): def __init__(self, rtyper, item_repr, listitem=None): self.rtyper = rtyper @@ -49,6 +49,8 @@ self.listitem = listitem self.list_cache = {} # setup() needs to be called to finish this initialization + self.ll_concat = ll_concat + self.ll_extend = ll_extend self.list_builder = ListBuilder() def _setup_repr_final(self): @@ -184,7 +186,7 @@ return 1 # bad but not used alone -class ListRepr(BaseListRepr): +class ListRepr(AbstractListRepr, BaseListRepr): def _setup_repr(self): if 'item_repr' not in self.__dict__: @@ -403,11 +405,6 @@ return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) - def rtype_add((r_lst1, r_lst2), hop): - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - return hop.gendirectcall(ll_concat, cRESLIST, v_lst1, v_lst2) - def rtype_eq((r_lst1, r_lst2), hop): assert r_lst1.item_repr == r_lst2.item_repr v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) @@ -422,14 +419,6 @@ def ll_both_none(lst1, lst2): return not lst1 and not lst2 -class __extend__(pairtype(ListRepr, BaseListRepr)): - - def rtype_inplace_add((r_lst1, r_lst2), hop): - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - hop.gendirectcall(ll_extend, v_lst1, v_lst2) - return v_lst1 - - # ____________________________________________________________ # # Low-level methods. These can be run for testing, but are meant to Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 5 17:05:23 2006 @@ -1,27 +1,28 @@ from pypy.annotation.pairtype import pairtype -from pypy.rpython.rlist import AbstractListRepr, AbstractListIteratorRepr, \ - rtype_newlist +from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ + AbstractListIteratorRepr, rtype_newlist from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.rmodel import inputconst, externalvsinternal from pypy.rpython.ootypesystem import ootype from pypy.rpython.ootypesystem.riterable import iterator_type -class BaseListRepr(AbstractListRepr): +class BaseListRepr(AbstractBaseListRepr): def __init__(self, rtyper, item_repr, listitem=None): self.rtyper = rtyper - # XXX do we need something like this for ootypes? - #self.LIST = GcForwardReference() if not isinstance(item_repr, Repr): # not computed yet, done by setup() assert callable(item_repr) self._item_repr_computer = item_repr - self.lowleveltype = ootype.ForwardReference() + self.LIST = ootype.ForwardReference() else: - self.lowleveltype = ootype.List(item_repr.lowleveltype) + self.LIST = ootype.List(item_repr.lowleveltype) self.external_item_repr, self.item_repr = \ externalvsinternal(rtyper, item_repr) + self.lowleveltype = self.LIST self.listitem = listitem self.list_cache = {} + self.ll_concat = ll_concat + self.ll_extend = ll_extend # setup() needs to be called to finish this initialization def _setup_repr(self): @@ -48,8 +49,11 @@ def make_iterator_repr(self): return ListIteratorRepr(self) -ListRepr = BaseListRepr -FixedSizeListRepr = BaseListRepr +class ListRepr(AbstractListRepr, BaseListRepr): + + pass + +FixedSizeListRepr = ListRepr class __extend__(pairtype(BaseListRepr, IntegerRepr)): @@ -71,7 +75,7 @@ resulttype=ootype.Void) return v_result -# These helpers are trivial but help encapsulation +# These helpers are sometimes trivial but help encapsulation def ll_newlist(LIST): return ootype.new(LIST) @@ -79,6 +83,28 @@ def ll_append(lst, item): lst.append(item) +def ll_extend(l1, l2): + # This is a bit inefficient, could also add extend to the list interface + len2 = l2.length() + i = 0 + while i < len2: + l1.append(l2.getitem(i)) + i += 1 + +def ll_concat(RESLIST, l1, l2): + len1 = l1.length() + len2 = l2.length() + l = ootype.new(RESLIST) + i = 0 + while i < len1: + l.append(l1.getitem(i)) + i += 1 + i = 0 + while i < len2: + l.append(l2.getitem(i)) + i += 1 + return l + # ____________________________________________________________ # # Iteration. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Wed Apr 5 17:05:23 2006 @@ -1,3 +1,4 @@ +from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.rpython.rmodel import Repr, IteratorRepr, inputconst from pypy.rpython.lltypesystem import lltype @@ -28,11 +29,15 @@ return self.__class__, self.listdef.listitem -class AbstractListRepr(Repr): +class AbstractBaseListRepr(Repr): def recast(self, llops, v): return llops.convertvar(v, self.item_repr, self.external_item_repr) +class AbstractListRepr(AbstractBaseListRepr): + + pass + def rtype_newlist(hop): nb_args = hop.nb_args r_list = hop.r_result @@ -53,6 +58,21 @@ def dum_checkidx(): pass def dum_nocheck(): pass + +class __extend__(pairtype(AbstractBaseListRepr, AbstractBaseListRepr)): + + def rtype_add((r_lst1, r_lst2), hop): + v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) + cRESLIST = hop.inputconst(lltype.Void, hop.r_result.LIST) + return hop.gendirectcall(hop.r_result.ll_concat, cRESLIST, v_lst1, v_lst2) + +class __extend__(pairtype(AbstractListRepr, AbstractBaseListRepr)): + + def rtype_inplace_add((r_lst1, r_lst2), hop): + v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) + hop.gendirectcall(r_lst1.ll_extend, v_lst1, v_lst2) + return v_lst1 + # ____________________________________________________________ # # Iteration. Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Wed Apr 5 17:05:23 2006 @@ -234,25 +234,22 @@ res = interpret(dummyfn, [5]) assert res == 5 -def tolst(l): - return map(None, l.ll_items())[:l.ll_length()] - -def test_add(): - def dummyfn(): - l = [5] - l += [6,7] - return l + [8] - res = interpret(dummyfn, []) - assert tolst(res) == [5, 6, 7, 8] - - def dummyfn(): - l = [5] - l += [6,7] - l2 = l + [8] - l2.append(9) - return l2 - res = interpret(dummyfn, []) - assert tolst(res) == [5, 6, 7, 8, 9] + def test_add(self): + def dummyfn(): + l = [5] + l += [6,7] + return l + [8] + res = interpret(dummyfn, [], type_system=self.ts) + assert self.ll_to_list(res) == [5, 6, 7, 8] + + def dummyfn(): + l = [5] + l += [6,7] + l2 = l + [8] + l2.append(9) + return l2 + res = interpret(dummyfn, [], type_system=self.ts) + assert self.ll_to_list(res) == [5, 6, 7, 8, 9] def test_slice(): def dummyfn(): @@ -303,6 +300,9 @@ res = interpret(dummyfn, []) assert res == 0 +def tolst(l): + return map(None, l.ll_items())[:l.ll_length()] + def test_setslice(): def dummyfn(): l = [10, 9, 8, 7] @@ -1053,7 +1053,12 @@ ts = "lltype" + def ll_to_list(self, l): + return map(None, l.ll_items())[:l.ll_length()] + class TestOotypeRtyping(BaseTestListRtyping): ts = "ootype" + def ll_to_list(self, l): + return l._list[:] From afayolle at codespeak.net Wed Apr 5 17:12:04 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Wed, 5 Apr 2006 17:12:04 +0200 (CEST) Subject: [pypy-svn] r25395 - pypy/dist/pypy/objspace/logic Message-ID: <20060405151204.5A292101FE@code0.codespeak.net> Author: afayolle Date: Wed Apr 5 17:11:59 2006 New Revision: 25395 Modified: pypy/dist/pypy/objspace/logic/distributor.py Log: Ported NaiveDistributor Removed EnumeratorDistributor Changed DefautDistributor to NaiveDistributor Refactored distribute method for fun, profit and duplication eradication Modified: pypy/dist/pypy/objspace/logic/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/logic/distributor.py (original) +++ pypy/dist/pypy/objspace/logic/distributor.py Wed Apr 5 17:11:59 2006 @@ -19,7 +19,7 @@ def set_space(self, space): self.cs = space - def findSmallestDomain(self): + def find_smallest_domain(self): """returns the variable having the smallest domain. (or one of such varibles if there is a tie) """ @@ -34,25 +34,26 @@ return best def nb_subdomains(self): - """return number of sub domains to explore""" + """return number of possible splits""" return self.nb_subspaces + + def distribute(self, choice): - raise NotImplementedError("Use a concrete implementation of " - "the Distributor interface") + variable = self._find_distribution_variable() + self._do_distribute(self.cs.dom(variable), choice) + for const in self.cs.dependant_constraints(variable): + self.cs.event_set.add(const) + + def _find_distribution_variable(self): + return self.find_smallest_domain() + + def _do_distribute(self, domain, choice): + """remove values from domain depending on choice""" + raise NotImplementedError + + -## def distribute(self, verbose=0): -## """do the minimal job and let concrete class distribute variables -## """ -## self.verbose = verbose -## variables = self.cs.get_variables_with_a_domain() -## replicas = [] -## for i in range(self.nb_subdomains()): -## replicas.append(arrange_domains(self.cs, variables)) -## modified_domains = self._distribute(*replicas) -## for domain in modified_domains: -## domain.reset_flags() -## return replicas class NaiveDistributor(AbstractDistributor): @@ -60,17 +61,12 @@ The first new domain has a size of one, and the second has all the other values""" - def distribute(self, dom1, dom2): - """See AbstractDistributor""" - raise NotImplementedError - variable = self.findSmallestDomain(dom1) - values = dom1[variable].get_values() - if self.verbose: - print 'Distributing domain for variable', variable, \ - 'at value', values[0] - dom1[variable].remove_values(values[1:]) - dom2[variable].remove_value(values[0]) - return (dom1[variable], dom2[variable]) + def _do_distribute(self, domain, choice): + values = domain.get_values() + if choice == 0: + domain.remove_values(values[1:]) + else: + domain.remove_value(values[0]) class SplitDistributor(AbstractDistributor): @@ -84,8 +80,7 @@ self.__to_split = None def nb_subdomains(self): - """See AbstractDistributor""" - self.__to_split = self.findSmallestDomain() + self.__to_split = self.find_smallest_domain() if self.nb_subspaces: return min(self.nb_subspaces, self.cs.dom(self.__to_split).size()) @@ -93,18 +88,14 @@ return self.cs.dom(self.__to_split).size() - def distribute(self, choice): - variable = self.findSmallestDomain() + def _do_distribute(self, domain, choice): nb_subspaces = self.nb_subdomains() - values = self.cs.dom(variable).get_values() + values = domain.get_values() nb_elts = max(1, len(values)*1./nb_subspaces) start, end = (int(math.floor(choice * nb_elts)), int(math.floor((choice + 1) * nb_elts))) - self.cs.dom(variable).remove_values(values[:start]) - self.cs.dom(variable).remove_values(values[end:]) - - for const in self.cs.dependant_constraints(variable): - self.cs.event_set.add(const) + domain.remove_values(values[:start]) + domain.remove_values(values[end:]) class DichotomyDistributor(SplitDistributor): @@ -113,11 +104,4 @@ def __init__(self, c_space): SplitDistributor.__init__(self, c_space, 2) - -class EnumeratorDistributor(SplitDistributor): - """distributes domains by splitting the smallest domain - in domains of size 1.""" - def __init__(self, c_space): - SplitDistributor.__init__(self, c_space, 0) - -DefaultDistributor = DichotomyDistributor +DefaultDistributor = NaiveDistributor From auc at codespeak.net Wed Apr 5 17:17:06 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 17:17:06 +0200 (CEST) Subject: [pypy-svn] r25396 - pypy/dist/pypy/objspace Message-ID: <20060405151706.A03DA10208@code0.codespeak.net> Author: auc Date: Wed Apr 5 17:17:05 2006 New Revision: 25396 Modified: pypy/dist/pypy/objspace/logic.py Log: Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 17:17:05 2006 @@ -553,10 +553,10 @@ if not space.eq_w(w_x, w_y): w_d1 = w_x.getdict() w_d2 = w_y.getdict() - if None not in (w_d1, w_d2): - return space.unify(w_d1, w_d2) - else: + if (w_d1 == None) or (w_d2 == None): fail(space, w_x, w_y) + else: + return space.unify(w_d1, w_d2) return space.w_None def unify__Var_Var(space, w_x, w_y): @@ -621,6 +621,7 @@ if space.is_true(space.is_nb_(w_xi, w_yi)): continue space.unify(w_xi, w_yi) + return space.w_None unify_mm = StdObjSpaceMultiMethod('unify', 2) From mwh at codespeak.net Wed Apr 5 17:21:51 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 17:21:51 +0200 (CEST) Subject: [pypy-svn] r25397 - in pypy/dist/pypy: . translator/c/test translator/tool Message-ID: <20060405152151.3641B10209@code0.codespeak.net> Author: mwh Date: Wed Apr 5 17:21:49 2006 New Revision: 25397 Modified: pypy/dist/pypy/conftest.py pypy/dist/pypy/translator/c/test/test_genc.py pypy/dist/pypy/translator/tool/cbuild.py Log: (pedronis, mwh) add a --gc option to pypy's conftest that allows one to select which gc policy test_genc uses by default (should probably be placed differently...) poke at the method test_genc uses to build its extensions so that at least using the boehm gc policy works (the framework doesn't, yet). Modified: pypy/dist/pypy/conftest.py ============================================================================== --- pypy/dist/pypy/conftest.py (original) +++ pypy/dist/pypy/conftest.py Wed Apr 5 17:21:49 2006 @@ -41,6 +41,10 @@ help="""select compiling approach. see pypy/doc/README.compiling"""), Option('--view', action="store_true", dest="view", default=False, help="view translation tests' flow graphs with Pygame"), + Option('--gc', action="store", default=None, + type="choice", dest="gcpolicy", + choices=['ref', 'boehm', 'none', 'framework'], + help="GcPolicy class to use for genc tests"), ) _SPACECACHE={} Modified: pypy/dist/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_genc.py (original) +++ pypy/dist/pypy/translator/c/test/test_genc.py Wed Apr 5 17:21:49 2006 @@ -26,7 +26,8 @@ targetdir = udir.join(modulename).ensure(dir=1) gen_source(db, modulename, str(targetdir), defines={'COUNT_OP_MALLOCS': 1}) m = make_module_from_c(targetdir.join(modulename+'.c'), - include_dirs = [os.path.dirname(autopath.this_dir)]) + include_dirs = [os.path.dirname(autopath.this_dir)], + libraries = db.gcpolicy.gc_libraries()) return m def compile(fn, argtypes, view=False, gcpolicy=None, backendopt=True): @@ -36,6 +37,20 @@ t.buildrtyper().specialize() if backendopt: backend_optimizations(t) + if gcpolicy is None: + polname = conftest.option.gcpolicy + if polname is not None: + from pypy.translator.c import gc + if polname == 'boehm': + gcpolicy = gc.BoehmGcPolicy + elif polname == 'ref': + gcpolicy = gc.RefcountingGcPolicy + elif polname == 'none': + gcpolicy = gc.NoneGcPolicy + elif polname == 'framework': + gcpolicy = gc.FrameworkGcPolicy + else: + assert False, "unknown gc policy %r"%polname db = LowLevelDatabase(t, gcpolicy=gcpolicy) entrypoint = db.get(pyobjectptr(fn)) db.complete() Modified: pypy/dist/pypy/translator/tool/cbuild.py ============================================================================== --- pypy/dist/pypy/translator/tool/cbuild.py (original) +++ pypy/dist/pypy/translator/tool/cbuild.py Wed Apr 5 17:21:49 2006 @@ -153,10 +153,10 @@ finally: lastdir.chdir() -def make_module_from_c(cfile, include_dirs=None): +def make_module_from_c(cfile, include_dirs=None, libraries=[]): cfile = py.path.local(cfile) modname = cfile.purebasename - compile_c_module([cfile], modname, include_dirs) + compile_c_module([cfile], modname, include_dirs, libraries) return import_module_from_directory(cfile.dirpath(), modname) def import_module_from_directory(dir, modname): From mwh at codespeak.net Wed Apr 5 17:25:02 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 17:25:02 +0200 (CEST) Subject: [pypy-svn] r25398 - pypy/dist/pypy/translator/c/test Message-ID: <20060405152502.22EF41020B@code0.codespeak.net> Author: mwh Date: Wed Apr 5 17:25:01 2006 New Revision: 25398 Modified: pypy/dist/pypy/translator/c/test/test_genc.py Log: (pedronis, mwh) add another test that tested the last fix to gctransform (which is best tested with boehm) also supply a boehm-specific version of one of the other tests. Modified: pypy/dist/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_genc.py (original) +++ pypy/dist/pypy/translator/c/test/test_genc.py Wed Apr 5 17:25:01 2006 @@ -315,17 +315,42 @@ assert f1() == 1 # this test shows if we have a problem with refcounting PyObject -def test_refcount_pyobj(): - def prob_with_pyobj(b): - return 3, b +if conftest.option.gc == 'boehm': + def test_refcount_pyobj(): + from pypy.rpython.lltypesystem.lloperation import llop + def prob_with_pyobj(b): + return 3, b + def collect(): + llop.gc__collect(Void) + f = compile(prob_with_pyobj, [object]) + c = compile(collect, []) + from sys import getrefcount as g + obj = None + before = g(obj) + f(obj) + f(obj) + f(obj) + f(obj) + f(obj) + c() + c() + c() + c() + c() + after = g(obj) + assert abs(before - after) < 5 +else: + def test_refcount_pyobj(): + def prob_with_pyobj(b): + return 3, b - f = compile(prob_with_pyobj, [object]) - from sys import getrefcount as g - obj = None - before = g(obj) - f(obj) - after = g(obj) - assert before == after + f = compile(prob_with_pyobj, [object]) + from sys import getrefcount as g + obj = None + before = g(obj) + f(obj) + after = g(obj) + assert before == after def test_refcount_pyobj_setfield(): import weakref, gc @@ -345,3 +370,20 @@ wref = f(weakref.ref, C) gc.collect() assert not wref() + +def test_refcount_pyobj_setfield_increfs(): + class S(object): + def __init__(self): + self.p = None + def goo(objfact): + s = S() + b = objfact() + s.p = b + return s + def foo(objfact): + s = goo(objfact) + return s.p + f = compile(foo, [object], backendopt=False) + class C(object): + pass + print f(C) From auc at codespeak.net Wed Apr 5 17:36:42 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 17:36:42 +0200 (CEST) Subject: [pypy-svn] r25399 - pypy/dist/pypy/objspace/logic Message-ID: <20060405153642.479841020D@code0.codespeak.net> Author: auc Date: Wed Apr 5 17:36:41 2006 New Revision: 25399 Modified: pypy/dist/pypy/objspace/logic/distributor.py Log: Modified: pypy/dist/pypy/objspace/logic/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/logic/distributor.py (original) +++ pypy/dist/pypy/objspace/logic/distributor.py Wed Apr 5 17:36:41 2006 @@ -77,15 +77,15 @@ def __init__(self, c_space, nb_subspaces=3): AbstractDistributor.__init__(self, c_space, nb_subspaces) - self.__to_split = None + def nb_subdomains(self): - self.__to_split = self.find_smallest_domain() + to_split = self.find_smallest_domain() if self.nb_subspaces: return min(self.nb_subspaces, - self.cs.dom(self.__to_split).size()) + self.cs.dom(to_split).size()) else: - return self.cs.dom(self.__to_split).size() + return self.cs.dom(to_split).size() def _do_distribute(self, domain, choice): From mwh at codespeak.net Wed Apr 5 17:40:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 17:40:53 +0200 (CEST) Subject: [pypy-svn] r25400 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060405154053.C3BFE1020F@code0.codespeak.net> Author: mwh Date: Wed Apr 5 17:40:52 2006 New Revision: 25400 Modified: pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/test/test_genc.py Log: (pedronis, mwh) make the default selection of gcpolicy in database.py consider --gc fix embarrassing typo in test_genc Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Wed Apr 5 17:40:52 2006 @@ -14,6 +14,7 @@ from pypy.translator.c.support import log from pypy.translator.c.extfunc import do_the_getting from pypy.translator.c.exceptiontransform import ExceptionTransformer +from pypy import conftest # ____________________________________________________________ @@ -39,7 +40,20 @@ self.pyobjmaker = PyObjMaker(self.namespace, self.get, translator, instantiators) if gcpolicy is None: from pypy.translator.c import gc - gcpolicy = gc.RefcountingGcPolicy + polname = conftest.option.gcpolicy + if polname is not None: + if polname == 'boehm': + gcpolicy = gc.BoehmGcPolicy + elif polname == 'ref': + gcpolicy = gc.RefcountingGcPolicy + elif polname == 'none': + gcpolicy = gc.NoneGcPolicy + elif polname == 'framework': + gcpolicy = gc.FrameworkGcPolicy + else: + assert False, "unknown gc policy %r"%polname + else: + gcpolicy = gc.RefcountingGcPolicy if translator is None or translator.rtyper is None: self.exctransformer = None else: Modified: pypy/dist/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_genc.py (original) +++ pypy/dist/pypy/translator/c/test/test_genc.py Wed Apr 5 17:40:52 2006 @@ -37,20 +37,6 @@ t.buildrtyper().specialize() if backendopt: backend_optimizations(t) - if gcpolicy is None: - polname = conftest.option.gcpolicy - if polname is not None: - from pypy.translator.c import gc - if polname == 'boehm': - gcpolicy = gc.BoehmGcPolicy - elif polname == 'ref': - gcpolicy = gc.RefcountingGcPolicy - elif polname == 'none': - gcpolicy = gc.NoneGcPolicy - elif polname == 'framework': - gcpolicy = gc.FrameworkGcPolicy - else: - assert False, "unknown gc policy %r"%polname db = LowLevelDatabase(t, gcpolicy=gcpolicy) entrypoint = db.get(pyobjectptr(fn)) db.complete() @@ -315,7 +301,7 @@ assert f1() == 1 # this test shows if we have a problem with refcounting PyObject -if conftest.option.gc == 'boehm': +if conftest.option.gcpolicy == 'boehm': def test_refcount_pyobj(): from pypy.rpython.lltypesystem.lloperation import llop def prob_with_pyobj(b): From arigo at codespeak.net Wed Apr 5 17:43:54 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 17:43:54 +0200 (CEST) Subject: [pypy-svn] r25401 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060405154354.961BB10211@code0.codespeak.net> Author: arigo Date: Wed Apr 5 17:43:52 2006 New Revision: 25401 Modified: pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/c/test/test_lltyped.py Log: (arre, arigo) Implemented FixedSizeArrays in genc. Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Wed Apr 5 17:43:52 2006 @@ -1,13 +1,14 @@ from pypy.rpython.lltypesystem.lltype import \ Primitive, Ptr, typeOf, RuntimeTypeInfo, \ Struct, Array, FuncType, PyObject, Void, \ - ContainerType, OpaqueType + ContainerType, OpaqueType, FixedSizeArray from pypy.rpython.lltypesystem import lltype from pypy.rpython.lltypesystem.llmemory import Address from pypy.rpython.memory.lladdress import NULL from pypy.translator.c.primitive import PrimitiveName, PrimitiveType from pypy.translator.c.primitive import PrimitiveErrorValue from pypy.translator.c.node import StructDefNode, ArrayDefNode +from pypy.translator.c.node import FixedSizeArrayDefNode from pypy.translator.c.node import ContainerNodeFactory, ExtTypeOpaqueDefNode from pypy.translator.c.support import cdecl, CNameManager, ErrorValue from pypy.translator.c.pyobj import PyObjMaker @@ -72,7 +73,10 @@ node = self.structdefnodes[key] except KeyError: if isinstance(T, Struct): - node = StructDefNode(self, T, varlength) + if isinstance(T, FixedSizeArray): + node = FixedSizeArrayDefNode(self, T) + else: + node = StructDefNode(self, T, varlength) elif isinstance(T, Array): node = ArrayDefNode(self, T, varlength) elif isinstance(T, OpaqueType) and hasattr(T, '_exttypeinfo'): @@ -93,7 +97,7 @@ node = self.gettypedefnode(T, varlength=varlength) if who_asks is not None: who_asks.dependencies[node] = True - return 'struct %s @' % node.name + return node.gettype() elif T == PyObject: return 'PyObject @' elif isinstance(T, FuncType): Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Wed Apr 5 17:43:52 2006 @@ -4,9 +4,11 @@ from pypy.translator.c.support import llvalue_from_constant, gen_assignments from pypy.objspace.flow.model import Variable, Constant, Block from pypy.objspace.flow.model import c_last_exception -from pypy.rpython.lltypesystem.lltype import \ - Ptr, PyObject, Void, Bool, Signed, Unsigned, SignedLongLong, UnsignedLongLong,Char, UniChar, pyobjectptr, Struct, Array - +from pypy.rpython.lltypesystem.lltype import Ptr, PyObject, Void, Bool, Signed +from pypy.rpython.lltypesystem.lltype import Unsigned, SignedLongLong +from pypy.rpython.lltypesystem.lltype import UnsignedLongLong, Char, UniChar +from pypy.rpython.lltypesystem.lltype import pyobjectptr +from pypy.rpython.lltypesystem.lltype import Struct, Array, FixedSizeArray PyObjPtr = Ptr(PyObject) LOCALVAR = 'l_%s' @@ -396,21 +398,44 @@ return self.OP_GETFIELD(op, ampersand='&') def OP_GETARRAYSIZE(self, op): - return '%s = %s->length;' % (self.expr(op.result), - self.expr(op.args[0])) + ARRAY = self.lltypemap(op.args[0]).TO + if isinstance(ARRAY, FixedSizeArray): + return '%s = %d;' % (self.expr(op.result), + ARRAY.length) + else: + return '%s = %s->length;' % (self.expr(op.result), + self.expr(op.args[0])) def OP_GETARRAYITEM(self, op): - return self.generic_get(op, '%s->items[%s]' % (self.expr(op.args[0]), - self.expr(op.args[1]))) + ARRAY = self.lltypemap(op.args[0]).TO + items = self.expr(op.args[0]) + if isinstance(ARRAY, FixedSizeArray): + items = '(*%s)' % (items,) + else: + items += '->items' + return self.generic_get(op, '%s[%s]' % (items, + self.expr(op.args[1]))) def OP_SETARRAYITEM(self, op): - return self.generic_set(op, '%s->items[%s]' % (self.expr(op.args[0]), - self.expr(op.args[1]))) + ARRAY = self.lltypemap(op.args[0]).TO + items = self.expr(op.args[0]) + if isinstance(ARRAY, FixedSizeArray): + items = '(*%s)' % (items,) + else: + items += '->items' + return self.generic_set(op, '%s[%s]' % (items, + self.expr(op.args[1]))) def OP_GETARRAYSUBSTRUCT(self, op): - return '%s = %s->items + %s;' % (self.expr(op.result), - self.expr(op.args[0]), - self.expr(op.args[1])) + ARRAY = self.lltypemap(op.args[0]).TO + items = self.expr(op.args[0]) + if isinstance(ARRAY, FixedSizeArray): + items = '*%s' % (items,) + else: + items += '->items' + return '%s = %s + %s;' % (self.expr(op.result), + items, + self.expr(op.args[1])) def OP_PTR_NONZERO(self, op): return '%s = (%s != NULL);' % (self.expr(op.result), Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Wed Apr 5 17:43:52 2006 @@ -448,7 +448,8 @@ print >> f, '/*** Structure definitions ***/' print >> f for node in structdeflist: - print >> f, 'struct %s;' % node.name + if node.name: + print >> f, 'struct %s;' % node.name print >> f for node in structdeflist: for line in node.definition(): Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Wed Apr 5 17:43:52 2006 @@ -1,6 +1,6 @@ from __future__ import generators from pypy.rpython.lltypesystem.lltype import \ - Struct, Array, FuncType, PyObjectType, typeOf, \ + Struct, Array, FixedSizeArray, FuncType, PyObjectType, typeOf, \ GcStruct, GcArray, GC_CONTAINER, ContainerType, \ parentlink, Ptr, PyObject, Void, OpaqueType, Float, \ RuntimeTypeInfo, getRuntimeTypeInfo, Char @@ -50,8 +50,6 @@ self.dependencies = {} self.prefix = somelettersfrom(STRUCT._name) + '_' - gcpolicy = db.gcpolicy - def setup(self): # this computes self.fields self.fields = [] @@ -86,6 +84,9 @@ return self.gcinfo gcinfo = defaultproperty(computegcinfo) + def gettype(self): + return 'struct %s @' % self.name + def c_struct_field_name(self, name): return self.prefix + name @@ -173,6 +174,9 @@ return self.gcinfo gcinfo = defaultproperty(computegcinfo) + def gettype(self): + return 'struct %s @' % self.name + def access_expr(self, baseexpr, index): return '%s.items[%d]' % (baseexpr, index) @@ -228,6 +232,61 @@ yield '-1' +class FixedSizeArrayDefNode: + gcinfo = None + name = None + + def __init__(self, db, FIXEDARRAY): + self.db = db + self.FIXEDARRAY = FIXEDARRAY + self.LLTYPE = FIXEDARRAY + self.dependencies = {} + self.itemtypename = db.gettype(FIXEDARRAY.OF, who_asks=self) + + def setup(self): + """Loops are forbidden by ForwardReference.become() because + there is no way to declare them in C.""" + + def gettype(self): + FIXEDARRAY = self.FIXEDARRAY + return self.itemtypename.replace('@', '(@)[%d]' % FIXEDARRAY.length) + + def access_expr(self, baseexpr, index): + return '%s[%d]' % (baseexpr, index) + + def definition(self): + return [] # no declaration is needed + + def visitor_lines(self, prefix, on_item): + FIXEDARRAY = self.FIXEDARRAY + # we need a unique name for this C variable, or at least one that does + # not collide with the expression in 'prefix' + i = 0 + varname = 'p0' + while prefix.find(varname) >= 0: + i += 1 + varname = 'p%d' % i + body = list(on_item('(*%s)' % varname, FIXEDARRAY.OF)) + if body: + yield '{' + yield '\t%s = %s;' % (cdecl(self.itemtypename, '*' + varname), + prefix) + yield '\t%s = %s + %d;' % (cdecl(self.itemtypename, + '*%s_end' % varname), + varname, + FIXEDARRAY.length) + yield '\twhile (%s != %s_end) {' % (varname, varname) + for line in body: + yield '\t\t' + line + yield '\t\t%s++;' % varname + yield '\t}' + yield '}' + + def debug_offsets(self): + # XXX not implemented + return [] + + class ExtTypeOpaqueDefNode: "For OpaqueTypes created by pypy.rpython.extfunctable.ExtTypeInfo." @@ -387,6 +446,34 @@ assert not USESLOTS or '__dict__' not in dir(ArrayNode) +class FixedSizeArrayNode(ContainerNode): + nodekind = 'array' + if USESLOTS: + __slots__ = () + + def basename(self): + return self.T._name + + def enum_dependencies(self): + for name in self.T._names: # _names == ['item0', 'item1', ...] + yield getattr(self.obj, name) + + def getlength(self): + return 1 # not variable-sized! + + def initializationexpr(self, decoration=''): + is_empty = True + yield '{' + # _names == ['item0', 'item1', ...] + for j, name in enumerate(self.T._names): + value = getattr(self.obj, name) + lines = generic_initializationexpr(self.db, value, + '%s[%d]' % (self.name, j), + '%s%d' % (decoration, j)) + for line in lines: + yield '\t' + line + yield '}' + def generic_initializationexpr(db, value, access_expr, decoration): if isinstance(typeOf(value), ContainerType): node = db.getcontainernode(value) @@ -603,6 +690,7 @@ GcStruct: StructNode, Array: ArrayNode, GcArray: ArrayNode, + FixedSizeArray: FixedSizeArrayNode, FuncType: FuncNode, OpaqueType: opaquenode_factory, PyObjectType: PyObjectNode, Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Wed Apr 5 17:43:52 2006 @@ -23,3 +23,59 @@ return s.a.v + s.b.v fn = self.getcompiled(llf) assert fn() == 18 + + def test_fixedsizearray(self): + S = Struct("s", ('v', Signed)) + A7 = FixedSizeArray(Signed, 7) + A3 = FixedSizeArray(S, 3) + A42 = FixedSizeArray(A7, 6) + BIG = GcStruct("big", ("a7", A7), ("a3", A3), ("a42", A42)) + def llf(): + big = malloc(BIG) + a7 = big.a7 + a3 = big.a3 + a42 = big.a42 + a7[0] = -1 + a7[6] = -2 + a3[0].v = -3 + a3[2].v = -4 + a42[0][0] = -5 + a42[5][6] = -6 + assert a7[0] == -1 + assert a7[6] == -2 + assert a3[0].v == -3 + assert a3[2].v == -4 + assert a42[0][0] == -5 + assert a42[5][6] == -6 + return len(a42)*100 + len(a42[4]) + fn = self.getcompiled(llf) + res = fn() + assert fn() == 607 + + def test_recursivearray(self): + A = ForwardReference() + A.become(FixedSizeArray(Struct("S", ('a', Ptr(A))), 5)) + TREE = GcStruct("TREE", ("root", A), ("other", A)) + def llf(): + tree = malloc(TREE) + tree.root[0].a = tree.root + tree.root[1].a = tree.other + assert tree.root[0].a[0].a[0].a[0].a[0].a[1].a == tree.other + fn = self.getcompiled(llf) + fn() + + def test_prebuilt_array(self): + A = FixedSizeArray(Signed, 5) + a = malloc(A, immortal=True) + a[0] = 8 + a[1] = 5 + a[2] = 12 + a[3] = 12 + a[4] = 15 + def llf(): + s = '' + for i in range(5): + s += chr(64+a[i]) + assert s == "HELLO" + fn = self.getcompiled(llf) + fn() From arigo at codespeak.net Wed Apr 5 17:45:42 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 17:45:42 +0200 (CEST) Subject: [pypy-svn] r25402 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060405154542.69DEE1020E@code0.codespeak.net> Author: arigo Date: Wed Apr 5 17:45:41 2006 New Revision: 25402 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: (arre, arigo) This rctypes test passes now. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Wed Apr 5 17:45:41 2006 @@ -126,7 +126,6 @@ class Test_compilation: def test_compile_array_access(self): - py.test.skip("in-progress") def access_array(): my_array = c_int_10() my_array[0] = 1 From mwh at codespeak.net Wed Apr 5 18:11:58 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 18:11:58 +0200 (CEST) Subject: [pypy-svn] r25403 - in pypy/dist/pypy/rpython: . lltypesystem Message-ID: <20060405161158.A22C110216@code0.codespeak.net> Author: mwh Date: Wed Apr 5 18:11:57 2006 New Revision: 25403 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py Log: add call_boehm_gc_alloc to the operation table to get the l3interp tests passing again. not a nice solution, but the l3interp's status is sufficiently vague that it doesn't seem worth working on it to remove this particular hack from it now Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Wed Apr 5 18:11:57 2006 @@ -640,6 +640,9 @@ def op_yield_current_frame_to_caller(self): raise NotImplementedError("yield_current_frame_to_caller") + def op_call_boehm_gc_alloc(self): + raise NotImplementedError("call_boehm_gc_alloc") + # operations on pyobjects! for opname in opimpls.keys(): exec py.code.Source(""" Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Wed Apr 5 18:11:57 2006 @@ -255,6 +255,10 @@ 'ptr_iszero': LLOp(canfold=True), 'cast_ptr_to_int': LLOp(sideeffects=False), + # _________ XXX l3interp hacks ___________ + + 'call_boehm_gc_alloc': LLOp(canraise=(MemoryError,)), + # __________ address operations __________ 'raw_malloc': LLOp(canraise=(MemoryError,)), From auc at codespeak.net Wed Apr 5 18:43:43 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 18:43:43 +0200 (CEST) Subject: [pypy-svn] r25404 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060405164343.E625B1021A@code0.codespeak.net> Author: auc Date: Wed Apr 5 18:43:42 2006 New Revision: 25404 Added: pypy/dist/pypy/objspace/constraint/ - copied from r25396, pypy/dist/pypy/objspace/logic/ pypy/dist/pypy/objspace/constraint/distributor.py - copied unchanged from r25399, pypy/dist/pypy/objspace/logic/distributor.py pypy/dist/pypy/objspace/constraint/test/test_fd.py Modified: pypy/dist/pypy/objspace/constraint/domain.py Log: finite domain at interp level Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/logic/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Wed Apr 5 18:43:42 2006 @@ -1,7 +1,8 @@ from pypy.interpreter.error import OperationError -from pypy.interpreter import baseobjspace, gateway +from pypy.interpreter import baseobjspace, typedef from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.gateway import interp2app from pypy.objspace.std.objspace import W_Object @@ -34,54 +35,54 @@ if self.size() == 0: raise ConsistencyFailure() -W_AbstractDomain.typedef = TypeDef("W_AbstractDomain", - reset_flags = interp2app(W_AbstractDomain.w_reset_flags) +W_AbstractDomain.typedef = typedef.TypeDef("W_AbstractDomain", + reset_flags = interp2app(W_AbstractDomain.w_reset_flags), has_changed = interp2app(W_AbstractDomain.w_has_changed)) -class W_FiniteDomain(AbstractDomain): +class W_FiniteDomain(W_AbstractDomain): """ Variable Domain with a finite set of possible values """ - def __init__(self, values): + def __init__(self, space, w_values): """values is a list of values in the domain This class uses a dictionnary to make sure that there are no duplicate values""" - AbstractDomain.__init__(self) - self.set_values(values) + W_AbstractDomain.__init__(self, space) + self.set_values(w_values) - def set_values(self, values): - self._values = set(values) + def set_values(self, w_values): + self._values = set(w_values.wrappeditems) - def w_remove_value(self, value): + def w_remove_value(self, w_value): """Remove value of domain and check for consistency""" - self._values.remove(value) + self._values.remove(w_value) self._value_removed() - def w_remove_values(self, values): + def w_remove_values(self, w_values): """Remove values of domain and check for consistency""" - if values: - for val in values : + if w_values: + for val in w_values : self._values.remove(val) self._value_removed() - __delitem__ = remove_value + __delitem__ = w_remove_value def w_size(self): """computes the size of a finite domain""" - return len(self._values) - __len__ = size + return self._space.newint(len(self._values)) + __len__ = w_size def w_get_values(self): """return all the values in the domain in an indexable sequence""" - return list(self._values) + return self._space.newlist([x for x in self._values]) def __iter__(self): return iter(self._values) def w_copy(self): """clone the domain""" - return FiniteDomain(self) + return W_FiniteDomain(self, self._space) def __repr__(self): return '' % str(self.get_values()) @@ -97,5 +98,9 @@ if other is None: return self.get_values() return self._values & other._values -W_FiniteDomain.typedef = TypeDef("W_FiniteDomain", - +W_FiniteDomain.typedef = typedef.TypeDef("W_FiniteDomain", + remove_value = interp2app(W_FiniteDomain.w_remove_value), + remove_values = interp2app(W_FiniteDomain.w_remove_values), + get_values = interp2app(W_FiniteDomain.w_get_values), + copy = interp2app(W_FiniteDomain.w_copy), + size = interp2app(W_FiniteDomain.w_size)) Added: pypy/dist/pypy/objspace/constraint/test/test_fd.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/test/test_fd.py Wed Apr 5 18:43:42 2006 @@ -0,0 +1,14 @@ +from pypy.conftest import gettestobjspace + +class UnificationFailure(Exception): pass + +class AppTest_FD(object): + + def setup_class(cls): + cls.space = gettestobjspace('logic') + + def test_instantiate(self): + fd = FiniteDomain([1, 2, 3]) + assert fd.size() == 3 + assert set(fd.get_values()) == set([1, 2, 3]) + #fd2 = fd.copy() From auc at codespeak.net Wed Apr 5 18:47:11 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 5 Apr 2006 18:47:11 +0200 (CEST) Subject: [pypy-svn] r25405 - pypy/dist/pypy/objspace Message-ID: <20060405164711.A05911021B@code0.codespeak.net> Author: auc Date: Wed Apr 5 18:47:06 2006 New Revision: 25405 Modified: pypy/dist/pypy/objspace/logic.py Log: Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Wed Apr 5 18:47:06 2006 @@ -753,6 +753,15 @@ return proxy +#------ constraints ----------------- + +from pypy.objspace.constraint import domain + +W_FiniteDomain = domain.W_FiniteDomain + +def make_fd(space, w_values): + return domain.W_FiniteDomain(space, w_values) +app_make_fd = gateway.interp2app(make_fd) #-- THE SPACE --------------------------------------- @@ -804,8 +813,8 @@ space.wrap(app_alias_of)) space.setitem(space.builtin.w_dict, space.wrap('is_aliased'), space.wrap(app_is_aliased)) -## space.setitem(space.builtin.w_dict, space.wrap('disp'), -## space.wrap(app_disp)) + space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), + space.wrap(app_make_fd)) space.setitem(space.builtin.w_dict, space.wrap('bind'), space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), @@ -833,3 +842,10 @@ space.wrap(app_wait_needed)) patch_space_in_place(space, 'logic', proxymaker) return space + + + + + + + From afayolle at codespeak.net Wed Apr 5 18:50:12 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Wed, 5 Apr 2006 18:50:12 +0200 (CEST) Subject: [pypy-svn] r25406 - pypy/dist/pypy/objspace/constraint Message-ID: <20060405165012.B70FA1021D@code0.codespeak.net> Author: afayolle Date: Wed Apr 5 18:50:11 2006 New Revision: 25406 Added: pypy/dist/pypy/objspace/constraint/__init__.py (contents, props changed) Log: (auc) forgot that one Added: pypy/dist/pypy/objspace/constraint/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/__init__.py Wed Apr 5 18:50:11 2006 @@ -0,0 +1 @@ +#empty From arigo at codespeak.net Wed Apr 5 18:57:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 18:57:58 +0200 (CEST) Subject: [pypy-svn] r25407 - in pypy/dist/pypy: annotation jit/codegen/llvm rpython rpython/lltypesystem rpython/rctypes rpython/rctypes/test translator/c/test Message-ID: <20060405165758.9D0FE10226@code0.codespeak.net> Author: arigo Date: Wed Apr 5 18:57:55 2006 New Revision: 25407 Modified: pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/jit/codegen/llvm/rgenop.py pypy/dist/pypy/rpython/extregistry.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/translator/c/test/test_wrapping.py Log: (arre, arigo) Remove the 'compute_result_annotation' support from bookkeeper.py. Replaced all usages with extregistry usage. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Wed Apr 5 18:57:55 2006 @@ -359,10 +359,10 @@ result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) elif extregistry.is_registered(x): result = extregistry.lookup(x).get_annotation(tp, x) - elif hasattr(x, "compute_result_annotation"): - result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) - elif hasattr(tp, "compute_annotation"): - result = tp.compute_annotation() +## elif hasattr(x, "compute_result_annotation"): +## result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) +## elif hasattr(tp, "compute_annotation"): +## result = tp.compute_annotation() elif tp in EXTERNAL_TYPE_ANALYZERS: result = SomeExternalObject(tp) elif isinstance(x, lltype._ptr): @@ -515,8 +515,8 @@ return s_None elif t in EXTERNAL_TYPE_ANALYZERS: return SomeExternalObject(t) - elif hasattr(t, "compute_annotation"): - return t.compute_annotation() +## elif hasattr(t, "compute_annotation"): +## return t.compute_annotation() elif extregistry.is_registered_type(t): return extregistry.lookup_type(t).get_annotation(t) elif t.__module__ != '__builtin__' and t not in self.pbctypes: Modified: pypy/dist/pypy/jit/codegen/llvm/rgenop.py ============================================================================== --- pypy/dist/pypy/jit/codegen/llvm/rgenop.py (original) +++ pypy/dist/pypy/jit/codegen/llvm/rgenop.py Wed Apr 5 18:57:55 2006 @@ -227,18 +227,25 @@ LINKPAIR = lltype.GcStruct('tuple2', *fields) # helpers -def setannotation(func, TYPE): - func.compute_result_annotation = lambda *args_s: TYPE - -def setspecialize(func): - # for now - def specialize_as_direct_call(hop): - FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], hop.r_result.lowleveltype) - args_v = hop.inputargs(*hop.args_r) - funcptr = lltype.functionptr(FUNCTYPE, func.__name__, _callable=func) - cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) - return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) - func.specialize = specialize_as_direct_call +def setannotation(func, annotation, specialize_as_constant=False): + if specialize_as_constant: + def specialize(hop): + llvalue = func(hop.args_s[0].const) + return hop.inputconst(lltype.typeOf(llvalue), llvalue) + else: + # specialize as direct_call + def specialize(hop): + FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], + hop.r_result.lowleveltype) + args_v = hop.inputargs(*hop.args_r) + funcptr = lltype.functionptr(FUNCTYPE, func.__name__, + _callable=func) + cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) + return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) + + extregistry.register_value(func, + compute_result_annotation = annotation, + specialize_call = specialize) # annotations from pypy.annotation import model as annmodel @@ -251,35 +258,17 @@ setannotation(geninputarg, s_ConstOrVar) setannotation(genop, s_ConstOrVar) setannotation(genconst, s_ConstOrVar) -revealconst.compute_result_annotation = lambda s_T, s_gv: annmodel.lltype_to_annotation(s_T.const) +setannotation(revealconst, lambda s_T, s_gv: annmodel.lltype_to_annotation( + s_T.const)) +setannotation(isconst, annmodel.SomeBool()) setannotation(closeblock1, s_Link) setannotation(closeblock2, s_LinkPair) setannotation(closelink, None) setannotation(closereturnlink, None) -# specialize -setspecialize(initblock) -setspecialize(geninputarg) -setspecialize(genop) -setspecialize(genconst) -setspecialize(revealconst) -setspecialize(closeblock1) -setspecialize(closeblock2) -setspecialize(closelink) -setspecialize(closereturnlink) +setannotation(isptrtype, annmodel.SomeBool()) # XXX(for now) void constant constructors -setannotation(constFieldName, s_ConstOrVar) -setannotation(constTYPE, s_ConstOrVar) -setannotation(placeholder, s_ConstOrVar) - -def set_specialize_void_constant_constructor(func): - # for now - def specialize_as_constant(hop): - llvalue = func(hop.args_s[0].const) - return hop.inputconst(lltype.typeOf(llvalue), llvalue) - func.specialize = specialize_as_constant - -set_specialize_void_constant_constructor(placeholder) -set_specialize_void_constant_constructor(constFieldName) -set_specialize_void_constant_constructor(constTYPE) +setannotation(constFieldName, s_ConstOrVar, specialize_as_constant=True) +setannotation(constTYPE, s_ConstOrVar, specialize_as_constant=True) +setannotation(placeholder, s_ConstOrVar, specialize_as_constant=True) Modified: pypy/dist/pypy/rpython/extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/extregistry.py (original) +++ pypy/dist/pypy/rpython/extregistry.py Wed Apr 5 18:57:55 2006 @@ -26,16 +26,18 @@ def create_annotation_callable(annotation): from pypy.annotation import model as annmodel - if isinstance(annotation, annmodel.SomeObject): + if isinstance(annotation, annmodel.SomeObject) or annotation is None: s_result = annotation def annotation(*args): return s_result return annotation -def create_entry(compute_result_annotation=None, compute_annotation=None, +undefined = object() + +def create_entry(compute_result_annotation=undefined, compute_annotation=None, specialize_call=None, get_repr=None): - if compute_result_annotation is not None: + if compute_result_annotation is not undefined: compute_result_annotation = create_annotation_callable( compute_result_annotation) return ExtRegistryFunc(compute_result_annotation, specialize_call) @@ -44,14 +46,17 @@ get_repr) def register_value(value, **kwargs): + assert value not in EXT_REGISTRY_BY_VALUE EXT_REGISTRY_BY_VALUE[value] = create_entry(**kwargs) return EXT_REGISTRY_BY_VALUE[value] def register_type(t, **kwargs): + assert t not in EXT_REGISTRY_BY_TYPE EXT_REGISTRY_BY_TYPE[t] = create_entry(**kwargs) return EXT_REGISTRY_BY_TYPE[t] def register_metatype(t, **kwargs): + assert t not in EXT_REGISTRY_BY_METATYPE EXT_REGISTRY_BY_METATYPE[t] = create_entry(**kwargs) return EXT_REGISTRY_BY_METATYPE[t] Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Wed Apr 5 18:57:55 2006 @@ -2,6 +2,9 @@ The table of all LL operations. """ +from pypy.rpython import extregistry + + class LLOp(object): def __init__(self, sideeffects=True, canfold=False, canraise=(), pyobj=False): @@ -25,6 +28,11 @@ # The operation manipulates PyObjects self.pyobj = pyobj + # XXX refactor extregistry to allow a single registration + extregistry.register_value(self, + compute_result_annotation = self.compute_result_annotation, + specialize_call = self.specialize) + # __________ make the LLOp instances callable from LL helpers __________ __name__ = property(lambda self: 'llop_'+self.opname) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Wed Apr 5 18:57:55 2006 @@ -66,9 +66,10 @@ bltintyper = rtyper.type_system.rbuiltin.\ BUILTIN_TYPER[self.builtinfunc] except (KeyError, TypeError): - if hasattr(self.builtinfunc,"specialize"): - bltintyper = self.builtinfunc.specialize - elif extregistry.is_registered(self.builtinfunc): +## if hasattr(self.builtinfunc,"specialize"): +## bltintyper = self.builtinfunc.specialize +## else + if extregistry.is_registered(self.builtinfunc): entry = extregistry.lookup(self.builtinfunc) bltintyper = entry.specialize_call else: Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Wed Apr 5 18:57:55 2006 @@ -10,21 +10,21 @@ from pypy.rpython.error import TyperError from pypy.rpython.rctypes.rmodel import CTypesValueRepr -ctypes_annotation_list = [ - (c_char, lltype.Char), - (c_byte, lltype.Signed), - (c_ubyte, lltype.Unsigned), - (c_short, lltype.Signed), - (c_ushort, lltype.Unsigned), - (c_int, lltype.Signed), - (c_uint, lltype.Unsigned), - (c_long, lltype.Signed), - (c_ulong, lltype.Unsigned), - (c_longlong, lltype.SignedLongLong), - (c_ulonglong, lltype.UnsignedLongLong), - (c_float, lltype.Float), - (c_double, lltype.Float), -] +ctypes_annotation_list = { + c_char: lltype.Char, + c_byte: lltype.Signed, + c_ubyte: lltype.Unsigned, + c_short: lltype.Signed, + c_ushort: lltype.Unsigned, + c_int: lltype.Signed, + c_uint: lltype.Unsigned, + c_long: lltype.Signed, + c_ulong: lltype.Unsigned, + c_longlong: lltype.SignedLongLong, + c_ulonglong: lltype.UnsignedLongLong, + c_float: lltype.Float, + c_double: lltype.Float, +}.items() # nb. platform-dependent duplicate ctypes are removed class PrimitiveRepr(CTypesValueRepr): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Apr 5 18:57:55 2006 @@ -264,7 +264,9 @@ # result should be an integer assert s.knowntype == int t.buildrtyper().specialize() - #d#t.view() + + if conftest.option.view: + a.translator.view() def x_test_compile_simple(self): fn = compile(o_atoi, [str]) Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Wed Apr 5 18:57:55 2006 @@ -10,6 +10,7 @@ from pypy.rpython.module.support import init_opaque_object from pypy.rpython.module.support import to_opaque_object, from_opaque_object from pypy.rpython.module.support import from_rstr +from pypy.rpython import extregistry # for debugging, sanity checks in non-RPython code @@ -254,18 +255,25 @@ # helpers -def setannotation(func, TYPE): - func.compute_result_annotation = lambda *args_s: TYPE - -def setspecialize(func): - # for now - def specialize_as_direct_call(hop): - FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], hop.r_result.lowleveltype) - args_v = hop.inputargs(*hop.args_r) - funcptr = lltype.functionptr(FUNCTYPE, func.__name__, _callable=func) - cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) - return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) - func.specialize = specialize_as_direct_call +def setannotation(func, annotation, specialize_as_constant=False): + if specialize_as_constant: + def specialize(hop): + llvalue = func(hop.args_s[0].const) + return hop.inputconst(lltype.typeOf(llvalue), llvalue) + else: + # specialize as direct_call + def specialize(hop): + FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], + hop.r_result.lowleveltype) + args_v = hop.inputargs(*hop.args_r) + funcptr = lltype.functionptr(FUNCTYPE, func.__name__, + _callable=func) + cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) + return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) + + extregistry.register_value(func, + compute_result_annotation = annotation, + specialize_call = specialize) # annotations from pypy.annotation import model as annmodel @@ -278,7 +286,8 @@ setannotation(geninputarg, s_ConstOrVar) setannotation(genop, s_ConstOrVar) setannotation(genconst, s_ConstOrVar) -revealconst.compute_result_annotation = lambda s_T, s_gv: annmodel.lltype_to_annotation(s_T.const) +setannotation(revealconst, lambda s_T, s_gv: annmodel.lltype_to_annotation( + s_T.const)) setannotation(isconst, annmodel.SomeBool()) setannotation(closeblock1, s_Link) setannotation(closeblock2, s_LinkPair) @@ -287,32 +296,7 @@ setannotation(isptrtype, annmodel.SomeBool()) -# specialize -setspecialize(initblock) -setspecialize(geninputarg) -setspecialize(genop) -setspecialize(genconst) -setspecialize(revealconst) -setspecialize(isconst) -setspecialize(closeblock1) -setspecialize(closeblock2) -setspecialize(closelink) -setspecialize(closereturnlink) - -setspecialize(isptrtype) - # XXX(for now) void constant constructors -setannotation(constFieldName, s_ConstOrVar) -setannotation(constTYPE, s_ConstOrVar) -setannotation(placeholder, s_ConstOrVar) - -def set_specialize_void_constant_constructor(func): - # for now - def specialize_as_constant(hop): - llvalue = func(hop.args_s[0].const) - return hop.inputconst(lltype.typeOf(llvalue), llvalue) - func.specialize = specialize_as_constant - -set_specialize_void_constant_constructor(placeholder) -set_specialize_void_constant_constructor(constFieldName) -set_specialize_void_constant_constructor(constTYPE) +setannotation(constFieldName, s_ConstOrVar, specialize_as_constant=True) +setannotation(constTYPE, s_ConstOrVar, specialize_as_constant=True) +setannotation(placeholder, s_ConstOrVar, specialize_as_constant=True) Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Wed Apr 5 18:57:55 2006 @@ -32,7 +32,6 @@ global t # allow us to view later t = TranslationContext(do_imports_immediately=False) - do_register(t) t.buildannotator() rtyper = t.buildrtyper() bk = rtyper.annotator.bookkeeper @@ -192,44 +191,33 @@ return null -""" -The following registers the new mappings. The registration -had to be done from a function, because we need to pass the -translator in. - -Note that these registrations are for explicit wrapping/unwrapping -which was the first thing I tried. Meanwhile similar functionality -has been added to rclass.py to support automatic wrapping. -""" - -def do_register(t): - def compute_annotation_unwrap(s_wrapper, s_class): - assert hasattr(s_class, 'descriptions'), 'need a class in unwrap 2nd arg' - descs = s_class.descriptions - assert len(descs) == 1, 'missing specialisation, classdesc not unique!' - for desc in descs.keys(): - classdef = desc.getuniqueclassdef() - return annmodel.SomeInstance(classdef) - - extregistry.register_value(ll_create_pywrapper, - compute_result_annotation=annmodel.SomePtr(lltype.Ptr(lltype.PyObject)), - specialize_call=rtype_wrap_object_create) - - extregistry.register_value(ll_fetch_pywrapper, - compute_result_annotation=annmodel.SomePtr(lltype.Ptr(lltype.PyObject)), - specialize_call=rtype_wrap_object_fetch) - - extregistry.register_value(ll_call_destructor, - compute_result_annotation=lambda *args: None, - specialize_call=rtype_destruct_object) - - extregistry.register_value(wrap_obj, - compute_result_annotation=annmodel.SomeObject(), - specialize_call=rtype_wrap_object) - - extregistry.register_value(unwrap_obj, - compute_result_annotation=compute_annotation_unwrap, - specialize_call=rtype_unwrap_object) +def compute_annotation_unwrap(s_wrapper, s_class): + assert hasattr(s_class, 'descriptions'), 'need a class in unwrap 2nd arg' + descs = s_class.descriptions + assert len(descs) == 1, 'missing specialisation, classdesc not unique!' + for desc in descs.keys(): + classdef = desc.getuniqueclassdef() + return annmodel.SomeInstance(classdef) + +extregistry.register_value(ll_create_pywrapper, + compute_result_annotation=annmodel.SomePtr(lltype.Ptr(lltype.PyObject)), + specialize_call=rtype_wrap_object_create) + +extregistry.register_value(ll_fetch_pywrapper, + compute_result_annotation=annmodel.SomePtr(lltype.Ptr(lltype.PyObject)), + specialize_call=rtype_wrap_object_fetch) + +extregistry.register_value(ll_call_destructor, + compute_result_annotation=lambda *args: None, + specialize_call=rtype_destruct_object) + +extregistry.register_value(wrap_obj, + compute_result_annotation=annmodel.SomeObject(), + specialize_call=rtype_wrap_object) + +extregistry.register_value(unwrap_obj, + compute_result_annotation=compute_annotation_unwrap, + specialize_call=rtype_unwrap_object) # _______________________________________________- # the actual tests @@ -344,8 +332,9 @@ def wraptest(obj): return obj -wraptest.compute_result_annotation = lambda *args: annmodel.SomeObject() -wraptest.specialize = rtype_wraptest +extregistry.register_value(wraptest, + compute_result_annotation = annmodel.SomeObject(), + specialize_call = rtype_wraptest) # not sure what to do with the above. # use genpickle facility to produce a plain function? @@ -398,4 +387,4 @@ if __name__=='__main__': test_expose_classes() - \ No newline at end of file + From antocuni at codespeak.net Wed Apr 5 19:08:53 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 5 Apr 2006 19:08:53 +0200 (CEST) Subject: [pypy-svn] r25408 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060405170853.D6E5810227@code0.codespeak.net> Author: antocuni Date: Wed Apr 5 19:08:30 2006 New Revision: 25408 Added: pypy/dist/pypy/translator/cli/class_.py (contents, props changed) pypy/dist/pypy/translator/cli/node.py (contents, props changed) pypy/dist/pypy/translator/cli/test/test_oo.py (contents, props changed) Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/ilgenerator.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py Log: Initial class support Added: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/class_.py Wed Apr 5 19:08:30 2006 @@ -0,0 +1,36 @@ +from pypy.translator.cli.node import Node +from pypy.translator.cli.function import Function +from pypy.translator.cli import cts + +class Class(Node): + def __init__(self, classdef): + self.classdef = classdef + + def get_name(self): + return self.classdef._name + + def render(self, ilasm): + self.ilasm = ilasm + name = self.get_name().replace('__main__.', '') # TODO: handle modules + ilasm.begin_class(name) # TODO: handle base class + for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): + # TODO: handle default values + ilasm.field(f_name, cts.lltype_to_cts(f_type)) + + self._ctor() + + for m_name, m_meth in self.classdef._methods.iteritems(): + # TODO: handle static methods + # TODO: should __init__ be rendered as a constructor? + f = Function(m_meth.graph, m_name, is_method = True) + f.render(ilasm) + + ilasm.end_class() + + def _ctor(self): + self.ilasm.begin_function('.ctor', [], 'void', False, 'specialname', 'rtspecialname', 'instance') + self.ilasm.opcode('ldarg.0') + self.ilasm.call('instance void object::.ctor()') # TODO: base class + self.ilasm.opcode('ret') + self.ilasm.end_function() + Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Wed Apr 5 19:08:30 2006 @@ -46,9 +46,12 @@ def lltype_to_cts(t): - # TODO: handle instances more accurately if isinstance(t, Instance): - return 'object' + name = t._name.replace('__main__.', '') # TODO: modules + if name in ('Object_meta', 'Object'): + return 'object' + + return 'class %s' % name return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) @@ -76,10 +79,15 @@ else: return ilasm_type, str(const.value) -def graph_to_signature(graph): +def graph_to_signature(graph, is_method = False, func_name = None): ret_type, ret_var = llvar_to_cts(graph.getreturnvar()) - func_name = graph.name - arg_types = [lltype_to_cts(arg.concretetype) for arg in graph.getargs()] + func_name = func_name or graph.name + + args = graph.getargs() + if is_method: + args = args[1:] + + arg_types = [lltype_to_cts(arg.concretetype) for arg in args] arg_list = ', '.join(arg_types) return '%s %s(%s)' % (ret_type, func_name, arg_list) Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Wed Apr 5 19:08:30 2006 @@ -1,34 +1,35 @@ +try: + set +except NameError: + from sets import Set as set + from pypy.objspace.flow import model as flowmodel from pypy.rpython.lltypesystem.lltype import Void +from pypy.rpython.ootypesystem.ootype import Instance from pypy.translator.cli.option import getoption from pypy.translator.cli import cts from pypy.translator.cli.opcodes import opcodes from pypy.translator.cli.metavm import InstructionList, Generator +from pypy.translator.cli.node import Node from pypy.tool.ansi_print import ansi_log import py log = py.log.Producer("cli") py.log.setconsumer("cli", ansi_log) - -class Node(object): - def get_name(self): - pass - - def render(self, ilasm): - pass - - class Function(Node, Generator): - def __init__(self, graph, is_entrypoint = False): + def __init__(self, graph, name = None, is_method = False, is_entrypoint = False): self.graph = graph + self.name = name or graph.name + self.is_method = is_method self.is_entrypoint = is_entrypoint self.blocknum = {} + self.classdefs = set() self._set_args() self._set_locals() def get_name(self): - return self.graph.name + return self.name def _is_return_block(self, block): return (not block.exits) and len(block.inputargs) == 1 @@ -40,11 +41,23 @@ self.ilasm = ilasm graph = self.graph returntype, returnvar = cts.llvar_to_cts(graph.getreturnvar()) - - self.ilasm.begin_function(graph.name, self.args, returntype, self.is_entrypoint) + + if self.is_method: + args = self.args[1:] # self is implicit + meth_type = 'virtual' + else: + args = self.args + meth_type = 'static' + + self.ilasm.begin_function(self.name, args, returntype, self.is_entrypoint, meth_type) self.ilasm.locals(self.locals) + return_blocks = [] for block in graph.iterblocks(): + if self._is_return_block(block): + return_blocks.append(block) + continue + self.ilasm.label(self._get_block_name(block)) handle_exc = (block.exitswitch == flowmodel.c_last_exception) @@ -52,15 +65,10 @@ self.ilasm.begin_try() for op in block.operations: + self._search_for_classes(op) self._render_op(op) - # check for codeless blocks - if self._is_return_block(block): - return_var = block.inputargs[0] - if return_var.concretetype is not Void: - self.load(return_var) - self.ilasm.opcode('ret') - elif self._is_raise_block(block): + if self._is_raise_block(block): exc = block.inputargs[1] self.load(exc) self.ilasm.opcode('throw') @@ -111,15 +119,16 @@ self.load(block.exitswitch) self.ilasm.branch_if(link.exitcase, target_label) + # render return blocks at the end just to please the .NET + # runtime that seems to need a return statement at the end of + # the function + for block in return_blocks: + self.ilasm.label(self._get_block_name(block)) + return_var = block.inputargs[0] + if return_var.concretetype is not Void: + self.load(return_var) + self.ilasm.opcode('ret') - # add a block that will never be executed, just to please the - # .NET runtime that seems to need a return statement at the - # end of the function - if returntype != 'void': - ilasm_type = cts.ctstype_to_ilasm(returntype) - self.ilasm.opcode('ldc.%s 0' % ilasm_type) - - self.ilasm.opcode('ret') self.ilasm.end_function() def _setup_link(self, link): @@ -179,6 +188,18 @@ def _get_block_name(self, block): return 'block%s' % self.blocknum[block] + def _search_for_classes(self, op): + for arg in op.args: + lltype = None + if isinstance(arg, flowmodel.Variable): + lltype = arg.concretetype + elif isinstance(arg, flowmodel.Constant): + lltype = arg.value + + if isinstance(lltype, Instance): + self.classdefs.add(lltype) + + def _render_op(self, op): instr_list = opcodes.get(op.opname, None) if instr_list is not None: @@ -191,10 +212,24 @@ else: assert False, 'Unknown opcode: %s ' % op + def field_name(self, obj, field): + class_name = self.class_name(obj) + field_type = cts.lltype_to_cts(obj._field_type(field)) + return '%s %s::%s' % (field_type, class_name, field) + + def ctor_name(self, ooinstance): + return 'instance void class %s::.ctor()' % self.class_name(ooinstance) + # following methods belongs to the Generator interface - def function_name(self, graph): - return cts.graph_to_signature(graph) + def function_signature(self, graph): + return cts.graph_to_signature(graph, False) + + def method_signature(self, graph, name): + return cts.graph_to_signature(graph, True, name) + + def class_name(self, ooinstance): + return ooinstance._name.replace('__main__.', '') # TODO: modules def emit(self, instr, *args): self.ilasm.opcode(instr, *args) @@ -202,10 +237,28 @@ def call(self, func_name): self.ilasm.call(func_name) + def new(self, obj): + self.ilasm.new(self.ctor_name(obj)) + + def set_field(self, obj, name): + self.ilasm.opcode('stfld ' + self.field_name(obj, name)) + + def get_field(self, obj, name): + self.ilasm.opcode('ldfld ' + self.field_name(obj, name)) + + def call_method(self, obj, name): + owner, meth = obj._lookup(name) + full_name = '%s::%s' % (self.class_name(obj), name) + self.ilasm.call_method(self.method_signature(meth.graph, full_name)) + def load(self, v): if isinstance(v, flowmodel.Variable): if v.name in self.argset: - self.ilasm.opcode('ldarg', repr(v.name)) + selftype, selfname = self.args[0] + if self.is_method and v.name == selfname: + self.ilasm.opcode('ldarg.0') # special case for 'self' + else: + self.ilasm.opcode('ldarg', repr(v.name)) else: self.ilasm.opcode('ldloc', repr(v.name)) Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Wed Apr 5 19:08:30 2006 @@ -1,8 +1,15 @@ +try: + set +except NameError: + from sets import Set as set + import sys +from types import MethodType from pypy.translator.cli import conftest from pypy.translator.cli.ilgenerator import IlasmGenerator from pypy.translator.cli.function import Function +from pypy.translator.cli.class_ import Class from pypy.translator.cli.option import getoption @@ -23,7 +30,8 @@ def __init__(self, tmpdir, translator, entrypoint = None): self.tmpdir = tmpdir self.translator = translator - self.entrypoint = entrypoint + self.entrypoint = entrypoint + self.classdefs = set() if entrypoint is None: self.assembly_name = self.translator.graphs[0].name @@ -39,6 +47,7 @@ self.ilasm = IlasmGenerator(out, self.assembly_name) self.gen_all_functions() + self.gen_classes() out.close() return self.tmpfile.strpath @@ -50,9 +59,15 @@ for graph in self.translator.graphs: # TODO: remove this test - if graph.name == 'll_issubclass__Instance_Object_meta__Instance_Object_meta_': + if graph.name.startswith('ll_'): continue if '.' not in graph.name: # it's not a method f = Function(graph) f.render(self.ilasm) + self.classdefs.update(f.classdefs) + + def gen_classes(self): + for classdef in self.classdefs: + c = Class(classdef) + c.render(self.ilasm) Modified: pypy/dist/pypy/translator/cli/ilgenerator.py ============================================================================== --- pypy/dist/pypy/translator/cli/ilgenerator.py (original) +++ pypy/dist/pypy/translator/cli/ilgenerator.py Wed Apr 5 19:08:30 2006 @@ -7,9 +7,8 @@ self._startblock = startblock self._endblock = endblock - def write(self, s, indent = None): - if indent is None: - indent = self._indent + def write(self, s, indent = 0): + indent = self._indent + (indent * self._indentstep) if self._bol: self._out.write(' ' * indent) @@ -43,14 +42,29 @@ def close(self): self.out.close() - def begin_function(self, name, args, returntype, is_entry_point = False): - # TODO: .maxstack + def begin_class(self, name, base = None): + if base is None: + base = '[mscorlib]System.Object' + + self.code.writeline('.class public %s extends %s' % (name, base)) + self.code.openblock() + + def end_class(self): + self.code.closeblock() - arglist = ', '.join(['%s %s' % arg for arg in args]) + def field(self, name, type_): + self.code.writeline('.field public %s %s' %(type_, name)) - self.code.writeline('.method static public %s %s(%s) il managed' % (returntype, name, arglist)) + def begin_function(self, name, arglist, returntype, is_entrypoint = False, *args): + # TODO: .maxstack + + attributes = ' '.join(args) + arglist = ', '.join(['%s %s' % arg for arg in arglist]) + self.code.writeline('.method public %s %s %s(%s) il managed' %\ + (attributes, returntype, name, arglist)) + self.code.openblock() - if is_entry_point: + if is_entrypoint: self.code.writeline('.entrypoint') def end_function(self): @@ -78,7 +92,7 @@ def label(self, lbl): self.code.writeline() - self.code.write(lbl + ':', indent=0) + self.code.write(lbl + ':', indent=-1) self.code.writeline() def leave(self, lbl): @@ -98,6 +112,12 @@ def call(self, func): self.code.writeline('call ' + func) + def call_method(self, meth): + self.code.writeline('callvirt instance ' + meth) + + def new(self, class_): + self.code.writeline('newobj ' + class_) + def opcode(self, opcode, *args): self.code.write(opcode + ' ') self.code.writeline(' '.join(map(str, args))) Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Wed Apr 5 19:08:30 2006 @@ -1,5 +1,5 @@ class Generator(object): - def function_name(self, graph): + def function_signature(self, graph): pass def emit(self, instr, *args): @@ -50,14 +50,50 @@ class _Call(MicroInstruction): def render(self, generator, op): - func_name = generator.function_name(op.args[0].value.graph) + func_sig = generator.function_signature(op.args[0].value.graph) # push parameters for func_arg in op.args[1:]: generator.load(func_arg) - generator.call(func_name) + generator.call(func_sig) + +class _New(MicroInstruction): + def render(self, generator, op): + generator.new(op.args[0].value) + +class _SetField(MicroInstruction): + def render(self, generator, op): + this, field, value = op.args + if field.value == 'meta': + return # TODO + + generator.load(this) + generator.load(value) + generator.set_field(this.concretetype, field.value) + +class _GetField(MicroInstruction): + def render(self, generator, op): + this, field = op.args + generator.load(this) + generator.get_field(this.concretetype, field.value) + +class _CallMethod(MicroInstruction): + def render(self, generator, op): + method = op.args[0] + this = op.args[1] + + # push parameters + for func_arg in op.args[1:]: + generator.load(func_arg) + + generator.call_method(this.concretetype, method.value) + PushAllArgs = _PushAllArgs() StoreResult = _StoreResult() Call = _Call() +New = _New() +SetField = _SetField() +GetField = _GetField() +CallMethod = _CallMethod() Added: pypy/dist/pypy/translator/cli/node.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/node.py Wed Apr 5 19:08:30 2006 @@ -0,0 +1,6 @@ +class Node(object): + def get_name(self): + pass + + def render(self, ilasm): + pass Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Wed Apr 5 19:08:30 2006 @@ -1,4 +1,5 @@ -from pypy.translator.cli.metavm import PushArg, PushAllArgs, StoreResult, Call, InstructionList +from pypy.translator.cli.metavm import PushArg, PushAllArgs,\ + StoreResult, Call, InstructionList, New, SetField, GetField, CallMethod # some useful instruction patterns Not = ['ldc.i4.0', 'ceq'] @@ -9,6 +10,13 @@ opcodes = { + # __________ object oriented operations __________ + 'new': [New], + 'oosetfield': [SetField], + 'oogetfield': [GetField], + 'oosend': [CallMethod], + + 'same_as': DoNothing, # TODO: does same_as really do nothing else than renaming? 'direct_call': [Call], 'indirect_call': None, # when is it generated? Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Wed Apr 5 19:08:30 2006 @@ -23,14 +23,16 @@ else: print 'OK' -def foo(x): - return x+1 +class A: + def __init__(self, x): + self.x = x + + def foo(self, a, b): + return self.x def bar(x, y): - if x > y: - return x+y+foo(2) - else: - return x-y + a = A(x) + return a.foo(x, y) f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Wed Apr 5 19:08:30 2006 @@ -7,7 +7,7 @@ from pypy.translator.translator import TranslationContext from pypy.translator.cli.option import getoption from pypy.translator.cli.gencli import GenCli -from pypy.translator.cli.function import Node +from pypy.translator.cli.node import Node from pypy.translator.cli.cts import graph_to_signature from pypy.translator.cli.cts import llvar_to_cts @@ -36,7 +36,7 @@ return 'main' def render(self, ilasm): - ilasm.begin_function('main', [('string[]', 'argv')], 'void', True) + ilasm.begin_function('main', [('string[]', 'argv')], 'void', True, 'static') # TODO: only int32 and bool are tested for i, arg in enumerate(self.graph.getargs()): Added: pypy/dist/pypy/translator/cli/test/test_oo.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/test/test_oo.py Wed Apr 5 19:08:30 2006 @@ -0,0 +1,24 @@ +from pypy.translator.cli.test.runtest import check + +class MyClass: + def __init__(self, x, y): + self.x = x + self.y = y + + def compute(self): + return self.x + self.y + + def compute_and_multiply(self, factor): + return self.compute() * factor + +def oo_compute(x, y): + obj = MyClass(x, y) + return obj.compute() + +def oo_compute_multiply(x, y): + obj = MyClass(x, y) + return obj.compute_and_multiply(2) + +def test_oo(): + yield check, oo_compute, [int, int], (42, 13) + yield check, oo_compute_multiply, [int, int], (42, 13) From mwh at codespeak.net Wed Apr 5 19:10:45 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 19:10:45 +0200 (CEST) Subject: [pypy-svn] r25409 - pypy/dist/pypy/translator/c/src Message-ID: <20060405171045.CD9341022A@code0.codespeak.net> Author: mwh Date: Wed Apr 5 19:10:43 2006 New Revision: 25409 Modified: pypy/dist/pypy/translator/c/src/pyobj.h Log: fix semi-colon placement in the OP_LEN macro. Modified: pypy/dist/pypy/translator/c/src/pyobj.h ============================================================================== --- pypy/dist/pypy/translator/c/src/pyobj.h (original) +++ pypy/dist/pypy/translator/c/src/pyobj.h Wed Apr 5 19:10:43 2006 @@ -25,12 +25,12 @@ #define OP_IS_TRUE(x,r) op_bool(r,PyObject_IsTrue(x)) #define OP_NONZERO(x,r) op_bool(r,PyObject_IsTrue(x)) -#define OP_LEN(x,r) { \ +#define OP_LEN(x,r) do { \ int _retval = PyObject_Size(x); \ - if (_retval < 0) { CFAIL() }; \ + if (_retval < 0) { CFAIL(); } \ else \ r = PyInt_FromLong(_retval); \ - } + } while (0) #define OP_NEG(x,r) if (!(r=PyNumber_Negative(x))) CFAIL() #define OP_POS(x,r) if (!(r=PyNumber_Positive(x))) CFAIL() #define OP_INVERT(x,r) if (!(r=PyNumber_Invert(x))) CFAIL() From mwh at codespeak.net Wed Apr 5 19:29:39 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 19:29:39 +0200 (CEST) Subject: [pypy-svn] r25410 - in pypy/dist/pypy: rpython rpython/lltypesystem translator/c Message-ID: <20060405172939.8419D10222@code0.codespeak.net> Author: mwh Date: Wed Apr 5 19:29:37 2006 New Revision: 25410 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/translator/c/funcgen.py Log: (pedronis, mwh) re-introduce bare_setfield for test_wrapping. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Wed Apr 5 19:29:37 2006 @@ -391,6 +391,8 @@ write_barrier = gc.get_funcptr_write_barrier() result = self.op_direct_call(write_barrier, *args) + op_bare_setfield = op_setfield + def op_getarrayitem(self, array, index): return array[index] Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Wed Apr 5 19:29:37 2006 @@ -255,6 +255,7 @@ 'getsubstruct': LLOp(canfold=True), 'getarraysubstruct': LLOp(canfold=True), 'setfield': LLOp(), + 'bare_setfield': LLOp(), 'setarrayitem': LLOp(), 'cast_pointer': LLOp(canfold=True), 'ptr_eq': LLOp(canfold=True), Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Wed Apr 5 19:29:37 2006 @@ -394,6 +394,8 @@ return self.generic_set(op, '%s->%s' % (self.expr(op.args[0]), fieldname)) + OP_BARE_SETFIELD = OP_SETFIELD + def OP_GETSUBSTRUCT(self, op): return self.OP_GETFIELD(op, ampersand='&') From arigo at codespeak.net Wed Apr 5 20:06:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 5 Apr 2006 20:06:55 +0200 (CEST) Subject: [pypy-svn] r25412 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060405180655.75F291022E@code0.codespeak.net> Author: arigo Date: Wed Apr 5 20:06:52 2006 New Revision: 25412 Added: pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: (arre, arigo) (dinner time!) Completed an rarray test: missing conversion from ints to c_long. Started function support (mostly by copying code from implementation.py). Pushed and pulled a bit and 42 fell out. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Wed Apr 5 20:06:52 2006 @@ -765,16 +765,10 @@ # Note: The following works for index either pointers and arrays, # because both have a _type_ attribute that contains the type of the # object pointed to or in the case of an array the element type. - assert hasattr(s_cto.knowntype, '_type_') - assert extregistry.is_registered_type(s_cto.knowntype._type_) - entry = extregistry.lookup_type(s_cto.knowntype._type_) - if hasattr(entry, 'lowleveltype'): - # special case for reading primitives out of arrays - return lltype_to_annotation(entry.lowleveltype) - else: - return SomeCTypesObject( - s_cto.knowntype._type_, - memorystate=SomeCTypesObject.MEMORYALIAS) + result_ctype = s_cto.knowntype._type_ + s_result = SomeCTypesObject(result_ctype, + memorystate=SomeCTypesObject.MEMORYALIAS) + return s_result.return_annotation() class __extend__(pairtype(SomeCTypesObject, SomeSlice)): def setitem((s_cto, s_slice), s_iterable): Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Wed Apr 5 20:06:52 2006 @@ -443,6 +443,21 @@ def can_be_none(self): return False + def return_annotation(self): + """Returns either 'self' or the annotation of the unwrapped version + of this ctype, following the logic used when ctypes operations + return a value. + """ + from pypy.rpython import extregistry + assert extregistry.is_registered_type(self.knowntype) + entry = extregistry.lookup_type(self.knowntype) + if hasattr(entry, 'lowleveltype'): + # special case for returning primitives + return lltype_to_annotation(entry.lowleveltype) + else: + return self + + class SomeImpossibleValue(SomeObject): """The empty set. Instances are placeholders for objects that will never show up at run-time, e.g. elements of an empty list.""" Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Apr 5 20:06:52 2006 @@ -24,6 +24,7 @@ import pypy.rpython.rctypes.rarray import pypy.rpython.rctypes.rprimitive import pypy.rpython.rctypes.rpointer +import pypy.rpython.rctypes.rfunc # ctypes_annotation_list contains various attributes that # are used by the pypy annotation. @@ -81,79 +82,6 @@ create_ctypes_annotations() -CFuncPtrType = type(ctypes.CFUNCTYPE(None)) - -def cfuncptrtype_compute_annotation(type, instance): - from pypy.annotation.model import ll_to_annotation_map - - def compute_result_annotation(*args_s): - """ - Answer the annotation of the external function's result - """ - - # results of external function calls *must* be in the registry - # XXX: goden: check metatype too? - assert extregistry.is_registered_type(instance.restype) - - entry = extregistry.lookup_type(instance.restype) - - # XXX: goden: this probably isn't right - return ll_to_annotation_map.get(entry.lowleveltype) - - # Take 3, Check whether we can get away with the cheap - # precomputed solution and if not it, use a special - # attribute with the memory state - try: - return instance.restype.annotator_type - except AttributeError: - return SomeCTypesObject( - instance.restype, - instance.restype.external_function_result_memorystate ) - # Take 2, looks like we need another level of indirection - # That's to complicated - #o#return self.restype.compute_external_function_result_annotator_type() - # TODO: Check whether the function returns a pointer - # an correct the memory state appropriately - try: - return instance.restype.annotator_type - except AttributeError: - return SomeCTypesObject(instance.restype) - - return SomeBuiltin(compute_result_annotation, - methodname=instance.__name__) - -def cfuncptrtype_specialize_call(hop): - # this is necessary to get the original function pointer when specializing - # the metatype - cfuncptr = hop.spaceop.args[0].value - - def convert_params(backend, param_info_list): - assert "c" == backend.lower() - assert cfuncptr.argtypes is not None - answer = [] - for ctype_type, (ll_type, arg_name) in zip(cfuncptr.argtypes, - param_info_list): - if ll_type == ctype_type.ll_type: - answer.append(arg_name) - else: - answer.append(ctype_type.wrap_arg(ll_type, arg_name)) - return answer - - assert extregistry.is_registered_type(cfuncptr.restype) - - # results of external function calls *must* be in the registry - entry = extregistry.lookup_type(cfuncptr.restype) - - return hop.llops.gencapicall( - cfuncptr.__name__, - hop.args_v, - resulttype = entry.lowleveltype, - _callable=None, - convert_params = convert_params ) - -extregistry.register_metatype(CFuncPtrType, - compute_annotation=cfuncptrtype_compute_annotation, - specialize_call=cfuncptrtype_specialize_call) class FunctionPointerTranslation(object): Added: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Wed Apr 5 20:06:52 2006 @@ -0,0 +1,46 @@ +from pypy.annotation import model as annmodel +from pypy.rpython import extregistry + +import ctypes + + +CFuncPtrType = type(ctypes.CFUNCTYPE(None)) + +def cfuncptrtype_compute_annotation(type, instance): + + def compute_result_annotation(*args_s): + """ + Answer the annotation of the external function's result + """ + result_ctype = instance.restype + s_result = annmodel.SomeCTypesObject(result_ctype, + annmodel.SomeCTypesObject.OWNSMEMORY) + return s_result.return_annotation() + + return annmodel.SomeBuiltin(compute_result_annotation, + methodname=instance.__name__) + +def cfuncptrtype_specialize_call(hop): + # this is necessary to get the original function pointer when specializing + # the metatype + assert hop.spaceop.opname == "simple_call" + cfuncptr = hop.spaceop.args[0].value + + args_r = [] + for ctype in cfuncptr.argtypes: + s_arg = annmodel.SomeCTypesObject(ctype, + annmodel.SomeCTypesObject.MEMORYALIAS) + r_arg = hop.rtyper.getrepr(s_arg) + args_r.append(r_arg) + + vlist = hop.inputargs(*args_r) + unwrapped_args_v = [r_arg.getvalue(hop.llops, v) + for r_arg, v in zip(args_r, vlist)] + + ll_func = cfuncptr.llinterp_friendly_version + v_result = hop.llops.gendirectcall(ll_func, *unwrapped_args_v) + return v_result + +extregistry.register_metatype(CFuncPtrType, + compute_annotation=cfuncptrtype_compute_annotation, + specialize_call=cfuncptrtype_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Wed Apr 5 20:06:52 2006 @@ -26,8 +26,8 @@ # traked by our GC around the raw 'c_data_type'-shaped # data. # - # * 'owner_lowleveltype' is the lowleveltype of the repr for the same - # ctype but for ownsmemory=True. + # * 'r_memoryowner.lowleveltype' is the lowleveltype of the repr for the + # same ctype but for ownsmemory=True. def __init__(self, rtyper, s_ctypesobject, ll_type): # s_ctypesobject: the annotation to represent @@ -49,20 +49,23 @@ self.c_data_type = self.get_c_data_type(ll_type) content_keepalives = self.get_content_keepalives() - self.owner_lowleveltype = lltype.Ptr( - lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data", self.c_data_type ), - *content_keepalives - ) - ) if self.ownsmemory: - self.lowleveltype = self.owner_lowleveltype + self.r_memoryowner = self + self.lowleveltype = lltype.Ptr( + lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), + ( "c_data", self.c_data_type ), + *content_keepalives + ) + ) else: + s_memoryowner = SomeCTypesObject(ctype, + SomeCTypesObject.OWNSMEMORY) + self.r_memoryowner = rtyper.getrepr(s_memoryowner) self.lowleveltype = lltype.Ptr( lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data_ref", lltype.Ptr(self.c_data_type) ), - ( "c_data_owner_keepalive", self.owner_lowleveltype ), - *content_keepalives + ( "c_data_ref", lltype.Ptr(self.c_data_type) ), + ( "c_data_owner_keepalive", self.r_memoryowner.lowleveltype ), + *content_keepalives ) ) self.const_cache = {} # store generated const values+original value @@ -88,7 +91,7 @@ inputargs = [v_box, inputconst(lltype.Void, "c_data_owner_keepalive")] return llops.genop('getfield', inputargs, - self.owner_lowleveltype) + self.r_memoryowner.lowleveltype) def allocate_instance(self, llops): c1 = inputconst(lltype.Void, self.lowleveltype.TO) @@ -104,7 +107,8 @@ inputargs = [v_box, inputconst(lltype.Void, "c_data_ref"), v_c_data] llops.genop('setfield', inputargs) if v_c_data_owner is not None: - assert v_c_data_owner.concretetype == self.owner_lowleveltype + assert (v_c_data_owner.concretetype == + self.r_memoryowner.lowleveltype) inputargs = [v_box, inputconst(lltype.Void, "c_data_owner_keepalive"), v_c_data_owner] Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Wed Apr 5 20:06:52 2006 @@ -22,7 +22,8 @@ def get_content_keepalives(self): "Return an extra keepalive field used for the pointer's contents." - return [('keepalive_contents', self.r_contents.owner_lowleveltype)] + return [('keepalive_contents', + self.r_contents.r_memoryowner.lowleveltype)] def setkeepalive(self, llops, v_box, v_owner): inputargs = [v_box, inputconst(lltype.Void, 'keepalive_contents'), Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Wed Apr 5 20:06:52 2006 @@ -6,7 +6,7 @@ from pypy.rpython.rmodel import Repr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype -from pypy.rpython.rmodel import IntegerRepr +from pypy.rpython.rmodel import IntegerRepr, FloatRepr, CharRepr from pypy.rpython.error import TyperError from pypy.rpython.rctypes.rmodel import CTypesValueRepr @@ -49,7 +49,7 @@ try: return self.const_cache[key][0] except KeyError: - p = lltype.malloc(self.owner_lowleveltype.TO) + p = lltype.malloc(self.r_memoryowner.lowleveltype.TO) p.c_data.value = value if self.ownsmemory: result = p @@ -78,6 +78,14 @@ self.setvalue(hop.llops, v_primitive, v_value) +class __extend__(pairtype(IntegerRepr, PrimitiveRepr)): + def convert_from_to((r_from, r_to), v, llops): + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + r_temp.setvalue(llops, v_owned_box, v) + return llops.convertvar(v_owned_box, r_temp, r_to) + + def primitive_specialize_call(hop): r_primitive = hop.r_result v_result = r_primitive.allocate_instance(hop.llops) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Wed Apr 5 20:06:52 2006 @@ -34,16 +34,17 @@ a.translator.view() def test_annotate_array_access(self): - def access_array(): + def access_array(n): my_array = c_int_10() my_array[0] = c_int(1) my_array[1] = 2 + my_array[2] = n return my_array[0] t = TranslationContext() a = t.buildannotator() - s = a.build_types(access_array, []) + s = a.build_types(access_array, [int]) assert s.knowntype == int if conftest.option.view: @@ -114,14 +115,15 @@ assert len(c_data) == 10 def test_specialize_array_access(self): - def access_array(): + def access_array(n): my_array = c_int_10() my_array[0] = 1 my_array[1] = c_int(1) + my_array[2] = n return my_array[0] - res = interpret(access_array, []) + res = interpret(access_array, [44]) assert res == 1 class Test_compilation: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Apr 5 20:06:52 2006 @@ -257,7 +257,7 @@ # result should be an integer assert s.knowntype == int - def test_specialize_simple(self): + def x_test_specialize_simple(self): t = TranslationContext() a = t.buildannotator() s = a.build_types(o_atoi, [str]) Added: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Wed Apr 5 20:06:52 2006 @@ -0,0 +1,36 @@ +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.rpython.rctypes.test.test_rctypes import mylib +from pypy.rpython.test.test_llinterp import interpret +from pypy import conftest + +from ctypes import c_long + + +labs = mylib.labs +labs.restype = c_long +labs.argtypes = [c_long] + +def ll_labs(n): + return abs(n) + +labs.llinterp_friendly_version = ll_labs + + +def test_labs(n=6): + assert labs(n) == abs(n) + assert labs(c_long(0)) == 0 + assert labs(-42) == 42 + return labs(n) + +class Test_annotation: + def test_annotate_labs(self): + a = RPythonAnnotator() + s = a.build_types(test_labs, [int]) + assert s.knowntype == int + if conftest.option.view: + a.translator.view() + +class Test_specialization: + def test_specialize_labs(self): + res = interpret(test_labs, [-11]) + assert res == 11 From mwh at codespeak.net Wed Apr 5 20:11:01 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 20:11:01 +0200 (CEST) Subject: [pypy-svn] r25413 - in pypy/dist/pypy/translator: backendopt c Message-ID: <20060405181101.5D1851022F@code0.codespeak.net> Author: mwh Date: Wed Apr 5 20:10:59 2006 New Revision: 25413 Modified: pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/c/exceptiontransform.py Log: (mwh, pedronis) don't keep on recreating raiseanalyzers for inlining purposes, instead use the one that the exceptiontransformer already has. Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Wed Apr 5 20:10:59 2006 @@ -2,7 +2,6 @@ from pypy.translator.simplify import join_blocks, cleanup_graph from pypy.translator.simplify import get_graph from pypy.translator.unsimplify import copyvar, split_block -from pypy.translator.backendopt import canraise from pypy.objspace.flow.model import Variable, Constant, Block, Link from pypy.objspace.flow.model import SpaceOperation, c_last_exception from pypy.objspace.flow.model import traverse, mkentrymap, checkgraph @@ -103,7 +102,7 @@ class BaseInliner(object): def __init__(self, translator, graph, inline_guarded_calls=False, - inline_guarded_calls_no_matter_what=False): + inline_guarded_calls_no_matter_what=False, raise_analyzer=None): self.translator = translator self.graph = graph self.inline_guarded_calls = inline_guarded_calls @@ -111,7 +110,8 @@ # it is used by the exception transformation self.inline_guarded_calls_no_matter_what = inline_guarded_calls_no_matter_what if inline_guarded_calls: - self.raise_analyzer = canraise.RaiseAnalyzer(translator) + assert raise_analyzer is not None + self.raise_analyzer = raise_analyzer else: self.raise_analyzer = None @@ -398,9 +398,11 @@ class Inliner(BaseInliner): def __init__(self, translator, graph, inline_func, inline_guarded_calls=False, - inline_guarded_calls_no_matter_what=False): + inline_guarded_calls_no_matter_what=False, raise_analyzer=None): BaseInliner.__init__(self, translator, graph, - inline_guarded_calls, inline_guarded_calls_no_matter_what) + inline_guarded_calls, + inline_guarded_calls_no_matter_what, + raise_analyzer) self.inline_func = inline_func # to simplify exception matching join_blocks(graph) @@ -412,9 +414,11 @@ class OneShotInliner(BaseInliner): def __init__(self, translator, graph, inline_guarded_calls=False, - inline_guarded_calls_no_matter_what=False): + inline_guarded_calls_no_matter_what=False, raise_analyzer=None): BaseInliner.__init__(self, translator, graph, - inline_guarded_calls, inline_guarded_calls_no_matter_what) + inline_guarded_calls, + inline_guarded_calls_no_matter_what, + raise_analyzer) def search_for_calls(self, block): pass Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Wed Apr 5 20:10:59 2006 @@ -202,7 +202,10 @@ #non-exception case block.exits[0].exitcase = block.exits[0].llexitcase = None # use the dangerous second True flag :-) - inliner = inline.OneShotInliner(self.translator, graph, True, True) + inliner = inline.OneShotInliner(self.translator, graph, + inline_guarded_calls=True, + inline_guarded_calls_no_matter_what=True, + raise_analyzer=self.raise_analyzer) inliner.inline_once(block, len(block.operations)-1) #block.exits[0].exitcase = block.exits[0].llexitcase = False From mwh at codespeak.net Wed Apr 5 20:17:08 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 5 Apr 2006 20:17:08 +0200 (CEST) Subject: [pypy-svn] r25414 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060405181708.7405E10233@code0.codespeak.net> Author: mwh Date: Wed Apr 5 20:17:07 2006 New Revision: 25414 Added: pypy/dist/pypy/translator/c/stacklesstransform.py (contents, props changed) pypy/dist/pypy/translator/c/test/test_stacklesstransform.py (contents, props changed) Log: very first glimmerings of a stackless transformer. doesn't actually do anything at all much, just computes the lltype.Struct suitable for storing away a bunch of variables. Added: pypy/dist/pypy/translator/c/stacklesstransform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/c/stacklesstransform.py Wed Apr 5 20:17:07 2006 @@ -0,0 +1,54 @@ +from pypy.rpython.lltypesystem import lltype, llmemory + +STORAGE_TYPES = [llmemory.Address, + lltype.Signed, + lltype.Float, + lltype.SignedLongLong] +STORAGE_FIELDS = ['addr', + 'long', + 'float', + 'longlong'] + +def storage_type(T): + """Return the type used to save values of this type + """ + if T is lltype.Void: + return None + elif T is lltype.Float: + return 2 + elif T in [lltype.SignedLongLong, lltype.UnsignedLongLong]: + return 3 + elif T is llmemory.Address or isinstance(T, lltype.Ptr): + return 0 + elif isinstance(T, lltype.Primitive): + return 1 + else: + raise Exception("don't know about %r" % (T,)) + + +class StacklessTransfomer(object): + def __init__(self, translator): + self.translator = translator + + self.frametypes = {} + + def frame_type_for_vars(self, vars): + types = [storage_type(v.concretetype) for v in vars] + counts = dict.fromkeys(range(len(STORAGE_TYPES)), 0) + for t in types: + counts[t] = counts[t] + 1 + keys = counts.keys() + keys.sort() + key = tuple([counts[k] for k in keys]) + if key in self.frametypes: + return self.frametypes[key] + else: + fields = [] + for i, k in enumerate(key): + for j in range(k): + fields.append(('state_%s_%d'%(STORAGE_FIELDS[i], j), STORAGE_TYPES[i])) + T = lltype.Struct("state_%d_%d_%d_%d"%tuple(key), *fields) + self.frametypes[key] = T + return T + + Added: pypy/dist/pypy/translator/c/test/test_stacklesstransform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/c/test/test_stacklesstransform.py Wed Apr 5 20:17:07 2006 @@ -0,0 +1,22 @@ +from pypy.translator.c.stacklesstransform import StacklessTransfomer +from pypy.rpython.memory.gctransform import varoftype +from pypy.rpython.lltypesystem import lltype, llmemory + +def test_frame_types(): + st = StacklessTransfomer(None) + + signed = varoftype(lltype.Signed) + ptr = varoftype(lltype.Ptr(lltype.GcStruct("S"))) + addr = varoftype(llmemory.Address) + float = varoftype(lltype.Float) + longlong = varoftype(lltype.SignedLongLong) + + ft4vars = st.frame_type_for_vars + + s1 = ft4vars([signed]) + assert len(s1._flds) == 1 + + s2_1 = ft4vars([signed, ptr]) + s2_2 = ft4vars([ptr, signed]) + + assert s2_1 is s2_2 From afayolle at codespeak.net Wed Apr 5 20:19:23 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Wed, 5 Apr 2006 20:19:23 +0200 (CEST) Subject: [pypy-svn] r25415 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060405181923.995B510233@code0.codespeak.net> Author: afayolle Date: Wed Apr 5 20:19:22 2006 New Revision: 25415 Modified: pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_fd.py Log: wrapped most useful methods in domain.py Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Wed Apr 5 20:19:22 2006 @@ -6,7 +6,6 @@ from pypy.objspace.std.objspace import W_Object -# ? from pypy.objspace.std.listobject import W_ListObject, W_TupleObject @@ -21,19 +20,23 @@ def __init__(self, space): self._space = space - self.__changed = 0 + self.__changed = self._space.newbool(False) def w_reset_flags(self): - self.__changed = 0 + self.__changed = self._space.newbool(False) def w_has_changed(self): return self.__changed + def w_size(self): + pass + def _value_removed(self): """The implementation of remove_value should call this method""" - self.__changed = 1 - if self.size() == 0: - raise ConsistencyFailure() + self.__changed = self._space.newbool(True) + if self._space.eq_w(self.w_size(), self._space.newint(0)): + raise OperationError(self._space.w_RuntimeError, + self._space.wrap('ConsistencyFailure')) W_AbstractDomain.typedef = typedef.TypeDef("W_AbstractDomain", reset_flags = interp2app(W_AbstractDomain.w_reset_flags), @@ -61,8 +64,9 @@ def w_remove_values(self, w_values): """Remove values of domain and check for consistency""" - if w_values: - for val in w_values : + if self._space.is_true(self._space.gt(self._space.len(w_values), + self._space.newint(0))) : + for val in w_values.wrappeditems : self._values.remove(val) self._value_removed() __delitem__ = w_remove_value @@ -82,25 +86,28 @@ def w_copy(self): """clone the domain""" - return W_FiniteDomain(self, self._space) + return W_FiniteDomain(self._space, self.w_get_values()) def __repr__(self): return '' % str(self.get_values()) - def __eq__(self, other): - if other is NoDom: return False - return self._values == other._values - - def __ne__(self, other): - return not self == other - - def intersection(self, other): - if other is None: return self.get_values() - return self._values & other._values + def __eq__(self, w_other): + if w_other is NoDom: return False + return self._values == w_other._values + + def __ne__(self, w_other): + return not self == w_other + + # FIXME: this does not work, but we don't need it yet + def w_intersection(self, w_other): + assert isinstance(w_other, W_FiniteDomain) + return self._space.newlist([x for x in (set(self.w_get_values()) & set(w_other.w_get_values()))]) W_FiniteDomain.typedef = typedef.TypeDef("W_FiniteDomain", + W_AbstractDomain.typedef, remove_value = interp2app(W_FiniteDomain.w_remove_value), remove_values = interp2app(W_FiniteDomain.w_remove_values), get_values = interp2app(W_FiniteDomain.w_get_values), copy = interp2app(W_FiniteDomain.w_copy), +# intersection = interp2app(W_FiniteDomain.w_intersection), size = interp2app(W_FiniteDomain.w_size)) Modified: pypy/dist/pypy/objspace/constraint/test/test_fd.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_fd.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_fd.py Wed Apr 5 20:19:22 2006 @@ -1,7 +1,5 @@ from pypy.conftest import gettestobjspace -class UnificationFailure(Exception): pass - class AppTest_FD(object): def setup_class(cls): @@ -12,3 +10,55 @@ assert fd.size() == 3 assert set(fd.get_values()) == set([1, 2, 3]) #fd2 = fd.copy() + + def test_copy(self): + fd = FiniteDomain([1, 2, 3]) + clone = fd.copy() + assert set(clone.get_values()) == set([1, 2, 3]) + + def test_remove_value(self): + fd = FiniteDomain([1, 2, 3]) + fd.remove_value(2) + assert fd.size() == 2 + assert set(fd.get_values()) == set([1, 3]) + assert fd.has_changed() + + def test_remove_all_values(self): + fd = FiniteDomain([3]) + # FIXME: check this is a ConsistencyFailure + raises(Exception, fd.remove_value, 3) + + + def test_remove_values(self): + fd = FiniteDomain([1, 2, 3]) + fd.remove_values([1, 2]) + assert fd.size() == 1 + assert set(fd.get_values()) == set([3,]) + assert fd.has_changed() + + def test_remove_values_empty_list(self): + fd = FiniteDomain([1, 2, 3]) + assert not(fd.has_changed()) + fd.remove_values([]) + assert fd.size() == 3 + assert not(fd.has_changed()) + + + def notest_logical_variable_in_domain(self): + '''Logical variables do not play well with sets, skip for now''' + X = newvar() + fd = FiniteDomain([X, 2]) + assert fd.size() == 2 + unify(X,42) + fd.remove_value(42) + assert fd.size() == 1 + assert fd.has_changed() + + def notest_intersection(self): + """not used for now""" + fd1 = FiniteDomain([1, 2, 3]) + fd2 = FiniteDomain([2,3,4]) + assert fd1.intersection(fd2) == set([2,3]) + assert fd2.intersection(fd1) == set([3,2]) + + From antocuni at codespeak.net Wed Apr 5 20:41:59 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 5 Apr 2006 20:41:59 +0200 (CEST) Subject: [pypy-svn] r25417 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060405184159.0DD9710235@code0.codespeak.net> Author: antocuni Date: Wed Apr 5 20:41:46 2006 New Revision: 25417 Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/ilgenerator.py pypy/dist/pypy/translator/cli/test/compile.py Log: Added namespace support Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Wed Apr 5 20:41:46 2006 @@ -5,14 +5,17 @@ class Class(Node): def __init__(self, classdef): self.classdef = classdef + self.namespace, self.name = cts.split_class_name(classdef._name) def get_name(self): - return self.classdef._name + return self.name def render(self, ilasm): self.ilasm = ilasm - name = self.get_name().replace('__main__.', '') # TODO: handle modules - ilasm.begin_class(name) # TODO: handle base class + if self.namespace: + ilasm.begin_namespace(self.namespace) + + ilasm.begin_class(self.name) # TODO: handle base class for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): # TODO: handle default values ilasm.field(f_name, cts.lltype_to_cts(f_type)) @@ -27,6 +30,9 @@ ilasm.end_class() + if self.namespace: + ilasm.end_namespace() + def _ctor(self): self.ilasm.begin_function('.ctor', [], 'void', False, 'specialname', 'rtspecialname', 'instance') self.ilasm.opcode('ldarg.0') Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Wed Apr 5 20:41:46 2006 @@ -92,6 +92,13 @@ return '%s %s(%s)' % (ret_type, func_name, arg_list) +def split_class_name(class_name): + parts = class_name.rsplit('.', 1) + if len(parts) == 2: + return parts + else: + return None, parts[0] + _pyexception_to_cts = { exceptions.Exception: '[mscorlib]System.Exception', exceptions.OverflowError: '[mscorlib]System.OverflowException' Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Wed Apr 5 20:41:46 2006 @@ -229,7 +229,7 @@ return cts.graph_to_signature(graph, True, name) def class_name(self, ooinstance): - return ooinstance._name.replace('__main__.', '') # TODO: modules + return ooinstance._name def emit(self, instr, *args): self.ilasm.opcode(instr, *args) @@ -249,6 +249,8 @@ def call_method(self, obj, name): owner, meth = obj._lookup(name) full_name = '%s::%s' % (self.class_name(obj), name) + # TODO: there are cases when we don't need callvirt but a + # plain call is sufficient self.ilasm.call_method(self.method_signature(meth.graph, full_name)) def load(self, v): Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Wed Apr 5 20:41:46 2006 @@ -61,7 +61,7 @@ # TODO: remove this test if graph.name.startswith('ll_'): continue - + if '.' not in graph.name: # it's not a method f = Function(graph) f.render(self.ilasm) Modified: pypy/dist/pypy/translator/cli/ilgenerator.py ============================================================================== --- pypy/dist/pypy/translator/cli/ilgenerator.py (original) +++ pypy/dist/pypy/translator/cli/ilgenerator.py Wed Apr 5 20:41:46 2006 @@ -42,6 +42,13 @@ def close(self): self.out.close() + def begin_namespace(self, name): + self.code.writeline('.namespace ' + name) + self.code.openblock() + + def end_namespace(self): + self.code.closeblock() + def begin_class(self, name, base = None): if base is None: base = '[mscorlib]System.Object' Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Wed Apr 5 20:41:46 2006 @@ -1,5 +1,4 @@ #!/bin/env python - import sys import py from pypy.rpython.rarithmetic import r_int, r_uint, r_ulonglong, r_longlong, ovfcheck From afayolle at codespeak.net Wed Apr 5 23:29:07 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Wed, 5 Apr 2006 23:29:07 +0200 (CEST) Subject: [pypy-svn] r25423 - pypy/dist/pypy/objspace/constraint Message-ID: <20060405212907.5F59610238@code0.codespeak.net> Author: afayolle Date: Wed Apr 5 23:29:06 2006 New Revision: 25423 Modified: pypy/dist/pypy/objspace/constraint/distributor.py Log: added missing constructor to NaiveDistributor Modified: pypy/dist/pypy/objspace/constraint/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/distributor.py Wed Apr 5 23:29:06 2006 @@ -61,6 +61,9 @@ The first new domain has a size of one, and the second has all the other values""" + def __init__(self, c_space): + AbstractDistributor.__init__(self, c_space, 2) + def _do_distribute(self, domain, choice): values = domain.get_values() if choice == 0: From ericvrp at codespeak.net Wed Apr 5 23:32:08 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 5 Apr 2006 23:32:08 +0200 (CEST) Subject: [pypy-svn] r25424 - pypy/dist/pypy/translator/llvm/pyllvm Message-ID: <20060405213208.ECDDA1023A@code0.codespeak.net> Author: ericvrp Date: Wed Apr 5 23:32:08 2006 New Revision: 25424 Modified: pypy/dist/pypy/translator/llvm/pyllvm/cc.py Log: Use ctypes.cdll.LoadLibrary which seems to work better on Linux Modified: pypy/dist/pypy/translator/llvm/pyllvm/cc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pyllvm/cc.py (original) +++ pypy/dist/pypy/translator/llvm/pyllvm/cc.py Wed Apr 5 23:32:08 2006 @@ -8,7 +8,7 @@ d = __file__[:__file__.find("pyllvm")] + "llvmcapi/" #XXX does this load once or every time? try: - self.library = cdll.load(d + libname + ".so") + self.library = cdll.LoadLibrary(d + libname + ".so") except: raise Exception("llvmcapi not found: run 'python setup.py build_ext -i' in " + d) self.restype = restype From antocuni at codespeak.net Wed Apr 5 23:39:06 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 5 Apr 2006 23:39:06 +0200 (CEST) Subject: [pypy-svn] r25425 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060405213906.7FF731023B@code0.codespeak.net> Author: antocuni Date: Wed Apr 5 23:38:52 2006 New Revision: 25425 Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/test_oo.py Log: Added support for inheritance Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Wed Apr 5 23:38:52 2006 @@ -10,12 +10,19 @@ def get_name(self): return self.name + def get_base_class(self): + base_class = self.classdef._superclass + if base_class is None or base_class._name == 'Object': + return '[mscorlib]System.Object' + else: + return base_class._name + def render(self, ilasm): self.ilasm = ilasm if self.namespace: ilasm.begin_namespace(self.namespace) - ilasm.begin_class(self.name) # TODO: handle base class + ilasm.begin_class(self.name, self.get_base_class()) for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): # TODO: handle default values ilasm.field(f_name, cts.lltype_to_cts(f_type)) @@ -23,7 +30,7 @@ self._ctor() for m_name, m_meth in self.classdef._methods.iteritems(): - # TODO: handle static methods + # TODO: handle static, class and unbound methods # TODO: should __init__ be rendered as a constructor? f = Function(m_meth.graph, m_name, is_method = True) f.render(ilasm) @@ -36,7 +43,7 @@ def _ctor(self): self.ilasm.begin_function('.ctor', [], 'void', False, 'specialname', 'rtspecialname', 'instance') self.ilasm.opcode('ldarg.0') - self.ilasm.call('instance void object::.ctor()') # TODO: base class + self.ilasm.call('instance void %s::.ctor()' % self.get_base_class()) self.ilasm.opcode('ret') self.ilasm.end_function() Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Wed Apr 5 23:38:52 2006 @@ -47,6 +47,7 @@ self.ilasm = IlasmGenerator(out, self.assembly_name) self.gen_all_functions() + self.find_superclasses() self.gen_classes() out.close() return self.tmpfile.strpath @@ -67,6 +68,18 @@ f.render(self.ilasm) self.classdefs.update(f.classdefs) + def find_superclasses(self): + classdefs = set() + pendings = self.classdefs + + while pendings: + classdef = pendings.pop() + if classdef not in classdefs and classdef is not None: + classdefs.add(classdef) + pendings.add(classdef._superclass) + + self.classdefs = classdefs + def gen_classes(self): for classdef in self.classdefs: c = Class(classdef) Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Wed Apr 5 23:38:52 2006 @@ -15,6 +15,8 @@ 'oosetfield': [SetField], 'oogetfield': [GetField], 'oosend': [CallMethod], + 'ooupcast': DoNothing, + 'oodowncast': DoNothing, # TODO: is it really safe? 'same_as': DoNothing, # TODO: does same_as really do nothing else than renaming? Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Wed Apr 5 23:38:52 2006 @@ -22,16 +22,16 @@ else: print 'OK' -class A: +class Base: def __init__(self, x): self.x = x - def foo(self, a, b): - return self.x +class Derived(Base): + pass def bar(x, y): - a = A(x) - return a.foo(x, y) + a = Derived(42) + f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cli/test/test_oo.py Wed Apr 5 23:38:52 2006 @@ -1,5 +1,13 @@ from pypy.translator.cli.test.runtest import check +def test_oo(): + for name, func in globals().iteritems(): + if not name.startswith('oo_'): + continue + + yield check, func, [int, int], (42, 13) + + class MyClass: def __init__(self, x, y): self.x = x @@ -11,6 +19,14 @@ def compute_and_multiply(self, factor): return self.compute() * factor +class MyDerivedClass(MyClass): + def __init__(self, x, y): + self.x = x + self.y = y + + def compute(self): + return self.x - self.y + def oo_compute(x, y): obj = MyClass(x, y) return obj.compute() @@ -19,6 +35,14 @@ obj = MyClass(x, y) return obj.compute_and_multiply(2) -def test_oo(): - yield check, oo_compute, [int, int], (42, 13) - yield check, oo_compute_multiply, [int, int], (42, 13) +def oo_inheritance(x, y): + obj = MyDerivedClass(x, y) + return obj.compute_and_multiply(2) + +def helper(obj): + return obj.compute() + +def oo_liskov(x, y): + base = MyClass(x, y) + derived = MyDerivedClass(x, y) + return helper(base) + helper(derived) From tismer at codespeak.net Thu Apr 6 05:03:14 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 6 Apr 2006 05:03:14 +0200 (CEST) Subject: [pypy-svn] r25432 - pypy/dist/pypy/translator/backendopt Message-ID: <20060406030314.BE39E1023E@code0.codespeak.net> Author: tismer Date: Thu Apr 6 05:03:11 2006 New Revision: 25432 Modified: pypy/dist/pypy/translator/backendopt/canraise.py Log: disabled one's debug print Modified: pypy/dist/pypy/translator/backendopt/canraise.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/canraise.py (original) +++ pypy/dist/pypy/translator/backendopt/canraise.py Thu Apr 6 05:03:11 2006 @@ -19,7 +19,7 @@ def can_raise(self, op, seen=None): if op.opname == "direct_call": graph = get_graph(op.args[0], self.translator) - print "graph", graph + ##print "graph", graph if graph is None: return True return self.direct_call_can_raise(graph, seen) From tismer at codespeak.net Thu Apr 6 06:14:57 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 6 Apr 2006 06:14:57 +0200 (CEST) Subject: [pypy-svn] r25433 - in pypy/dist/pypy: objspace/flow translator/c/test Message-ID: <20060406041457.9F33010219@code0.codespeak.net> Author: tismer Date: Thu Apr 6 06:14:54 2006 New Revision: 25433 Modified: pypy/dist/pypy/objspace/flow/objspace.py pypy/dist/pypy/translator/c/test/test_wrapping.py Log: protect flow space from write access to globals. (actually, the intention is to detect global imports and resolve them as constant) Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Thu Apr 6 06:14:54 2006 @@ -603,6 +603,16 @@ FlowObjSpace.regular_getattr = FlowObjSpace.getattr FlowObjSpace.getattr = getattr + # protect us from globals access + def setitem(self, w_obj, w_key, w_val): + ec = self.getexecutioncontext() + if not (ec and w_obj is ec.w_globals): + return self.regular_setitem(w_obj, w_key, w_val) + raise SyntaxError, "attempt to write global attribute %r in %r" % (w_key, ec.graph.func) + + FlowObjSpace.regular_setitem = FlowObjSpace.setitem + FlowObjSpace.setitem = setitem + override() # ______________________________________________________________________ Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Thu Apr 6 06:14:54 2006 @@ -343,8 +343,15 @@ ### XXX write up the rules how to use this ### +def t2(): + global sys + import sys + def setup_new_module(mod, modname): # note the name clash with py.test on setup_module + #t1() + #t2() + return from types import module m = module(modname) allobjs = mod.__dict__.values() From ericvrp at codespeak.net Thu Apr 6 10:08:16 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 6 Apr 2006 10:08:16 +0200 (CEST) Subject: [pypy-svn] r25434 - in pypy/dist/pypy: jit/codegen/llvm translator/llvm/pyllvm Message-ID: <20060406080816.01ADC1021F@code0.codespeak.net> Author: ericvrp Date: Thu Apr 6 10:08:16 2006 New Revision: 25434 Modified: pypy/dist/pypy/jit/codegen/llvm/rgenop.py pypy/dist/pypy/translator/llvm/pyllvm/cc.py Log: * ctypes load (instead of LoadLibrary) was correct after all, I was using an older ctypes version on the other linux server. (need ctypes >=0.9.9.3) * small changes to codegen/llvm/rgenop.py to stay in sync with rpython/rgenop.py Modified: pypy/dist/pypy/jit/codegen/llvm/rgenop.py ============================================================================== --- pypy/dist/pypy/jit/codegen/llvm/rgenop.py (original) +++ pypy/dist/pypy/jit/codegen/llvm/rgenop.py Thu Apr 6 10:08:16 2006 @@ -10,12 +10,17 @@ from pypy.rpython.module.support import init_opaque_object from pypy.rpython.module.support import to_opaque_object, from_opaque_object from pypy.rpython.module.support import from_rstr +from pypy.rpython import extregistry from pypy.jit.codegen.llvm.jitcode import JITcode # for debugging, sanity checks in non-RPython code reveal = from_opaque_object +def isptrtype(gv_type): + c = from_opaque_object(gv_type) + return isinstance(c.value, lltype.Ptr) + def initblock(opaqueptr): init_opaque_object(opaqueptr, flowmodel.Block([])) @@ -26,6 +31,8 @@ def geninputarg(blockcontainer, gv_CONCRETE_TYPE): block = from_opaque_object(blockcontainer.obj) + assert not block.operations, "block already contains operations" + assert block.exits == [], "block already closed" CONCRETE_TYPE = from_opaque_object(gv_CONCRETE_TYPE).value v = flowmodel.Variable() v.concretetype = CONCRETE_TYPE @@ -50,6 +57,7 @@ if not isinstance(opname, str): opname = from_rstr(opname) block = from_opaque_object(blockcontainer.obj) + assert block.exits == [], "block already closed" RESULT_TYPE = from_opaque_object(gv_RESULT_TYPE).value opvars = _inputvars(vars) v = flowmodel.Variable() @@ -82,7 +90,11 @@ return llmemory.cast_ptr_to_adr(c.value) else: return lltype.cast_primitive(T, c.value) - + +def isconst(gv_value): + c = from_opaque_object(gv_value) + return isinstance(c, flowmodel.Constant) + # XXX # temporary interface; it's unclera if genop itself should change to ease dinstinguishing # Void special args from the rest. Or there should be variation for the ops involving them @@ -196,8 +208,6 @@ from pypy.rpython.llinterp import LLInterpreter if viewbefore: gengraph.show() - #llinterp = LLInterpreter(PseudoRTyper()) - #return llinterp.eval_graph(gengraph, args) jitcode = JITcode(PseudoRTyper()) return jitcode.eval_graph(gengraph, args) @@ -226,6 +236,25 @@ fields = tuple(zip(fieldnames, lltypes)) LINKPAIR = lltype.GcStruct('tuple2', *fields) +# support constants and types + +nullvar = lltype.nullptr(CONSTORVAR.TO) +gv_Void = constTYPE(lltype.Void) + +# VARLIST +def ll_fixed_items(l): + return l + +def ll_fixed_length(l): + return len(l) + +VARLIST = lltype.Ptr(lltype.GcArray(CONSTORVAR, + adtmeths = { + "ll_items": ll_fixed_items, + "ll_length": ll_fixed_length + })) + + # helpers def setannotation(func, annotation, specialize_as_constant=False): if specialize_as_constant: Modified: pypy/dist/pypy/translator/llvm/pyllvm/cc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/pyllvm/cc.py (original) +++ pypy/dist/pypy/translator/llvm/pyllvm/cc.py Thu Apr 6 10:08:16 2006 @@ -8,7 +8,7 @@ d = __file__[:__file__.find("pyllvm")] + "llvmcapi/" #XXX does this load once or every time? try: - self.library = cdll.LoadLibrary(d + libname + ".so") + self.library = cdll.load(d + libname + ".so") except: raise Exception("llvmcapi not found: run 'python setup.py build_ext -i' in " + d) self.restype = restype From auc at codespeak.net Thu Apr 6 12:41:19 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 6 Apr 2006 12:41:19 +0200 (CEST) Subject: [pypy-svn] r25444 - in pypy/dist/pypy/objspace: . test Message-ID: <20060406104119.EC4ED10240@code0.codespeak.net> Author: auc Date: Thu Apr 6 12:41:18 2006 New Revision: 25444 Modified: pypy/dist/pypy/objspace/logic.py pypy/dist/pypy/objspace/test/test_logicobjspace.py Log: logic objspace is translatable Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Thu Apr 6 12:41:18 2006 @@ -553,7 +553,7 @@ if not space.eq_w(w_x, w_y): w_d1 = w_x.getdict() w_d2 = w_y.getdict() - if (w_d1 == None) or (w_d2 == None): + if None in [w_d1, w_d2]: fail(space, w_x, w_y) else: return space.unify(w_d1, w_d2) @@ -755,13 +755,13 @@ #------ constraints ----------------- -from pypy.objspace.constraint import domain +## from pypy.objspace.constraint import domain -W_FiniteDomain = domain.W_FiniteDomain +## W_FiniteDomain = domain.W_FiniteDomain -def make_fd(space, w_values): - return domain.W_FiniteDomain(space, w_values) -app_make_fd = gateway.interp2app(make_fd) +## def make_fd(space, w_values): +## return domain.W_FiniteDomain(space, w_values) +## app_make_fd = gateway.interp2app(make_fd) #-- THE SPACE --------------------------------------- @@ -813,8 +813,8 @@ space.wrap(app_alias_of)) space.setitem(space.builtin.w_dict, space.wrap('is_aliased'), space.wrap(app_is_aliased)) - space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), - space.wrap(app_make_fd)) +## space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), +## space.wrap(app_make_fd)) space.setitem(space.builtin.w_dict, space.wrap('bind'), space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), Modified: pypy/dist/pypy/objspace/test/test_logicobjspace.py ============================================================================== --- pypy/dist/pypy/objspace/test/test_logicobjspace.py (original) +++ pypy/dist/pypy/objspace/test/test_logicobjspace.py Thu Apr 6 12:41:18 2006 @@ -271,22 +271,24 @@ assert T == 45 def notest_wait_two(self): + """this seems to trigger an + infinite loop in the + greenlet machinery + """ def sleep(X, Barrier): - print "sleep" wait(X) bind(Barrier, True) def wait_two(X, Y): - print "wait two" Z = newvar() uthread(sleep, X, Z) uthread(sleep, Y, Z) - print "on barrier" wait(Z) if is_free(Y): return 1 return 2 - + + print X, Y = newvar(), newvar() disp(X) disp(Y) From auc at codespeak.net Thu Apr 6 15:59:48 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 6 Apr 2006 15:59:48 +0200 (CEST) Subject: [pypy-svn] r25448 - in pypy/dist/pypy/objspace: . constraint logic Message-ID: <20060406135948.C516A1023F@code0.codespeak.net> Author: auc Date: Thu Apr 6 15:59:46 2006 New Revision: 25448 Removed: pypy/dist/pypy/objspace/logic/ Modified: pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/logic.py Log: domain : set -> dict, intersection as a mm Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Thu Apr 6 15:59:46 2006 @@ -1,13 +1,15 @@ from pypy.interpreter.error import OperationError -from pypy.interpreter import baseobjspace, typedef +from pypy.interpreter import baseobjspace, typedef, gateway from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.gateway import interp2app -from pypy.objspace.std.objspace import W_Object - from pypy.objspace.std.listobject import W_ListObject, W_TupleObject +from pypy.objspace.std.model import StdObjSpaceMultiMethod + + +all_mms = {} class ConsistencyFailure(Exception): """The repository is not in a consistent state""" @@ -52,14 +54,16 @@ This class uses a dictionnary to make sure that there are no duplicate values""" W_AbstractDomain.__init__(self, space) + self._values = {} self.set_values(w_values) def set_values(self, w_values): - self._values = set(w_values.wrappeditems) + for w_v in w_values.wrappeditems: + self._values[w_v] = True def w_remove_value(self, w_value): """Remove value of domain and check for consistency""" - self._values.remove(w_value) + self._values.pop(w_value) self._value_removed() def w_remove_values(self, w_values): @@ -67,7 +71,7 @@ if self._space.is_true(self._space.gt(self._space.len(w_values), self._space.newint(0))) : for val in w_values.wrappeditems : - self._values.remove(val) + self._values.pop(val) self._value_removed() __delitem__ = w_remove_value @@ -98,10 +102,19 @@ def __ne__(self, w_other): return not self == w_other - # FIXME: this does not work, but we don't need it yet - def w_intersection(self, w_other): - assert isinstance(w_other, W_FiniteDomain) - return self._space.newlist([x for x in (set(self.w_get_values()) & set(w_other.w_get_values()))]) +# function bolted into the space to serve as constructor +def make_fd(space, w_values): + return W_FiniteDomain(space, w_values) +app_make_fd = gateway.interp2app(make_fd) + + +def intersection__FiniteDomain_FiniteDomain(space, w_fd1, w_fd2): + return make_fd(w_fd1.w_get_values() + w_fd2.w_get_values()) + +intersection_mm = StdObjSpaceMultiMethod('intersection', 2) +intersection_mm.register(intersection__FiniteDomain_FiniteDomain, + W_FiniteDomain, W_FiniteDomain) +all_mms['intersection'] = intersection_mm W_FiniteDomain.typedef = typedef.TypeDef("W_FiniteDomain", W_AbstractDomain.typedef, @@ -109,5 +122,5 @@ remove_values = interp2app(W_FiniteDomain.w_remove_values), get_values = interp2app(W_FiniteDomain.w_get_values), copy = interp2app(W_FiniteDomain.w_copy), -# intersection = interp2app(W_FiniteDomain.w_intersection), size = interp2app(W_FiniteDomain.w_size)) + Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Thu Apr 6 15:59:46 2006 @@ -551,7 +551,7 @@ def unify__Root_Root(space, w_x, w_y): if not space.eq_w(w_x, w_y): - w_d1 = w_x.getdict() + w_d1 = w_x.getdict() #returns wrapped dict or unwrapped None ... w_d2 = w_y.getdict() if None in [w_d1, w_d2]: fail(space, w_x, w_y) @@ -753,15 +753,11 @@ return proxy -#------ constraints ----------------- +#------ domains ----------------- -## from pypy.objspace.constraint import domain +from pypy.objspace.constraint import domain +all_mms.update(domain.all_mms) -## W_FiniteDomain = domain.W_FiniteDomain - -## def make_fd(space, w_values): -## return domain.W_FiniteDomain(space, w_values) -## app_make_fd = gateway.interp2app(make_fd) #-- THE SPACE --------------------------------------- @@ -813,8 +809,8 @@ space.wrap(app_alias_of)) space.setitem(space.builtin.w_dict, space.wrap('is_aliased'), space.wrap(app_is_aliased)) -## space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), -## space.wrap(app_make_fd)) + space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), + space.wrap(domain.app_make_fd)) space.setitem(space.builtin.w_dict, space.wrap('bind'), space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), @@ -844,8 +840,3 @@ return space - - - - - From cfbolz at codespeak.net Thu Apr 6 16:51:48 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 16:51:48 +0200 (CEST) Subject: [pypy-svn] r25449 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060406145148.423B01023D@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 16:51:44 2006 New Revision: 25449 Modified: pypy/dist/pypy/translator/backendopt/propagate.py pypy/dist/pypy/translator/backendopt/test/test_propagate.py Log: clean up propagate to be a bit less insane: * use lltypesystem.lloperation * do not use visit also implement a transform that removes superfluous cast_pointers (which speeds up refcounting and the transformer gcs). Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Thu Apr 6 16:51:44 2006 @@ -1,7 +1,7 @@ from pypy.objspace.flow.model import Block, Variable, Constant, c_last_exception from pypy.objspace.flow.model import traverse, mkentrymap, checkgraph from pypy.objspace.flow.model import SpaceOperation -from pypy.rpython.lltypesystem.lltype import Bool +from pypy.rpython.lltypesystem import lltype, lloperation from pypy.rpython.llinterp import LLInterpreter, LLFrame from pypy.translator import simplify from pypy.translator.simplify import get_graph @@ -23,22 +23,20 @@ inlining of functions that return a bool.""" entrymap = mkentrymap(graph) candidates = {} - def visit(block): - if not isinstance(block, Block): - return + for block in graph.iterblocks(): if (isinstance(block.exitswitch, Variable) and - block.exitswitch.concretetype is Bool): + block.exitswitch.concretetype is lltype.Bool): for val, link in enumerate(block.exits): val = bool(val) - if block.exitswitch not in link.args: + try: + index = link.args.index(block.exitswitch) + except ValueError: continue if len(link.target.operations) > 0: continue - index = link.args.index(block.exitswitch) var = link.target.inputargs[index] if link.target.exitswitch is var: candidates[block] = val - traverse(visit, graph) for block, val in candidates.iteritems(): link = block.exits[val] args = [] @@ -52,29 +50,25 @@ link.args = args if candidates: print "rewiring links in graph", graph.name - checkgraph(graph) + simplify.join_blocks(graph) return True return False def coalesce_links(graph): - candidates = {} - def visit(block): - if not isinstance(block, Block): - return + done = False + for block in graph.iterblocks(): if len(block.exits) != 2: - return + continue if block.exitswitch == c_last_exception: - return - if (block.exits[0].args == block.exits[1].args and + continue + if not (block.exits[0].args == block.exits[1].args and block.exits[0].target is block.exits[1].target): - candidates[block] = True - traverse(visit, graph) - for block in candidates: + continue + done = True block.exitswitch = None block.exits = block.exits[:1] block.exits[0].exitcase = None - if candidates: - print "coalescing links in graph", graph.name + if done: return True else: return False @@ -84,7 +78,7 @@ if all blocks leading to it have the same constant in that position""" entrymap = mkentrymap(graph) candidates = [] - changed = [False] + changed = False for block, ingoing in entrymap.iteritems(): if block in [graph.returnblock, graph.exceptblock]: continue @@ -108,26 +102,13 @@ del block.inputargs[i] op = SpaceOperation("same_as", [const], var) block.operations.insert(0, op) - changed[0] = True - if changed[0]: + changed = True + if changed: remove_same_as(graph) checkgraph(graph) return True return False -_op = """getarrayitem setarrayitem malloc malloc_varsize flavored_malloc - flavored_free getfield setfield getsubstruct getarraysubstruct - getarraysize raw_malloc raw_free raw_memcopy raw_load - raw_store direct_call indirect_call cast_pointer - cast_ptr_to_int""".split() -from pypy.objspace.flow.operation import FunctionByName -_op += FunctionByName.keys() #operations with PyObjects are dangerous -cannot_constant_fold = {} -for opname in _op: - cannot_constant_fold[opname] = True -del _op -del FunctionByName - class TooManyOperations(Exception): pass @@ -149,14 +130,12 @@ """do constant folding if the arguments of an operations are constants""" lli = LLInterpreter(translator.rtyper) llframe = LLFrame(graph, None, lli) - changed = [False] - def visit(block): - if not isinstance(block, Block): - return + changed = False + for block in graph.iterblocks(): for i, op in enumerate(block.operations): if sum([isinstance(arg, Variable) for arg in op.args]): continue - if op.opname not in cannot_constant_fold: + if lloperation.LL_OPERATIONS[op.opname].canfold: print "folding operation", op, "in graph", graph.name try: llframe.eval_operation(op) @@ -168,8 +147,9 @@ res.concretetype = op.result.concretetype block.operations[i].opname = "same_as" block.operations[i].args = [res] - changed[0] = True - elif op.opname == "direct_call": + changed = True + # disabling the code for now, since it is not correct + elif 0: #op.opname == "direct_call": called_graph = get_graph(op.args[0], translator) if (called_graph is not None and simplify.has_no_side_effects(translator, called_graph) and @@ -189,10 +169,9 @@ res.concretetype = op.result.concretetype block.operations[i].opname = "same_as" block.operations[i].args = [res] - changed[0] = True + changed = True block.operations = [op for op in block.operations if op is not None] - traverse(visit, graph) - if changed[0]: + if changed: remove_same_as(graph) propagate_consts(graph) checkgraph(graph) @@ -208,7 +187,7 @@ return usedvars = {} for op in block.operations: - if op.opname in cannot_constant_fold: + if not lloperation.LL_OPERATIONS[op.opname].canfold: return for arg in op.args: if (isinstance(arg, Variable) and arg in block.inputargs): @@ -270,14 +249,67 @@ else: return False +def remove_duplicate_casts(graph, translator): + simplify.join_blocks(graph) + num_removed = 0 + # remove chains of casts + for block in graph.iterblocks(): + comes_from = {} + for op in block.operations: + if op.opname == "cast_pointer": + if op.args[0] in comes_from: + from_var = comes_from[op.args[0]] + comes_from[op.result] = from_var + if from_var.concretetype == op.result.concretetype: + op.opname = "same_as" + op.args = [from_var] + num_removed += 1 + else: + op.args = [from_var] + else: + comes_from[op.result] = op.args[0] + if num_removed: + remove_same_as(graph) + # remove duplicate casts + for block in graph.iterblocks(): + available = {} + for op in block.operations: + if op.opname == "cast_pointer": + key = (op.args[0], op.result.concretetype) + if key in available: + op.opname = "same_as" + op.args = [available[key]] + num_removed += 1 + else: + available[key] = op.result + if num_removed: + remove_same_as(graph) + # remove casts with unused results + for block in graph.iterblocks(): + used = {} + for link in block.exits: + for arg in link.args: + used[arg] = True + for i, op in list(enumerate(block.operations))[::-1]: + if op.opname == "cast_pointer" and op.result not in used: + del block.operations[i] + num_removed += 1 + else: + for arg in op.args: + used[arg] = True + print "removed %s cast_pointers in %s" % (num_removed, graph.name) + return num_removed + def propagate_all(translator): for graph in translator.graphs: def prop(): - changed = rewire_links(graph) - changed = changed or propagate_consts(graph) - changed = changed or coalesce_links(graph) - changed = changed or do_atmost(100, constant_folding, graph, - translator) - changed = changed or partial_folding(graph, translator) + changed = False + changed = rewire_links(graph) or changed + changed = propagate_consts(graph) or changed + changed = coalesce_links(graph) or changed + changed = do_atmost(100, constant_folding, graph, + translator) or changed + changed = partial_folding(graph, translator) or changed + changed = remove_duplicate_casts(graph, translator) or changed return changed do_atmost(10, prop) Modified: pypy/dist/pypy/translator/backendopt/test/test_propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_propagate.py Thu Apr 6 16:51:44 2006 @@ -1,8 +1,10 @@ +import py from pypy.translator.translator import TranslationContext, graphof from pypy.translator.backendopt.propagate import * from pypy.translator.backendopt.all import backend_optimizations from pypy.rpython.llinterp import LLInterpreter - +from pypy.rpython.memory.test.test_gctransform import getops +from pypy import conftest def get_graph(fn, signature, inline_threshold=True): t = TranslationContext() @@ -11,6 +13,8 @@ backend_optimizations(t, inline_threshold=inline_threshold, ssa_form=False, propagate=False) graph = graphof(t, fn) + if conftest.option.view: + t.view() return graph, t def check_graph(graph, args, expected_result, t): @@ -36,9 +40,38 @@ return ret graph, t = get_graph(g, [int]) propagate_consts(graph) + if conftest.option.view: + t.view() assert len(graph.startblock.exits[0].args) == 4 check_graph(graph, [100], g(100), t) - + +def test_rewire_links(): + def f(x): + return x != 1 and x != 5 and x != 42 + def g(x): + ret = x + if f(x): + ret += 1 + else: + ret += 2 + return ret + graph, t = get_graph(g, [int]) + rewire_links(graph) + if conftest.option.view: + t.view() + block = graph.startblock + for i in range(3): + op = block.exits[False].target.operations[0] + assert op.opname == "int_add" + assert op.args[1].value == 2 + block = block.exits[True].target + assert block.operations[0].opname == "int_add" + assert block.operations[0].args[1].value == 1 + check_graph(graph, [0], g(0), t) + check_graph(graph, [1], g(1), t) + check_graph(graph, [5], g(5), t) + check_graph(graph, [42], g(42), t) + def test_dont_fold_return(): def f(x): return @@ -54,10 +87,14 @@ return 1 + f(x) graph, t = get_graph(g, [int]) constant_folding(graph, t) + if conftest.option.view: + t.view() assert len(graph.startblock.operations) == 0 check_graph(graph, [1], g(1), t) def test_constant_fold_call(): + # fix the logic and try again :-) + py.test.skip("constant folding calls is disabled, for sanity reasons") def s(x): res = 0 i = 1 @@ -70,6 +107,8 @@ graph, t = get_graph(g, [int], inline_threshold=0) while constant_folding(graph, t): pass + if conftest.option.view: + t.view() assert len(graph.startblock.operations) == 1 check_graph(graph, [10], g(10), t) @@ -86,9 +125,26 @@ graph, t = get_graph(g, [int]) partial_folding(graph, t) constant_folding(graph, t) + if conftest.option.view: + t.view() assert len(graph.startblock.operations) == 1 check_graph(graph, [10], g(10), t) +def test_coalesce_links(): + def f(x): + y = 1 + if x: + y += 1 + else: + y += 2 + return 4 + graph, t = get_graph(f, [int]) + simplify.eliminate_empty_blocks(graph) + coalesce_links(graph) + if conftest.option.view: + t.view() + assert len(graph.startblock.exits) == 1 + def getitem(l, i): #LookupError, KeyError if not isinstance(i, int): raise TypeError @@ -108,6 +164,8 @@ return 4 graph, t = get_graph(fn, [int]) coalesce_links(graph) + if conftest.option.view: + t.view() check_graph(graph, [-1], fn(-1), t) def list_default_argument(i1, l1=[0]): @@ -122,3 +180,46 @@ backend_optimizations(t, propagate=True, ssa_form=False) for i in range(10): check_graph(graph, [i], call_list_default_argument(i), t) + if conftest.option.view: + t.view() + +def test_remove_duplicate_casts(): + class A(object): + def __init__(self, x, y): + self.x = x + self.y = y + def getsum(self): + return self.x + self.y + class B(A): + def __init__(self, x, y, z): + A.__init__(self, x, y) + self.z = z + def getsum(self): + return self.x + self.y + self.z + def f(x, switch): + a = A(x, x + 1) + b = B(x, x + 1, x + 2) + if switch: + c = A(x, x + 1) + else: + c = B(x, x + 1, x + 2) + return a.x + a.y + b.x + b.y + b.z + c.getsum() + assert f(10, True) == 75 + graph, t = get_graph(f, [int, bool], 1) + num_cast_pointer = len(getops(graph)['cast_pointer']) + changed = remove_duplicate_casts(graph, t) + assert changed + ops = getops(graph) + assert len(ops['cast_pointer']) < num_cast_pointer + print len(ops['cast_pointer']), num_cast_pointer + graph_getsum = graphof(t, B.getsum.im_func) + num_cast_pointer = len(getops(graph_getsum)['cast_pointer']) + changed = remove_duplicate_casts(graph_getsum, t) + assert changed + if conftest.option.view: + t.view() + check_graph(graph, [10, True], 75, t) + ops = getops(graph_getsum) + assert len(ops['cast_pointer']) < num_cast_pointer + print len(ops['cast_pointer']), num_cast_pointer + From cfbolz at codespeak.net Thu Apr 6 16:53:10 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 16:53:10 +0200 (CEST) Subject: [pypy-svn] r25450 - pypy/dist/pypy/translator Message-ID: <20060406145310.AC74910245@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 16:53:08 2006 New Revision: 25450 Modified: pypy/dist/pypy/translator/simplify.py Log: use the table of operations here too since it is much more complete Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Thu Apr 6 16:53:08 2006 @@ -369,18 +369,12 @@ # _____________________________________________________________________ # decide whether a function has side effects -lloperations_with_side_effects = {"setfield": True, - "setarrayitem": True, - } class HasSideEffects(Exception): pass -# XXX: this could even be improved: -# if setfield and setarrayitem only occur on things that are malloced -# in this function then the function still does not have side effects - def has_no_side_effects(translator, graph, seen=None): + from pypy.rpython.lltypesystem import lloperation #is the graph specialized? if no we can't say anything #don't cache the result though if translator.rtyper is None: @@ -397,8 +391,6 @@ if not isinstance(block, Block): return for op in block.operations: - if op.opname in lloperations_with_side_effects: - raise HasSideEffects if op.opname == "direct_call": g = get_graph(op.args[0], translator) if g is None: @@ -406,6 +398,10 @@ if not has_no_side_effects(translator, g, seen + [graph]): raise HasSideEffects elif op.opname == "indirect_call": + # XXX can be improved: look at all graphs, + # if they are there + raise HasSideEffects + elif lloperation.LL_OPERATIONS[op.opname].sideeffects: raise HasSideEffects traverse(visit, graph) except HasSideEffects: From cfbolz at codespeak.net Thu Apr 6 16:56:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 16:56:49 +0200 (CEST) Subject: [pypy-svn] r25451 - pypy/dist/pypy/translator/backendopt/test Message-ID: <20060406145649.948D01024A@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 16:56:48 2006 New Revision: 25451 Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py Log: (pedronis, cfbolz) fix test Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Thu Apr 6 16:56:48 2006 @@ -3,6 +3,7 @@ import os from pypy.objspace.flow.model import traverse, Block, Link, Variable, Constant from pypy.objspace.flow.model import last_exception, checkgraph +from pypy.translator.backendopt import canraise from pypy.translator.backendopt.inline import inline_function, CannotInline from pypy.translator.backendopt.inline import auto_inlining, Inliner from pypy.translator.backendopt.inline import collect_called_graphs @@ -51,7 +52,12 @@ sanity_check(t) # also check before inlining (so we don't blame it) if option.view: t.view() - inliner = Inliner(t, graphof(t, in_func), func, inline_guarded_calls) + if inline_guarded_calls: + raise_analyzer = canraise.RaiseAnalyzer(t) + else: + raise_analyzer = None + inliner = Inliner(t, graphof(t, in_func), func, inline_guarded_calls, + raise_analyzer=raise_analyzer) inliner.inline_all() # inline_function(t, func, graphof(t, in_func)) if option.view: From ale at codespeak.net Thu Apr 6 17:33:25 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 6 Apr 2006 17:33:25 +0200 (CEST) Subject: [pypy-svn] r25453 - pypy/dist/pypy/doc Message-ID: <20060406153325.6816110088@code0.codespeak.net> Author: ale Date: Thu Apr 6 17:33:23 2006 New Revision: 25453 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: Fix rest issues Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Thu Apr 6 17:33:23 2006 @@ -320,7 +320,7 @@ Let us look at a small relational program written respectively in Prolog, Oz and extended Python. -Prolog : +Prolog :: Soft(beige). Soft(coral). @@ -333,7 +333,7 @@ Suit(Shirt, Pants, Socks) :- Contrast(Shirt, Pants), Contrast(Pants, Socks), Shirt != Socks. -Oz : +Oz :: fun {Soft} choice beige [] coral end end fun {Hard} choice mauve [] ochre end end @@ -351,7 +351,7 @@ suit(Shirt Pants Socks) end -Python : +Python :: def soft(): choice: 'beige' or: 'coral' From mwh at codespeak.net Thu Apr 6 17:41:02 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 6 Apr 2006 17:41:02 +0200 (CEST) Subject: [pypy-svn] r25454 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060406154102.E5AA910203@code0.codespeak.net> Author: mwh Date: Thu Apr 6 17:41:01 2006 New Revision: 25454 Modified: pypy/dist/pypy/translator/c/stacklesstransform.py pypy/dist/pypy/translator/c/test/test_stacklesstransform.py Log: add a header to the saved frame states. Modified: pypy/dist/pypy/translator/c/stacklesstransform.py ============================================================================== --- pypy/dist/pypy/translator/c/stacklesstransform.py (original) +++ pypy/dist/pypy/translator/c/stacklesstransform.py Thu Apr 6 17:41:01 2006 @@ -25,6 +25,10 @@ else: raise Exception("don't know about %r" % (T,)) +state_header = lltype.Struct('state_header', + ('f_back', lltype.Ptr(lltype.ForwardReference())), + ('signed', lltype.Signed)) +state_header.f_back.TO.become(state_header) class StacklessTransfomer(object): def __init__(self, translator): @@ -47,7 +51,9 @@ for i, k in enumerate(key): for j in range(k): fields.append(('state_%s_%d'%(STORAGE_FIELDS[i], j), STORAGE_TYPES[i])) - T = lltype.Struct("state_%d_%d_%d_%d"%tuple(key), *fields) + T = lltype.Struct("state_%d_%d_%d_%d"%tuple(key), + ('header', state_header), + *fields) self.frametypes[key] = T return T Modified: pypy/dist/pypy/translator/c/test/test_stacklesstransform.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_stacklesstransform.py (original) +++ pypy/dist/pypy/translator/c/test/test_stacklesstransform.py Thu Apr 6 17:41:01 2006 @@ -14,7 +14,8 @@ ft4vars = st.frame_type_for_vars s1 = ft4vars([signed]) - assert len(s1._flds) == 1 + assert 'header' in s1._flds + assert len(s1._flds) == 2 s2_1 = ft4vars([signed, ptr]) s2_2 = ft4vars([ptr, signed]) From cfbolz at codespeak.net Thu Apr 6 18:09:19 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 18:09:19 +0200 (CEST) Subject: [pypy-svn] r25457 - in pypy/dist/pypy: rpython/memory translator/backendopt translator/backendopt/test Message-ID: <20060406160919.E318610242@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 18:09:17 2006 New Revision: 25457 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/backendopt/test/test_inline.py Log: (pedronis, cfbolz): fix the insertion of keepalive in the non-exceptional case. make the skipped test pass Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Thu Apr 6 18:09:17 2006 @@ -5,6 +5,7 @@ c_last_exception, FunctionGraph, Block, Link, checkgraph from pypy.translator.unsimplify import insert_empty_block from pypy.translator.translator import graphof +from pypy.translator.backendopt.support import var_needsgc, needs_conservative_livevar_calculation from pypy.annotation import model as annmodel from pypy.rpython import rmodel, rptr, annlowlevel from pypy.rpython.memory import gc, lladdress @@ -13,13 +14,6 @@ NEVER_RAISING_OPS = ['gc_protect', 'gc_unprotect'] -def var_needsgc(var): - if hasattr(var, 'concretetype'): - vartype = var.concretetype - return isinstance(vartype, lltype.Ptr) and vartype._needsgc() - else: - # assume PyObjPtr - return True def var_ispyobj(var): if hasattr(var, 'concretetype'): @@ -652,18 +646,6 @@ for a in gc_pointers_inside(v.items[i], adr + llmemory.itemoffsetof(t, i)): yield a -def needs_conservative_livevar_calculation(block): - from pypy.rpython.lltypesystem import rclass - vars = block.getvariables() - for var in vars: - TYPE = getattr(var, "concretetype", lltype.Ptr(lltype.PyObject)) - if isinstance(TYPE, lltype.Ptr) and not var_needsgc(var): - try: - lltype.castable(TYPE, rclass.CLASSTYPE) - except lltype.InvalidCast: - return True - else: - return False class FrameworkGCTransformer(BoehmGCTransformer): Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Thu Apr 6 18:09:17 2006 @@ -9,7 +9,7 @@ from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void from pypy.rpython import rmodel from pypy.tool.algo import sparsemat -from pypy.translator.backendopt.support import log +from pypy.translator.backendopt.support import log, needs_conservative_livevar_calculation, var_needsgc BASE_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -351,6 +351,24 @@ def do_inline(self, block, index_operation): afterblock = split_block(self.translator, self.graph, block, index_operation) + conservative_keepalives = needs_conservative_livevar_calculation(block) + if conservative_keepalives: + keep_alive_vars = [var for var in block.getvariables() + if var_needsgc(var)] + # XXX you could maybe remove more, if the variables are kept + # alive by something else. but this is sometimes hard to know + for i, var in enumerate(keep_alive_vars): + try: + index = block.exits[0].args.index(var) + newvar = afterblock.inputargs[index] + except ValueError: + block.exits[0].args.append(var) + newvar = copyvar(self.translator, var) + afterblock.inputargs.append(newvar) + keep_alive_vars[i] = newvar + else: + keep_alive_vars = [var for var in afterblock.operations[0].args + if var_needsgc(var)] # these variables have to be passed along all the links in the inlined # graph because the original function needs them in the blocks after # the inlined function @@ -377,7 +395,7 @@ linktoinlined.target = copiedstartblock linktoinlined.args = passon_args afterblock.inputargs = [self.op.result] + afterblock.inputargs - afterblock.operations = self.generate_keepalive(afterblock.inputargs) + afterblock.operations[1:] + afterblock.operations = afterblock.operations[1:] if self.graph_to_inline.returnblock in self.entrymap: self.rewire_returnblock(afterblock) if self.graph_to_inline.exceptblock in self.entrymap: @@ -386,6 +404,7 @@ assert afterblock.exits[0].exitcase is None afterblock.exits = [afterblock.exits[0]] afterblock.exitswitch = None + afterblock.operations.extend(self.generate_keepalive(keep_alive_vars)) self.search_for_calls(afterblock) self.search_for_calls(block) Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Thu Apr 6 18:09:17 2006 @@ -1,5 +1,5 @@ import py -from pypy.rpython.lltypesystem.lltype import functionptr, FuncType, typeOf +from pypy.rpython.lltypesystem import lltype from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log @@ -20,10 +20,31 @@ def annotate(translator, func, result, args): args = [arg.concretetype for arg in args] graph = translator.rtyper.annotate_helper(func, args) - fptr = functionptr(FuncType(args, result.concretetype), func.func_name, graph=graph) - c = inputconst(typeOf(fptr), fptr) + fptr = lltype.functionptr(lltype.FuncType(args, result.concretetype), func.func_name, graph=graph) + c = inputconst(lltype.typeOf(fptr), fptr) return c +def var_needsgc(var): + if hasattr(var, 'concretetype'): + vartype = var.concretetype + return isinstance(vartype, lltype.Ptr) and vartype._needsgc() + else: + # assume PyObjPtr + return True + +def needs_conservative_livevar_calculation(block): + from pypy.rpython.lltypesystem import rclass + vars = block.getvariables() + for var in vars: + TYPE = getattr(var, "concretetype", lltype.Ptr(lltype.PyObject)) + if isinstance(TYPE, lltype.Ptr) and not var_needsgc(var): + try: + lltype.castable(TYPE, rclass.CLASSTYPE) + except lltype.InvalidCast: + return True + else: + return False + def md5digest(translator): import md5 m = md5.new() Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Thu Apr 6 18:09:17 2006 @@ -435,7 +435,6 @@ py.test.raises(CannotInline, check_inline, x3, x4, []) def test_keepalive_hard_case(): - py.test.skip("fix this :(") from pypy.rpython.lltypesystem import lltype Y = lltype.Struct('y', ('n', lltype.Signed)) X = lltype.GcStruct('x', ('y', Y)) From mwh at codespeak.net Thu Apr 6 18:10:15 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 6 Apr 2006 18:10:15 +0200 (CEST) Subject: [pypy-svn] r25458 - pypy/dist/pypy/translator/c Message-ID: <20060406161015.E8DEF1024E@code0.codespeak.net> Author: mwh Date: Thu Apr 6 18:10:15 2006 New Revision: 25458 Modified: pypy/dist/pypy/translator/c/stacklesstransform.py Log: name that field more sensibly (thanks pedronis) Modified: pypy/dist/pypy/translator/c/stacklesstransform.py ============================================================================== --- pypy/dist/pypy/translator/c/stacklesstransform.py (original) +++ pypy/dist/pypy/translator/c/stacklesstransform.py Thu Apr 6 18:10:15 2006 @@ -27,7 +27,7 @@ state_header = lltype.Struct('state_header', ('f_back', lltype.Ptr(lltype.ForwardReference())), - ('signed', lltype.Signed)) + ('state', lltype.Signed)) state_header.f_back.TO.become(state_header) class StacklessTransfomer(object): From arigo at codespeak.net Thu Apr 6 19:05:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 6 Apr 2006 19:05:48 +0200 (CEST) Subject: [pypy-svn] r25459 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060406170548.ECAFF10253@code0.codespeak.net> Author: arigo Date: Thu Apr 6 19:05:48 2006 New Revision: 25459 Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: (arre, arigo) Compilation of calls to external ctypes functions. Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Thu Apr 6 19:05:48 2006 @@ -25,6 +25,7 @@ # the metatype assert hop.spaceop.opname == "simple_call" cfuncptr = hop.spaceop.args[0].value + fnname = cfuncptr.__name__ args_r = [] for ctype in cfuncptr.argtypes: @@ -36,9 +37,14 @@ vlist = hop.inputargs(*args_r) unwrapped_args_v = [r_arg.getvalue(hop.llops, v) for r_arg, v in zip(args_r, vlist)] + s_res = annmodel.SomeCTypesObject(cfuncptr.restype, + annmodel.SomeCTypesObject.OWNSMEMORY) + r_res = hop.rtyper.getrepr(s_res) - ll_func = cfuncptr.llinterp_friendly_version - v_result = hop.llops.gendirectcall(ll_func, *unwrapped_args_v) + ll_func = getattr(cfuncptr, 'llinterp_friendly_version', None) + v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, + resulttype = r_res.ll_type, + _callable = ll_func) return v_result extregistry.register_metatype(CFuncPtrType, Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Thu Apr 6 19:05:48 2006 @@ -1,6 +1,7 @@ from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rctypes.test.test_rctypes import mylib from pypy.rpython.test.test_llinterp import interpret +from pypy.translator.c.test.test_genc import compile from pypy import conftest from ctypes import c_long @@ -34,3 +35,9 @@ def test_specialize_labs(self): res = interpret(test_labs, [-11]) assert res == 11 + +class Test_compile: + def test_compile_labs(self): + fn = compile(test_labs, [int]) + res = fn(-11) + assert res == 11 From cfbolz at codespeak.net Thu Apr 6 19:14:27 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 19:14:27 +0200 (CEST) Subject: [pypy-svn] r25460 - in pypy/dist/pypy/translator: backendopt c Message-ID: <20060406171427.80FA910257@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 19:14:26 2006 New Revision: 25460 Modified: pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/c/exceptiontransform.py Log: (pedronis, cfbolz): factor out the logic that inserts keepalives. use the same logic in exceptionstransformer Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Thu Apr 6 19:14:26 2006 @@ -9,7 +9,7 @@ from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void from pypy.rpython import rmodel from pypy.tool.algo import sparsemat -from pypy.translator.backendopt.support import log, needs_conservative_livevar_calculation, var_needsgc +from pypy.translator.backendopt.support import log, split_block_with_keepalive, generate_keepalive BASE_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -217,17 +217,6 @@ newlink.llexitcase = link.llexitcase return newlink - def generate_keepalive(self, vars): - keepalive_ops = [] - for v in vars: - if isinstance(v, Constant): - continue - if v.concretetype._is_atomic(): - continue - v_keepalive = Variable() - v_keepalive.concretetype = Void - keepalive_ops.append(SpaceOperation('keepalive', [v], v_keepalive)) - return keepalive_ops def find_args_in_exceptional_case(self, link, block, etype, evalue, afterblock, passon_vars): linkargs = [] @@ -297,7 +286,7 @@ for exceptionlink in afterblock.exits[1:]: if exc_match(eclass, exceptionlink.llexitcase): passon_vars = self.passon_vars(link.prevblock) - copiedblock.operations += self.generate_keepalive(passon_vars) + copiedblock.operations += generate_keepalive(passon_vars) copiedlink.target = exceptionlink.target linkargs = self.find_args_in_exceptional_case( exceptionlink, link.prevblock, etype, evalue, afterblock, passon_vars) @@ -346,29 +335,12 @@ del blocks[-1].exits[0].llexitcase linkargs = copiedexceptblock.inputargs copiedexceptblock.closeblock(Link(linkargs, blocks[0])) - copiedexceptblock.operations += self.generate_keepalive(linkargs) + copiedexceptblock.operations += generate_keepalive(linkargs) def do_inline(self, block, index_operation): - afterblock = split_block(self.translator, self.graph, block, index_operation) - conservative_keepalives = needs_conservative_livevar_calculation(block) - if conservative_keepalives: - keep_alive_vars = [var for var in block.getvariables() - if var_needsgc(var)] - # XXX you could maybe remove more, if the variables are kept - # alive by something else. but this is sometimes hard to know - for i, var in enumerate(keep_alive_vars): - try: - index = block.exits[0].args.index(var) - newvar = afterblock.inputargs[index] - except ValueError: - block.exits[0].args.append(var) - newvar = copyvar(self.translator, var) - afterblock.inputargs.append(newvar) - keep_alive_vars[i] = newvar - else: - keep_alive_vars = [var for var in afterblock.operations[0].args - if var_needsgc(var)] + afterblock = split_block_with_keepalive( + self.translator, self.graph, block, index_operation) # these variables have to be passed along all the links in the inlined # graph because the original function needs them in the blocks after # the inlined function @@ -404,7 +376,6 @@ assert afterblock.exits[0].exitcase is None afterblock.exits = [afterblock.exits[0]] afterblock.exitswitch = None - afterblock.operations.extend(self.generate_keepalive(keep_alive_vars)) self.search_for_calls(afterblock) self.search_for_calls(block) Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Thu Apr 6 19:14:26 2006 @@ -2,6 +2,9 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log +from pypy.translator.unsimplify import split_block, copyvar +from pypy.objspace.flow.model import Constant, Variable, SpaceOperation +from pypy.rpython.lltypesystem import lltype log = py.log.Producer("backendopt") py.log.setconsumer("backendopt", ansi_log) @@ -45,6 +48,44 @@ else: return False +def generate_keepalive(vars): + keepalive_ops = [] + for v in vars: + if isinstance(v, Constant): + continue + if v.concretetype._is_atomic(): + continue + v_keepalive = Variable() + v_keepalive.concretetype = lltype.Void + keepalive_ops.append(SpaceOperation('keepalive', [v], v_keepalive)) + return keepalive_ops + +def split_block_with_keepalive(translator, graph, block, index_operation, + keep_alive_op_args=True): + afterblock = split_block(translator, graph, block, index_operation) + conservative_keepalives = needs_conservative_livevar_calculation(block) + if conservative_keepalives: + keep_alive_vars = [var for var in block.getvariables() + if var_needsgc(var)] + # XXX you could maybe remove more, if the variables are kept + # alive by something else. but this is sometimes hard to know + for i, var in enumerate(keep_alive_vars): + try: + index = block.exits[0].args.index(var) + newvar = afterblock.inputargs[index] + except ValueError: + block.exits[0].args.append(var) + newvar = copyvar(translator, var) + afterblock.inputargs.append(newvar) + keep_alive_vars[i] = newvar + elif keep_alive_op_args: + keep_alive_vars = [var for var in afterblock.operations[0].args + if var_needsgc(var)] + else: + keep_alive_vars = [] + afterblock.operations.extend(generate_keepalive(keep_alive_vars)) + return afterblock + def md5digest(translator): import md5 m = md5.new() Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Thu Apr 6 19:14:26 2006 @@ -1,11 +1,11 @@ from pypy.translator.simplify import join_blocks, cleanup_graph from pypy.translator.unsimplify import copyvar, split_block -from pypy.translator.backendopt import canraise, inline +from pypy.translator.backendopt import canraise, inline, support from pypy.objspace.flow.model import Block, Constant, Variable, Link, \ c_last_exception, SpaceOperation, checkgraph, FunctionGraph from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.memory.lladdress import NULL -from pypy.rpython.memory.gctransform import varoftype, var_needsgc +from pypy.rpython.memory.gctransform import varoftype from pypy.rpython import rclass from pypy.rpython.rarithmetic import r_uint, r_longlong, r_ulonglong from pypy.annotation import model as annmodel @@ -29,20 +29,6 @@ return Constant(lltype.nullptr(T.TO), T) assert 0, "not implemented yet" -def insert_keepalives_along(translator, link, vars): - vars = [v for v in vars if v not in link.args] - link.args.extend(vars) - newvars = [copyvar(translator, v) for v in vars] - block = link.target - block.inputargs.extend(newvars) - block.operations[0:0] = [SpaceOperation('keepalive', [v], - varoftype(lltype.Void)) - for v in newvars] - -def vars_to_keepalive(block): - # XXX make cleverer - return [v for v in block.getvariables() if var_needsgc(v)] - class ExceptionTransformer(object): def __init__(self, translator): self.translator = translator @@ -163,13 +149,11 @@ if not self.raise_analyzer.can_raise(op): continue - afterblock = split_block(self.translator, graph, block, i+1) + afterblock = support.split_block_with_keepalive( + self.translator, graph, block, i+1, False) if lastblock is block: lastblock = afterblock - insert_keepalives_along(self.translator, block.exits[0], - vars_to_keepalive(block)) - self.gen_exc_check(block, graph.returnblock) #non-exception case From cfbolz at codespeak.net Thu Apr 6 19:55:40 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 19:55:40 +0200 (CEST) Subject: [pypy-svn] r25461 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060406175540.B8A6F1008D@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 19:55:39 2006 New Revision: 25461 Modified: pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/backendopt/test/test_inline.py Log: (pedronis, cfbolz): fix placement of keepalives Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Thu Apr 6 19:55:39 2006 @@ -2,8 +2,8 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log -from pypy.translator.unsimplify import split_block, copyvar -from pypy.objspace.flow.model import Constant, Variable, SpaceOperation +from pypy.translator.unsimplify import split_block, copyvar, insert_empty_block +from pypy.objspace.flow.model import Constant, Variable, SpaceOperation, c_last_exception from pypy.rpython.lltypesystem import lltype log = py.log.Producer("backendopt") @@ -83,7 +83,15 @@ if var_needsgc(var)] else: keep_alive_vars = [] - afterblock.operations.extend(generate_keepalive(keep_alive_vars)) + if afterblock.exitswitch == c_last_exception: + for link in afterblock.exits: + betweenblock = insert_empty_block(translator, link) + fresh_vars = [copyvar(translator, var) for var in keep_alive_vars] + betweenblock.inputargs.extend(fresh_vars) + link.args.extend(keep_alive_vars) + betweenblock.operations = generate_keepalive(fresh_vars) + else: + afterblock.operations.extend(generate_keepalive(keep_alive_vars)) return afterblock def md5digest(translator): Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Thu Apr 6 19:55:39 2006 @@ -453,3 +453,20 @@ eval_func = check_inline(g, f, []) res = eval_func([]) assert res == 5 + +def test_correct_keepalive_placement(): + def h(x): + if not x: + raise ValueError + return 1 + def f(x): + s = "a %s" % (x, ) + try: + h(len(s)) + except ValueError: + pass + return -42 + eval_func, t = check_auto_inlining(f, [int]) + res = eval_func([42]) + assert res == -42 + From tismer at codespeak.net Thu Apr 6 22:35:09 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 6 Apr 2006 22:35:09 +0200 (CEST) Subject: [pypy-svn] r25464 - pypy/dist/pypy/objspace/flow Message-ID: <20060406203509.659BA1025E@code0.codespeak.net> Author: tismer Date: Thu Apr 6 22:35:07 2006 New Revision: 25464 Modified: pypy/dist/pypy/objspace/flow/objspace.py Log: temporary, preparing "constant import into globals" Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Thu Apr 6 22:35:07 2006 @@ -63,6 +63,9 @@ #self.make_sys() # objects which should keep their SomeObjectness self.not_really_const = NOT_REALLY_CONST + # variables which might in turn turn into constants. + # purpose: allow for importing into globals. + self.maybe_const = {} # variable -> constant def enter_cache_building_mode(self): # when populating the caches, the flow space switches to @@ -595,11 +598,15 @@ def override(): def getattr(self, w_obj, w_name): + # handling special things like sys + # (maybe this will vanish with a unique import logic) if w_obj in self.not_really_const: const_w = self.not_really_const[w_obj] if w_name not in const_w: return self.do_operation_with_implicit_exceptions('getattr', w_obj, w_name) + # tracking variables which might be constants return self.regular_getattr(w_obj, w_name) + FlowObjSpace.regular_getattr = FlowObjSpace.getattr FlowObjSpace.getattr = getattr From auc at codespeak.net Thu Apr 6 23:17:37 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 6 Apr 2006 23:17:37 +0200 (CEST) Subject: [pypy-svn] r25465 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060406211737.C620010260@code0.codespeak.net> Author: auc Date: Thu Apr 6 23:17:36 2006 New Revision: 25465 Added: pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py Modified: pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_fd.py Log: ruminations about constraint exprs simplification Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Thu Apr 6 23:17:36 2006 @@ -93,23 +93,34 @@ return W_FiniteDomain(self._space, self.w_get_values()) def __repr__(self): - return '' % str(self.get_values()) + return '' % str(self.w_get_values()) def __eq__(self, w_other): - if w_other is NoDom: return False - return self._values == w_other._values + if not isinstance(w_other, W_FiniteDomain): + return self._space.w_False + return self._space.newbool(self._space.eq_w(self._values, w_other._values)) def __ne__(self, w_other): return not self == w_other # function bolted into the space to serve as constructor def make_fd(space, w_values): - return W_FiniteDomain(space, w_values) + return space.wrap(W_FiniteDomain(space, w_values)) app_make_fd = gateway.interp2app(make_fd) +def intersection(space, w_fd1, w_fd2): + assert isinstance(w_fd1, W_FiniteDomain) + assert isinstance(w_fd2, W_FiniteDomain) + return space.intersection(w_fd1, w_fd2) +app_intersection = gateway.interp2app(intersection) + + def intersection__FiniteDomain_FiniteDomain(space, w_fd1, w_fd2): - return make_fd(w_fd1.w_get_values() + w_fd2.w_get_values()) + w_v1 = w_fd1._values + w_res = [w_v for w_v in w_fd2.w_get_values().wrappeditems + if w_v in w_v1] + return make_fd(space, space.newlist(w_res)) intersection_mm = StdObjSpaceMultiMethod('intersection', 2) intersection_mm.register(intersection__FiniteDomain_FiniteDomain, @@ -122,5 +133,6 @@ remove_values = interp2app(W_FiniteDomain.w_remove_values), get_values = interp2app(W_FiniteDomain.w_get_values), copy = interp2app(W_FiniteDomain.w_copy), - size = interp2app(W_FiniteDomain.w_size)) + size = interp2app(W_FiniteDomain.w_size), + __eq__ = interp2app(W_FiniteDomain.__eq__)) Added: pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py Thu Apr 6 23:17:36 2006 @@ -0,0 +1,47 @@ + + +#-- with a new expr construct (lambda on steroids) + +def queens_problem(n): + s = newspace() + queens = [] + for i in range(n): + queens.append(s.var(FiniteDomain([(i,j) + for i in range(n) + for j in range(n)]))) + for i in range(n): + for j in range(i,n): + s.add(expr queens[i], queens[j]: queens[i][0] != queen[j][0]) #-> s.add(make_expression([queen[i], queen[j]], + ## '%s[0] != %s[0]'%[q.name for q in [queen[i], queen[j]]) + s.add(expr queens[i], queens[j]: queens[i][1] != queen[j][1]) + s.add(AllDistinct(queens)) + return s + + +#-- redefining python operators + +class Variable: + def __getitem__(self, key): + return ViewVariable(self, + def __neq__(self, other): + return NotEqualConstraint(self, other) + def __add__(self, other): + pass + def __radd__(self, other): + pass + + +class QueensProblem(ComputationSpace): + def __init__(self, n): + vars = [Variable(FiniteDomain([(i,j) + for i in range(n) + for j in range(n)]) + ) + for k in n] + constraints = [] + for i in range(n): + for j in range(i,n): + constraint.append(vars[i][0] != vars[j][0]) + ComputationSpace.__init__(self, vars, constraints) + + Modified: pypy/dist/pypy/objspace/constraint/test/test_fd.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_fd.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_fd.py Thu Apr 6 23:17:36 2006 @@ -54,11 +54,11 @@ assert fd.size() == 1 assert fd.has_changed() - def notest_intersection(self): + def test_intersection(self): """not used for now""" fd1 = FiniteDomain([1, 2, 3]) - fd2 = FiniteDomain([2,3,4]) - assert fd1.intersection(fd2) == set([2,3]) - assert fd2.intersection(fd1) == set([3,2]) + fd2 = FiniteDomain([2, 3, 4]) + assert intersection(fd1, fd2) == FiniteDomain([2, 3]) + assert intersection(fd2, fd1) == FiniteDomain([3, 2]) From cfbolz at codespeak.net Thu Apr 6 23:19:12 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 6 Apr 2006 23:19:12 +0200 (CEST) Subject: [pypy-svn] r25466 - pypy/dist/pypy/translator/backendopt Message-ID: <20060406211912.5632D10263@code0.codespeak.net> Author: cfbolz Date: Thu Apr 6 23:19:11 2006 New Revision: 25466 Modified: pypy/dist/pypy/translator/backendopt/support.py Log: hum. getting keepalive right for hopefully the final time Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Thu Apr 6 23:19:11 2006 @@ -80,7 +80,7 @@ keep_alive_vars[i] = newvar elif keep_alive_op_args: keep_alive_vars = [var for var in afterblock.operations[0].args - if var_needsgc(var)] + if isinstance(var, Variable) and var_needsgc(var)] else: keep_alive_vars = [] if afterblock.exitswitch == c_last_exception: From mwh at codespeak.net Fri Apr 7 09:56:02 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 7 Apr 2006 09:56:02 +0200 (CEST) Subject: [pypy-svn] r25472 - pypy/dist/pypy/translator/c Message-ID: <20060407075602.C3D6D10249@code0.codespeak.net> Author: mwh Date: Fri Apr 7 09:56:01 2006 New Revision: 25472 Modified: pypy/dist/pypy/translator/c/exceptiontransform.py Log: fold a long line Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Fri Apr 7 09:56:01 2006 @@ -161,7 +161,9 @@ if need_exc_matching: assert lastblock.exitswitch == c_last_exception if not self.raise_analyzer.can_raise(lastblock.operations[-1]): - print "XXX: operation %s cannot raise, but has exception guarding in graph %s" % (lastblock.operations[-1], graph) + print ("XXX: operation %s cannot raise, but has exception" + " guarding in graph %s" % (lastblock.operations[-1], + graph)) lastblock.exitswitch = None lastblock.exits = [lastblock.exits[0]] lastblock.exits[0].exitcase = None From mwh at codespeak.net Fri Apr 7 10:12:01 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 7 Apr 2006 10:12:01 +0200 (CEST) Subject: [pypy-svn] r25473 - pypy/dist/pypy/translator/c Message-ID: <20060407081201.9B4AD10241@code0.codespeak.net> Author: mwh Date: Fri Apr 7 10:11:59 2006 New Revision: 25473 Modified: pypy/dist/pypy/translator/c/stacklesstransform.py Log: small forward steps Modified: pypy/dist/pypy/translator/c/stacklesstransform.py ============================================================================== --- pypy/dist/pypy/translator/c/stacklesstransform.py (original) +++ pypy/dist/pypy/translator/c/stacklesstransform.py Fri Apr 7 10:11:59 2006 @@ -1,4 +1,5 @@ from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython import rarithmetic STORAGE_TYPES = [llmemory.Address, lltype.Signed, @@ -25,15 +26,27 @@ else: raise Exception("don't know about %r" % (T,)) -state_header = lltype.Struct('state_header', +STATE_HEADER = lltype.Struct('state_header', ('f_back', lltype.Ptr(lltype.ForwardReference())), ('state', lltype.Signed)) -state_header.f_back.TO.become(state_header) +STATE_HEADER.f_back.TO.become(STATE_HEADER) class StacklessTransfomer(object): def __init__(self, translator): self.translator = translator + + null_frame_p = lltype.nullptr(STATE_HEADER) + class StacklessData: + def __init__(self): + top = null_frame_p + bottom = null_frame_p + restart_substate = 0 + retval_long = 0 + retval_longlong = rarithmetic.r_longlong(0) + retval_double = 0.0 + retval_void_p = llmemory.fakeaddress(None) + self.frametypes = {} def frame_type_for_vars(self, vars): @@ -52,9 +65,12 @@ for j in range(k): fields.append(('state_%s_%d'%(STORAGE_FIELDS[i], j), STORAGE_TYPES[i])) T = lltype.Struct("state_%d_%d_%d_%d"%tuple(key), - ('header', state_header), + ('header', STATE_HEADER), *fields) self.frametypes[key] = T return T - + + def transform_graph(self, graph): + for block in graph.iterblocks(): + pass From hpk at codespeak.net Fri Apr 7 10:29:07 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 7 Apr 2006 10:29:07 +0200 (CEST) Subject: [pypy-svn] r25474 - pypy/branch/stacklesscfg Message-ID: <20060407082907.A711C10252@code0.codespeak.net> Author: hpk Date: Fri Apr 7 10:29:00 2006 New Revision: 25474 Added: pypy/branch/stacklesscfg/ - copied from r25473, pypy/dist/ Log: (hopefully short term) branch for doing stackless transformations on flow graph level instead of at gen-c level From afayolle at codespeak.net Fri Apr 7 10:30:30 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 10:30:30 +0200 (CEST) Subject: [pypy-svn] r25475 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060407083030.D4A8010261@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 10:30:24 2006 New Revision: 25475 Added: pypy/dist/pypy/objspace/constraint/computationspace.py (contents, props changed) pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (contents, props changed) Log: started working on computation spaces Added: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Fri Apr 7 10:30:24 2006 @@ -0,0 +1,27 @@ +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter import baseobjspace, typedef, gateway +from pypy.interpreter.gateway import interp2app + +from pypy.objspace.constraint.domain import W_AbstractDomain + +all_mms = {} + +class W_ComputationSpace(Wrappable): + def __init__(self, obj_space): + self._space = obj_space + self.domains = self._space.newdict({}) + + def w_var(self, w_name, w_domain): + name = self._space.str_w(w_name) + assert isinstance(w_domain, W_AbstractDomain) + assert name not in self.domains + self.domains[name] = w_domain + + +W_ComputationSpace.typedef = typedef.TypeDef("W_ComputationSpace", + var = interp2app(W_ComputationSpace.w_var), + ) + +def newspace(space): + return W_ComputationSpace(space) +app_newspace = gateway.interp2app(newspace) Added: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Fri Apr 7 10:30:24 2006 @@ -0,0 +1,19 @@ +from pypy.conftest import gettestobjspace + + +class AppTest_ComputationSpace(object): + + def setup_class(cls): + cls.space = gettestobjspace('logic') + + def test_instantiate(self): + cspace = newspace() + assert str(type(cspace)) == "" + + def test_var(self): + cspace = newspace() + cspace.var("foo", FiniteDomain([1,2,3])) + + + + From auc at codespeak.net Fri Apr 7 10:47:26 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 7 Apr 2006 10:47:26 +0200 (CEST) Subject: [pypy-svn] r25476 - pypy/dist/pypy/doc Message-ID: <20060407084726.1D48A10261@code0.codespeak.net> Author: auc Date: Fri Apr 7 10:47:24 2006 New Revision: 25476 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: typo Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Fri Apr 7 10:47:24 2006 @@ -381,7 +381,7 @@ solving. Basically, a computation space is like an independant world in which a specific, unique combination of choices will be tried and eventually a -locally- unique solution be produced. The solver uses as -many computation spaces as neccessary to eventually enumerate all +many computation spaces as necessary to eventually enumerate all possible solutions. From auc at codespeak.net Fri Apr 7 10:48:31 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 7 Apr 2006 10:48:31 +0200 (CEST) Subject: [pypy-svn] r25477 - pypy/dist/pypy/objspace Message-ID: <20060407084831.4F54210265@code0.codespeak.net> Author: auc Date: Fri Apr 7 10:48:30 2006 New Revision: 25477 Modified: pypy/dist/pypy/objspace/logic.py Log: typeorder for intersection mm Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Fri Apr 7 10:48:30 2006 @@ -17,7 +17,7 @@ # this collects all multimethods to be made part of the Space all_mms = {} W_Root = baseobjspace.W_Root - +Wrappable = baseobjspace.Wrappable USE_COROUTINES = True HAVE_GREENLETS = True @@ -758,6 +758,7 @@ from pypy.objspace.constraint import domain all_mms.update(domain.all_mms) +W_FiniteDomain = domain.W_FiniteDomain #-- THE SPACE --------------------------------------- @@ -774,8 +775,8 @@ space = std.Space(*args, **kwds) # multimethods hack - #space.model.typeorder[W_Var] = [(baseobjspace.W_Root, None)] - space.model.typeorder[W_Var] = [(W_Var, None), (baseobjspace.W_Root, None)] + space.model.typeorder[W_Var] = [(W_Var, None), (W_Root, None)] # None means no conversion + space.model.typeorder[W_FiniteDomain] = [(W_FiniteDomain, None), (W_Root, None)] for name in all_mms.keys(): exprargs, expr, miniglobals, fallback = ( all_mms[name].install_not_sliced(space.model.typeorder, baked_perform_call=False)) @@ -809,12 +810,15 @@ space.wrap(app_alias_of)) space.setitem(space.builtin.w_dict, space.wrap('is_aliased'), space.wrap(app_is_aliased)) - space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), - space.wrap(domain.app_make_fd)) space.setitem(space.builtin.w_dict, space.wrap('bind'), space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), space.wrap(app_unify)) + #-- domain ----- + space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), + space.wrap(domain.app_make_fd)) + space.setitem(space.builtin.w_dict, space.wrap('intersection'), + space.wrap(domain.app_intersection)) if USE_COROUTINES: import os def exitfunc(): From cfbolz at codespeak.net Fri Apr 7 11:02:08 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 11:02:08 +0200 (CEST) Subject: [pypy-svn] r25478 - pypy/dist/pypy/translator/c/test Message-ID: <20060407090208.D38DF10269@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 11:02:06 2006 New Revision: 25478 Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: (pedronis, cfbolz): wuargh! this test was always broken and never tested something useful. It is fixed now and actually passes. Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Fri Apr 7 11:02:06 2006 @@ -66,18 +66,21 @@ s.b_dels += 1 class C(A): pass - def f(): + def f(x=int): A() B() C() A() B() C() - return s.a_dels * 10 + s.b_dels - res = f() - assert res == 42 + if x: + return s.a_dels * 10 + s.b_dels + else: + return -1 fn = self.getcompiled(f) - res = fn() + res = f(1) + assert res == 42 + res = fn(1) assert res == 42 From cfbolz at codespeak.net Fri Apr 7 11:04:56 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 11:04:56 +0200 (CEST) Subject: [pypy-svn] r25479 - in pypy/dist/pypy: rpython rpython/memory translator/c Message-ID: <20060407090456.0BAB81026B@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 11:04:52 2006 New Revision: 25479 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/c/exceptiontransform.py pypy/dist/pypy/translator/c/funcgen.py Log: (pedronis, cfbolz): don't protect the call to RPyExceptionOccured Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Apr 7 11:04:52 2006 @@ -421,6 +421,8 @@ frame = self.__class__(graph, args, self.llinterpreter, self) return frame.eval() + op_safe_call = op_direct_call + def op_indirect_call(self, f, *args): graphs = args[-1] args = args[:-1] Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 7 11:04:52 2006 @@ -208,6 +208,9 @@ result.extend(self.pop_alive(oldval)) return result + def replace_safe_call(self, op, livevars, block): + return [SpaceOperation("direct_call", op.args, op.result)] + def annotate_helper(self, ll_helper, ll_args, ll_result): assert not self.finished args_s = map(annmodel.lltype_to_annotation, ll_args) Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Fri Apr 7 11:04:52 2006 @@ -249,7 +249,7 @@ var_exc_occured = Variable() var_exc_occured.concretetype = lltype.Bool - block.operations.append(SpaceOperation("direct_call", [self.rpyexc_occured_ptr], var_exc_occured)) + block.operations.append(SpaceOperation("safe_call", [self.rpyexc_occured_ptr], var_exc_occured)) block.exitswitch = var_exc_occured #exception occurred case l = Link([error_value(returnblock.inputargs[0].concretetype)], returnblock) Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Apr 7 11:04:52 2006 @@ -354,6 +354,7 @@ # the following works since the extra arguments that indirect_call has # is of type Void, which is removed by OP_DIRECT_CALL OP_INDIRECT_CALL = OP_DIRECT_CALL + OP_SAFE_CALL = OP_DIRECT_CALL #XXX: add a trivial gc transformer def check_directcall_result(self, op): return None From afayolle at codespeak.net Fri Apr 7 11:04:59 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 11:04:59 +0200 (CEST) Subject: [pypy-svn] r25480 - pypy/dist/pypy/objspace/constraint Message-ID: <20060407090459.C8E8710272@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 11:04:58 2006 New Revision: 25480 Modified: pypy/dist/pypy/objspace/constraint/constraint.py Log: started working on the port Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Fri Apr 7 11:04:58 2006 @@ -44,8 +44,9 @@ def __eq__(self, other): #FIXME and parent if not isinstance(other, self.__class__): return False return self._variables == other._variables + -class BasicConstraint(object): +class W_BasicConstraint(object): """A BasicConstraint, which is never queued by the Repository A BasicConstraint affects only one variable, and will be entailed on the first call to narrow()""" @@ -356,32 +357,32 @@ return Expression(vars, formula) -class Equals(BasicConstraint): +class W_Equals(W_BasicConstraint): """A basic constraint variable == constant value""" def __init__(self, variable, reference): - BasicConstraint.__init__(self, variable, reference, operator.eq) + W_BasicConstraint.__init__(self, variable, reference, operator.eq) -class NotEquals(BasicConstraint): +class W_NotEquals(W_BasicConstraint): """A basic constraint variable != constant value""" def __init__(self, variable, reference): - BasicConstraint.__init__(self, variable, reference, operator.ne) + W_BasicConstraint.__init__(self, variable, reference, operator.ne) -class LesserThan(BasicConstraint): +class W_LesserThan(W_BasicConstraint): """A basic constraint variable < constant value""" def __init__(self, variable, reference): - BasicConstraint.__init__(self, variable, reference, operator.lt) + W_BasicConstraint.__init__(self, variable, reference, operator.lt) -class LesserOrEqual(BasicConstraint): +class W_LesserOrEqual(W_BasicConstraint): """A basic constraint variable <= constant value""" def __init__(self, variable, reference): - BasicConstraint.__init__(self, variable, reference, operator.le) + W_BasicConstraint.__init__(self, variable, reference, operator.le) -class GreaterThan(BasicConstraint): +class W_GreaterThan(W_BasicConstraint): """A basic constraint variable > constant value""" def __init__(self, variable, reference): - BasicConstraint.__init__(self, variable, reference, operator.gt) + W_BasicConstraint.__init__(self, variable, reference, operator.gt) -class GreaterOrEqual(BasicConstraint): +class W_GreaterOrEqual(W_BasicConstraint): """A basic constraint variable >= constant value""" def __init__(self, variable, reference): - BasicConstraint.__init__(self, variable, reference, operator.ge) + W_BasicConstraint.__init__(self, variable, reference, operator.ge) From afayolle at codespeak.net Fri Apr 7 11:05:14 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 11:05:14 +0200 (CEST) Subject: [pypy-svn] r25481 - pypy/dist/pypy/objspace/constraint Message-ID: <20060407090514.D3B3610270@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 11:05:12 2006 New Revision: 25481 Modified: pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py Log: updated syntax proposal Modified: pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py (original) +++ pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py Fri Apr 7 11:05:12 2006 @@ -6,13 +6,15 @@ s = newspace() queens = [] for i in range(n): - queens.append(s.var(FiniteDomain([(i,j) + queens.append(s.var('q%02d'%i, + FiniteDomain([(i,j) for i in range(n) for j in range(n)]))) for i in range(n): for j in range(i,n): - s.add(expr queens[i], queens[j]: queens[i][0] != queen[j][0]) #-> s.add(make_expression([queen[i], queen[j]], - ## '%s[0] != %s[0]'%[q.name for q in [queen[i], queen[j]]) + s.add(expr queens[i], queens[j]: queens[i][0] != queen[j][0]) + ## -> s.add(make_expression([queen[i], queen[j]], + ## '%s[0] != %s[0]'%[q.name for q in [queen[i], queen[j]]) s.add(expr queens[i], queens[j]: queens[i][1] != queen[j][1]) s.add(AllDistinct(queens)) return s From auc at codespeak.net Fri Apr 7 11:14:27 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 7 Apr 2006 11:14:27 +0200 (CEST) Subject: [pypy-svn] r25482 - in pypy/dist/pypy/objspace: . constraint constraint/test Message-ID: <20060407091427.C6CA810234@code0.codespeak.net> Author: auc Date: Fri Apr 7 11:14:26 2006 New Revision: 25482 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py pypy/dist/pypy/objspace/logic.py Log: instanciation, variable creation Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Fri Apr 7 11:14:26 2006 @@ -1,3 +1,4 @@ +from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter import baseobjspace, typedef, gateway from pypy.interpreter.gateway import interp2app @@ -14,13 +15,15 @@ def w_var(self, w_name, w_domain): name = self._space.str_w(w_name) assert isinstance(w_domain, W_AbstractDomain) - assert name not in self.domains - self.domains[name] = w_domain + if w_name in self.domains.content: + raise OperationError(self._space.w_RuntimeError, + self._space.wrap("Name already used")) + self.domains.content[w_name] = w_domain W_ComputationSpace.typedef = typedef.TypeDef("W_ComputationSpace", - var = interp2app(W_ComputationSpace.w_var), - ) + var = interp2app(W_ComputationSpace.w_var)) + def newspace(space): return W_ComputationSpace(space) Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Fri Apr 7 11:14:26 2006 @@ -13,6 +13,8 @@ def test_var(self): cspace = newspace() cspace.var("foo", FiniteDomain([1,2,3])) + #FIXME: raise the good exc. type + raises(Exception, cspace.var, "foo", FiniteDomain([1,2,3])) Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Fri Apr 7 11:14:26 2006 @@ -760,6 +760,13 @@ W_FiniteDomain = domain.W_FiniteDomain +#------ computation space ------- + +from pypy.objspace.constraint import computationspace +all_mms.update(computationspace.all_mms) + +W_ComputationSpace = computationspace.W_ComputationSpace + #-- THE SPACE --------------------------------------- #class UnificationError(w_RuntimeError): @@ -814,7 +821,10 @@ space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), space.wrap(app_unify)) - #-- domain ----- + #-- comp space --- + space.setitem(space.builtin.w_dict, space.wrap('newspace'), + space.wrap(computationspace.app_newspace)) + #-- domain ------- space.setitem(space.builtin.w_dict, space.wrap('FiniteDomain'), space.wrap(domain.app_make_fd)) space.setitem(space.builtin.w_dict, space.wrap('intersection'), From cfbolz at codespeak.net Fri Apr 7 11:38:07 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 11:38:07 +0200 (CEST) Subject: [pypy-svn] r25483 - in pypy/dist/pypy: rpython/memory translator/c translator/c/test Message-ID: <20060407093807.3C9AF10264@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 11:38:06 2006 New Revision: 25483 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/test/test_genc.py Log: (pedronis, cfbolz): * cleanup some places that worked for non-rtyped graphs * reintroduced the MinimalGCTransformer for sanity reasons Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 7 11:38:06 2006 @@ -32,6 +32,7 @@ def __init__(self, translator, inline=False): self.translator = translator self.seen_graphs = {} + self.minimal_transform = {} if translator: self.mixlevelannotator = MixLevelHelperAnnotator(translator.rtyper) else: @@ -40,17 +41,22 @@ self.graphs_to_inline = {} def get_lltype_of_exception_value(self): - if self.translator is not None and self.translator.rtyper is not None: - exceptiondata = self.translator.rtyper.getexceptiondata() - return exceptiondata.lltype_of_exception_value - else: - return lltype.Ptr(lltype.PyObject) + exceptiondata = self.translator.rtyper.getexceptiondata() + return exceptiondata.lltype_of_exception_value + + def need_minimal_transform(self, graph): + self.seen_graphs[graph] = True + self.minimal_transform[graph] = True def transform(self, graphs): for graph in graphs: self.transform_graph(graph) def transform_graph(self, graph): + if graph in self.minimal_transform: + MinimalGCTransformer(self.translator).transform_graph(graph) + del self.minimal_transform[graph] + return if graph in self.seen_graphs: return self.seen_graphs[graph] = True @@ -224,20 +230,18 @@ def inittime_helper(self, ll_helper, ll_args, ll_result): graph, ptr = self.annotate_helper(ll_helper, ll_args, ll_result) - self.seen_graphs[graph] = True + self.need_minimal_transform(graph) return Constant(ptr, lltype.Ptr(lltype.FuncType(ll_args, ll_result))) def finish(self): self.finished = True - if self.translator and self.translator.rtyper: - self.mixlevelannotator.finish() - + self.mixlevelannotator.finish() class MinimalGCTransformer(GCTransformer): - def push_alive_nopyobj(self, var): + def push_alive(self, var): return [] - def pop_alive_nopyobj(self, var): + def pop_alive(self, var): return [] @@ -307,14 +311,13 @@ dealloc(adr) def ll_no_pointer_dealloc(adr): llop.gc_free(lltype.Void, adr) - if self.translator is not None and self.translator.rtyper is not None: - self.increfptr = self.inittime_helper( - ll_incref, [llmemory.Address], lltype.Void) - self.decref_ptr = self.inittime_helper( - ll_decref, [llmemory.Address, lltype.Ptr(ADDRESS_VOID_FUNC)], - lltype.Void) - self.no_pointer_dealloc_ptr = self.inittime_helper( - ll_no_pointer_dealloc, [llmemory.Address], lltype.Void) + self.increfptr = self.inittime_helper( + ll_incref, [llmemory.Address], lltype.Void) + self.decref_ptr = self.inittime_helper( + ll_decref, [llmemory.Address, lltype.Ptr(ADDRESS_VOID_FUNC)], + lltype.Void) + self.no_pointer_dealloc_ptr = self.inittime_helper( + ll_no_pointer_dealloc, [llmemory.Address], lltype.Void) # cache graphs: self.decref_funcptrs = {} self.static_deallocator_funcptrs = {} @@ -465,8 +468,7 @@ this = d['ll_deallocator'] g, fptr = self.annotate_helper(this, [llmemory.Address], lltype.Void) # the produced deallocator graph does not need to be transformed - self.seen_graphs[g] = True - + self.need_minimal_transform(g) self.static_deallocator_funcptrs[TYPE] = fptr for p in find_gc_ptrs_in_type(TYPE): self.static_deallocation_funcptr_for_type(p.TO) @@ -498,7 +500,7 @@ gcheader.signed[0] = 0 llop.gc_call_rtti_destructor(lltype.Void, rtti, addr) g, fptr = self.annotate_helper(ll_dealloc, [llmemory.Address], lltype.Void) - self.seen_graphs[g] = True + self.need_minimal_transform(g) self.dynamic_deallocator_funcptrs[TYPE] = fptr self.queryptr2dynamic_deallocator_funcptr[queryptr._obj] = fptr @@ -573,7 +575,7 @@ return None def finish(self): - self.mixlevelannotator.finish() + super(BoehmGCTransformer, self).finish() def finalizer_funcptr_for_type(self, TYPE): if TYPE in self.finalizer_funcptrs: @@ -626,7 +628,7 @@ g = fptr = None if g: - self.seen_graphs[g] = True + self.need_minimal_transform(g) self.finalizer_funcptrs[TYPE] = fptr return fptr @@ -807,7 +809,7 @@ self.graphs_to_inline[malloc_fixedsize_graph] = True def graph2funcptr(self, graph): - self.seen_graphs[graph] = True + self.need_minimal_transform(graph) return const_funcptr_fromgraph(graph) def get_type_id(self, TYPE): Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Apr 7 11:38:06 2006 @@ -354,7 +354,6 @@ # the following works since the extra arguments that indirect_call has # is of type Void, which is removed by OP_DIRECT_CALL OP_INDIRECT_CALL = OP_DIRECT_CALL - OP_SAFE_CALL = OP_DIRECT_CALL #XXX: add a trivial gc transformer def check_directcall_result(self, op): return None Modified: pypy/dist/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_genc.py (original) +++ pypy/dist/pypy/translator/c/test/test_genc.py Fri Apr 7 11:38:06 2006 @@ -51,23 +51,6 @@ return res return checking_fn - -def test_untyped_func(): - def f(x): - return x+1 - graph = TranslationContext().buildflowgraph(f) - - F = FuncType([Ptr(PyObject)], Ptr(PyObject)) - S = GcStruct('testing', ('fptr', Ptr(F))) - f = functionptr(F, "f", graph=graph) - s = malloc(S) - s.fptr = f - db = LowLevelDatabase() - db.get(s) - db.complete() - compile_db(db) - - def test_func_as_pyobject(): def f(x): return x*2 From afayolle at codespeak.net Fri Apr 7 11:44:17 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 11:44:17 +0200 (CEST) Subject: [pypy-svn] r25484 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060407094417.C46D81026A@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 11:44:16 2006 New Revision: 25484 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: added W_Variable Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Fri Apr 7 11:44:16 2006 @@ -7,6 +7,13 @@ all_mms = {} +class W_Variable(Wrappable): + def __init__(self, obj_space, name): + self._space = obj_space + self.name = self._space.str_w(name) + +W_Variable.typedef = typedef.TypeDef("W_Variable") + class W_ComputationSpace(Wrappable): def __init__(self, obj_space): self._space = obj_space @@ -19,6 +26,7 @@ raise OperationError(self._space.w_RuntimeError, self._space.wrap("Name already used")) self.domains.content[w_name] = w_domain + return W_Variable(self._space, w_name) W_ComputationSpace.typedef = typedef.TypeDef("W_ComputationSpace", Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Fri Apr 7 11:44:16 2006 @@ -12,10 +12,10 @@ def test_var(self): cspace = newspace() - cspace.var("foo", FiniteDomain([1,2,3])) + v = cspace.var("foo", FiniteDomain([1,2,3])) + assert str(v).startswith(' Author: auc Date: Fri Apr 7 11:54:46 2006 New Revision: 25485 Modified: pypy/dist/pypy/objspace/constraint/domain.py Log: Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Fri Apr 7 11:54:46 2006 @@ -54,36 +54,36 @@ This class uses a dictionnary to make sure that there are no duplicate values""" W_AbstractDomain.__init__(self, space) - self._values = {} + self._values = space.newdict([]) self.set_values(w_values) def set_values(self, w_values): for w_v in w_values.wrappeditems: - self._values[w_v] = True + self._values.content[w_v] = self._space.w_True def w_remove_value(self, w_value): """Remove value of domain and check for consistency""" - self._values.pop(w_value) + del self._values.content[w_value] self._value_removed() def w_remove_values(self, w_values): """Remove values of domain and check for consistency""" if self._space.is_true(self._space.gt(self._space.len(w_values), self._space.newint(0))) : - for val in w_values.wrappeditems : - self._values.pop(val) + for w_val in w_values.wrappeditems : + del self._values.content [w_val] self._value_removed() __delitem__ = w_remove_value def w_size(self): """computes the size of a finite domain""" - return self._space.newint(len(self._values)) + return self._space.newint(len(self._values.content)) __len__ = w_size def w_get_values(self): """return all the values in the domain in an indexable sequence""" - return self._space.newlist([x for x in self._values]) + return self._space.newlist([x for x in self._values.content]) def __iter__(self): return iter(self._values) @@ -97,8 +97,9 @@ def __eq__(self, w_other): if not isinstance(w_other, W_FiniteDomain): - return self._space.w_False - return self._space.newbool(self._space.eq_w(self._values, w_other._values)) + return self._space.newbool(False) + return self._space.eq(self._values, w_other._values) + def __ne__(self, w_other): return not self == w_other @@ -117,7 +118,7 @@ def intersection__FiniteDomain_FiniteDomain(space, w_fd1, w_fd2): - w_v1 = w_fd1._values + w_v1 = w_fd1._values.content w_res = [w_v for w_v in w_fd2.w_get_values().wrappeditems if w_v in w_v1] return make_fd(space, space.newlist(w_res)) From cfbolz at codespeak.net Fri Apr 7 12:04:53 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 12:04:53 +0200 (CEST) Subject: [pypy-svn] r25486 - pypy/dist/pypy/rpython/memory Message-ID: <20060407100453.B169A10262@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 12:04:52 2006 New Revision: 25486 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: hum. revert some of my last checkin, to make test_database.py happy. there needs to be some cleanup, but not now :-( Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 7 12:04:52 2006 @@ -41,8 +41,11 @@ self.graphs_to_inline = {} def get_lltype_of_exception_value(self): - exceptiondata = self.translator.rtyper.getexceptiondata() - return exceptiondata.lltype_of_exception_value + if self.translator is not None: + exceptiondata = self.translator.rtyper.getexceptiondata() + return exceptiondata.lltype_of_exception_value + else: + return lltype.Ptr(lltype.PyObject) def need_minimal_transform(self, graph): self.seen_graphs[graph] = True @@ -235,7 +238,8 @@ def finish(self): self.finished = True - self.mixlevelannotator.finish() + if self.translator is not None: + self.mixlevelannotator.finish() class MinimalGCTransformer(GCTransformer): def push_alive(self, var): @@ -311,13 +315,14 @@ dealloc(adr) def ll_no_pointer_dealloc(adr): llop.gc_free(lltype.Void, adr) - self.increfptr = self.inittime_helper( - ll_incref, [llmemory.Address], lltype.Void) - self.decref_ptr = self.inittime_helper( - ll_decref, [llmemory.Address, lltype.Ptr(ADDRESS_VOID_FUNC)], - lltype.Void) - self.no_pointer_dealloc_ptr = self.inittime_helper( - ll_no_pointer_dealloc, [llmemory.Address], lltype.Void) + if self.translator: + self.increfptr = self.inittime_helper( + ll_incref, [llmemory.Address], lltype.Void) + self.decref_ptr = self.inittime_helper( + ll_decref, [llmemory.Address, lltype.Ptr(ADDRESS_VOID_FUNC)], + lltype.Void) + self.no_pointer_dealloc_ptr = self.inittime_helper( + ll_no_pointer_dealloc, [llmemory.Address], lltype.Void) # cache graphs: self.decref_funcptrs = {} self.static_deallocator_funcptrs = {} From afayolle at codespeak.net Fri Apr 7 12:13:05 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 12:13:05 +0200 (CEST) Subject: [pypy-svn] r25487 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060407101305.EF9291026F@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 12:13:00 2006 New Revision: 25487 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: added cspace.dom(w_var) Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Fri Apr 7 12:13:00 2006 @@ -3,14 +3,16 @@ from pypy.interpreter import baseobjspace, typedef, gateway from pypy.interpreter.gateway import interp2app +from pypy.objspace.std.stringobject import W_StringObject + from pypy.objspace.constraint.domain import W_AbstractDomain all_mms = {} class W_Variable(Wrappable): - def __init__(self, obj_space, name): + def __init__(self, obj_space, w_name): self._space = obj_space - self.name = self._space.str_w(name) + self.name = w_name W_Variable.typedef = typedef.TypeDef("W_Variable") @@ -20,7 +22,7 @@ self.domains = self._space.newdict({}) def w_var(self, w_name, w_domain): - name = self._space.str_w(w_name) + assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) if w_name in self.domains.content: raise OperationError(self._space.w_RuntimeError, @@ -28,9 +30,14 @@ self.domains.content[w_name] = w_domain return W_Variable(self._space, w_name) - -W_ComputationSpace.typedef = typedef.TypeDef("W_ComputationSpace", - var = interp2app(W_ComputationSpace.w_var)) + def w_dom(self, w_variable): + assert isinstance(w_variable, W_Variable) + return self.domains.content[w_variable.name] + +W_ComputationSpace.typedef = typedef.TypeDef( + "W_ComputationSpace", + var = interp2app(W_ComputationSpace.w_var), + dom = interp2app(W_ComputationSpace.w_dom)) def newspace(space): Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Fri Apr 7 12:13:00 2006 @@ -18,4 +18,11 @@ raises(Exception, cspace.var, "foo", FiniteDomain([1,2,3])) + def test_dom(self): + cspace = newspace() + domain = FiniteDomain([1,2,3]) + v = cspace.var("foo", domain) + assert cspace.dom(v) is domain + + From cfbolz at codespeak.net Fri Apr 7 12:20:03 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 12:20:03 +0200 (CEST) Subject: [pypy-svn] r25488 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060407102003.9529810271@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 12:20:02 2006 New Revision: 25488 Modified: pypy/dist/pypy/translator/backendopt/propagate.py pypy/dist/pypy/translator/backendopt/removenoops.py pypy/dist/pypy/translator/backendopt/test/test_propagate.py pypy/dist/pypy/translator/backendopt/test/test_removenoops.py Log: add an transformation that removes superfluous keepalives. move some stuff from propagate to removenoops Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Fri Apr 7 12:20:02 2006 @@ -249,56 +249,6 @@ else: return False -def remove_duplicate_casts(graph, translator): - simplify.join_blocks(graph) - num_removed = 0 - # remove chains of casts - for block in graph.iterblocks(): - comes_from = {} - for op in block.operations: - if op.opname == "cast_pointer": - if op.args[0] in comes_from: - from_var = comes_from[op.args[0]] - comes_from[op.result] = from_var - if from_var.concretetype == op.result.concretetype: - op.opname = "same_as" - op.args = [from_var] - num_removed += 1 - else: - op.args = [from_var] - else: - comes_from[op.result] = op.args[0] - if num_removed: - remove_same_as(graph) - # remove duplicate casts - for block in graph.iterblocks(): - available = {} - for op in block.operations: - if op.opname == "cast_pointer": - key = (op.args[0], op.result.concretetype) - if key in available: - op.opname = "same_as" - op.args = [available[key]] - num_removed += 1 - else: - available[key] = op.result - if num_removed: - remove_same_as(graph) - # remove casts with unused results - for block in graph.iterblocks(): - used = {} - for link in block.exits: - for arg in link.args: - used[arg] = True - for i, op in list(enumerate(block.operations))[::-1]: - if op.opname == "cast_pointer" and op.result not in used: - del block.operations[i] - num_removed += 1 - else: - for arg in op.args: - used[arg] = True - print "removed %s cast_pointers in %s" % (num_removed, graph.name) - return num_removed def propagate_all(translator): for graph in translator.graphs: @@ -310,6 +260,5 @@ changed = do_atmost(100, constant_folding, graph, translator) or changed changed = partial_folding(graph, translator) or changed - changed = remove_duplicate_casts(graph, translator) or changed return changed do_atmost(10, prop) Modified: pypy/dist/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/removenoops.py Fri Apr 7 12:20:02 2006 @@ -1,6 +1,8 @@ from pypy.objspace.flow.model import Block, Variable, Constant from pypy.objspace.flow.model import traverse from pypy.rpython.lltypesystem.lltype import Void +from pypy.translator import simplify +from pypy import conftest def remove_unaryops(graph, opnames): """Removes unary low-level ops with a name appearing in the opnames list. @@ -63,6 +65,67 @@ args = [arg for arg in op.args if arg.concretetype is not Void] op.args = args + +def remove_duplicate_casts(graph, translator): + simplify.join_blocks(graph) + num_removed = 0 + # remove chains of casts + for block in graph.iterblocks(): + comes_from = {} + for op in block.operations: + if op.opname == "cast_pointer": + if op.args[0] in comes_from: + from_var = comes_from[op.args[0]] + comes_from[op.result] = from_var + if from_var.concretetype == op.result.concretetype: + op.opname = "same_as" + op.args = [from_var] + num_removed += 1 + else: + op.args = [from_var] + else: + comes_from[op.result] = op.args[0] + if num_removed: + remove_same_as(graph) + # remove duplicate casts + for block in graph.iterblocks(): + available = {} + for op in block.operations: + if op.opname == "cast_pointer": + key = (op.args[0], op.result.concretetype) + if key in available: + op.opname = "same_as" + op.args = [available[key]] + num_removed += 1 + else: + available[key] = op.result + if num_removed: + remove_same_as(graph) + # remove casts with unused results + for block in graph.iterblocks(): + used = {} + for link in block.exits: + for arg in link.args: + used[arg] = True + for i, op in list(enumerate(block.operations))[::-1]: + if op.opname == "cast_pointer" and op.result not in used: + del block.operations[i] + num_removed += 1 + else: + for arg in op.args: + used[arg] = True + print "removed %s cast_pointers in %s" % (num_removed, graph.name) + return num_removed + +def remove_superfluous_keep_alive(graph): + for block in graph.iterblocks(): + used = {} + for i, op in list(enumerate(block.operations))[::-1]: + if op.opname == "keepalive": + if op.args[0] in used: + del block.operations[i] + else: + used[op.args[0]] = True ##def rename_extfunc_calls(translator): ## from pypy.rpython.extfunctable import table as extfunctable Modified: pypy/dist/pypy/translator/backendopt/test/test_propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_propagate.py Fri Apr 7 12:20:02 2006 @@ -183,43 +183,4 @@ if conftest.option.view: t.view() -def test_remove_duplicate_casts(): - class A(object): - def __init__(self, x, y): - self.x = x - self.y = y - def getsum(self): - return self.x + self.y - class B(A): - def __init__(self, x, y, z): - A.__init__(self, x, y) - self.z = z - def getsum(self): - return self.x + self.y + self.z - def f(x, switch): - a = A(x, x + 1) - b = B(x, x + 1, x + 2) - if switch: - c = A(x, x + 1) - else: - c = B(x, x + 1, x + 2) - return a.x + a.y + b.x + b.y + b.z + c.getsum() - assert f(10, True) == 75 - graph, t = get_graph(f, [int, bool], 1) - num_cast_pointer = len(getops(graph)['cast_pointer']) - changed = remove_duplicate_casts(graph, t) - assert changed - ops = getops(graph) - assert len(ops['cast_pointer']) < num_cast_pointer - print len(ops['cast_pointer']), num_cast_pointer - graph_getsum = graphof(t, B.getsum.im_func) - num_cast_pointer = len(getops(graph_getsum)['cast_pointer']) - changed = remove_duplicate_casts(graph_getsum, t) - assert changed - if conftest.option.view: - t.view() - check_graph(graph, [10, True], 75, t) - ops = getops(graph_getsum) - assert len(ops['cast_pointer']) < num_cast_pointer - print len(ops['cast_pointer']), num_cast_pointer - + Modified: pypy/dist/pypy/translator/backendopt/test/test_removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_removenoops.py Fri Apr 7 12:20:02 2006 @@ -1,12 +1,14 @@ from pypy.translator.backendopt.removenoops import remove_void, remove_same_as, \ - remove_unaryops + remove_unaryops, remove_duplicate_casts, remove_superfluous_keep_alive from pypy.translator.backendopt.inline import inline_function +from pypy.translator.backendopt.test.test_propagate import getops, get_graph, check_graph from pypy.translator.translator import TranslationContext, graphof from pypy.translator.test.snippet import simple_method from pypy.objspace.flow.model import checkgraph, flatten, Block -from pypy.rpython.lltypesystem.lltype import Void, Signed +from pypy.rpython.lltypesystem import lltype from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.llinterp import LLInterpreter +from pypy import conftest import py log = py.log.Producer('test_backendoptimization') @@ -26,7 +28,7 @@ for graph in t.graphs: assert checkgraph(graph) is None for arg in graph.startblock.inputargs: - assert arg.concretetype is not Void + assert arg.concretetype is not lltype.Void interp = LLInterpreter(t.rtyper) assert interp.eval_graph(graphof(t, f), [0]) == 1 @@ -40,7 +42,7 @@ #for fieldname in self.struct._names: #XXX helper (in lltype?) should remove these voids # type_ = getattr(struct, fieldname) # log('fieldname=%(fieldname)s , type_=%(type_)s' % locals()) - # assert _type is not Void + # assert _type is not lltype.Void #interp = LLInterpreter(t.flowgraphs, t.rtyper) #assert interp.eval_function(f, [0]) == 1 @@ -75,9 +77,9 @@ # oodowncast in dynamically typed languages, but it's easier to test # it with operations on ints here. def f(x): - i = llop.int_invert(Signed, x) - i = llop.int_add(Signed, x, 1) - return llop.int_neg(Signed, i) + i = llop.int_invert(lltype.Signed, x) + i = llop.int_add(lltype.Signed, x, 1) + return llop.int_neg(lltype.Signed, i) t = TranslationContext() t.buildannotator().build_types(f, [int]) t.buildrtyper().specialize() @@ -89,3 +91,57 @@ result = interp.eval_graph(f_graph, [-2]) assert result == -1 +def test_remove_keepalive(): + S = lltype.GcStruct("s", ("f", lltype.Signed)) + def f(): + s1 = lltype.malloc(S) + llop.keepalive(lltype.Void, s1) + s2 = lltype.malloc(S) + llop.keepalive(lltype.Void, s1) + llop.keepalive(lltype.Void, s2) + return id(s1) + id(s2) + graph, t = get_graph(f, []) + remove_superfluous_keep_alive(graph) + ops = getops(graph) + assert len(ops['keepalive']) == 2 + +def test_remove_duplicate_casts(): + class A(object): + def __init__(self, x, y): + self.x = x + self.y = y + def getsum(self): + return self.x + self.y + class B(A): + def __init__(self, x, y, z): + A.__init__(self, x, y) + self.z = z + def getsum(self): + return self.x + self.y + self.z + def f(x, switch): + a = A(x, x + 1) + b = B(x, x + 1, x + 2) + if switch: + c = A(x, x + 1) + else: + c = B(x, x + 1, x + 2) + return a.x + a.y + b.x + b.y + b.z + c.getsum() + assert f(10, True) == 75 + graph, t = get_graph(f, [int, bool], 1) + num_cast_pointer = len(getops(graph)['cast_pointer']) + changed = remove_duplicate_casts(graph, t) + assert changed + ops = getops(graph) + assert len(ops['cast_pointer']) < num_cast_pointer + print len(ops['cast_pointer']), num_cast_pointer + graph_getsum = graphof(t, B.getsum.im_func) + num_cast_pointer = len(getops(graph_getsum)['cast_pointer']) + changed = remove_duplicate_casts(graph_getsum, t) + assert changed + if conftest.option.view: + t.view() + check_graph(graph, [10, True], 75, t) + ops = getops(graph_getsum) + assert len(ops['cast_pointer']) < num_cast_pointer + print len(ops['cast_pointer']), num_cast_pointer + From cfbolz at codespeak.net Fri Apr 7 12:25:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 12:25:52 +0200 (CEST) Subject: [pypy-svn] r25489 - in pypy/dist/pypy/translator: backendopt c Message-ID: <20060407102552.5240210272@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 12:25:51 2006 New Revision: 25489 Modified: pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/c/exceptiontransform.py Log: remove superfluous keepalives after inlining and the excpetion transform Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Fri Apr 7 12:25:51 2006 @@ -1,5 +1,5 @@ from pypy.translator.backendopt.raisingop2direct_call import raisingop2direct_call -from pypy.translator.backendopt.removenoops import remove_same_as +from pypy.translator.backendopt.removenoops import remove_same_as, remove_superfluous_keep_alive from pypy.translator.backendopt.inline import auto_inlining from pypy.translator.backendopt.malloc import remove_simple_mallocs from pypy.translator.backendopt.ssa import SSI_to_SSA @@ -44,6 +44,8 @@ # inline functions in each other if inline_threshold: auto_inlining(translator, inline_threshold) + for graph in translator.graphs: + remove_superfluous_keep_alive(graph) if PRINT_STATISTICS: print "after inlining:" Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Fri Apr 7 12:25:51 2006 @@ -1,6 +1,6 @@ from pypy.translator.simplify import join_blocks, cleanup_graph from pypy.translator.unsimplify import copyvar, split_block -from pypy.translator.backendopt import canraise, inline, support +from pypy.translator.backendopt import canraise, inline, support, removenoops from pypy.objspace.flow.model import Block, Constant, Variable, Link, \ c_last_exception, SpaceOperation, checkgraph, FunctionGraph from pypy.rpython.lltypesystem import lltype, llmemory @@ -131,6 +131,7 @@ self.transform_block(graph, block) self.transform_except_block(graph, graph.exceptblock) cleanup_graph(graph) + removenoops.remove_superfluous_keep_alive(graph) def transform_block(self, graph, block): if block is graph.exceptblock: From afayolle at codespeak.net Fri Apr 7 12:46:05 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 12:46:05 +0200 (CEST) Subject: [pypy-svn] r25490 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060407104605.0839B10277@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 12:46:04 2006 New Revision: 25490 Added: pypy/dist/pypy/objspace/constraint/test/test_constraint.py (contents, props changed) Modified: pypy/dist/pypy/objspace/constraint/constraint.py Log: can now call AllDistinct.affected_variables Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Fri Apr 7 12:46:04 2006 @@ -1,4 +1,14 @@ -from variable import NoDom +from pypy.interpreter.error import OperationError +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter import baseobjspace, typedef, gateway +from pypy.interpreter.gateway import interp2app + +from pypy.objspace.std.listobject import W_ListObject + + +from pypy.objspace.constraint.computationspace import W_ComputationSpace + +#from variable import NoDom import operator #-- Exceptions --------------------------------------- @@ -14,29 +24,60 @@ #-- Constraints ------------------------------------------ -class AbstractConstraint(object): +class W_Constraint(Wrappable): + def __init__(self, object_space, computation_space): + self._space = object_space + assert isinstance(computation_space, W_ComputationSpace) + self.cs = computation_space + + def w_affected_variables(self): + pass + + def w_is_variable_relevant(self, w_var): + pass + + def w_estimate_cost(self): + pass + + def w_copy_to(self, w_computation_space): + pass + + def w_revise(self): + pass + +W_Constraint.typedef = typedef.TypeDef( + "W_Constraint", + affected_variables = interp2app(W_Constraint.w_affected_variables), + is_variable_relevant = interp2app(W_Constraint.w_is_variable_relevant), + estimate_cost = interp2app(W_Constraint.w_estimate_cost), + copy_to = interp2app(W_Constraint.w_copy_to), + revise = interp2app(W_Constraint.w_revise) + ) + + +class W_AbstractConstraint(W_Constraint): - def __init__(self, c_space, variables): + def __init__(self, object_space, c_space, w_variables): """variables is a list of variables which appear in the formula""" - self.cs = c_space + W_Constraint.__init__(self, object_space, c_space) + assert isinstance(w_variables, W_ListObject) + assert self._space.is_true(self._space.gt(self._space.len(w_variables), self._space.newint(1))) self._names_to_vars = {} - for var in variables: - if self.cs.dom(var) == NoDom: - raise DomainlessVariables + for var in w_variables.wrappeditems: self._names_to_vars[var.name] = var - self._variables = variables + self._variables = w_variables - def affected_variables(self): + def w_affected_variables(self): """ Return a list of all variables affected by this constraint """ return self._variables - def isVariableRelevant(self, variable): + def w_is_variable_relevant(self, w_variable): return variable in self._variables - def estimate_cost(self): + def w_estimate_cost(self): """Return an estimate of the cost of the narrowing of the constraint""" return reduce(operator.mul, - [self.cs.dom(var).size() for var in self._variables]) + [self.cs.w_dom(var).size() for var in self._variables]) def copy_to(self, space): return self.__class__(space, self._variables) @@ -45,15 +86,22 @@ if not isinstance(other, self.__class__): return False return self._variables == other._variables - -class W_BasicConstraint(object): +W_AbstractConstraint.typedef = typedef.TypeDef("W_AbstractConstraint", + affected_variables = interp2app(W_AbstractConstraint.w_affected_variables), + is_variable_relevant = interp2app(W_AbstractConstraint.w_is_variable_relevant), + estimate_cost = interp2app(W_AbstractConstraint.w_estimate_cost), + copy_to = interp2app(W_AbstractConstraint.w_copy_to), + revise = interp2app(W_AbstractConstraint.w_revise) +) +class W_BasicConstraint(W_Constraint): """A BasicConstraint, which is never queued by the Repository A BasicConstraint affects only one variable, and will be entailed on the first call to narrow()""" - def __init__(self, variable, reference, operator): + def __init__(self, object_space, variable, reference, operator): """variables is a list of variables on which the constraint is applied""" + W_Constraint.__init__(self, object_space) self._variable = variable self._reference = reference self._operator = operator @@ -61,22 +109,22 @@ def __repr__(self): return '<%s %s %s>'% (self.__class__, self._variable, self._reference) - def copy_to(self, space): + def w_copy_to(self, w_computation_space): raise NotImplementedError - def isVariableRelevant(self, variable): + def w_is_variable_relevant(self, w_variable): return variable == self._variable - def estimateCost(self, domains): + def w_estimate_cost(self): return 0 # get in the first place in the queue - def affectedVariables(self): + def w_affected_variables(self): return [self._variable] def getVariable(self): return self._variable - def revise(self, domains): + def w_revise(self, w_domains): domain = domains[self._variable] operator = self._operator ref = self._reference @@ -92,6 +140,15 @@ def __eq__(self, other): raise NotImplementedError +W_BasicConstraint.typedef = typedef.TypeDef( + "W_BasicConstraint", + affected_variables = interp2app(W_BasicConstraint.w_affected_variables), + is_variable_relevant = interp2app(W_BasicConstraint.w_is_variable_relevant), + estimate_cost = interp2app(W_BasicConstraint.w_estimate_cost), + copy_to = interp2app(W_BasicConstraint.w_copy_to), + revise = interp2app(W_BasicConstraint.w_revise) + ) + def make_lambda_head(vars): var_ids = ','.join([var.name for var in vars]) return 'lambda ' + var_ids + ':' @@ -102,22 +159,22 @@ return expr -class AllDistinct(AbstractConstraint): +class W_AllDistinct(W_AbstractConstraint): """Contraint: all values must be distinct""" - def __init__(self, c_space, variables): - assert len(variables)>1 - AbstractConstraint.__init__(self, c_space, variables) + def __init__(self, object_space, w_c_space, w_variables): + W_AbstractConstraint.__init__(self, object_space, w_c_space, w_variables) + assert len(w_variables.wrappeditems)>1 # worst case complexity - self.__cost = len(variables) * (len(variables) - 1) / 2 + self.__cost = len(w_variables.wrappeditems) * (len(w_variables.wrappeditems) - 1) / 2 def __repr__(self): return '' % str(self._variables) - def copy_to(self, space): + def w_copy_to(self, w_computation_space): return self.__class__(space, self._variables) - def estimateCost(self, domains): + def w_estimate_cost(self): return self.__cost def test_solution(self, sol): @@ -127,9 +184,9 @@ value_set = set(values) return len(value_set) == len(sol) - def revise(self): - variables = [(self.cs.dom(variable).size(), - variable, self.cs.dom(variable)) + def w_revise(self): + variables = [(self.cs.w_dom(variable).size(), + variable, self.cs.w_dom(variable)) for variable in self._variables] variables.sort() @@ -166,8 +223,18 @@ return 1 +W_AllDistinct.typedef = typedef.TypeDef( + "W_AllDistinct", W_AbstractConstraint.typedef, + estimate_cost = interp2app(W_AllDistinct.w_estimate_cost), + revise = interp2app(W_Constraint.w_revise) + ) + +# function bolted into the space to serve as constructor +def make_alldistinct(objectspace, w_computationspace, w_variables): + return objectspace.wrap(W_AllDistinct(objectspace, w_computationspace, w_variables)) +app_make_alldistinct = gateway.interp2app(make_alldistinct) -class Expression(AbstractConstraint): +class W_Expression(W_AbstractConstraint): """A constraint represented as a python expression.""" _FILTER_CACHE = {} @@ -209,9 +276,9 @@ variables = [] kwargs = {} for variable in self._variables: - domain = self.cs.dom(variable) - values = domain.get_values() - variables.append((domain.size(), [variable, values, 0, len(values)])) + domain = self.cs.w_dom(variable) + values = domain.w_get_values() + variables.append((domain.w_size(), [variable, values, 0, len(values)])) kwargs[variable.name] = values[0] # sort variables to instanciate those with fewer possible values first variables.sort() @@ -253,7 +320,7 @@ try: for var, keep in result_cache.iteritems(): - domain = self.cs.dom(self._names_to_vars[var]) + domain = self.cs.w_dom(self._names_to_vars[var]) domain.remove_values([val for val in domain if val not in keep]) except ConsistencyFailure: @@ -275,7 +342,11 @@ def __repr__(self): return '<%s>' % self.formula -class BinaryExpression(Expression): +W_Expression.typedef = typedef.TypeDef("W_Expression", + W_AbstractConstraint.typedef) + + +class BinaryExpression(W_Expression): """A binary constraint represented as a python expression This implementation uses a narrowing algorithm optimized for Added: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Fri Apr 7 12:46:04 2006 @@ -0,0 +1,19 @@ +from pypy.conftest import gettestobjspace + +class AppTest_AllDistinct(object): + + def setup_class(cls): + cls.space = gettestobjspace('logic') + + + def test_instantiate(self): + cspace = newspace() + v1 = cspace.var('v1', FiniteDomain([1,2])) + v2 = cspace.var('v2', FiniteDomain([1,2])) + cstr = AllDistinct(cspace, [v1, v2]) + variables = cstr.affected_variables() + assert variables is not None + print variables + assert len(variables) == 2 + assert v1 in variables + assert v2 in variables From afayolle at codespeak.net Fri Apr 7 13:03:07 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Fri, 7 Apr 2006 13:03:07 +0200 (CEST) Subject: [pypy-svn] r25491 - pypy/dist/pypy/objspace Message-ID: <20060407110307.A64FE10276@code0.codespeak.net> Author: afayolle Date: Fri Apr 7 13:03:06 2006 New Revision: 25491 Modified: pypy/dist/pypy/objspace/logic.py Log: expose constraint module Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Fri Apr 7 13:03:06 2006 @@ -202,6 +202,7 @@ return W_Var() app_newvar = gateway.interp2app(newvar) + def wait__Root(space, w_obj): return w_obj @@ -754,19 +755,20 @@ #------ domains ----------------- - from pypy.objspace.constraint import domain all_mms.update(domain.all_mms) W_FiniteDomain = domain.W_FiniteDomain -#------ computation space ------- - +#-------- computationspace -------------- from pypy.objspace.constraint import computationspace all_mms.update(computationspace.all_mms) W_ComputationSpace = computationspace.W_ComputationSpace +# ---- constraints +from pypy.objspace.constraint import constraint + #-- THE SPACE --------------------------------------- #class UnificationError(w_RuntimeError): @@ -817,6 +819,10 @@ space.wrap(app_alias_of)) space.setitem(space.builtin.w_dict, space.wrap('is_aliased'), space.wrap(app_is_aliased)) + space.setitem(space.builtin.w_dict, space.wrap('newspace'), + space.wrap(computationspace.app_newspace)) + space.setitem(space.builtin.w_dict, space.wrap('AllDistinct'), + space.wrap(constraint.app_make_alldistinct)) space.setitem(space.builtin.w_dict, space.wrap('bind'), space.wrap(app_bind)) space.setitem(space.builtin.w_dict, space.wrap('unify'), From mwh at codespeak.net Fri Apr 7 13:29:10 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 7 Apr 2006 13:29:10 +0200 (CEST) Subject: [pypy-svn] r25492 - in pypy/extradoc/talk/accu2006: . accu-2006.key accu-2006.key/thumbs Message-ID: <20060407112910.B6CF01027A@code0.codespeak.net> Author: mwh Date: Fri Apr 7 13:28:49 2006 New Revision: 25492 Modified: pypy/extradoc/talk/accu2006/accu-2006.key/index.apxl.gz pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st0-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st1-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st12-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st2.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-2.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-3.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-4.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-5.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st4.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st6.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-3.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-5.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-6.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-7.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-8.tiff pypy/extradoc/talk/accu2006/accu-2006.pdf Log: updates to my accu talk. to be previewed in leysin soon (i hope :) Modified: pypy/extradoc/talk/accu2006/accu-2006.key/index.apxl.gz ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st0-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st1-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st12-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st2.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-2.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-3.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-4.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-5.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st4.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st6.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-3.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-5.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-6.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-7.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7-8.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.pdf ============================================================================== Binary files. No diff available. From auc at codespeak.net Fri Apr 7 13:47:01 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 7 Apr 2006 13:47:01 +0200 (CEST) Subject: [pypy-svn] r25493 - in pypy/dist/pypy/objspace: . constraint constraint/test Message-ID: <20060407114701.954C01027C@code0.codespeak.net> Author: auc Date: Fri Apr 7 13:47:00 2006 New Revision: 25493 Modified: pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py pypy/dist/pypy/objspace/constraint/test/test_fd.py pypy/dist/pypy/objspace/logic.py Log: estimated_cost and revise Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Fri Apr 7 13:47:00 2006 @@ -116,7 +116,7 @@ return variable == self._variable def w_estimate_cost(self): - return 0 # get in the first place in the queue + return self._space.newint(0) # get in the first place in the queue def w_affected_variables(self): return [self._variable] @@ -168,14 +168,14 @@ # worst case complexity self.__cost = len(w_variables.wrappeditems) * (len(w_variables.wrappeditems) - 1) / 2 - def __repr__(self): - return '' % str(self._variables) + def w__repr__(self): + return self._space.newstring('' % str(self._variables)) def w_copy_to(self, w_computation_space): return self.__class__(space, self._variables) def w_estimate_cost(self): - return self.__cost + return self._space.newint(self.__cost) def test_solution(self, sol): """test a solution against this constraint @@ -185,20 +185,20 @@ return len(value_set) == len(sol) def w_revise(self): - variables = [(self.cs.w_dom(variable).size(), + variables = [(self._space.int_w(self.cs.w_dom(variable).w_size()), variable, self.cs.w_dom(variable)) - for variable in self._variables] - + for variable in self._variables.wrappeditems] + variables.sort() # if a domain has a size of 1, # then the value must be removed from the other domains for size, var, dom in variables: - if dom.size() == 1: + if self._space.eq_w(dom.w_size(), self._space.newint(1)): print "AllDistinct removes values" for _siz, _var, _dom in variables: - if _var != var: + if not self._space.eq_w(_var, var): try: - _dom.remove_value(dom.get_values()[0]) + _dom.w_remove_value(dom.w_get_values().wrappeditems[0]) except KeyError: # we ignore errors caused by the removal of # non existing values @@ -207,26 +207,29 @@ # if there are less values than variables, the constraint fails values = {} for size, var, dom in variables: - for val in dom: + for val in dom.w_get_values().wrappeditems: values[val] = 0 if len(values) < len(variables): print "AllDistinct failed" - raise ConsistencyFailure() + raise OperationError(self._space.w_RuntimeError, + self._space.wrap("Consistency Failure")) +# raise ConsistencyFailure() # the constraint is entailed if all domains have a size of 1 for variable in variables: - if variable[2].size() != 1: - return 0 + if self._space.is_true(self._space.ne(variable[2].w_size(), self._space.newint(1))): + return self._space.newint(0) # Question : did we *really* completely check # our own alldistinctness predicate ? - return 1 + return self._space.newint(1) W_AllDistinct.typedef = typedef.TypeDef( "W_AllDistinct", W_AbstractConstraint.typedef, estimate_cost = interp2app(W_AllDistinct.w_estimate_cost), - revise = interp2app(W_Constraint.w_revise) + revise = interp2app(W_AllDistinct.w_revise), + __repr__ = interp2app(W_AllDistinct.w__repr__) ) # function bolted into the space to serve as constructor Modified: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Fri Apr 7 13:47:00 2006 @@ -13,7 +13,46 @@ cstr = AllDistinct(cspace, [v1, v2]) variables = cstr.affected_variables() assert variables is not None - print variables assert len(variables) == 2 assert v1 in variables assert v2 in variables + + def test_estimated_cost(self): + csp = newspace() + v1 = csp.var('v1', FiniteDomain([1, 2])) + v2 = csp.var('v2', FiniteDomain([1, 2])) + cstr = AllDistinct(csp, [v1, v2]) + assert cstr.estimate_cost() == 1 + + def notest_repr(self): + csp = newspace() + v1 = csp.var('v1', FiniteDomain([1, 2])) + v2 = csp.var('v2', FiniteDomain([1, 2])) + cstr = AllDistinct(csp, [v1, v2]) + print cstr + + def test_revise(self): + csp = newspace() + v1 = csp.var('v1', FiniteDomain([1, 2])) + v2 = csp.var('v2', FiniteDomain([1, 2])) + cstr = AllDistinct(csp, [v1, v2]) + assert cstr.revise() == 0 # not entailed + + v3 = csp.var('v3', FiniteDomain([1])) + v4 = csp.var('v4', FiniteDomain([2])) + cstr = AllDistinct(csp, [v3, v4]) + assert cstr.revise() == 1 # entailed + + v5 = csp.var('v5', FiniteDomain([1])) + v6 = csp.var('v6', FiniteDomain([1])) + cstr = AllDistinct(csp, [v5, v6]) + raises(Exception, cstr.revise) + + cstr = AllDistinct(csp, [v1, v5]) + assert cstr.revise() == 1 + assert csp.dom(v1).get_values() == [2] + + v7 = csp.var('v7', FiniteDomain([1, 2])) + v8 = csp.var('v8', FiniteDomain([1, 2])) + cstr = AllDistinct(csp, [v2, v7, v8]) + raises(Exception, cstr.revise) Modified: pypy/dist/pypy/objspace/constraint/test/test_fd.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_fd.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_fd.py Fri Apr 7 13:47:00 2006 @@ -9,7 +9,6 @@ fd = FiniteDomain([1, 2, 3]) assert fd.size() == 3 assert set(fd.get_values()) == set([1, 2, 3]) - #fd2 = fd.copy() def test_copy(self): fd = FiniteDomain([1, 2, 3]) @@ -28,7 +27,6 @@ # FIXME: check this is a ConsistencyFailure raises(Exception, fd.remove_value, 3) - def test_remove_values(self): fd = FiniteDomain([1, 2, 3]) fd.remove_values([1, 2]) Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Fri Apr 7 13:47:00 2006 @@ -786,6 +786,8 @@ # multimethods hack space.model.typeorder[W_Var] = [(W_Var, None), (W_Root, None)] # None means no conversion space.model.typeorder[W_FiniteDomain] = [(W_FiniteDomain, None), (W_Root, None)] + + for name in all_mms.keys(): exprargs, expr, miniglobals, fallback = ( all_mms[name].install_not_sliced(space.model.typeorder, baked_perform_call=False)) From antocuni at codespeak.net Fri Apr 7 15:11:38 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 7 Apr 2006 15:11:38 +0200 (CEST) Subject: [pypy-svn] r25494 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060407131138.265C91027E@code0.codespeak.net> Author: antocuni Date: Fri Apr 7 15:11:23 2006 New Revision: 25494 Added: pypy/dist/pypy/translator/cli/database.py (contents, props changed) Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py pypy/dist/pypy/translator/cli/test/test_oo.py Log: Added support for unbound methods Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Fri Apr 7 15:11:23 2006 @@ -3,7 +3,8 @@ from pypy.translator.cli import cts class Class(Node): - def __init__(self, classdef): + def __init__(self, db, classdef): + self.db = db self.classdef = classdef self.namespace, self.name = cts.split_class_name(classdef._name) @@ -30,9 +31,8 @@ self._ctor() for m_name, m_meth in self.classdef._methods.iteritems(): - # TODO: handle static, class and unbound methods - # TODO: should __init__ be rendered as a constructor? - f = Function(m_meth.graph, m_name, is_method = True) + # TODO: handle class and unbound methods + f = Function(self.db, m_meth.graph, m_name, is_method = True) f.render(ilasm) ilasm.end_class() Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Fri Apr 7 15:11:23 2006 @@ -47,10 +47,10 @@ def lltype_to_cts(t): if isinstance(t, Instance): - name = t._name.replace('__main__.', '') # TODO: modules - if name in ('Object_meta', 'Object'): + name = t._name + if 'Object_meta' in name or 'Object' in name: # TODO return 'object' - + return 'class %s' % name return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Added: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/database.py Fri Apr 7 15:11:23 2006 @@ -0,0 +1,18 @@ +try: + set +except NameError: + from sets import Set as set + + +class LowLevelDatabase(object): + def __init__(self): + self.classes = set() + self.pending_graphs = [] + self.functions = {} # graph --> function_name + self.methods = {} # graph --> method_name + + def record_function(self, graph, name): + self.functions[graph] = name + + def function_name(self, graph): + return self.functions.get(graph, None) Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Fri Apr 7 15:11:23 2006 @@ -18,13 +18,13 @@ py.log.setconsumer("cli", ansi_log) class Function(Node, Generator): - def __init__(self, graph, name = None, is_method = False, is_entrypoint = False): + def __init__(self, db, graph, name = None, is_method = False, is_entrypoint = False): + self.db = db self.graph = graph self.name = name or graph.name self.is_method = is_method self.is_entrypoint = is_entrypoint self.blocknum = {} - self.classdefs = set() self._set_args() self._set_locals() @@ -130,6 +130,10 @@ self.ilasm.opcode('ret') self.ilasm.end_function() + if self.is_method: + pass # TODO + else: + self.db.record_function(self.graph, self.name) def _setup_link(self, link): target = link.target @@ -197,7 +201,7 @@ lltype = arg.value if isinstance(lltype, Instance): - self.classdefs.add(lltype) + self.db.classes.add(lltype) def _render_op(self, op): @@ -225,8 +229,9 @@ def function_signature(self, graph): return cts.graph_to_signature(graph, False) - def method_signature(self, graph, name): - return cts.graph_to_signature(graph, True, name) + def method_signature(self, graph, class_name, name): + full_name = '%s::%s' % (class_name, name) + return cts.graph_to_signature(graph, True, full_name) def class_name(self, ooinstance): return ooinstance._name @@ -234,7 +239,8 @@ def emit(self, instr, *args): self.ilasm.opcode(instr, *args) - def call(self, func_name): + def call(self, graph, func_name): + self.db.pending_graphs.append(graph) self.ilasm.call(func_name) def new(self, obj): @@ -248,10 +254,10 @@ def call_method(self, obj, name): owner, meth = obj._lookup(name) - full_name = '%s::%s' % (self.class_name(obj), name) + signature = self.method_signature(meth.graph, self.class_name(obj), name) # TODO: there are cases when we don't need callvirt but a # plain call is sufficient - self.ilasm.call_method(self.method_signature(meth.graph, full_name)) + self.ilasm.call_method(signature) def load(self, v): if isinstance(v, flowmodel.Variable): @@ -265,8 +271,9 @@ self.ilasm.opcode('ldloc', repr(v.name)) elif isinstance(v, flowmodel.Constant): - iltype, ilvalue = cts.llconst_to_ilasm(v) - self.ilasm.opcode('ldc.' + iltype, ilvalue) + if v.concretetype != Void: + iltype, ilvalue = cts.llconst_to_ilasm(v) + self.ilasm.opcode('ldc.' + iltype, ilvalue) else: assert False Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Fri Apr 7 15:11:23 2006 @@ -1,8 +1,3 @@ -try: - set -except NameError: - from sets import Set as set - import sys from types import MethodType @@ -11,6 +6,7 @@ from pypy.translator.cli.function import Function from pypy.translator.cli.class_ import Class from pypy.translator.cli.option import getoption +from pypy.translator.cli.database import LowLevelDatabase class Tee(object): @@ -31,7 +27,7 @@ self.tmpdir = tmpdir self.translator = translator self.entrypoint = entrypoint - self.classdefs = set() + self.db = LowLevelDatabase() if entrypoint is None: self.assembly_name = self.translator.graphs[0].name @@ -46,41 +42,41 @@ out = Tee(sys.stdout, out) self.ilasm = IlasmGenerator(out, self.assembly_name) - self.gen_all_functions() + self.gen_entrypoint() self.find_superclasses() self.gen_classes() + self.gen_functions() out.close() return self.tmpfile.strpath - def gen_all_functions(self): + def gen_entrypoint(self): if self.entrypoint: - self.entrypoint.render(self.ilasm) - - # generate code for all 'global' functions, i.e., those who are not methods. - for graph in self.translator.graphs: + self.db.pending_graphs += self.entrypoint.render(self.ilasm) + else: + self.db.pending_graphs.append(self.translator.graphs[0]) - # TODO: remove this test - if graph.name.startswith('ll_'): - continue + self.gen_functions() - if '.' not in graph.name: # it's not a method - f = Function(graph) + def gen_functions(self): + while self.db.pending_graphs: + graph = self.db.pending_graphs.pop() + if self.db.function_name(graph) is None: + f = Function(self.db, graph) f.render(self.ilasm) - self.classdefs.update(f.classdefs) def find_superclasses(self): - classdefs = set() - pendings = self.classdefs + classes = set() + pendings = self.db.classes while pendings: classdef = pendings.pop() - if classdef not in classdefs and classdef is not None: - classdefs.add(classdef) + if classdef not in classes and classdef is not None: + classes.add(classdef) pendings.add(classdef._superclass) - self.classdefs = classdefs + self.db.classes = classes def gen_classes(self): - for classdef in self.classdefs: - c = Class(classdef) + for classdef in self.db.classes: + c = Class(self.db, classdef) c.render(self.ilasm) Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Fri Apr 7 15:11:23 2006 @@ -50,13 +50,32 @@ class _Call(MicroInstruction): def render(self, generator, op): - func_sig = generator.function_signature(op.args[0].value.graph) + graph = op.args[0].value.graph + cls = getattr(graph.func, 'class_', None) - # push parameters - for func_arg in op.args[1:]: + self._render_function(generator, graph, op.args) +## if cls is None: +## self._render_function(generator, graph, op.args) +## else: +## self._render_unbound_meth(generator, cls, graph, op.args) + + def _render_unbound_meth(self, generator, cls, graph, args): + 0/0 + this = args[1] + # TODO: make sure that 'this' is compatible with 'cls' + for func_arg in args[1:]: generator.load(func_arg) - generator.call(func_sig) + # TODO: it doesn't work if cls is in another namespace + cls_name, meth_name = graph.name.rsplit('.', 1) + meth_sig = generator.method_signature(graph, cls_name, meth_name) + generator.call(graph, meth_sig) # TODO + + def _render_function(self, generator, graph, args): + func_sig = generator.function_signature(graph) + for func_arg in args[1:]: # push parameters + generator.load(func_arg) + generator.call(graph, func_sig) class _New(MicroInstruction): def render(self, generator, op): Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Fri Apr 7 15:11:23 2006 @@ -27,10 +27,15 @@ self.x = x class Derived(Base): - pass + def __init__(self, x): + Base.__init__(self, x) + +def foo(x): + return x+1 def bar(x, y): - a = Derived(42) + a = Derived(x) + return a.x f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Fri Apr 7 15:11:23 2006 @@ -54,6 +54,7 @@ ilasm.call('void class [mscorlib]System.Console::WriteLine(%s)' % ret_type) ilasm.opcode('ret') ilasm.end_function() + return [self.graph] def __convert_method(self, arg_type): _conv = { Modified: pypy/dist/pypy/translator/cli/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cli/test/test_oo.py Fri Apr 7 15:11:23 2006 @@ -19,10 +19,13 @@ def compute_and_multiply(self, factor): return self.compute() * factor + def static_meth(x, y): + return x*y + static_meth = staticmethod(static_meth) + class MyDerivedClass(MyClass): def __init__(self, x, y): - self.x = x - self.y = y + MyClass.__init__(self, x, y) def compute(self): return self.x - self.y @@ -46,3 +49,10 @@ base = MyClass(x, y) derived = MyDerivedClass(x, y) return helper(base) + helper(derived) + +def oo_static_method(x, y): + base = MyClass(x, y) + derived = MyDerivedClass(x, y) + return base.static_meth(x,y) + derived.static_meth(x, y)\ + + MyClass.static_meth(x, y) + MyDerivedClass.static_meth(x, y) + From cfbolz at codespeak.net Fri Apr 7 15:18:08 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 15:18:08 +0200 (CEST) Subject: [pypy-svn] r25495 - in pypy/dist/pypy: rpython rpython/memory translator/backendopt translator/backendopt/test translator/c Message-ID: <20060407131808.1620610094@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 15:18:05 2006 New Revision: 25495 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/removenoops.py pypy/dist/pypy/translator/backendopt/test/test_inline.py pypy/dist/pypy/translator/backendopt/test/test_propagate.py pypy/dist/pypy/translator/backendopt/test/test_removenoops.py pypy/dist/pypy/translator/c/exceptiontransform.py Log: (pedronis, cfbolz): introduced an optimizations remove_duplicate_casts to reduce the size of the live variable set. For this we needed to make the exception heuristic in inlining cleverer. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 7 15:18:05 2006 @@ -38,6 +38,8 @@ else: self.mixlevelannotator = None self.inline = inline + if inline: + self.lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() self.graphs_to_inline = {} def get_lltype_of_exception_value(self): @@ -87,7 +89,8 @@ from pypy.translator.backendopt import inline for inline_graph in self.graphs_to_inline: try: - inline.inline_function(self.translator, inline_graph, graph) + inline.inline_function(self.translator, inline_graph, graph, + self.lltype_to_classdef) except inline.CannotInline: pass checkgraph(graph) Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Fri Apr 7 15:18:05 2006 @@ -102,6 +102,12 @@ def getexceptiondata(self): return self.exceptiondata # built at the end of specialize() + def lltype_to_classdef_mapping(self): + result = {} + for (classdef, _), repr in self.instance_reprs.iteritems(): + result[repr.lowleveltype] = classdef + return result + def makekey(self, s_obj): return pair(self.type_system, s_obj).rtyper_makekey(self) Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Fri Apr 7 15:18:05 2006 @@ -1,5 +1,5 @@ from pypy.translator.backendopt.raisingop2direct_call import raisingop2direct_call -from pypy.translator.backendopt.removenoops import remove_same_as, remove_superfluous_keep_alive +from pypy.translator.backendopt import removenoops from pypy.translator.backendopt.inline import auto_inlining from pypy.translator.backendopt.malloc import remove_simple_mallocs from pypy.translator.backendopt.ssa import SSI_to_SSA @@ -29,9 +29,10 @@ # remove obvious no-ops for graph in translator.graphs: - remove_same_as(graph) + removenoops.remove_same_as(graph) simplify.eliminate_empty_blocks(graph) simplify.transform_dead_op_vars(graph, translator) + removenoops.remove_duplicate_casts(graph, translator) if PRINT_STATISTICS: print "after no-op removal:" @@ -45,7 +46,8 @@ if inline_threshold: auto_inlining(translator, inline_threshold) for graph in translator.graphs: - remove_superfluous_keep_alive(graph) + removenoops.remove_superfluous_keep_alive(graph) + removenoops.remove_duplicate_casts(graph, translator) if PRINT_STATISTICS: print "after inlining:" @@ -58,7 +60,7 @@ count = remove_simple_mallocs(graph) if count: # remove typical leftovers from malloc removal - remove_same_as(graph) + removenoops.remove_same_as(graph) simplify.eliminate_empty_blocks(graph) simplify.transform_dead_op_vars(graph, translator) tot += count Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Fri Apr 7 15:18:05 2006 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import SpaceOperation, c_last_exception from pypy.objspace.flow.model import traverse, mkentrymap, checkgraph from pypy.annotation import model as annmodel -from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void +from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void, Ptr from pypy.rpython import rmodel from pypy.tool.algo import sparsemat from pypy.translator.backendopt.support import log, split_block_with_keepalive, generate_keepalive @@ -67,25 +67,28 @@ except StopIteration: return False -def inline_function(translator, inline_func, graph): - inliner = Inliner(translator, graph, inline_func) +def inline_function(translator, inline_func, graph, lltype_to_classdef): + inliner = Inliner(translator, graph, inline_func, lltype_to_classdef) return inliner.inline_all() +def simple_inline_function(translator, inline_func, graph): + inliner = Inliner(translator, graph, inline_func, + translator.rtyper.lltype_to_classdef_mapping()) + return inliner.inline_all() + + def _find_exception_type(block): #XXX slightly brittle: find the exception type for simple cases #(e.g. if you do only raise XXXError) by doing pattern matching - ops = [op for op in block.operations if op.opname != 'keepalive'] - if (len(ops) < 6 or - ops[-6].opname != "malloc" or ops[-5].opname != "cast_pointer" or - ops[-4].opname != "setfield" or ops[-3].opname != "cast_pointer" or - ops[-2].opname != "getfield" or ops[-1].opname != "cast_pointer" or - len(block.exits) != 1 or block.exits[0].args[0] != ops[-2].result or - block.exitswitch is not None or - block.exits[0].args[1] != ops[-1].result or - not isinstance(ops[-4].args[1], Constant) or - ops[-4].args[1].value != "typeptr"): + currvar = block.exits[0].args[1] + for op in block.operations[::-1]: + if op.opname in ("same_as", "cast_pointer") and op.result is currvar: + currvar = op.args[0] + elif op.opname == "malloc" and op.result is currvar: + break + else: return None, None - return ops[-4].args[2].value, block.exits[0] + return Ptr(op.args[0].value), block.exits[0] def does_raise_directly(graph, raise_analyzer): """ this function checks, whether graph contains operations which can raise @@ -101,7 +104,8 @@ return False class BaseInliner(object): - def __init__(self, translator, graph, inline_guarded_calls=False, + def __init__(self, translator, graph, lltype_to_classdef, + inline_guarded_calls=False, inline_guarded_calls_no_matter_what=False, raise_analyzer=None): self.translator = translator self.graph = graph @@ -114,6 +118,7 @@ self.raise_analyzer = raise_analyzer else: self.raise_analyzer = None + self.lltype_to_classdef = lltype_to_classdef def inline_all(self): count = 0 @@ -274,22 +279,27 @@ def rewire_exceptblock_with_guard(self, afterblock, copiedexceptblock): # this rewiring does not always succeed. in the cases where it doesn't # there will be generic code inserted + from pypy.rpython.lltypesystem import rclass exc_match = self.translator.rtyper.getexceptiondata().fn_exception_match for link in self.entrymap[self.graph_to_inline.exceptblock]: copiedblock = self.copy_block(link.prevblock) - eclass, copiedlink = _find_exception_type(copiedblock) + VALUE, copiedlink = _find_exception_type(copiedblock) #print copiedblock.operations - if eclass is None: + if VALUE is None or VALUE not in self.lltype_to_classdef: continue - etype = copiedlink.args[0] - evalue = copiedlink.args[1] + classdef = self.lltype_to_classdef[VALUE] + rtyper = self.translator.rtyper + classrepr = rclass.getclassrepr(rtyper, classdef) + vtable = classrepr.getruntime() + var_etype = copiedlink.args[0] + var_evalue = copiedlink.args[1] for exceptionlink in afterblock.exits[1:]: - if exc_match(eclass, exceptionlink.llexitcase): + if exc_match(vtable, exceptionlink.llexitcase): passon_vars = self.passon_vars(link.prevblock) copiedblock.operations += generate_keepalive(passon_vars) copiedlink.target = exceptionlink.target linkargs = self.find_args_in_exceptional_case( - exceptionlink, link.prevblock, etype, evalue, afterblock, passon_vars) + exceptionlink, link.prevblock, var_etype, var_evalue, afterblock, passon_vars) copiedlink.args = linkargs break @@ -387,9 +397,9 @@ class Inliner(BaseInliner): - def __init__(self, translator, graph, inline_func, inline_guarded_calls=False, + def __init__(self, translator, graph, inline_func, lltype_to_classdef, inline_guarded_calls=False, inline_guarded_calls_no_matter_what=False, raise_analyzer=None): - BaseInliner.__init__(self, translator, graph, + BaseInliner.__init__(self, translator, graph, lltype_to_classdef, inline_guarded_calls, inline_guarded_calls_no_matter_what, raise_analyzer) @@ -403,21 +413,10 @@ self.block_to_index.setdefault(block, {})[i] = g class OneShotInliner(BaseInliner): - def __init__(self, translator, graph, inline_guarded_calls=False, - inline_guarded_calls_no_matter_what=False, raise_analyzer=None): - BaseInliner.__init__(self, translator, graph, - inline_guarded_calls, - inline_guarded_calls_no_matter_what, - raise_analyzer) - def search_for_calls(self, block): pass -def _inline_function(translator, graph, block, index_operation): - inline_func = block.operations[index_operation].args[0].value._obj._callable - inliner = Inliner(translator, graph, inline_func) - # ____________________________________________________________ # # Automatic inlining @@ -509,6 +508,7 @@ fiboheap = [(0.0, graph) for graph in callers] valid_weight = {} couldnt_inline = {} + lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() while fiboheap: weight, graph = fiboheap[0] @@ -529,7 +529,8 @@ continue sys.stdout.flush() try: - res = bool(inline_function(translator, graph, parentgraph)) + res = bool(inline_function(translator, graph, parentgraph, + lltype_to_classdef)) except CannotInline: couldnt_inline[graph] = True res = CannotInline Modified: pypy/dist/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/removenoops.py Fri Apr 7 15:18:05 2006 @@ -1,6 +1,7 @@ from pypy.objspace.flow.model import Block, Variable, Constant from pypy.objspace.flow.model import traverse from pypy.rpython.lltypesystem.lltype import Void +from pypy.translator.backendopt.support import log from pypy.translator import simplify from pypy import conftest @@ -114,7 +115,8 @@ else: for arg in op.args: used[arg] = True - print "removed %s cast_pointers in %s" % (num_removed, graph.name) + log.removecasts( + "removed %s cast_pointers in %s" % (num_removed, graph.name)) return num_removed def remove_superfluous_keep_alive(graph): Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Fri Apr 7 15:18:05 2006 @@ -4,7 +4,7 @@ from pypy.objspace.flow.model import traverse, Block, Link, Variable, Constant from pypy.objspace.flow.model import last_exception, checkgraph from pypy.translator.backendopt import canraise -from pypy.translator.backendopt.inline import inline_function, CannotInline +from pypy.translator.backendopt.inline import simple_inline_function, CannotInline from pypy.translator.backendopt.inline import auto_inlining, Inliner from pypy.translator.backendopt.inline import collect_called_graphs from pypy.translator.backendopt.inline import measure_median_execution_cost @@ -56,10 +56,11 @@ raise_analyzer = canraise.RaiseAnalyzer(t) else: raise_analyzer = None - inliner = Inliner(t, graphof(t, in_func), func, inline_guarded_calls, + inliner = Inliner(t, graphof(t, in_func), func, + t.rtyper.lltype_to_classdef_mapping(), + inline_guarded_calls, raise_analyzer=raise_analyzer) inliner.inline_all() -# inline_function(t, func, graphof(t, in_func)) if option.view: t.view() sanity_check(t) @@ -282,7 +283,7 @@ break else: assert 0, "cannot find ll_rangenext_*() function" - inline_function(t, graph, graphof(t, f)) + simple_inline_function(t, graph, graphof(t, f)) sanity_check(t) interp = LLInterpreter(t.rtyper) result = interp.eval_graph(graphof(t, f), [10]) Modified: pypy/dist/pypy/translator/backendopt/test/test_propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_propagate.py Fri Apr 7 15:18:05 2006 @@ -6,12 +6,13 @@ from pypy.rpython.memory.test.test_gctransform import getops from pypy import conftest -def get_graph(fn, signature, inline_threshold=True): +def get_graph(fn, signature, inline_threshold=True, all_opts=True): t = TranslationContext() t.buildannotator().build_types(fn, signature) t.buildrtyper().specialize() - backend_optimizations(t, inline_threshold=inline_threshold, - ssa_form=False, propagate=False) + if all_opts: + backend_optimizations(t, inline_threshold=inline_threshold, + ssa_form=False, propagate=False) graph = graphof(t, fn) if conftest.option.view: t.view() Modified: pypy/dist/pypy/translator/backendopt/test/test_removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_removenoops.py Fri Apr 7 15:18:05 2006 @@ -1,6 +1,6 @@ from pypy.translator.backendopt.removenoops import remove_void, remove_same_as, \ remove_unaryops, remove_duplicate_casts, remove_superfluous_keep_alive -from pypy.translator.backendopt.inline import inline_function +from pypy.translator.backendopt.inline import simple_inline_function from pypy.translator.backendopt.test.test_propagate import getops, get_graph, check_graph from pypy.translator.translator import TranslationContext, graphof from pypy.translator.test.snippet import simple_method @@ -60,7 +60,7 @@ t.buildrtyper().specialize() # now we make the 'if True' appear f_graph = graphof(t, f) - inline_function(t, nothing, f_graph) + simple_inline_function(t, nothing, f_graph) # here, the graph looks like v21=same_as(True); exitswitch: v21 remove_same_as(f_graph) t.checkgraphs() @@ -127,7 +127,7 @@ c = B(x, x + 1, x + 2) return a.x + a.y + b.x + b.y + b.z + c.getsum() assert f(10, True) == 75 - graph, t = get_graph(f, [int, bool], 1) + graph, t = get_graph(f, [int, bool], 1, False) num_cast_pointer = len(getops(graph)['cast_pointer']) changed = remove_duplicate_casts(graph, t) assert changed Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Fri Apr 7 15:18:05 2006 @@ -113,6 +113,8 @@ lltype.Ptr(RPYEXC_RAISE)) mixlevelannotator.finish() + + self.lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() def transform_completely(self): for graph in self.translator.graphs: @@ -189,10 +191,10 @@ #non-exception case block.exits[0].exitcase = block.exits[0].llexitcase = None # use the dangerous second True flag :-) - inliner = inline.OneShotInliner(self.translator, graph, - inline_guarded_calls=True, - inline_guarded_calls_no_matter_what=True, - raise_analyzer=self.raise_analyzer) + inliner = inline.OneShotInliner( + self.translator, graph, self.lltype_to_classdef, + inline_guarded_calls=True, inline_guarded_calls_no_matter_what=True, + raise_analyzer=self.raise_analyzer) inliner.inline_once(block, len(block.operations)-1) #block.exits[0].exitcase = block.exits[0].llexitcase = False From auc at codespeak.net Fri Apr 7 15:24:47 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 7 Apr 2006 15:24:47 +0200 (CEST) Subject: [pypy-svn] r25496 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060407132447.8CEC210281@code0.codespeak.net> Author: auc Date: Fri Apr 7 15:24:45 2006 New Revision: 25496 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py Log: made the constraints stateless Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Fri Apr 7 15:24:45 2006 @@ -7,8 +7,11 @@ from pypy.objspace.constraint.domain import W_AbstractDomain + all_mms = {} +#-- Variables --------------------------- + class W_Variable(Wrappable): def __init__(self, obj_space, w_name): self._space = obj_space @@ -16,10 +19,44 @@ W_Variable.typedef = typedef.TypeDef("W_Variable") + +#-- Constraints ------------------------- + +class W_Constraint(Wrappable): + def __init__(self, object_space): + self._space = object_space + + def w_affected_variables(self): + pass + + def w_is_variable_relevant(self, w_var): + pass + + def w_estimate_cost(self, w_cs): + pass + + def w_copy_to(self, w_computation_space): + pass + + def w_revise(self, w_cs): + pass + +W_Constraint.typedef = typedef.TypeDef( + "W_Constraint", + affected_variables = interp2app(W_Constraint.w_affected_variables), + is_variable_relevant = interp2app(W_Constraint.w_is_variable_relevant), + estimate_cost = interp2app(W_Constraint.w_estimate_cost), + copy_to = interp2app(W_Constraint.w_copy_to), + revise = interp2app(W_Constraint.w_revise) + ) + +#-- Computation space ------------------- + class W_ComputationSpace(Wrappable): def __init__(self, obj_space): self._space = obj_space self.domains = self._space.newdict({}) + self.constraints = self._space.newdict({}) def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) @@ -34,10 +71,15 @@ assert isinstance(w_variable, W_Variable) return self.domains.content[w_variable.name] + def w_tell(self, w_constraint): + assert isinstance(w_constraint, W_Constraint) + self.constraints.content[w_constraint] = self._space.w_True + W_ComputationSpace.typedef = typedef.TypeDef( "W_ComputationSpace", var = interp2app(W_ComputationSpace.w_var), - dom = interp2app(W_ComputationSpace.w_dom)) + dom = interp2app(W_ComputationSpace.w_dom), + tell = interp2app(W_ComputationSpace.w_tell)) def newspace(space): Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Fri Apr 7 15:24:45 2006 @@ -5,8 +5,9 @@ from pypy.objspace.std.listobject import W_ListObject - from pypy.objspace.constraint.computationspace import W_ComputationSpace +from pypy.objspace.constraint.computationspace import W_Constraint + #from variable import NoDom import operator @@ -24,42 +25,12 @@ #-- Constraints ------------------------------------------ -class W_Constraint(Wrappable): - def __init__(self, object_space, computation_space): - self._space = object_space - assert isinstance(computation_space, W_ComputationSpace) - self.cs = computation_space - - def w_affected_variables(self): - pass - - def w_is_variable_relevant(self, w_var): - pass - - def w_estimate_cost(self): - pass - - def w_copy_to(self, w_computation_space): - pass - - def w_revise(self): - pass - -W_Constraint.typedef = typedef.TypeDef( - "W_Constraint", - affected_variables = interp2app(W_Constraint.w_affected_variables), - is_variable_relevant = interp2app(W_Constraint.w_is_variable_relevant), - estimate_cost = interp2app(W_Constraint.w_estimate_cost), - copy_to = interp2app(W_Constraint.w_copy_to), - revise = interp2app(W_Constraint.w_revise) - ) - class W_AbstractConstraint(W_Constraint): - def __init__(self, object_space, c_space, w_variables): + def __init__(self, object_space, w_variables): """variables is a list of variables which appear in the formula""" - W_Constraint.__init__(self, object_space, c_space) + W_Constraint.__init__(self, object_space) assert isinstance(w_variables, W_ListObject) assert self._space.is_true(self._space.gt(self._space.len(w_variables), self._space.newint(1))) self._names_to_vars = {} @@ -74,13 +45,12 @@ def w_is_variable_relevant(self, w_variable): return variable in self._variables - def w_estimate_cost(self): + def w_estimate_cost(self, w_cs): """Return an estimate of the cost of the narrowing of the constraint""" + assert isinstance(w_cs, W_ComputationSpace) return reduce(operator.mul, - [self.cs.w_dom(var).size() for var in self._variables]) - - def copy_to(self, space): - return self.__class__(space, self._variables) + [w_cs.w_dom(var).size() + for var in self._variables]) def __eq__(self, other): #FIXME and parent if not isinstance(other, self.__class__): return False @@ -93,61 +63,8 @@ copy_to = interp2app(W_AbstractConstraint.w_copy_to), revise = interp2app(W_AbstractConstraint.w_revise) ) -class W_BasicConstraint(W_Constraint): - """A BasicConstraint, which is never queued by the Repository - A BasicConstraint affects only one variable, and will be entailed - on the first call to narrow()""" - - def __init__(self, object_space, variable, reference, operator): - """variables is a list of variables on which - the constraint is applied""" - W_Constraint.__init__(self, object_space) - self._variable = variable - self._reference = reference - self._operator = operator - - def __repr__(self): - return '<%s %s %s>'% (self.__class__, self._variable, self._reference) - - def w_copy_to(self, w_computation_space): - raise NotImplementedError - - def w_is_variable_relevant(self, w_variable): - return variable == self._variable - - def w_estimate_cost(self): - return self._space.newint(0) # get in the first place in the queue - - def w_affected_variables(self): - return [self._variable] - - def getVariable(self): - return self._variable - - def w_revise(self, w_domains): - domain = domains[self._variable] - operator = self._operator - ref = self._reference - try: - for val in domain.get_values() : - if not operator(val, ref) : - domain.remove_value(val) - except ConsistencyFailure: - raise ConsistencyFailure('inconsistency while applying %s' % \ - repr(self)) - return 1 - def __eq__(self, other): - raise NotImplementedError -W_BasicConstraint.typedef = typedef.TypeDef( - "W_BasicConstraint", - affected_variables = interp2app(W_BasicConstraint.w_affected_variables), - is_variable_relevant = interp2app(W_BasicConstraint.w_is_variable_relevant), - estimate_cost = interp2app(W_BasicConstraint.w_estimate_cost), - copy_to = interp2app(W_BasicConstraint.w_copy_to), - revise = interp2app(W_BasicConstraint.w_revise) - ) def make_lambda_head(vars): var_ids = ','.join([var.name for var in vars]) @@ -162,8 +79,8 @@ class W_AllDistinct(W_AbstractConstraint): """Contraint: all values must be distinct""" - def __init__(self, object_space, w_c_space, w_variables): - W_AbstractConstraint.__init__(self, object_space, w_c_space, w_variables) + def __init__(self, object_space, w_variables): + W_AbstractConstraint.__init__(self, object_space, w_variables) assert len(w_variables.wrappeditems)>1 # worst case complexity self.__cost = len(w_variables.wrappeditems) * (len(w_variables.wrappeditems) - 1) / 2 @@ -171,10 +88,8 @@ def w__repr__(self): return self._space.newstring('' % str(self._variables)) - def w_copy_to(self, w_computation_space): - return self.__class__(space, self._variables) - - def w_estimate_cost(self): + def w_estimate_cost(self, w_cs): + assert isinstance(w_cs, W_ComputationSpace) return self._space.newint(self.__cost) def test_solution(self, sol): @@ -184,9 +99,10 @@ value_set = set(values) return len(value_set) == len(sol) - def w_revise(self): - variables = [(self._space.int_w(self.cs.w_dom(variable).w_size()), - variable, self.cs.w_dom(variable)) + def w_revise(self, w_cs): + assert isinstance(w_cs, W_ComputationSpace) + variables = [(self._space.int_w(w_cs.w_dom(variable).w_size()), + variable, w_cs.w_dom(variable)) for variable in self._variables.wrappeditems] variables.sort() @@ -233,8 +149,8 @@ ) # function bolted into the space to serve as constructor -def make_alldistinct(objectspace, w_computationspace, w_variables): - return objectspace.wrap(W_AllDistinct(objectspace, w_computationspace, w_variables)) +def make_alldistinct(objectspace, w_variables): + return objectspace.wrap(W_AllDistinct(objectspace, w_variables)) app_make_alldistinct = gateway.interp2app(make_alldistinct) class W_Expression(W_AbstractConstraint): @@ -275,11 +191,11 @@ return result_cache - def _assign_values(self): + def _assign_values(self, w_cs): variables = [] kwargs = {} for variable in self._variables: - domain = self.cs.w_dom(variable) + domain = w_cs.w_dom(variable) values = domain.w_get_values() variables.append((domain.w_size(), [variable, values, 0, len(values)])) kwargs[variable.name] = values[0] @@ -302,13 +218,13 @@ # it's over go_on = 0 - def revise(self): - # removed domain arg. (auc, ale) + def revise(self, w_cs): """generic propagation algorithm for n-ary expressions""" + assert isinstance(w_cs, W_ComputationSpace) maybe_entailed = 1 ffunc = self.filterFunc result_cache = self._init_result_cache() - for kwargs in self._assign_values(): + for kwargs in self._assign_values(w_cs): if maybe_entailed: for var, val in kwargs.iteritems(): if val not in result_cache[var]: @@ -323,7 +239,7 @@ try: for var, keep in result_cache.iteritems(): - domain = self.cs.w_dom(self._names_to_vars[var]) + domain = w_cs.w_dom(self._names_to_vars[var]) domain.remove_values([val for val in domain if val not in keep]) except ConsistencyFailure: @@ -349,6 +265,7 @@ W_AbstractConstraint.typedef) +# completely unported class BinaryExpression(W_Expression): """A binary constraint represented as a python expression @@ -431,6 +348,65 @@ return Expression(vars, formula) + + + + + + +# have a look at this later ... (really needed ?) +class W_BasicConstraint(W_Constraint): + """A BasicConstraint, which is never queued by the Repository + A BasicConstraint affects only one variable, and will be entailed + on the first call to narrow()""" + + def __init__(self, object_space, variable, reference, operator): + """variables is a list of variables on which + the constraint is applied""" + W_Constraint.__init__(self, object_space) + self._variable = variable + self._reference = reference + self._operator = operator + + def __repr__(self): + return '<%s %s %s>'% (self.__class__, self._variable, self._reference) + + def w_is_variable_relevant(self, w_variable): + return variable == self._variable + + def w_estimate_cost(self): + return self._space.newint(0) # get in the first place in the queue + + def w_affected_variables(self): + return [self._variable] + + def getVariable(self): + return self._variable + + def w_revise(self, w_domains): + domain = domains[self._variable] + operator = self._operator + ref = self._reference + try: + for val in domain.get_values() : + if not operator(val, ref) : + domain.remove_value(val) + except ConsistencyFailure: + raise ConsistencyFailure('inconsistency while applying %s' % \ + repr(self)) + return 1 + + def __eq__(self, other): + raise NotImplementedError + +W_BasicConstraint.typedef = typedef.TypeDef( + "W_BasicConstraint", + affected_variables = interp2app(W_BasicConstraint.w_affected_variables), + is_variable_relevant = interp2app(W_BasicConstraint.w_is_variable_relevant), + estimate_cost = interp2app(W_BasicConstraint.w_estimate_cost), + copy_to = interp2app(W_BasicConstraint.w_copy_to), + revise = interp2app(W_BasicConstraint.w_revise) + ) class W_Equals(W_BasicConstraint): """A basic constraint variable == constant value""" def __init__(self, variable, reference): Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Fri Apr 7 15:24:45 2006 @@ -25,4 +25,9 @@ assert cspace.dom(v) is domain - + def test_tell(self): + csp = newspace() + v1 = csp.var('v1', FiniteDomain([1, 2])) + v2 = csp.var('v2', FiniteDomain([1, 2])) + cstr = AllDistinct([v1, v2]) + csp.tell(cstr) Modified: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Fri Apr 7 15:24:45 2006 @@ -10,7 +10,7 @@ cspace = newspace() v1 = cspace.var('v1', FiniteDomain([1,2])) v2 = cspace.var('v2', FiniteDomain([1,2])) - cstr = AllDistinct(cspace, [v1, v2]) + cstr = AllDistinct([v1, v2]) variables = cstr.affected_variables() assert variables is not None assert len(variables) == 2 @@ -21,38 +21,41 @@ csp = newspace() v1 = csp.var('v1', FiniteDomain([1, 2])) v2 = csp.var('v2', FiniteDomain([1, 2])) - cstr = AllDistinct(csp, [v1, v2]) - assert cstr.estimate_cost() == 1 + cstr = AllDistinct([v1, v2]) + assert cstr.estimate_cost(csp) == 1 def notest_repr(self): csp = newspace() v1 = csp.var('v1', FiniteDomain([1, 2])) v2 = csp.var('v2', FiniteDomain([1, 2])) - cstr = AllDistinct(csp, [v1, v2]) + cstr = AllDistinct([v1, v2]) print cstr def test_revise(self): csp = newspace() v1 = csp.var('v1', FiniteDomain([1, 2])) v2 = csp.var('v2', FiniteDomain([1, 2])) - cstr = AllDistinct(csp, [v1, v2]) - assert cstr.revise() == 0 # not entailed + cstr = AllDistinct([v1, v2]) + assert cstr.revise(csp) == 0 # not entailed v3 = csp.var('v3', FiniteDomain([1])) v4 = csp.var('v4', FiniteDomain([2])) - cstr = AllDistinct(csp, [v3, v4]) - assert cstr.revise() == 1 # entailed + cstr = AllDistinct([v3, v4]) + assert cstr.revise(csp) == 1 # entailed v5 = csp.var('v5', FiniteDomain([1])) v6 = csp.var('v6', FiniteDomain([1])) - cstr = AllDistinct(csp, [v5, v6]) - raises(Exception, cstr.revise) - - cstr = AllDistinct(csp, [v1, v5]) - assert cstr.revise() == 1 - assert csp.dom(v1).get_values() == [2] + cstr = AllDistinct([v5, v6]) + raises(Exception, cstr.revise, csp) v7 = csp.var('v7', FiniteDomain([1, 2])) v8 = csp.var('v8', FiniteDomain([1, 2])) - cstr = AllDistinct(csp, [v2, v7, v8]) - raises(Exception, cstr.revise) + cstr = AllDistinct([v2, v7, v8]) + raises(Exception, cstr.revise, csp) + + v9 = csp.var('v9', FiniteDomain([1])) + v10= csp.var('v10', FiniteDomain([1,2])) + cstr = AllDistinct([v9, v10]) + assert cstr.revise(csp) == 1 + assert csp.dom(v10).get_values() == [2] + From cfbolz at codespeak.net Fri Apr 7 15:29:45 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 15:29:45 +0200 (CEST) Subject: [pypy-svn] r25497 - pypy/dist/pypy/rpython/memory Message-ID: <20060407132945.7ABC810283@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 15:29:44 2006 New Revision: 25497 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: fight spam Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 7 15:29:44 2006 @@ -959,8 +959,6 @@ if var in needed: newlivevars.append(var) livevars = newlivevars - else: - print "block which needs conservative livevar calculation found" newops = list(self.push_roots(livevars)) index = len(newops) newops.append(op) From cfbolz at codespeak.net Fri Apr 7 15:39:16 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 15:39:16 +0200 (CEST) Subject: [pypy-svn] r25498 - pypy/dist/pypy/doc/discussion Message-ID: <20060407133916.EF4381027A@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 15:39:14 2006 New Revision: 25498 Added: pypy/dist/pypy/doc/discussion/translation-swamp.txt Log: (pedronis, cfbolz): list tasks that need to be tackled to be able to further work on performance improvements Added: pypy/dist/pypy/doc/discussion/translation-swamp.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/discussion/translation-swamp.txt Fri Apr 7 15:39:14 2006 @@ -0,0 +1,27 @@ +=================================================================== +List of things that need to be improved for translation to be saner +=================================================================== + + + * understand nondeterminism after rtyping + + * nondeterminism of insert stackcheck: + + * move it after inlining + * avoid inserting it multiple times in the same block + + * experiment with different heuristics: + + * weight backedges more + * consider size of outer function + * consider number of arguments + + * find a more deterministic inlining order (number of calls?) + + * experiment with using a base inlining threshold and then drive inlining by + malloc removal possibilities (using escape analysis) + + * move the inlining of gc helpers just before emitting the code. + throw the graph away + + * for gcc: use just one implement file From arigo at codespeak.net Fri Apr 7 16:08:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 16:08:49 +0200 (CEST) Subject: [pypy-svn] r25500 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060407140849.04F821027D@code0.codespeak.net> Author: arigo Date: Fri Apr 7 16:08:47 2006 New Revision: 25500 Added: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Log: (arre, arigo) A ctypes utility that discovers the layout of structures by invoking the C compiler on example code. Added: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/ctypes_platform.py Fri Apr 7 16:08:47 2006 @@ -0,0 +1,190 @@ +#! /usr/bin/env python + +import os, py +import ctypes +from pypy.translator.tool.cbuild import build_executable +from pypy.tool.udir import udir + + +def uniquefilepath(LAST=[0]): + i = LAST[0] + LAST[0] += 1 + return udir.join('ctypesplatcheck_%d.c' % i) + +alignment_types = [ + ctypes.c_short, + ctypes.c_int, + ctypes.c_long, + ctypes.c_float, + ctypes.c_double, + ctypes.c_char_p, + ctypes.c_void_p, + ctypes.c_longlong, + ctypes.c_wchar, + ctypes.c_wchar_p, + ] + +integer_class = [ctypes.c_byte, ctypes.c_ubyte, + ctypes.c_short, ctypes.c_ushort, + ctypes.c_int, ctypes.c_uint, + ctypes.c_long, ctypes.c_ulong, + ctypes.c_longlong, ctypes.c_ulonglong, + ] +float_class = [ctypes.c_float, ctypes.c_double] + +class Field(object): + def __init__(self, name, ctype): + self.name = name + self.ctype = ctype + def __repr__(self): + return '' % (self.name, self.ctype) + +def layout_addfield(layout, offset, ctype, prefix): + size = ctypes.sizeof(ctype) + name = prefix + i = 0 + while name in layout: + i += 1 + name = '%s_%d' % (prefix, i) + field = Field(name, ctype) + for i in range(offset, offset+size): + assert layout[i] is None, "%s overlaps %r" % (fieldname, layout[i]) + layout[i] = field + return field + +def size_and_sign(ctype): + return (ctypes.sizeof(ctype), + ctype in integer_class and ctype(-1).value > 0) + +def fixup_ctype(fieldtype, fieldname, expected_size_and_sign): + for typeclass in [integer_class, float_class]: + if fieldtype in typeclass: + for ctype in typeclass: + if size_and_sign(ctype) == expected_size_and_sign: + return ctype + raise TypeError("conflicting field type %r for %r" % (fieldtype, + fieldname)) + + +def getstruct(name, c_header_source, interesting_fields): + filepath = uniquefilepath() + f = filepath.open('w') + print >> f, '#include ' + print >> f, '#include /* for offsetof() */' + print >> f + print >> f, c_header_source + print >> f + print >> f, 'typedef %s ctypesplatcheck_t;' % (name,) + print >> f, 'typedef struct {' + print >> f, ' char c;' + print >> f, ' ctypesplatcheck_t s;' + print >> f, '} ctypesplatcheck2_t;' + print >> f + print >> f, 'void dump(char* key, int value) {' + print >> f, ' printf("%s: %d\\n", key, value);' + print >> f, '}' + print >> f + print >> f, 'int main(void) {' + print >> f, ' ctypesplatcheck_t s;' + print >> f, ' dump("align", offsetof(ctypesplatcheck2_t, s));' + print >> f, ' dump("size", sizeof(ctypesplatcheck_t));' + for fieldname, fieldtype in interesting_fields: + print >> f, ' dump("fldofs %s", offsetof(ctypesplatcheck_t, %s));'%( + fieldname, fieldname) + print >> f, ' dump("fldsize %s", sizeof(s.%s));' % ( + fieldname, fieldname) + if fieldtype in integer_class: + print >> f, ' s.%s = 0; s.%s = ~s.%s;' % (fieldname, + fieldname, + fieldname) + print >> f, ' dump("fldunsigned %s", s.%s > 0);' % (fieldname, + fieldname) + print >> f, ' return 0;' + print >> f, '}' + f.close() + + executable = build_executable([filepath]) + output = py.process.cmdexec(executable) + info = {} + for line in output.splitlines(): + key, value = line.strip().split(': ') + info[key] = int(value) + + alignment = 1 + layout = [None] * info['size'] + for fieldname, fieldtype in interesting_fields: + offset = info['fldofs ' + fieldname] + size = info['fldsize ' + fieldname] + sign = info.get('fldunsigned ' + fieldname, False) + if (size, sign) != size_and_sign(fieldtype): + fieldtype = fixup_ctype(fieldtype, fieldname, (size, sign)) + layout_addfield(layout, offset, fieldtype, fieldname) + alignment = max(alignment, ctypes.alignment(fieldtype)) + + # try to enforce the same alignment as the one of the original structure + if alignment < info['align']: + choices = [ctype for ctype in alignment_types + if ctypes.alignment(ctype) == info['align']] + assert choices, "unsupported alignment %d" % (info['align'],) + choices = [(ctypes.sizeof(ctype), i, ctype) + for i, ctype in enumerate(choices)] + csize, _, ctype = min(choices) + for i in range(0, info['size'] - csize + 1, info['align']): + if layout[i:i+csize] == [None] * csize: + layout_addfield(layout, i, ctype, '_alignment') + break + else: + raise AssertionError("unenforceable alignment %d" % ( + info['align'],)) + + n = 0 + for i, cell in enumerate(layout): + if cell is not None: + continue + layout_addfield(layout, i, ctypes.c_char, '_pad%d' % (n,)) + n += 1 + + # build the ctypes Structure + seen = {} + fields = [] + for cell in layout: + if cell in seen: + continue + fields.append((cell.name, cell.ctype)) + seen[cell] = True + + class S(ctypes.Structure): + _fields_ = fields + if name.startswith('struct '): + name = name[7:] + S.__name__ = name + return S + + +if __name__ == '__main__': + doc = """Example: + + ctypes_platform.py -h sys/types.h -h netinet/in.h + 'struct sockaddr_in' + sin_port c_int + """ + import sys, getopt + opts, args = getopt.gnu_getopt(sys.argv[1:], 'h:') + if not args: + print >> sys.stderr, doc + sys.exit(2) + assert len(args) % 2 == 1 + headers = [] + for opt, value in opts: + if opt == '-h': + headers.append('#include <%s>' % (value,)) + name = args[0] + fields = [] + for i in range(1, len(args), 2): + ctype = getattr(ctypes, args[i+1]) + fields.append((args[i], ctype)) + + S = getstruct(name, '\n'.join(headers), fields) + + for key, value in S._fields_: + print key, value Added: pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Fri Apr 7 16:08:47 2006 @@ -0,0 +1,85 @@ +import py, sys, struct +from pypy.rpython.rctypes import ctypes_platform +import ctypes + + +if sys.platform != 'linux2': + py.test.skip("the test must be adapted to your platform") + + +def test_dirent(): + dirent = ctypes_platform.getstruct("struct dirent", + """ + struct dirent /* for this example only, not the exact dirent */ + { + long d_ino; + int d_off; + unsigned short d_reclen; + char d_name[32]; + }; + """, + [("d_reclen", ctypes.c_ushort)]) + assert issubclass(dirent, ctypes.Structure) + ssize = (ctypes.sizeof(ctypes.c_long) + + ctypes.sizeof(ctypes.c_int) + + ctypes.sizeof(ctypes.c_ushort) + + 32) + extra_padding = (-ssize) % ctypes.alignment(ctypes.c_long) + + assert dirent._fields_ == [('_alignment', ctypes.c_long), + ('_pad0', ctypes.c_char), + ('_pad1', ctypes.c_char), + ('_pad2', ctypes.c_char), + ('_pad3', ctypes.c_char), + ('d_reclen', ctypes.c_ushort), + ] + [ + ('_pad%d' % n, ctypes.c_char) + for n in range(4, 4+32+extra_padding)] + assert ctypes.sizeof(dirent) == ssize + extra_padding + assert ctypes.alignment(dirent) == ctypes.alignment(ctypes.c_long) + +def test_fit_type(): + S = ctypes_platform.getstruct("struct S", + """ + struct S { + signed char c; + unsigned char uc; + short s; + unsigned short us; + int i; + unsigned int ui; + long l; + unsigned long ul; + long long ll; + unsigned long long ull; + float f; + double d; + }; + """, + [("c", ctypes.c_int), + ("uc", ctypes.c_int), + ("s", ctypes.c_uint), + ("us", ctypes.c_int), + ("i", ctypes.c_int), + ("ui", ctypes.c_int), + ("l", ctypes.c_int), + ("ul", ctypes.c_int), + ("ll", ctypes.c_int), + ("ull", ctypes.c_int), + ("f", ctypes.c_double), + ("d", ctypes.c_float)]) + assert issubclass(S, ctypes.Structure) + fields = dict(S._fields_) + assert fields["c"] == ctypes.c_byte + assert fields["uc"] == ctypes.c_ubyte + assert fields["s"] == ctypes.c_short + assert fields["us"] == ctypes.c_ushort + assert fields["i"] == ctypes.c_int + assert fields["ui"] == ctypes.c_uint + assert fields["l"] == ctypes.c_long + assert fields["ul"] == ctypes.c_ulong + assert fields["ll"] == ctypes.c_longlong + assert fields["ull"] == ctypes.c_ulonglong + assert fields["f"] == ctypes.c_float + assert fields["d"] == ctypes.c_double + From auc at codespeak.net Fri Apr 7 16:15:09 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 7 Apr 2006 16:15:09 +0200 (CEST) Subject: [pypy-svn] r25501 - in pypy/dist/pypy/objspace: . constraint constraint/test Message-ID: <20060407141509.17DFB10285@code0.codespeak.net> Author: auc Date: Fri Apr 7 16:15:07 2006 New Revision: 25501 Modified: pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py pypy/dist/pypy/objspace/logic.py Log: intermediate progress on expressions Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Fri Apr 7 16:15:07 2006 @@ -32,7 +32,7 @@ """variables is a list of variables which appear in the formula""" W_Constraint.__init__(self, object_space) assert isinstance(w_variables, W_ListObject) - assert self._space.is_true(self._space.gt(self._space.len(w_variables), self._space.newint(1))) + assert self._space.is_true(self._space.ge(self._space.len(w_variables), self._space.newint(1))) self._names_to_vars = {} for var in w_variables.wrappeditems: self._names_to_vars[var.name] = var @@ -66,13 +66,14 @@ -def make_lambda_head(vars): - var_ids = ','.join([var.name for var in vars]) +def make_lambda_head(space, w_vars): + var_ids = ','.join([space.str_w(var.name) for var in w_vars.wrappeditems]) return 'lambda ' + var_ids + ':' -def expand_expr_template(expr, vars): - for var in vars: - expr.replace(var.name, var.name + '.val') +def expand_expr_template(space, w_expr, w_vars): + return space.str_w(w_expr) + for w_var in w_vars.wrappeditems: + expr.replace(w_var.name, w_var.name + '.val') return expr @@ -157,18 +158,13 @@ """A constraint represented as a python expression.""" _FILTER_CACHE = {} - def __init__(self, c_space, variables, formula, typ='fd.Expression'): + def __init__(self, o_space, w_variables, w_formula): """variables is a list of variables which appear in the formula formula is a python expression that will be evaluated as a boolean""" - self.formula = formula - self.type = typ - AbstractConstraint.__init__(self, c_space, variables) - try: - self.filterFunc = Expression._FILTER_CACHE[formula] - except KeyError: - self.filterFunc = eval(make_lambda_head(variables) \ - + expand_expr_template(formula, variables), {}, {}) - Expression._FILTER_CACHE[formula] = self.filterFunc + self.formula = w_formula + W_AbstractConstraint.__init__(self, o_space, w_variables) + self.filter_func = eval(make_lambda_head(self._space, w_variables) \ + + self._space.str_w(w_formula), {}, {}) def test_solution(self, sol ): """test a solution against this constraint @@ -178,27 +174,24 @@ args.append( sol[var.name] ) return self.filterFunc( *args ) - - def copy_to(self, space): - return self.__class__(space, self._variables, - self.formula, self.type) - def _init_result_cache(self): """key = (variable,value), value = [has_success,has_failure]""" result_cache = {} - for var_name in self._variables: - result_cache[var_name.name] = {} + for var_name in self._variables.wrappeditems: + result_cache[self._space.str_w(var_name.name)] = {} return result_cache def _assign_values(self, w_cs): variables = [] kwargs = {} - for variable in self._variables: + for variable in self._variables.wrappeditems: domain = w_cs.w_dom(variable) values = domain.w_get_values() - variables.append((domain.w_size(), [variable, values, 0, len(values)])) - kwargs[variable.name] = values[0] + variables.append((self._space.int_w(domain.w_size()), + [variable, values, 0, + self._space.len(values)])) + kwargs[self._space.str_w(variable.name)] = values.wrappeditems[0] # sort variables to instanciate those with fewer possible values first variables.sort() @@ -218,11 +211,11 @@ # it's over go_on = 0 - def revise(self, w_cs): + def w_revise(self, w_cs): """generic propagation algorithm for n-ary expressions""" assert isinstance(w_cs, W_ComputationSpace) maybe_entailed = 1 - ffunc = self.filterFunc + ffunc = self.filter_func result_cache = self._init_result_cache() for kwargs in self._assign_values(w_cs): if maybe_entailed: @@ -250,19 +243,14 @@ pass return maybe_entailed - - def __eq__(self, other): - if not super(Expression, self).__eq__(other): return False - r1 = self.formula == other.formula - r2 = self.type == other.type - return r1 and r2 def __repr__(self): return '<%s>' % self.formula W_Expression.typedef = typedef.TypeDef("W_Expression", - W_AbstractConstraint.typedef) + W_AbstractConstraint.typedef, + revise = interp2app(W_Expression.w_revise)) # completely unported @@ -279,7 +267,7 @@ def copy_to(self, space): raise NotImplementedError - def revise(self, domains): + def w_revise(self, domains): """specialized narrowing algorithm for binary expressions Runs much faster than the generic version""" maybe_entailed = 1 @@ -289,7 +277,7 @@ var2 = self._variables[1] dom2 = domains[var2] values2 = dom2.get_values() - ffunc = self.filterFunc + ffunc = self.filter_func if dom2.size() < dom1.size(): var1, var2 = var2, var1 dom1, dom2 = dom2, dom1 @@ -325,34 +313,17 @@ return maybe_entailed -def make_expression(variables, formula, constraint_type=None): +def make_expression(o_space, w_variables, w_formula): """create a new constraint of type Expression or BinaryExpression The chosen class depends on the number of variables in the constraint""" # encode unicode - vars = [] - for var in variables: - if type(var) == type(u''): - vars.append(var.encode()) - else: - vars.append(var) - if len(vars) == 2: - if constraint_type is not None: - return BinaryExpression(vars, formula, constraint_type) - else: - return BinaryExpression(vars, formula) - + if o_space.eq_w(o_space.len(w_variables), o_space.newint(2)): + return W_BinaryExpression(o_space, w_variables, w_formula) else: - if constraint_type is not None: - return Expression(vars, formula, constraint_type) - else: - return Expression(vars, formula) - - - - - + return W_Expression(o_space, w_variables, w_formula) +app_make_expression = gateway.interp2app(make_expression) # have a look at this later ... (really needed ?) class W_BasicConstraint(W_Constraint): Modified: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Fri Apr 7 16:15:07 2006 @@ -59,3 +59,23 @@ assert cstr.revise(csp) == 1 assert csp.dom(v10).get_values() == [2] + +class AppTest_Expression(object): + + def setup_class(cls): + cls.space = gettestobjspace('logic') + + def test_instantiate(self): + csp = newspace() + v1 = csp.var('v1', FiniteDomain([1, 2])) + cstr = make_expression([v1], '2*v1==2') + assert str(cstr).startswith(' Author: arigo Date: Fri Apr 7 16:21:23 2006 New Revision: 25502 Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Log: (arre, arigo) Detect the precise type of externally-declared types (mostly for numeric types). Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/ctypes_platform.py Fri Apr 7 16:21:23 2006 @@ -66,11 +66,28 @@ fieldname)) +C_HEADER = """ +#include +#include /* for offsetof() */ + +void dump(char* key, int value) { + printf("%s: %d\\n", key, value); +} +""" + +def run_example_code(filepath): + executable = build_executable([filepath]) + output = py.process.cmdexec(executable) + info = {} + for line in output.splitlines(): + key, value = line.strip().split(': ') + info[key] = int(value) + return info + def getstruct(name, c_header_source, interesting_fields): filepath = uniquefilepath() f = filepath.open('w') - print >> f, '#include ' - print >> f, '#include /* for offsetof() */' + print >> f, C_HEADER print >> f print >> f, c_header_source print >> f @@ -80,10 +97,6 @@ print >> f, ' ctypesplatcheck_t s;' print >> f, '} ctypesplatcheck2_t;' print >> f - print >> f, 'void dump(char* key, int value) {' - print >> f, ' printf("%s: %d\\n", key, value);' - print >> f, '}' - print >> f print >> f, 'int main(void) {' print >> f, ' ctypesplatcheck_t s;' print >> f, ' dump("align", offsetof(ctypesplatcheck2_t, s));' @@ -103,12 +116,7 @@ print >> f, '}' f.close() - executable = build_executable([filepath]) - output = py.process.cmdexec(executable) - info = {} - for line in output.splitlines(): - key, value = line.strip().split(': ') - info[key] = int(value) + info = run_example_code(filepath) alignment = 1 layout = [None] * info['size'] @@ -161,6 +169,34 @@ return S +def getsimpletype(name, c_header_source, ctype_hint): + filepath = uniquefilepath() + f = filepath.open('w') + print >> f, C_HEADER + print >> f + print >> f, c_header_source + print >> f + print >> f, 'typedef %s ctypesplatcheck_t;' % (name,) + print >> f + print >> f, 'int main(void) {' + print >> f, ' ctypesplatcheck_t x;' + print >> f, ' dump("size", sizeof(ctypesplatcheck_t));' + if ctype_hint in integer_class: + print >> f, ' x = 0; x = ~x;' + print >> f, ' dump("unsigned", x > 0);' + print >> f, ' return 0;' + print >> f, '}' + f.close() + + info = run_example_code(filepath) + + size = info['size'] + sign = info.get('unsigned', False) + if (size, sign) != size_and_sign(ctype_hint): + ctype_hint = fixup_ctype(ctype_hint, name, (size, sign)) + return ctype_hint + + if __name__ == '__main__': doc = """Example: Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Fri Apr 7 16:21:23 2006 @@ -83,3 +83,8 @@ assert fields["f"] == ctypes.c_float assert fields["d"] == ctypes.c_double +def test_simple_type(): + ctype = ctypes_platform.getsimpletype('test_t', + 'typedef unsigned short test_t;', + ctypes.c_int) + assert ctype == ctypes.c_ushort From cfbolz at codespeak.net Fri Apr 7 16:23:32 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 16:23:32 +0200 (CEST) Subject: [pypy-svn] r25503 - in pypy/dist/pypy: objspace/flow rpython/memory translator/c Message-ID: <20060407142332.854E91028A@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 16:23:29 2006 New Revision: 25503 Modified: pypy/dist/pypy/objspace/flow/model.py pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/c/funcgen.py Log: (pedronis, cfbolz): only inline gc helpers directly before code generation. do this in a copy of the graph which you can throw away afterwards for memory reasons. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Fri Apr 7 16:23:29 2006 @@ -474,12 +474,14 @@ lst.append(link) return result -def copygraph(graph): +def copygraph(graph, shallow=False): "Make a copy of a flow graph." blockmap = {} varmap = {} def copyvar(v): + if shallow: + return v if isinstance(v, Variable): try: return varmap[v] @@ -497,6 +499,8 @@ newblock.operations = () else: def copyoplist(oplist): + if shallow: + return oplist[:] result = [] for op in oplist: copyop = SpaceOperation(op.opname, Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 7 16:23:29 2006 @@ -85,15 +85,18 @@ v = Variable('vanishing_exc_value') v.concretetype = self.get_lltype_of_exception_value() graph.exc_cleanup = (v, self.pop_alive(v)) + + def inline_helpers(self, graph): if self.inline: from pypy.translator.backendopt import inline for inline_graph in self.graphs_to_inline: try: + # XXX quite inefficient: we go over the function lots of times inline.inline_function(self.translator, inline_graph, graph, self.lltype_to_classdef) except inline.CannotInline: pass - checkgraph(graph) + checkgraph(graph) def transform_block(self, block): newops = [] Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Apr 7 16:23:29 2006 @@ -3,7 +3,7 @@ from pypy.translator.c.support import cdecl, ErrorValue from pypy.translator.c.support import llvalue_from_constant, gen_assignments from pypy.objspace.flow.model import Variable, Constant, Block -from pypy.objspace.flow.model import c_last_exception +from pypy.objspace.flow.model import c_last_exception, copygraph from pypy.rpython.lltypesystem.lltype import Ptr, PyObject, Void, Bool, Signed from pypy.rpython.lltypesystem.lltype import Unsigned, SignedLongLong from pypy.rpython.lltypesystem.lltype import UnsignedLongLong, Char, UniChar @@ -13,6 +13,8 @@ PyObjPtr = Ptr(PyObject) LOCALVAR = 'l_%s' +KEEP_INLINED_GRAPHS = False + class FunctionCodeGenerator(object): """ Collects information about a function which we have to generate @@ -27,7 +29,8 @@ lltypes functionname currentblock - blocknum""".split() + blocknum + oldgraph""".split() def __init__(self, graph, db, cpython_exc=False, functionname=None): self.graph = graph @@ -41,8 +44,15 @@ # apply the gc transformation self.db.gctransformer.transform_graph(self.graph) #self.graph.show() - - self.blocknum = {} + self.collect_var_and_types() + + for v in self.vars: + T = getattr(v, 'concretetype', PyObjPtr) + db.gettype(T) # force the type to be considered by the database + + self.lltypes = None + + def collect_var_and_types(self): # # collect all variables and constants used in the body, # and get their types now @@ -51,8 +61,7 @@ # Constants may hash and compare equal but have different lltypes mix = [self.graph.getreturnvar()] self.more_ll_values = [] - for block in graph.iterblocks(): - self.blocknum[block] = len(self.blocknum) + for block in self.graph.iterblocks(): mix.extend(block.inputargs) for op in block.operations: mix.extend(op.args) @@ -64,8 +73,8 @@ self.more_ll_values.append(link.llexitcase) elif link.exitcase is not None: mix.append(Constant(link.exitcase)) - if cpython_exc: - v, exc_cleanup_ops = graph.exc_cleanup + if self.cpython_exc: + v, exc_cleanup_ops = self.graph.exc_cleanup mix.append(v) for cleanupop in exc_cleanup_ops: mix.extend(cleanupop.args) @@ -77,15 +86,23 @@ if id(v) not in seen: uniquemix.append(v) seen[id(v)] = True - T = getattr(v, 'concretetype', PyObjPtr) - db.gettype(T) # force the type to be considered by the database self.vars = uniquemix - self.lltypes = None def name(self, cname): #virtual return cname def implementation_begin(self): + if self.db.gctransformer.inline: + newgraph = copygraph(self.graph, shallow=True) + self.oldgraph = self.graph + self.graph = newgraph + self.db.gctransformer.inline_helpers(newgraph) + self.collect_var_and_types() + else: + self.oldgraph = self.graph + self.blocknum = {} + for block in self.graph.iterblocks(): + self.blocknum[block] = len(self.blocknum) db = self.db lltypes = {} for v in self.vars: @@ -96,6 +113,8 @@ def implementation_end(self): self.lltypes = None + self.graph = self.oldgraph + del self.oldgraph def argnames(self): return [LOCALVAR % v.name for v in self.graph.getargs()] From arigo at codespeak.net Fri Apr 7 16:38:02 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 16:38:02 +0200 (CEST) Subject: [pypy-svn] r25504 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060407143802.13D801028B@code0.codespeak.net> Author: arigo Date: Fri Apr 7 16:38:01 2006 New Revision: 25504 Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Log: (arre, arigo) Retrieve the value of integer constants. Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/ctypes_platform.py Fri Apr 7 16:38:01 2006 @@ -197,6 +197,31 @@ return ctype_hint +def getconstantinteger(name, c_header_source): + filepath = uniquefilepath() + f = filepath.open('w') + print >> f, C_HEADER + print >> f + print >> f, c_header_source + print >> f + print >> f, 'int main(void) {' + print >> f, ' if ((%s) < 0) {' % (name,) + print >> f, ' long long x = (long long)(%s);' % (name,) + print >> f, ' printf("value: %lld\\n", x);' + print >> f, ' } else {' + print >> f, ' unsigned long long x = (unsigned long long)(%s);' % ( + name,) + print >> f, ' printf("value: %llu\\n", x);' + print >> f, ' }' + print >> f, ' return 0;' + print >> f, '}' + f.close() + + info = run_example_code(filepath) + + return info['value'] + + if __name__ == '__main__': doc = """Example: Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Fri Apr 7 16:38:01 2006 @@ -88,3 +88,14 @@ 'typedef unsigned short test_t;', ctypes.c_int) assert ctype == ctypes.c_ushort + +def test_constant_integer(): + value = ctypes_platform.getconstantinteger('BLAH', + '#define BLAH (6*7)') + assert value == 42 + value = ctypes_platform.getconstantinteger('BLAH', + '#define BLAH (-2147483648LL)') + assert value == -2147483648 + value = ctypes_platform.getconstantinteger('BLAH', + '#define BLAH (3333333333ULL)') + assert value == 3333333333 From arigo at codespeak.net Fri Apr 7 16:54:15 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 16:54:15 +0200 (CEST) Subject: [pypy-svn] r25507 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060407145415.A94411028D@code0.codespeak.net> Author: arigo Date: Fri Apr 7 16:54:12 2006 New Revision: 25507 Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Log: (arre, arigo) Helper getdefined(). Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/ctypes_platform.py Fri Apr 7 16:54:12 2006 @@ -222,6 +222,11 @@ return info['value'] +def getdefined(name, c_header_source): + return getconstantinteger('\n#ifdef %s\n1\n#else\n0\n#endif\n' % (name), + c_header_source) + + if __name__ == '__main__': doc = """Example: Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Fri Apr 7 16:54:12 2006 @@ -3,10 +3,6 @@ import ctypes -if sys.platform != 'linux2': - py.test.skip("the test must be adapted to your platform") - - def test_dirent(): dirent = ctypes_platform.getstruct("struct dirent", """ @@ -99,3 +95,10 @@ value = ctypes_platform.getconstantinteger('BLAH', '#define BLAH (3333333333ULL)') assert value == 3333333333 + +def test_defined(): + res = ctypes_platform.getdefined('ALFKJLKJFLKJFKLEJDLKEWMECEE', '') + assert not res + res = ctypes_platform.getdefined('ALFKJLKJFLKJFKLEJDLKEWMECEE', + '#define ALFKJLKJFLKJFKLEJDLKEWMECEE') + assert res From cfbolz at codespeak.net Fri Apr 7 16:54:46 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 16:54:46 +0200 (CEST) Subject: [pypy-svn] r25508 - pypy/dist/pypy/translator/c Message-ID: <20060407145446.2389F10292@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 16:54:45 2006 New Revision: 25508 Modified: pypy/dist/pypy/translator/c/genc.py Log: (pedronis, cfbolz): split the implement file only when on windows to make gcc more deterministic Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Fri Apr 7 16:54:45 2006 @@ -296,7 +296,8 @@ def getothernodes(self): return self.othernodes[:] - def splitnodesimpl(self, basecname, nodes, nextra, nbetween): + def splitnodesimpl(self, basecname, nodes, nextra, nbetween, + split_criteria=SPLIT_CRITERIA): # produce a sequence of nodes, grouped into files # which have no more than SPLIT_CRITERIA lines used = nextra @@ -306,7 +307,7 @@ if not impl: continue cost = len(impl) + nbetween - if used + cost > SPLIT_CRITERIA and part: + if used + cost > split_criteria and part: # split if criteria met, unless we would produce nothing. yield self.uniquecname(basecname), part part = [] @@ -318,6 +319,10 @@ yield self.uniquecname(basecname), part def gen_readable_parts_of_source(self, f): + if py.std.sys.platform != "win32": + split_criteria_big = py.std.sys.maxint + else: + split_criteria_big = SPLIT_CRITERIA if self.one_source_file: return gen_readable_parts_of_main_c_file(f, self.database, self.preimpl) @@ -409,7 +414,8 @@ nextralines = 8 + len(self.preimpl) + 4 + 1 for name, nodesimpl in self.splitnodesimpl('implement.c', self.funcnodes, - nextralines, 1): + nextralines, 1, + split_criteria_big): print >> f, '/* %s */' % name fc = self.makefile(name) print >> fc, '/***********************************************************/' From antocuni at codespeak.net Fri Apr 7 17:14:55 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 7 Apr 2006 17:14:55 +0200 (CEST) Subject: [pypy-svn] r25509 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060407151455.28DE81012D@code0.codespeak.net> Author: antocuni Date: Fri Apr 7 17:14:44 2006 New Revision: 25509 Added: pypy/dist/pypy/translator/cli/rte.py (contents, props changed) pypy/dist/pypy/translator/cli/src/ pypy/dist/pypy/translator/cli/src/pypy.cs Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/test_op.py Log: Added support for calling unbound methods Added support for 'abs' operation in opcodes.py. Added support for an automatically compiled runtime environment written in C# (see rte.py) Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Fri Apr 7 17:14:44 2006 @@ -7,13 +7,14 @@ self.db = db self.classdef = classdef self.namespace, self.name = cts.split_class_name(classdef._name) + #0/0 def get_name(self): return self.name def get_base_class(self): base_class = self.classdef._superclass - if base_class is None or base_class._name == 'Object': + if base_class is None: return '[mscorlib]System.Object' else: return base_class._name @@ -25,13 +26,13 @@ ilasm.begin_class(self.name, self.get_base_class()) for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): - # TODO: handle default values ilasm.field(f_name, cts.lltype_to_cts(f_type)) + # TODO: should the .ctor set the default values? self._ctor() for m_name, m_meth in self.classdef._methods.iteritems(): - # TODO: handle class and unbound methods + # TODO: handle class methods and attributes f = Function(self.db, m_meth.graph, m_name, is_method = True) f.render(ilasm) Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Fri Apr 7 17:14:44 2006 @@ -42,6 +42,10 @@ out = Tee(sys.stdout, out) self.ilasm = IlasmGenerator(out, self.assembly_name) + + # TODO: instance methods that are also called as unbound + # methods are rendered twice, once within the class and once + # as an external function. Fix this. self.gen_entrypoint() self.find_superclasses() self.gen_classes() Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Fri Apr 7 17:14:44 2006 @@ -8,6 +8,9 @@ def _not(op): return [PushAllArgs, op]+Not +def _abs(type_): + return [PushAllArgs, 'call %s class [mscorlib]System.Math::Abs(%s)' % (type_, type_)] + opcodes = { # __________ object oriented operations __________ @@ -40,14 +43,13 @@ 'int_is_true': DoNothing, 'int_neg': 'neg', 'int_neg_ovf': ['ldc.i4.0', PushAllArgs, 'sub.ovf'], - 'int_abs': None, # TODO - 'int_abs_ovf': None, # TODO + 'int_abs': _abs('int32'), + 'int_abs_ovf': _abs('int32'), 'int_invert': 'not', 'int_add': 'add', 'int_sub': 'sub', 'int_mul': 'mul', -# 'int_div': 'div', 'int_floordiv': 'div', 'int_mod': 'rem', 'int_lt': 'clt', @@ -64,7 +66,6 @@ 'int_add_ovf': 'add.ovf', 'int_sub_ovf': 'sub.ovf', 'int_mul_ovf': 'mul.ovf', -# 'int_div_ovf': None, 'int_floordiv_ovf': 'div', # these can't overflow! 'int_mod_ovf': 'rem', 'int_lt_ovf': 'clt', @@ -87,7 +88,7 @@ 'uint_is_true': DoNothing, 'uint_neg': None, # What's the meaning? - 'uint_abs': None, # TODO + 'uint_abs': _abs('unsigned int32'), # TODO: ? 'uint_invert': 'not', 'uint_add': 'add', @@ -111,14 +112,12 @@ 'float_is_true': [PushAllArgs, 'ldc.r8 0', 'ceq']+Not, 'float_neg': 'neg', - 'float_abs': None, # TODO + 'float_abs': _abs('float64'), 'float_add': 'add', 'float_sub': 'sub', 'float_mul': 'mul', -# 'float_div': 'div', 'float_truediv': 'div', -# 'float_floordiv': None, # TODO 'float_mod': 'rem', 'float_lt': 'clt', 'float_le': _not('cgt'), @@ -131,7 +130,7 @@ 'llong_is_true': [PushAllArgs, 'ldc.i8 0', 'ceq']+Not, 'llong_neg': 'neg', - 'llong_abs': None, # TODO + 'llong_abs': _abs('int64'), 'llong_invert': 'not', 'llong_add': 'add', @@ -150,7 +149,7 @@ 'ullong_is_true': [PushAllArgs, 'ldc.i8 0', 'ceq']+Not, 'ullong_neg': None, - 'ullong_abs': None, # TODO + 'ullong_abs': _abs('unsigned int64'), 'ullong_invert': 'not', 'ullong_add': 'add', Added: pypy/dist/pypy/translator/cli/rte.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/rte.py Fri Apr 7 17:14:44 2006 @@ -0,0 +1,42 @@ +""" +Support for an automatically compiled Run Time Environment. +The source of the RTE is in the src/ directory. +""" + +import os +import os.path +import subprocess + +SRC = 'pypy.cs' +DLL = 'pypy.dll' + +def _filename(name): + rel_path = os.path.join(os.path.dirname(__file__), 'src/' + name) + return os.path.abspath(rel_path) + +def get_pypy_dll(): + source = _filename(SRC) + dll = _filename(DLL) + + recompile = True + try: + source_stat = os.stat(source) + dll_stat = os.stat(dll) + if source_stat.st_mtime <= dll_stat.st_mtime: + recompile = False + except OSError: + pass + + if recompile: + compile(source) + + return dll + +def compile(source): + compiler = subprocess.Popen(['mcs', '/t:library', source], + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = compiler.communicate() + retval = compiler.wait() + + assert retval == 0, 'Failed to compile %s: the compiler said:\n %s' % (DLL, stderr) + Added: pypy/dist/pypy/translator/cli/src/pypy.cs ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/src/pypy.cs Fri Apr 7 17:14:44 2006 @@ -0,0 +1,8 @@ +using System; + +namespace PyPy +{ + public class Foo + { + } +} Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Fri Apr 7 17:14:44 2006 @@ -26,16 +26,25 @@ def __init__(self, x): self.x = x + def compute(self): + return self.x + class Derived(Base): def __init__(self, x): Base.__init__(self, x) -def foo(x): - return x+1 + def compute(self): + return self.x+1 + +def foo(cls, arg): + obj = cls(arg) + return obj.compute() def bar(x, y): - a = Derived(x) - return a.x + obj = Derived(x) + return obj.compute() + #return foo(Base, x) + foo(Derived, y) + f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/test_op.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_op.py (original) +++ pypy/dist/pypy/translator/cli/test/test_op.py Fri Apr 7 17:14:44 2006 @@ -55,3 +55,5 @@ def op_any_operations(x, y): return (x*y) + (x-y) + (x/y) +def op_any_abs(x, y): + return abs(x) From arigo at codespeak.net Fri Apr 7 17:22:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 17:22:10 +0200 (CEST) Subject: [pypy-svn] r25510 - in pypy/dist/pypy: . jit/tl/test rpython/l3interp/test translator/c/test translator/pyrex/test translator/test translator/tool Message-ID: <20060407152210.D638E10290@code0.codespeak.net> Author: arigo Date: Fri Apr 7 17:22:06 2006 New Revision: 25510 Modified: pypy/dist/pypy/conftest.py pypy/dist/pypy/jit/tl/test/test_tl.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py pypy/dist/pypy/translator/c/test/test_boehm.py pypy/dist/pypy/translator/c/test/test_exceptiontransform.py pypy/dist/pypy/translator/c/test/test_newgc.py pypy/dist/pypy/translator/c/test/test_typed.py pypy/dist/pypy/translator/pyrex/test/test_pyrextrans.py pypy/dist/pypy/translator/test/test_geninterp.py pypy/dist/pypy/translator/tool/cbuild.py Log: (arre, arigo) Removed skip_missing_compiler(). Test for this situation in conftest instead. Modified: pypy/dist/pypy/conftest.py ============================================================================== --- pypy/dist/pypy/conftest.py (original) +++ pypy/dist/pypy/conftest.py Fri Apr 7 17:22:06 2006 @@ -180,6 +180,15 @@ except OperationError, e: check_keyboard_interrupt(e) raise + except Exception, e: + cls = e.__class__ + while cls is not Exception: + if cls.__name__ == 'DistutilsPlatformError': + from distutils.errors import DistutilsPlatformError + if isinstance(e, DistutilsPlatformError): + py.test.skip('%s: %s' % (e.__class__.__name__, e)) + cls = cls.__bases__[0] + raise if 'pygame' in sys.modules: global _pygame_imported if not _pygame_imported: Modified: pypy/dist/pypy/jit/tl/test/test_tl.py ============================================================================== --- pypy/dist/pypy/jit/tl/test/test_tl.py (original) +++ pypy/dist/pypy/jit/tl/test/test_tl.py Fri Apr 7 17:22:06 2006 @@ -13,11 +13,10 @@ t.buildannotator(policy=pol).build_types(func, inputargs) t.buildrtyper().specialize() - from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.c import genc builder = genc.CExtModuleBuilder(t, func) builder.generate_source() - skip_missing_compiler(builder.compile) + builder.compile() builder.import_module() return builder.get_entry_point() Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Fri Apr 7 17:22:06 2006 @@ -24,11 +24,10 @@ if conftest.option.view: t.view() - from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.c import genc builder = genc.CExtModuleBuilder(t, func, gcpolicy=BoehmGcPolicy) builder.generate_source() - skip_missing_compiler(builder.compile) + builder.compile() builder.isolated_import() _cleanups.append(builder.cleanup) return builder.get_entry_point() Modified: pypy/dist/pypy/translator/c/test/test_boehm.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_boehm.py (original) +++ pypy/dist/pypy/translator/c/test/test_boehm.py Fri Apr 7 17:22:06 2006 @@ -1,6 +1,6 @@ import py from pypy.translator.translator import TranslationContext -from pypy.translator.tool.cbuild import skip_missing_compiler, check_boehm_presence +from pypy.translator.tool.cbuild import check_boehm_presence from pypy.translator.c.genc import CExtModuleBuilder from pypy import conftest @@ -39,7 +39,7 @@ mod = cbuilder.isolated_import() self._cleanups.append(cbuilder.cleanup) # schedule cleanup after test return cbuilder.get_entry_point() - return skip_missing_compiler(compile) + return compile() class TestUsingBoehm(AbstractTestClass): Modified: pypy/dist/pypy/translator/c/test/test_exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/test/test_exceptiontransform.py Fri Apr 7 17:22:06 2006 @@ -4,7 +4,6 @@ from pypy.translator.c import exceptiontransform, genc, gc from pypy.objspace.flow.model import c_last_exception from pypy.rpython.test.test_llinterp import get_interpreter -from pypy.translator.tool.cbuild import skip_missing_compiler from pypy import conftest import sys @@ -48,7 +47,7 @@ ## etrafo.transform_completely() builder = genc.CExtModuleBuilder(t, fn, gcpolicy=gc.RefcountingGcPolicy) builder.generate_source() - skip_missing_compiler(builder.compile) + builder.compile() builder.import_module() if conftest.option.view: t.view() Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Fri Apr 7 17:22:06 2006 @@ -3,7 +3,6 @@ import py from py.test import raises -from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.translator import TranslationContext from pypy.translator.c import genc, gc from pypy.rpython.lltypesystem import lltype @@ -18,7 +17,7 @@ t.buildrtyper().specialize() builder = genc.CExtModuleBuilder(t, fn, gcpolicy=gc.RefcountingGcPolicy) builder.generate_source() - skip_missing_compiler(builder.compile) + builder.compile() builder.import_module() if conftest.option.view: t.view() Modified: pypy/dist/pypy/translator/c/test/test_typed.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_typed.py (original) +++ pypy/dist/pypy/translator/c/test/test_typed.py Fri Apr 7 17:22:06 2006 @@ -6,7 +6,6 @@ from pypy import conftest from pypy.translator.test import snippet -from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.translator import TranslationContext from pypy.rpython.rarithmetic import r_uint, r_ulonglong, r_longlong, intmask @@ -34,7 +33,7 @@ from pypy.translator.c import genc builder = genc.CExtModuleBuilder(t, func) builder.generate_source() - skip_missing_compiler(builder.compile) + builder.compile() builder.import_module() return builder.get_entry_point() Modified: pypy/dist/pypy/translator/pyrex/test/test_pyrextrans.py ============================================================================== --- pypy/dist/pypy/translator/pyrex/test/test_pyrextrans.py (original) +++ pypy/dist/pypy/translator/pyrex/test/test_pyrextrans.py Fri Apr 7 17:22:06 2006 @@ -5,7 +5,6 @@ from pypy.objspace.flow.model import * from pypy.translator.tool.cbuild import build_cfunc from pypy.translator.tool.cbuild import make_module_from_pyxstring -from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.translator import TranslationContext from pypy.objspace.flow import FlowObjSpace @@ -32,7 +31,7 @@ 'simplify' : 1, 'dot' : dot, } - return skip_missing_compiler(build_cfunc, func, **options) + return build_cfunc(func, **options) def test_simple_func(self): cfunc = self.build_cfunc(snippet.simple_func) @@ -136,8 +135,7 @@ blobs.insert(0, code) pyxcode = '\n\n#_________________\n\n'.join(blobs) - mod = skip_missing_compiler( - make_module_from_pyxstring, name, udir, pyxcode) + mod = make_module_from_pyxstring(name, udir, pyxcode) return getattr(mod, name) def test_set_attr(self): Modified: pypy/dist/pypy/translator/test/test_geninterp.py ============================================================================== --- pypy/dist/pypy/translator/test/test_geninterp.py (original) +++ pypy/dist/pypy/translator/test/test_geninterp.py Fri Apr 7 17:22:06 2006 @@ -28,7 +28,6 @@ from pypy.tool.udir import udir from pypy.objspace.flow.model import * from pypy.translator.tool.cbuild import make_module_from_c -from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.geninterplevel import translate_as_module from pypy.translator.test import snippet from pypy.interpreter.error import OperationError Modified: pypy/dist/pypy/translator/tool/cbuild.py ============================================================================== --- pypy/dist/pypy/translator/tool/cbuild.py (original) +++ pypy/dist/pypy/translator/tool/cbuild.py Fri Apr 7 17:22:06 2006 @@ -188,13 +188,6 @@ cfile = pyxfile.new(ext='.c') return cfile -def skip_missing_compiler(fn, *args, **kwds): - from distutils.errors import DistutilsPlatformError - try: - return fn(*args, **kwds) - except DistutilsPlatformError, e: - py.test.skip('DistutilsPlatformError: %s' % (e,)) - def build_cfunc(func, simplify=1, dot=1, inputargtypes=None): """ return a pyrex-generated cfunction from the given func. From dialtone at codespeak.net Fri Apr 7 18:07:47 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Fri, 7 Apr 2006 18:07:47 +0200 (CEST) Subject: [pypy-svn] r25512 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060407160747.AA15610293@code0.codespeak.net> Author: dialtone Date: Fri Apr 7 18:07:46 2006 New Revision: 25512 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: update my info on tokyo sprint Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Fri Apr 7 18:07:46 2006 @@ -8,14 +8,14 @@ ==================== ============== ===================== Name Arrive/Depart Accomodation ==================== ============== ===================== -Beatrice D?ring 21/4 - 25/4 +Beatrice D??ring 21/4 - 25/4 Christian Tismer 22/4 - 30/4 Eric van Riet Paap 18/4 - 1/5 ? -Jacob Hall?n +Jacob Hall??n Laura Creighton Anders Lehmann 23/4 - 1/5 ? Niklaus Haldimann 20/4 - 1/5 ? -Anders Chrigstr?m 22/4 - 30/4 +Anders Chrigstr??m 22/4 - 30/4 Samuele Pedroni 22/4 - 30/4 ==================== ============== ===================== @@ -31,7 +31,7 @@ Lene Wagner Johan Hahn Amaury Forgeot d'Arc -Valentino Volonghi +Valentino Volonghi 22/4 - 1/5 Hotel Edoya (Ueno) Boris Feigin Bert Freudenberg ==================== ============== ===================== From mwh at codespeak.net Fri Apr 7 18:58:24 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 7 Apr 2006 18:58:24 +0200 (CEST) Subject: [pypy-svn] r25513 - in pypy/branch/stacklesscfg/pypy/translator/c: . test Message-ID: <20060407165824.C135A10282@code0.codespeak.net> Author: mwh Date: Fri Apr 7 18:58:23 2006 New Revision: 25513 Added: pypy/branch/stacklesscfg/pypy/translator/c/stacklesscode.py (contents, props changed) Modified: pypy/branch/stacklesscfg/pypy/translator/c/stacklesstransform.py pypy/branch/stacklesscfg/pypy/translator/c/test/test_stacklesstransform.py Log: (hpk, mwh) a start at an RPython main loop for stackless. a start at a transform to insert state saving blocks into a function more implicit XXXs than you could possibly imagine Added: pypy/branch/stacklesscfg/pypy/translator/c/stacklesscode.py ============================================================================== --- (empty file) +++ pypy/branch/stacklesscfg/pypy/translator/c/stacklesscode.py Fri Apr 7 18:58:23 2006 @@ -0,0 +1,87 @@ +from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython import rarithmetic + +STATE_HEADER = lltype.Struct('state_header', + ('f_back', lltype.Ptr(lltype.ForwardReference())), + ('state', lltype.Signed)) +STATE_HEADER.f_back.TO.become(STATE_HEADER) + +null_state = lltype.nullptr(STATE_HEADER) + +class StacklessData: + def __init__(self): + self.top = null_state + self.bottom = null_state + self.restart_substate = 0 + self.retval_long = 0 + self.retval_longlong = rarithmetic.r_longlong(0) + self.retval_double = 0.0 + self.retval_void_p = llmemory.fakeaddress(None) + self.exception = None + +global_state = StacklessData() + +class UnwindException(Exception): + def __init__(self): + self.frame = null_state + +void_void_func = lltype.FuncType([], lltype.Void) +long_void_func = lltype.FuncType([], lltype.Signed) +longlong_void_func = lltype.FuncType([], lltype.SignedLongLong) +pointer_void_func = lltype.FuncType([], llmemory.Address) +float_void_func = lltype.FuncType([], lltype.Float) + +def call_function(fn, signature): + pass +## if signature == 'void': +## fn2 = llmemory.cast_adr_to_ptr(fn, void_void_func) +## fn2() +## elif signature == 'long': +## fn3 = llmemory.cast_adr_to_ptr(fn, long_void_func) +## global_state.long_retval = fn3() +## elif signature == 'long long': +## fn4 = llmemory.cast_adr_to_ptr(fn, longlong_void_func) +## global_state.longlong_retval = fn4() +## elif signature == 'pointer': +## fn5 = llmemory.cast_adr_to_ptr(fn, pointer_void_func) +## global_state.pointer_retval = fn5() +## elif signature == 'float': +## fn6 = llmemory.cast_adr_to_ptr(fn, float_void_func) +## global_state.float_retval = fn6() + + +null_address = llmemory.fakeaddress(None) + +def decode_state(state): + return null_address, 'void', 0 + +def slp_main_loop(): + while 1: + pending = global_state.top + while 1: + back = pending.f_back + + state = pending.state + fn, signature, global_state.restart_substate = decode_state(state) + + try: + call_function(fn, signature) +## except UnwindException, u: +## pending = u.frame +## break + except Exception, e: + global_state.exception = e + else: + global_state.exception = None + + if not back: + if global_state.exception: + raise global_state.exception + return + + pending = back + global_state.top = pending + + if global_state.bottom: + assert global_state.bottom.f_back is None + global_state.bottom.f_back = back Modified: pypy/branch/stacklesscfg/pypy/translator/c/stacklesstransform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/c/stacklesstransform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/c/stacklesstransform.py Fri Apr 7 18:58:23 2006 @@ -1,5 +1,12 @@ from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython import rarithmetic +from pypy.translator.backendopt import support +from pypy.objspace.flow import model +from pypy.rpython.memory.gctransform import varoftype +from pypy.translator.unsimplify import copyvar +from pypy.annotation import model as annmodel +from pypy.rpython.annlowlevel import MixLevelHelperAnnotator +from pypy.translator.c import stacklesscode STORAGE_TYPES = [llmemory.Address, lltype.Signed, @@ -11,7 +18,7 @@ 'longlong'] def storage_type(T): - """Return the type used to save values of this type + """Return the index into STORAGE_TYPES """ if T is lltype.Void: return None @@ -26,28 +33,64 @@ else: raise Exception("don't know about %r" % (T,)) + + STATE_HEADER = lltype.Struct('state_header', ('f_back', lltype.Ptr(lltype.ForwardReference())), ('state', lltype.Signed)) STATE_HEADER.f_back.TO.become(STATE_HEADER) +null_state = lltype.nullptr(STATE_HEADER) + +## def func(x): +## return g() + x + 1 + +## STATE_func_0 = lltype.Struct('STATE_func_0', +## ('header', STATE_HEADER), +## ('saved_long_0', Signed)) + +## def func(x): +## if global_state.top: +## if global_state.restart_substate == 0: +## frame = cast_pointer(lltype.Ptr(STATE_func_0), global_state.top) +## x = frame.saved_long_0 +## retval = global_state.long_retval +## else: +## abort() +## else: +## try: +## retval = g(x) +## except UnwindException, u: +## state = lltype.raw_malloc(STATE_func_0) +## state.saved_long_0 = x +## state.header.f_back = u.frame +## state.header.state = XXX +## u.frame = state.header +## raise + +## return retval + x + 1 + + class StacklessTransfomer(object): def __init__(self, translator): self.translator = translator - null_frame_p = lltype.nullptr(STATE_HEADER) - - class StacklessData: - def __init__(self): - top = null_frame_p - bottom = null_frame_p - restart_substate = 0 - retval_long = 0 - retval_longlong = rarithmetic.r_longlong(0) - retval_double = 0.0 - retval_void_p = llmemory.fakeaddress(None) - + edata = translator.rtyper.getexceptiondata() self.frametypes = {} + self.curr_graph = None + + mixlevelannotator = MixLevelHelperAnnotator(translator.rtyper) + l2a = annmodel.lltype_to_annotation + + slp_main_loop_graph = mixlevelannotator.getgraph( + stacklesscode.slp_main_loop, [], l2a(lltype.Void)) + SLP_MAIN_LOOP_TYPE = lltype.FuncType([], lltype.Void) + self.slp_main_loop_type_ptr = model.Constant(lltype.functionptr( + SLP_MAIN_LOOP_TYPE, "slp_main_loop", + graph=slp_main_loop_graph), + SLP_MAIN_LOOP_TYPE) + + mixlevelannotator.finish() def frame_type_for_vars(self, vars): types = [storage_type(v.concretetype) for v in vars] @@ -71,6 +114,105 @@ return T def transform_graph(self, graph): + self.resume_points = [] + + assert self.curr_graph is None + self.curr_graph = graph + for block in graph.iterblocks(): - pass + self.transform_block(block) + if self.resume_points: + XXX + + self.curr_graph = None + + def transform_block(self, block): + i = 0 + + edata = self.translator.rtyper.getexceptiondata() + etype = edata.lltype_of_exception_type + evalue = edata.lltype_of_exception_value + + while i < len(block.operations): + op = block.operations[i] + if op.opname in ('direct_call', 'indirect_call'): + after_block = support.split_block_with_keepalive(self.translator, self.curr_graph, block, i+1) + link = block.exits[0] + + var_unwind_exception = varoftype(evalue) + + save_block = self.generate_save_block(link.args, var_unwind_exception) + + newlink = model.Link(link.args + [var_unwind_exception], save_block, model.Constant(SystemExit)) + block.exitswitch = model.c_last_exception + block.exits.append(newlink) + block = after_block + i = 0 + else: + i += 1 + + def generate_save_block(self, varstosave, var_unwind_exception): + edata = self.translator.rtyper.getexceptiondata() + etype = edata.lltype_of_exception_type + evalue = edata.lltype_of_exception_value + inputargs = [copyvar(self.translator, v) for v in varstosave] + var_unwind_exception0 = copyvar(self.translator, var_unwind_exception) + from pypy.rpython.rclass import getinstancerepr + var_unwind_exception = varoftype(getinstancerepr(self.translator.rtyper, + self.translator.annotator.bookkeeper.getuniqueclassdef(SystemExit)).lowleveltype) + + fields = [] + for i, v in enumerate(varstosave): + if v.concretetype is not lltype.Void: + fields.append(('field_%d'%(i,), v.concretetype)) + frame_type = lltype.Struct("S", + ('header', STATE_HEADER), + *fields) + + + save_state_block = model.Block(inputargs + [var_unwind_exception]) + saveops = save_state_block.operations + frame_state_var = varoftype(lltype.Ptr(frame_type)) + + saveops.append(model.SpaceOperation('malloc', + [model.Constant(frame_type, lltype.Void)], + frame_state_var)) + + saveops.extend(self.generate_saveops(frame_state_var, inputargs)) + +## state.header.f_back = u.frame +## state.header.state = XXX +## u.frame = state.header + header_var = varoftype(lltype.Ptr(STATE_HEADER)) + saveops.append(model.SpaceOperation("cast_pointer", [frame_state_var], header_var)) + var_unwind_exception_frame = varoftype(lltype.Ptr(STATE_HEADER)) + saveops.append(model.SpaceOperation("getfield", + [var_unwind_exception, model.Constant("frame", lltype.Void)], + var_unwind_exception_frame)) + saveops.append(model.SpaceOperation("setfield", + [header_var, model.Constant("f_back", lltype.Void), var_unwind_exception_frame], + varoftype(lltype.Void))) + saveops.append(model.SpaceOperation("setfield", + [var_unwind_exception, model.Constant("frame", lltype.Void)], + varoftype(lltype.Void))) + + save_state_block.closeblock(model.Link([varoftype(etype), varoftype(evalue)], + self.curr_graph.exceptblock)) + + return save_state_block + + + def generate_saveops(self, frame_state_var, varstosave): + frame_type = frame_state_var.concretetype.TO + ops = [] + for i, var in enumerate(varstosave): + t = storage_type(var.concretetype) + fname = model.Constant(frame_type._names[i+1], lltype.Void) + ops.append(model.SpaceOperation('setfield', + [frame_state_var, fname, var], + varoftype(lltype.Void))) + + return ops + + Modified: pypy/branch/stacklesscfg/pypy/translator/c/test/test_stacklesstransform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/c/test/test_stacklesstransform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/c/test/test_stacklesstransform.py Fri Apr 7 18:58:23 2006 @@ -1,23 +1,40 @@ from pypy.translator.c.stacklesstransform import StacklessTransfomer from pypy.rpython.memory.gctransform import varoftype from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.translator.translator import TranslationContext, graphof +from pypy import conftest -def test_frame_types(): - st = StacklessTransfomer(None) +## def test_frame_types(): +## st = StacklessTransfomer(None) - signed = varoftype(lltype.Signed) - ptr = varoftype(lltype.Ptr(lltype.GcStruct("S"))) - addr = varoftype(llmemory.Address) - float = varoftype(lltype.Float) - longlong = varoftype(lltype.SignedLongLong) +## signed = varoftype(lltype.Signed) +## ptr = varoftype(lltype.Ptr(lltype.GcStruct("S"))) +## addr = varoftype(llmemory.Address) +## float = varoftype(lltype.Float) +## longlong = varoftype(lltype.SignedLongLong) - ft4vars = st.frame_type_for_vars +## ft4vars = st.frame_type_for_vars - s1 = ft4vars([signed]) - assert 'header' in s1._flds - assert len(s1._flds) == 2 +## s1 = ft4vars([signed]) +## assert 'header' in s1._flds +## assert len(s1._flds) == 2 - s2_1 = ft4vars([signed, ptr]) - s2_2 = ft4vars([ptr, signed]) +## s2_1 = ft4vars([signed, ptr]) +## s2_2 = ft4vars([ptr, signed]) - assert s2_1 is s2_2 +## assert s2_1 is s2_2 + +def test_simple_transform(): + def g(): + return 2 + def example(x): + return g() + x + 1 + t = TranslationContext() + t.buildannotator().build_types(example, [int]) + t.buildrtyper().specialize() + example_graph = graphof(t, example) + st = StacklessTransfomer(t) + st.transform_graph(example_graph) + if conftest.option.view: + t.view() + From ac at codespeak.net Fri Apr 7 19:03:48 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 7 Apr 2006 19:03:48 +0200 (CEST) Subject: [pypy-svn] r25514 - in pypy/dist/pypy: interpreter interpreter/astcompiler tool Message-ID: <20060407170348.1967B10295@code0.codespeak.net> Author: ac Date: Fri Apr 7 19:03:47 2006 New Revision: 25514 Added: pypy/dist/pypy/tool/stdlib___future__.py - copied unchanged from r25373, pypy/dist/pypy/tool/__future__.py pypy/dist/pypy/tool/stdlib_opcode.py - copied unchanged from r25373, pypy/dist/pypy/tool/opcode.py Removed: pypy/dist/pypy/tool/__future__.py pypy/dist/pypy/tool/opcode.py Modified: pypy/dist/pypy/interpreter/astcompiler/pyassem.py pypy/dist/pypy/interpreter/pycompiler.py pypy/dist/pypy/interpreter/pyopcode.py Log: Move pypy/tool/{__future__,opcode}.py to pypy/tool/stdlib_{__future__,opcode}.py to avoid nameclashes. Modified: pypy/dist/pypy/interpreter/astcompiler/pyassem.py ============================================================================== --- pypy/dist/pypy/interpreter/astcompiler/pyassem.py (original) +++ pypy/dist/pypy/interpreter/astcompiler/pyassem.py Fri Apr 7 19:03:47 2006 @@ -7,7 +7,7 @@ import CO_OPTIMIZED, CO_NEWLOCALS, CO_VARARGS, CO_VARKEYWORDS from pypy.interpreter.pycode import PyCode from pypy.interpreter.baseobjspace import W_Root -from pypy.tool import opcode as pythonopcode +from pypy.tool import stdlib_opcode as pythonopcode class BlockSet: """A Set implementation specific to Blocks Modified: pypy/dist/pypy/interpreter/pycompiler.py ============================================================================== --- pypy/dist/pypy/interpreter/pycompiler.py (original) +++ pypy/dist/pypy/interpreter/pycompiler.py Fri Apr 7 19:03:47 2006 @@ -78,11 +78,11 @@ # faked compiler import warnings -from pypy.tool import __future__ +from pypy.tool import stdlib___future__ compiler_flags = 0 compiler_features = {} -for fname in __future__.all_feature_names: - flag = getattr(__future__, fname).compiler_flag +for fname in stdlib___future__.all_feature_names: + flag = getattr(stdlib___future__, fname).compiler_flag compiler_flags |= flag compiler_features[fname] = flag allowed_flags = compiler_flags | PyCF_DONT_IMPLY_DEDENT @@ -113,7 +113,7 @@ """Faked implementation of a compiler, using the underlying compile().""" def compile(self, source, filename, mode, flags): - flags |= __future__.generators.compiler_flag # always on (2.2 compat) + flags |= stdlib___future__.generators.compiler_flag # always on (2.2 compat) space = self.space try: old = self.setup_warn_explicit(warnings) @@ -208,7 +208,7 @@ from pyparser.pythonutil import AstBuilder, PYTHON_PARSER, TARGET_DICT from pypy.interpreter.pycode import PyCode - flags |= __future__.generators.compiler_flag # always on (2.2 compat) + flags |= stdlib___future__.generators.compiler_flag # always on (2.2 compat) space = self.space try: builder = AstBuilder(space=space) Modified: pypy/dist/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/dist/pypy/interpreter/pyopcode.py (original) +++ pypy/dist/pypy/interpreter/pyopcode.py Fri Apr 7 19:03:47 2006 @@ -14,7 +14,7 @@ from pypy.tool.sourcetools import func_with_new_name from pypy.rpython.objectmodel import we_are_translated from pypy.rpython.rarithmetic import intmask -from pypy.tool import opcode as pythonopcode +from pypy.tool import stdlib_opcode as pythonopcode def unaryoperation(operationname): """NOT_RPYTHON""" From hpk at codespeak.net Fri Apr 7 19:13:06 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 7 Apr 2006 19:13:06 +0200 (CEST) Subject: [pypy-svn] r25515 - in pypy/branch/stacklesscfg/pypy/translator: c c/test stackless stackless/test Message-ID: <20060407171306.86CCC10164@code0.codespeak.net> Author: hpk Date: Fri Apr 7 19:13:04 2006 New Revision: 25515 Added: pypy/branch/stacklesscfg/pypy/translator/stackless/ pypy/branch/stacklesscfg/pypy/translator/stackless/__init__.py (contents, props changed) pypy/branch/stacklesscfg/pypy/translator/stackless/code.py - copied unchanged from r25514, pypy/branch/stacklesscfg/pypy/translator/c/stacklesscode.py pypy/branch/stacklesscfg/pypy/translator/stackless/test/ pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py - copied, changed from r25514, pypy/branch/stacklesscfg/pypy/translator/c/test/test_stacklesstransform.py pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py - copied, changed from r25514, pypy/branch/stacklesscfg/pypy/translator/c/stacklesstransform.py Removed: pypy/branch/stacklesscfg/pypy/translator/c/stacklesscode.py pypy/branch/stacklesscfg/pypy/translator/c/stacklesstransform.py pypy/branch/stacklesscfg/pypy/translator/c/test/test_stacklesstransform.py Log: (mwh, hpk) move intermediate stackless transform code to a c-backend independent place (translator/stackless) Added: pypy/branch/stacklesscfg/pypy/translator/stackless/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/__init__.py Fri Apr 7 19:13:04 2006 @@ -0,0 +1 @@ +# From cfbolz at codespeak.net Fri Apr 7 19:22:46 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 19:22:46 +0200 (CEST) Subject: [pypy-svn] r25516 - pypy/dist/pypy/translator/c Message-ID: <20060407172246.F1DB11024F@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 19:22:45 2006 New Revision: 25516 Modified: pypy/dist/pypy/translator/c/genc.py Log: (pedronis, cfbolz): wuargh! strange, strange code: genc is nicely lazy everywhere, and in the end we just put the results from the generators into a list?! This checkin fixes this. Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Fri Apr 7 19:22:45 2006 @@ -300,23 +300,23 @@ split_criteria=SPLIT_CRITERIA): # produce a sequence of nodes, grouped into files # which have no more than SPLIT_CRITERIA lines - used = nextra - part = [] - for node in nodes: - impl = '\n'.join(list(node.implementation())).split('\n') - if not impl: - continue - cost = len(impl) + nbetween - if used + cost > split_criteria and part: - # split if criteria met, unless we would produce nothing. - yield self.uniquecname(basecname), part - part = [] - used = nextra - part.append( (node, impl) ) - used += cost - # generate left pieces - if part: - yield self.uniquecname(basecname), part + iternodes = iter(nodes) + done = [False] + def subiter(): + used = nextra + for node in iternodes: + impl = '\n'.join(list(node.implementation())).split('\n') + if not impl: + continue + cost = len(impl) + nbetween + yield node, impl + if used + cost > split_criteria: + # split if criteria met, unless we would produce nothing. + raise StopIteration + used += cost + done[0] = True + while not done[0]: + yield self.uniquecname(basecname), subiter() def gen_readable_parts_of_source(self, f): if py.std.sys.platform != "win32": @@ -389,7 +389,7 @@ fc.close() nextralines = 11 + 1 - for name, nodesimpl in self.splitnodesimpl('nonfuncnodes.c', + for name, nodeiter in self.splitnodesimpl('nonfuncnodes.c', self.othernodes, nextralines, 1): print >> f, '/* %s */' % name @@ -405,14 +405,14 @@ print >> fc, '#include "src/g_include.h"' print >> fc print >> fc, MARKER - for node, impl in nodesimpl: + for node, impl in nodeiter: print >> fc, '\n'.join(impl) print >> fc, MARKER print >> fc, '/***********************************************************/' fc.close() nextralines = 8 + len(self.preimpl) + 4 + 1 - for name, nodesimpl in self.splitnodesimpl('implement.c', + for name, nodeiter in self.splitnodesimpl('implement.c', self.funcnodes, nextralines, 1, split_criteria_big): @@ -432,7 +432,7 @@ print >> fc, '#include "src/g_include.h"' print >> fc print >> fc, MARKER - for node, impl in nodesimpl: + for node, impl in nodeiter: print >> fc, '\n'.join(impl) print >> fc, MARKER print >> fc, '/***********************************************************/' From arigo at codespeak.net Fri Apr 7 19:22:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 19:22:59 +0200 (CEST) Subject: [pypy-svn] r25517 - in pypy/dist/pypy: rpython translator/c translator/c/test Message-ID: <20060407172259.895AE1029B@code0.codespeak.net> Author: arigo Date: Fri Apr 7 19:22:58 2006 New Revision: 25517 Modified: pypy/dist/pypy/rpython/objectmodel.py pypy/dist/pypy/translator/c/primitive.py pypy/dist/pypy/translator/c/test/test_symbolic.py Log: (arre, arigo) Support for lazily-computed integers during the translation. The real value is only computed during code generation. Modified: pypy/dist/pypy/rpython/objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/objectmodel.py (original) +++ pypy/dist/pypy/rpython/objectmodel.py Fri Apr 7 19:22:58 2006 @@ -11,6 +11,19 @@ def lltype(self): return None +class ComputedIntSymbolic(Symbolic): + + def __init__(self, compute_fn): + self.compute_fn = compute_fn + + def annotation(self): + from pypy.annotation import model + return model.SomeInteger() + + def lltype(self): + from pypy.rpython.lltypesystem import lltype + return lltype.Signed + import new import weakref Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Fri Apr 7 19:22:58 2006 @@ -1,4 +1,5 @@ import sys +from pypy.rpython.objectmodel import Symbolic, ComputedIntSymbolic from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.lltypesystem.llmemory import Address, fakeaddress, \ AddressOffset, ItemOffset, ArrayItemsOffset, FieldOffset, \ @@ -35,6 +36,8 @@ return '0' elif type(value) == REFCOUNT_IMMORTAL: return 'REFCOUNT_IMMORTAL' + elif isinstance(value, ComputedIntSymbolic): + value = value.compute_fn() else: raise Exception("unimplemented symbolic %r"%value) if value == -sys.maxint-1: # blame C Modified: pypy/dist/pypy/translator/c/test/test_symbolic.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_symbolic.py (original) +++ pypy/dist/pypy/translator/c/test/test_symbolic.py Fri Apr 7 19:22:58 2006 @@ -2,6 +2,7 @@ from pypy import conftest from pypy.rpython.lltypesystem import llmemory, lltype from pypy.rpython.memory import lladdress +from pypy.rpython.objectmodel import ComputedIntSymbolic def getcompiled(f, args): t = Translation(f) @@ -72,4 +73,20 @@ res = fn() assert res == 51 +def test_computed_int_symbolic(): + too_early = True + def compute_fn(): + assert not too_early + return 7 + k = ComputedIntSymbolic(compute_fn) + def f(): + return k*6 + t = Translation(f) + t.rtype() + if conftest.option.view: + t.view() + too_early = False + fn = t.compile_c() + res = fn() + assert res == 42 From cfbolz at codespeak.net Fri Apr 7 19:27:21 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 19:27:21 +0200 (CEST) Subject: [pypy-svn] r25518 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060407172721.9DF061029E@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 19:27:20 2006 New Revision: 25518 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: put valentino into the right place :-) Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Fri Apr 7 19:27:20 2006 @@ -17,6 +17,7 @@ Niklaus Haldimann 20/4 - 1/5 ? Anders Chrigstr?m 22/4 - 30/4 Samuele Pedroni 22/4 - 30/4 +Valentino Volonghi 22/4 - 1/5 Hotel Edoya (Ueno) ==================== ============== ===================== People on the following list were present at previous sprints: @@ -31,7 +32,6 @@ Lene Wagner Johan Hahn Amaury Forgeot d'Arc -Valentino Volonghi 22/4 - 1/5 Hotel Edoya (Ueno) Boris Feigin Bert Freudenberg ==================== ============== ===================== From arigo at codespeak.net Fri Apr 7 19:31:09 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 19:31:09 +0200 (CEST) Subject: [pypy-svn] r25519 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060407173109.2F8C91029A@code0.codespeak.net> Author: arigo Date: Fri Apr 7 19:31:08 2006 New Revision: 25519 Modified: pypy/dist/pypy/translator/llvm/database.py pypy/dist/pypy/translator/llvm/test/test_symbolic.py Log: (arre, arigo) ComputedIntSymbolic support in genllvm. Modified: pypy/dist/pypy/translator/llvm/database.py ============================================================================== --- pypy/dist/pypy/translator/llvm/database.py (original) +++ pypy/dist/pypy/translator/llvm/database.py Fri Apr 7 19:31:08 2006 @@ -13,6 +13,7 @@ from pypy.rpython.lltypesystem import lltype, llmemory from pypy.objspace.flow.model import Constant, Variable from pypy.rpython.memory.lladdress import NULL +from pypy.rpython.objectmodel import Symbolic, ComputedIntSymbolic log = log.database @@ -366,8 +367,13 @@ # XXXXX things are happening in the gc world... # assert value == NULL repr = 'null' - elif isinstance(value, llmemory.AddressOffset): - return self.offset_str(value) + elif isinstance(value, Symbolic): + if isinstance(value, llmemory.AddressOffset): + return self.offset_str(value) + elif isinstance(value, ComputedIntSymbolic): + repr = '%d' % (value.compute_fn(),) + else: + raise NotImplementedError("symbolic: %r" % (value,)) else: repr = str(value) return repr Modified: pypy/dist/pypy/translator/llvm/test/test_symbolic.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_symbolic.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_symbolic.py Fri Apr 7 19:31:08 2006 @@ -3,6 +3,7 @@ from pypy import conftest from pypy.rpython.lltypesystem import llmemory, lltype from pypy.rpython.memory import lladdress +from pypy.rpython.objectmodel import ComputedIntSymbolic from pypy.translator.llvm.test.runtest import compile_function @@ -87,4 +88,13 @@ res = fn() assert res == 51 +def test_computed_int_symbolic(): + def compute_fn(): + return 7 + k = ComputedIntSymbolic(compute_fn) + def f(): + return k*6 + fn = compile_function(f, []) + res = fn() + assert res == 42 From hpk at codespeak.net Fri Apr 7 19:37:33 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 7 Apr 2006 19:37:33 +0200 (CEST) Subject: [pypy-svn] r25520 - pypy/branch/stacklesscfg/pypy/translator/stackless Message-ID: <20060407173733.010231029B@code0.codespeak.net> Author: hpk Date: Fri Apr 7 19:37:32 2006 New Revision: 25520 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/code.py Log: (mwh, hpk) small progress: annotating the call_function dispatching code Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/code.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/code.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/code.py Fri Apr 7 19:37:32 2006 @@ -25,30 +25,28 @@ def __init__(self): self.frame = null_state -void_void_func = lltype.FuncType([], lltype.Void) -long_void_func = lltype.FuncType([], lltype.Signed) -longlong_void_func = lltype.FuncType([], lltype.SignedLongLong) -pointer_void_func = lltype.FuncType([], llmemory.Address) -float_void_func = lltype.FuncType([], lltype.Float) +void_void_func = lltype.Ptr(lltype.FuncType([], lltype.Void)) +long_void_func = lltype.Ptr(lltype.FuncType([], lltype.Signed)) +longlong_void_func = lltype.Ptr(lltype.FuncType([], lltype.SignedLongLong)) +float_void_func = lltype.Ptr(lltype.FuncType([], lltype.Float)) +pointer_void_func = lltype.Ptr(lltype.FuncType([], llmemory.Address) ) def call_function(fn, signature): - pass -## if signature == 'void': -## fn2 = llmemory.cast_adr_to_ptr(fn, void_void_func) -## fn2() -## elif signature == 'long': -## fn3 = llmemory.cast_adr_to_ptr(fn, long_void_func) -## global_state.long_retval = fn3() -## elif signature == 'long long': -## fn4 = llmemory.cast_adr_to_ptr(fn, longlong_void_func) -## global_state.longlong_retval = fn4() -## elif signature == 'pointer': -## fn5 = llmemory.cast_adr_to_ptr(fn, pointer_void_func) -## global_state.pointer_retval = fn5() -## elif signature == 'float': -## fn6 = llmemory.cast_adr_to_ptr(fn, float_void_func) -## global_state.float_retval = fn6() - + if signature == 'void': + fn2 = llmemory.cast_adr_to_ptr(fn, void_void_func) + fn2() + elif signature == 'long': + fn3 = llmemory.cast_adr_to_ptr(fn, long_void_func) + global_state.long_retval = fn3() + elif signature == 'longlong': + fn3 = llmemory.cast_adr_to_ptr(fn, longlong_void_func) + global_state.longlong_retval = fn3() + elif signature == 'float': + fn3 = llmemory.cast_adr_to_ptr(fn, float_void_func) + global_state.float_retval = fn3() + elif signature == 'pointer': + fn5 = llmemory.cast_adr_to_ptr(fn, pointer_void_func) + global_state.pointer_retval = fn5() null_address = llmemory.fakeaddress(None) @@ -66,9 +64,9 @@ try: call_function(fn, signature) -## except UnwindException, u: -## pending = u.frame -## break + #except Exception, e: #KeyError, u: # XXX should be UnwindException + # #pending = u.frame + # break except Exception, e: global_state.exception = e else: From cfbolz at codespeak.net Fri Apr 7 19:44:50 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 19:44:50 +0200 (CEST) Subject: [pypy-svn] r25521 - pypy/dist/pypy/translator/test Message-ID: <20060407174450.0142C1006D@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 19:44:50 2006 New Revision: 25521 Modified: pypy/dist/pypy/translator/test/test_interactive.py Log: this is no longer supported Modified: pypy/dist/pypy/translator/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/translator/test/test_interactive.py (original) +++ pypy/dist/pypy/translator/test/test_interactive.py Fri Apr 7 19:44:50 2006 @@ -96,19 +96,6 @@ assert 'backendopt' not in t.driver.done - t = Translation(f, [int, int]) - t.disable(['annotate']) - t.source_c() - - assert 'annotate' not in t.driver.done and 'rtype' not in t.driver.done - - t = Translation(f, [int, int]) - t.disable(['rtype']) - t.source_c() - - assert 'annotate' in t.driver.done - assert 'rtype' not in t.driver.done and 'backendopt' not in t.driver.done - def test_simple_compile_c(): def f(x,y): return x+y From arigo at codespeak.net Fri Apr 7 20:32:09 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 7 Apr 2006 20:32:09 +0200 (CEST) Subject: [pypy-svn] r25522 - in pypy/dist/pypy/rpython: . lltypesystem test Message-ID: <20060407183209.3CAB010294@code0.codespeak.net> Author: arigo Date: Fri Apr 7 20:32:07 2006 New Revision: 25522 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py pypy/dist/pypy/rpython/lltypesystem/rclass.py pypy/dist/pypy/rpython/normalizecalls.py pypy/dist/pypy/rpython/test/test_normalizecalls.py Log: (arre, arigo) Turned the class range ids into ComputedIntSymbolics. Added a test that shows that mix-level helpers can now add subclasses of existing classes. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Apr 7 20:32:07 2006 @@ -2,6 +2,7 @@ from pypy.rpython.rarithmetic import intmask, r_uint, ovfcheck, r_longlong, r_ulonglong from pypy.rpython.lltypesystem import lltype, llmemory, lloperation from pypy.rpython.ootypesystem import ootype +from pypy.rpython.objectmodel import ComputedIntSymbolic import sys import math @@ -181,6 +182,8 @@ val = varorconst.value except AttributeError: val = self.bindings[varorconst] + if isinstance(val, ComputedIntSymbolic): + val = val.compute_fn() if varorconst.concretetype is not lltype.Void: try: val = lltype.enforce(varorconst.concretetype, val) Modified: pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py Fri Apr 7 20:32:07 2006 @@ -31,14 +31,15 @@ v_obj, v_cls = hop.inputargs(instance_repr, class_repr) if isinstance(v_cls, Constant): cls = v_cls.value - if cls.subclassrange_max == cls.subclassrange_min + 1: - # a class with no subclass - return hop.gendirectcall(rclass.ll_isinstance_exact, v_obj, v_cls) - else: - minid = hop.inputconst(lltype.Signed, cls.subclassrange_min) - maxid = hop.inputconst(lltype.Signed, cls.subclassrange_max) - return hop.gendirectcall(rclass.ll_isinstance_const, v_obj, minid, - maxid) + # XXX re-implement the following optimization + #if cls.subclassrange_max == cls.subclassrange_min + 1: + # # a class with no subclass + # return hop.gendirectcall(rclass.ll_isinstance_exact, v_obj, v_cls) + #else: + minid = hop.inputconst(lltype.Signed, cls.subclassrange_min) + maxid = hop.inputconst(lltype.Signed, cls.subclassrange_max) + return hop.gendirectcall(rclass.ll_isinstance_const, v_obj, minid, + maxid) else: return hop.gendirectcall(rclass.ll_isinstance, v_obj, v_cls) Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Fri Apr 7 20:32:07 2006 @@ -266,14 +266,15 @@ v_cls1, v_cls2 = hop.inputargs(class_repr, class_repr) if isinstance(v_cls2, Constant): cls2 = v_cls2.value - if cls2.subclassrange_max == cls2.subclassrange_min + 1: - # a class with no subclass - return hop.genop('ptr_eq', [v_cls1, v_cls2], resulttype=Bool) - else: - minid = hop.inputconst(Signed, cls2.subclassrange_min) - maxid = hop.inputconst(Signed, cls2.subclassrange_max) - return hop.gendirectcall(ll_issubclass_const, v_cls1, minid, - maxid) + # XXX re-implement the following optimization +## if cls2.subclassrange_max == cls2.subclassrange_min + 1: +## # a class with no subclass +## return hop.genop('ptr_eq', [v_cls1, v_cls2], resulttype=Bool) +## else: + minid = hop.inputconst(Signed, cls2.subclassrange_min) + maxid = hop.inputconst(Signed, cls2.subclassrange_max) + return hop.gendirectcall(ll_issubclass_const, v_cls1, minid, + maxid) else: v_cls1, v_cls2 = hop.inputargs(class_repr, class_repr) return hop.gendirectcall(ll_issubclass, v_cls1, v_cls2) Modified: pypy/dist/pypy/rpython/normalizecalls.py ============================================================================== --- pypy/dist/pypy/rpython/normalizecalls.py (original) +++ pypy/dist/pypy/rpython/normalizecalls.py Fri Apr 7 20:32:07 2006 @@ -9,7 +9,7 @@ from pypy.tool.sourcetools import func_with_new_name from pypy.rpython.error import TyperError from pypy.rpython.rmodel import needsgc -from pypy.rpython.objectmodel import instantiate +from pypy.rpython.objectmodel import instantiate, ComputedIntSymbolic def normalize_call_familes(annotator): for callfamily in annotator.bookkeeper.pbc_maximal_call_families.infos(): @@ -274,8 +274,26 @@ # ____________________________________________________________ -def assign_inheritance_ids(annotator): +class MinIdSymbolic(ComputedIntSymbolic): + def __init__(self, classdef): + self.classdef = classdef + def compute_fn(self): + if self.classdef.minid is self: + compute_inheritance_ids(self.classdef.bookkeeper) + return self.classdef.minid + +class MaxIdSymbolic(ComputedIntSymbolic): + def __init__(self, classdef): + self.classdef = classdef + def compute_fn(self): + if self.classdef.maxid is self: + compute_inheritance_ids(self.classdef.bookkeeper) + return self.classdef.maxid + +def compute_inheritance_ids(bookkeeper): def assign_id(classdef, nextid): + assert isinstance(classdef.minid, MinIdSymbolic) + assert isinstance(classdef.maxid, MaxIdSymbolic) classdef.minid = nextid nextid += 1 for subclass in classdef.subdefs: @@ -283,11 +301,16 @@ classdef.maxid = nextid return classdef.maxid id_ = 0 - for classdef in annotator.bookkeeper.classdefs: + for classdef in bookkeeper.classdefs: if classdef.basedef is None: - prevmaxid = getattr(classdef, 'maxid', sys.maxint) id_ = assign_id(classdef, id_) - assert id_ <= prevmaxid, "non-orthogonal class hierarchy growth" + +def assign_inheritance_ids(annotator): + for classdef in annotator.bookkeeper.classdefs: + if not hasattr(classdef, 'minid'): + classdef.minid = MinIdSymbolic(classdef) + if not hasattr(classdef, 'maxid'): + classdef.maxid = MaxIdSymbolic(classdef) # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/test/test_normalizecalls.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_normalizecalls.py (original) +++ pypy/dist/pypy/rpython/test/test_normalizecalls.py Fri Apr 7 20:32:07 2006 @@ -144,25 +144,27 @@ res = llinterp.eval_graph(graphof(translator, dummyfn), [2]) assert res == -2 + +class PBase: + def fn(self): + raise NotImplementedError +class PSub1(PBase): + def fn(self): + return 1 +class PSub2(PBase): + def fn(self): + return 2 +def prefn(n): + if n == 1: + x = PSub1() + else: + x = PSub2() + return x.fn() * 100 + isinstance(x, PSub2) + + class TestNormalizeAfterTheFact(TestNormalize): def rtype(self, fn, argtypes, resulttype): - class Base: - def fn(self): - raise NotImplementedError - class Sub1(Base): - def fn(self): - return 1 - class Sub2(Base): - def fn(self): - return 2 - def prefn(n): - if n == 1: - x = Sub1() - else: - x = Sub2() - return x.fn() - t = TranslationContext() a = t.buildannotator() a.build_types(prefn, [int]) @@ -182,9 +184,9 @@ # sanity check prefn llinterp = LLInterpreter(typer) res = llinterp.eval_graph(graphof(t, prefn), [1]) - assert res == 1 + assert res == 100 res = llinterp.eval_graph(graphof(t, prefn), [2]) - assert res == 2 + assert res == 201 t.checkgraphs() return t @@ -211,3 +213,24 @@ graph = annhelper.getgraph(f, [], annmodel.SomeInteger()) annhelper.finish() + def test_add_more_subclasses(self): + class Sub3(PBase): + def newmethod(self): + return 3 + def dummyfn(n): + x = Sub3() + return x.newmethod() + + translator = self.rtype(dummyfn, [int], int) + base_graph = graphof(translator, PBase.fn.im_func) + sub1_graph = graphof(translator, PSub1.fn.im_func) + sub2_graph = graphof(translator, PSub2.fn.im_func) + sub3_graph = graphof(translator, Sub3.fn.im_func) + dummyfn_graph = graphof(translator, dummyfn) + assert base_graph.getreturnvar().concretetype == lltype.Signed + assert sub1_graph.getreturnvar().concretetype == lltype.Signed + assert sub2_graph.getreturnvar().concretetype == lltype.Signed + assert sub3_graph.getreturnvar().concretetype == lltype.Signed + assert dummyfn_graph.getreturnvar().concretetype == lltype.Signed + + From cfbolz at codespeak.net Fri Apr 7 20:42:41 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 20:42:41 +0200 (CEST) Subject: [pypy-svn] r25523 - in pypy/dist/pypy: doc/discussion rpython translator translator/backendopt translator/c/test Message-ID: <20060407184241.7545E10286@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 20:42:39 2006 New Revision: 25523 Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt pypy/dist/pypy/rpython/annlowlevel.py pypy/dist/pypy/rpython/typesystem.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/c/test/test_typed.py pypy/dist/pypy/translator/driver.py pypy/dist/pypy/translator/interactive.py pypy/dist/pypy/translator/transform.py Log: (pedronis, cfbolz): do the (very indeterministic) stack check insertion as late as possible, just before code generation. Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/translation-swamp.txt (original) +++ pypy/dist/pypy/doc/discussion/translation-swamp.txt Fri Apr 7 20:42:39 2006 @@ -7,7 +7,7 @@ * nondeterminism of insert stackcheck: - * move it after inlining + * (TESTING) move it after inlining * avoid inserting it multiple times in the same block * experiment with different heuristics: @@ -24,4 +24,4 @@ * move the inlining of gc helpers just before emitting the code. throw the graph away - * for gcc: use just one implement file + * for gcc: use just one implement file (TESTING) Modified: pypy/dist/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/dist/pypy/rpython/annlowlevel.py (original) +++ pypy/dist/pypy/rpython/annlowlevel.py Fri Apr 7 20:42:39 2006 @@ -172,7 +172,7 @@ "originally specified: %r\n" " found by annotating: %r" % (graph, s_result, s_real_result)) - rtyper.type_system.perform_normalizations(rtyper, insert_stack_checks=False) + rtyper.type_system.perform_normalizations(rtyper) for r in self.delayedreprs: r.set_setup_delayed(False) for p, repr, obj in self.delayedconsts: Modified: pypy/dist/pypy/rpython/typesystem.py ============================================================================== --- pypy/dist/pypy/rpython/typesystem.py (original) +++ pypy/dist/pypy/rpython/typesystem.py Fri Apr 7 20:42:39 2006 @@ -62,16 +62,12 @@ in a graph.""" raise NotImplementedError() - def perform_normalizations(self, rtyper, insert_stack_checks=True): + def perform_normalizations(self, rtyper): """Prepare the annotator's internal data structures for rtyping with the specified type system. """ # default implementation - from pypy.translator.transform import insert_stackcheck from pypy.rpython.normalizecalls import perform_normalizations - # XXX stack checks cannot be inserted again - if insert_stack_checks: - insert_stackcheck(rtyper.annotator) perform_normalizations(rtyper) class LowLevelTypeSystem(TypeSystem): Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Fri Apr 7 20:42:39 2006 @@ -1,5 +1,6 @@ import py from pypy.rpython.lltypesystem import lltype +from pypy.translator.simplify import get_graph from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log from pypy.translator.unsimplify import split_block, copyvar, insert_empty_block @@ -94,6 +95,25 @@ afterblock.operations.extend(generate_keepalive(keep_alive_vars)) return afterblock +def calculate_call_graph(translator): + calls = {} + for graph in translator.graphs: + if getattr(getattr(graph, "func", None), "suggested_primitive", False): + continue + calls[graph] = {} + for block in graph.iterblocks(): + for op in block.operations: + if op.opname == "direct_call": + called_graph = get_graph(op.args[0], translator) + if called_graph is not None: + calls[graph][called_graph] = block + if op.opname == "indirect_call": + graphs = op.args[-1].value + if graphs is not None: + for called_graph in graphs: + calls[graph][called_graph] = block + return calls + def md5digest(translator): import md5 m = md5.new() Modified: pypy/dist/pypy/translator/c/test/test_typed.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_typed.py (original) +++ pypy/dist/pypy/translator/c/test/test_typed.py Fri Apr 7 20:42:39 2006 @@ -38,11 +38,13 @@ return builder.get_entry_point() def getcompiled(self, func, view=False): + from pypy.translator.transform import insert_ll_stackcheck t = self.annotatefunc(func) self.process(t) if view or conftest.option.view: t.view() t.checkgraphs() + insert_ll_stackcheck(t) return self.compilefunc(t, func) def test_set_attr(self): @@ -705,3 +707,18 @@ return a1.check_list_is_true() fn = self.getcompiled(f) assert fn() == 1 + + def test_infinite_recursion(self): + def f(x): + if x: + return f(x) + return 1 + def g(x=int): + try: + f(x) + except RuntimeError: + return 42 + return 1 + fn = self.getcompiled(g) + assert fn(0) == 1 + assert fn(1) == 42 Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Fri Apr 7 20:42:39 2006 @@ -205,6 +205,15 @@ task_backendopt = taskdef(task_backendopt, ['rtype'], "Back-end optimisations") + def task_stackcheckinsertion(self): + from pypy.translator.transform import insert_ll_stackcheck + insert_ll_stackcheck(self.translator) + + task_stackcheckinsertion = taskdef( + task_stackcheckinsertion, + ['?backendopt', 'rtype', 'annotate'], + "inserting stack checks") + def task_database_c(self): translator = self.translator opt = self.options @@ -237,7 +246,7 @@ self.database = database # task_database_c = taskdef(task_database_c, - ['?backendopt', '?rtype', '?annotate'], + ['stackcheckinsertion', '?backendopt', '?rtype', '?annotate'], "Creating database for generating c source") def task_source_c(self): # xxx messy @@ -301,7 +310,7 @@ log.llinterpret.event("result -> %s" % v) # task_llinterpret = taskdef(task_llinterpret, - ['?backendopt', 'rtype'], + ['stackcheckinsertion', '?backendopt', 'rtype'], "LLInterpreting") def task_source_llvm(self): @@ -320,7 +329,7 @@ self.log.info("written: %s" % (llvm_filename,)) # task_source_llvm = taskdef(task_source_llvm, - ['backendopt', 'rtype'], + ['stackcheckinsertion', 'backendopt', 'rtype'], "Generating llvm source") def task_compile_llvm(self): Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Fri Apr 7 20:42:39 2006 @@ -50,6 +50,7 @@ 'rtype': ['insist'], 'ootype': [], 'backendopt': ['merge_if_blocks'], + 'stackcheckinsertion': [], 'database_c': ['gc', 'stackless'], 'source_llvm': ['gc', 'stackless'], 'source_c': [], Modified: pypy/dist/pypy/translator/transform.py ============================================================================== --- pypy/dist/pypy/translator/transform.py (original) +++ pypy/dist/pypy/translator/transform.py Fri Apr 7 20:42:39 2006 @@ -13,6 +13,7 @@ from pypy.objspace.flow.model import c_last_exception, checkgraph from pypy.annotation import model as annmodel from pypy.rpython.rstack import stack_check +from pypy.rpython.lltypesystem import lltype def checkgraphs(self, blocks): seen = {} @@ -178,6 +179,41 @@ unwind_op = SpaceOperation('simple_call', [Constant(stack_check)], v) caller_block.operations.insert(0, unwind_op) +def insert_ll_stackcheck(translator): + from pypy.translator.backendopt.support import calculate_call_graph + from pypy.rpython.module.ll_stack import ll_stack_check + from pypy.tool.algo.graphlib import Edge, make_edge_dict, break_cycles + rtyper = translator.rtyper + graph = rtyper.annotate_helper(ll_stack_check, []) + rtyper.specialize_more_blocks() + stack_check_ptr = rtyper.getcallable(graph) + stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr)) + call_graph = calculate_call_graph(translator) + edges = [] + graphs_to_patch = {} + insert_in = {} + for caller, all_callees in call_graph.iteritems(): + for callee, block in all_callees.iteritems(): + if getattr(getattr(callee, 'func', None), + 'insert_stack_check_here', False): + insert_in[callee.startblock] = True + continue + edge = Edge(caller, callee) + edge.block = block + edges.append(edge) + + edgedict = make_edge_dict(edges) + for edge in break_cycles(edgedict, edgedict): + block = edge.block + insert_in[block] = True + + for block in insert_in: + v = Variable() + v.concretetype = lltype.Void + unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v) + block.operations.insert(0, unwind_op) + + default_extra_passes = [ transform_allocate, ] From ericvrp at codespeak.net Fri Apr 7 20:46:33 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 7 Apr 2006 20:46:33 +0200 (CEST) Subject: [pypy-svn] r25524 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060407184633.84B471029E@code0.codespeak.net> Author: ericvrp Date: Fri Apr 7 20:46:32 2006 New Revision: 25524 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: Tokyo accomodation Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Fri Apr 7 20:46:32 2006 @@ -9,12 +9,12 @@ Name Arrive/Depart Accomodation ==================== ============== ===================== Beatrice D?ring 21/4 - 25/4 -Christian Tismer 22/4 - 30/4 -Eric van Riet Paap 18/4 - 1/5 ? +Christian Tismer 22/4 - 30/4 Akihabare Washington Hotel +Eric van Riet Paap 18/4 - 1/5 Akihabare Washington Hotel Jacob Hall?n Laura Creighton -Anders Lehmann 23/4 - 1/5 ? -Niklaus Haldimann 20/4 - 1/5 ? +Anders Lehmann 23/4 - 1/5 Akihabare Washington Hotel +Niklaus Haldimann 20/4 - 1/5 Akihabare Washington Hotel Anders Chrigstr?m 22/4 - 30/4 Samuele Pedroni 22/4 - 30/4 Valentino Volonghi 22/4 - 1/5 Hotel Edoya (Ueno) From hpk at codespeak.net Fri Apr 7 20:53:23 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 7 Apr 2006 20:53:23 +0200 (CEST) Subject: [pypy-svn] r25525 - pypy/branch/stacklesscfg/pypy/translator/backendopt Message-ID: <20060407185323.22E9E102A1@code0.codespeak.net> Author: hpk Date: Fri Apr 7 20:53:22 2006 New Revision: 25525 Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py Log: no keepalive vars if no operations in afterblock Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py Fri Apr 7 20:53:22 2006 @@ -78,7 +78,7 @@ newvar = copyvar(translator, var) afterblock.inputargs.append(newvar) keep_alive_vars[i] = newvar - elif keep_alive_op_args: + elif keep_alive_op_args and afterblock.operations: keep_alive_vars = [var for var in afterblock.operations[0].args if isinstance(var, Variable) and var_needsgc(var)] else: From hpk at codespeak.net Fri Apr 7 20:54:07 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 7 Apr 2006 20:54:07 +0200 (CEST) Subject: [pypy-svn] r25526 - in pypy/branch/stacklesscfg/pypy/translator/stackless: . test Message-ID: <20060407185407.B2CB4102A5@code0.codespeak.net> Author: hpk Date: Fri Apr 7 20:54:06 2006 New Revision: 25526 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/code.py pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: progressing a bit more towards the stackless loop and passing a small example - but the graphs are still not completely valid -> but dinner now :) Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/code.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/code.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/code.py Fri Apr 7 20:54:06 2006 @@ -21,10 +21,6 @@ global_state = StacklessData() -class UnwindException(Exception): - def __init__(self): - self.frame = null_state - void_void_func = lltype.Ptr(lltype.FuncType([], lltype.Void)) long_void_func = lltype.Ptr(lltype.FuncType([], lltype.Signed)) longlong_void_func = lltype.Ptr(lltype.FuncType([], lltype.SignedLongLong)) @@ -53,33 +49,33 @@ def decode_state(state): return null_address, 'void', 0 +class UnwindException(Exception): + def __init__(self): + self.frame_top = null_state # points to frame that first caught + # the UnwindException + self.frame_bottom = null_state + # walking frame_top.f_back.f_back... goes to frame_bottom + # + def slp_main_loop(): - while 1: - pending = global_state.top - while 1: - back = pending.f_back - - state = pending.state - fn, signature, global_state.restart_substate = decode_state(state) - - try: - call_function(fn, signature) - #except Exception, e: #KeyError, u: # XXX should be UnwindException - # #pending = u.frame - # break - except Exception, e: - global_state.exception = e - else: - global_state.exception = None - - if not back: - if global_state.exception: - raise global_state.exception - return - - pending = back - global_state.top = pending - - if global_state.bottom: - assert global_state.bottom.f_back is None - global_state.bottom.f_back = back + currentframe = global_state.top + + while currentframe is not None: + nextframe = currentframe.f_back + framestate = currentframe.state + fn, signature, global_state.restart_substate = decode_state(framestate) + try: + call_function(fn, signature) + except UnwindException, u: #XXX annotation support needed + nextframe = u.frame_top + except Exception, e: + global_state.exception = e + else: + global_state.exception = None + + currentframe = nextframe + + if global_state.exception is not None: + raise global_state.exception + + Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py Fri Apr 7 20:54:06 2006 @@ -2,6 +2,7 @@ from pypy.rpython.memory.gctransform import varoftype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.translator import TranslationContext, graphof +from pypy.objspace.flow.model import checkgraph from pypy import conftest ## def test_frame_types(): @@ -25,16 +26,26 @@ ## assert s2_1 is s2_2 def test_simple_transform(): - def g(): + from pypy.translator.stackless.code import UnwindException + def check(x): + if x: + raise UnwindException # XXX or so + def g(x): + check(x) return 2 def example(x): - return g() + x + 1 + return g(x) + x + 1 t = TranslationContext() t.buildannotator().build_types(example, [int]) t.buildrtyper().specialize() - example_graph = graphof(t, example) st = StacklessTransfomer(t) - st.transform_graph(example_graph) + st.transform_graph(graphof(t, example)) + st.transform_graph(graphof(t, g)) + st.transform_graph(graphof(t, check)) + #for graph in t.graphs: + #checkgraph(graph) # XXX + #st.transform_graph(graph) + example_graph = graphof(t, example) if conftest.option.view: t.view() Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Fri Apr 7 20:54:06 2006 @@ -1,5 +1,5 @@ from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rpython import rarithmetic +from pypy.rpython import rarithmetic, rclass from pypy.translator.backendopt import support from pypy.objspace.flow import model from pypy.rpython.memory.gctransform import varoftype @@ -143,9 +143,14 @@ save_block = self.generate_save_block(link.args, var_unwind_exception) - newlink = model.Link(link.args + [var_unwind_exception], save_block, model.Constant(SystemExit)) + newlink = model.Link(link.args + [var_unwind_exception], + save_block, code.UnwindException) + r_case = rclass.get_type_repr(self.translator.rtyper) + newlink.llexitcase = r_case.convert_const(newlink.exitcase) block.exitswitch = model.c_last_exception - block.exits.append(newlink) + block.recloseblock(link, newlink) # exits.append(newlink) + # ARGH ... + block = after_block i = 0 else: @@ -159,7 +164,8 @@ var_unwind_exception0 = copyvar(self.translator, var_unwind_exception) from pypy.rpython.rclass import getinstancerepr var_unwind_exception = varoftype(getinstancerepr(self.translator.rtyper, - self.translator.annotator.bookkeeper.getuniqueclassdef(SystemExit)).lowleveltype) + self.translator.annotator.bookkeeper.getuniqueclassdef( + code.UnwindException)).lowleveltype) fields = [] for i, v in enumerate(varstosave): From ac at codespeak.net Fri Apr 7 20:54:45 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 7 Apr 2006 20:54:45 +0200 (CEST) Subject: [pypy-svn] r25527 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060407185445.823E5102A7@code0.codespeak.net> Author: ac Date: Fri Apr 7 20:54:44 2006 New Revision: 25527 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: Accomodation... Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Fri Apr 7 20:54:44 2006 @@ -15,8 +15,8 @@ Laura Creighton Anders Lehmann 23/4 - 1/5 Akihabare Washington Hotel Niklaus Haldimann 20/4 - 1/5 Akihabare Washington Hotel -Anders Chrigstr?m 22/4 - 30/4 -Samuele Pedroni 22/4 - 30/4 +Anders Chrigstr?m 22/4 - 30/4 Akihabare Washington Hotel +Samuele Pedroni 22/4 - 30/4 Akihabare Washington Hotel Valentino Volonghi 22/4 - 1/5 Hotel Edoya (Ueno) ==================== ============== ===================== From antocuni at codespeak.net Fri Apr 7 21:28:53 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 7 Apr 2006 21:28:53 +0200 (CEST) Subject: [pypy-svn] r25528 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060407192853.E810B1029E@code0.codespeak.net> Author: antocuni Date: Fri Apr 7 21:28:15 2006 New Revision: 25528 Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py Log: Some refactoring Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Fri Apr 7 21:28:15 2006 @@ -1,12 +1,13 @@ from pypy.translator.cli.node import Node from pypy.translator.cli.function import Function -from pypy.translator.cli import cts +from pypy.translator.cli.cts import CTS class Class(Node): def __init__(self, db, classdef): self.db = db + self.cts = CTS(db) self.classdef = classdef - self.namespace, self.name = cts.split_class_name(classdef._name) + self.namespace, self.name = self.cts.split_class_name(classdef._name) #0/0 def get_name(self): @@ -26,7 +27,7 @@ ilasm.begin_class(self.name, self.get_base_class()) for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): - ilasm.field(f_name, cts.lltype_to_cts(f_type)) + ilasm.field(f_name, self.cts.lltype_to_cts(f_type)) # TODO: should the .ctor set the default values? self._ctor() Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Fri Apr 7 21:28:15 2006 @@ -6,7 +6,7 @@ from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong -from pypy.rpython.ootypesystem.ootype import Instance +from pypy.rpython.ootypesystem.ootype import Instance, Class from pypy.translator.cli.option import getoption from pypy.tool.ansi_print import ansi_log @@ -23,17 +23,15 @@ UnsignedLongLong: 'unsigned int64', Bool: 'bool', Float: 'float64', + Class: '[mscorlib]System.Type', # TODO: check this } -_cts_to_ilasm = { - 'int32': 'i4', - 'unsigned int32': 'i4', - 'int64': 'i8', - 'unsigned int64': 'i8', - 'bool': 'i4', - 'float64': 'r8', +_pyexception_to_cts = { + exceptions.Exception: '[mscorlib]System.Exception', + exceptions.OverflowError: '[mscorlib]System.OverflowException' } + def _get_from_dict(d, key, error): try: return d[key] @@ -44,65 +42,43 @@ else: assert False, error +class CTS(object): + def __init__(self, db): + self.db = db -def lltype_to_cts(t): - if isinstance(t, Instance): - name = t._name - if 'Object_meta' in name or 'Object' in name: # TODO - return 'object' - - return 'class %s' % name - - return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) - -def lltype_to_ilasm(t): - return ctstype_to_ilasm(lltype_to_cts(t)) - -def ctstype_to_ilasm(t): - return _get_from_dict(_cts_to_ilasm, t, 'Unknown ilasm type %s' % t) - -def llvar_to_cts(var): - return lltype_to_cts(var.concretetype), var.name - -def llconst_to_cts(const): - return lltype_to_cts(const.concretetype), const.value - -def llconst_to_ilasm(const): - """ - Return the const as a string suitable for ilasm. - """ - ilasm_type = lltype_to_ilasm(const.concretetype) - if const.concretetype is Bool: - return ilasm_type, str(int(const.value)) - elif const.concretetype is Float: - return ilasm_type, repr(const.value) - else: - return ilasm_type, str(const.value) - -def graph_to_signature(graph, is_method = False, func_name = None): - ret_type, ret_var = llvar_to_cts(graph.getreturnvar()) - func_name = func_name or graph.name - - args = graph.getargs() - if is_method: - args = args[1:] - - arg_types = [lltype_to_cts(arg.concretetype) for arg in args] - arg_list = ', '.join(arg_types) - - return '%s %s(%s)' % (ret_type, func_name, arg_list) - -def split_class_name(class_name): - parts = class_name.rsplit('.', 1) - if len(parts) == 2: - return parts - else: - return None, parts[0] + def lltype_to_cts(self, t): + if isinstance(t, Instance): + name = t._name + self.db.classes.add(t) + return 'class %s' % name -_pyexception_to_cts = { - exceptions.Exception: '[mscorlib]System.Exception', - exceptions.OverflowError: '[mscorlib]System.OverflowException' - } + return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) + + def llvar_to_cts(self, var): + return self.lltype_to_cts(var.concretetype), var.name + + def llconst_to_cts(self, const): + return self.lltype_to_cts(const.concretetype), const.value + + def graph_to_signature(self, graph, is_method = False, func_name = None): + ret_type, ret_var = self.llvar_to_cts(graph.getreturnvar()) + func_name = func_name or graph.name + + args = graph.getargs() + if is_method: + args = args[1:] + + arg_types = [self.lltype_to_cts(arg.concretetype) for arg in args] + arg_list = ', '.join(arg_types) + + return '%s %s(%s)' % (ret_type, func_name, arg_list) + + def split_class_name(self, class_name): + parts = class_name.rsplit('.', 1) + if len(parts) == 2: + return parts + else: + return None, parts[0] -def pyexception_to_cts(exc): - return _get_from_dict(_pyexception_to_cts, exc, 'Unknown exception %s' % exc) + def pyexception_to_cts(self, exc): + return _get_from_dict(_pyexception_to_cts, exc, 'Unknown exception %s' % exc) Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Fri Apr 7 21:28:15 2006 @@ -10,9 +10,15 @@ self.pending_graphs = [] self.functions = {} # graph --> function_name self.methods = {} # graph --> method_name + self.consts = {} # value --> const_name def record_function(self, graph, name): self.functions[graph] = name def function_name(self, graph): return self.functions.get(graph, None) + + def record_const(self, const): + name = '__XXX__CONST__NAME__XXX__' # TODO + self.consts[const] = name + return name Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Fri Apr 7 21:28:15 2006 @@ -4,10 +4,11 @@ from sets import Set as set from pypy.objspace.flow import model as flowmodel -from pypy.rpython.lltypesystem.lltype import Void -from pypy.rpython.ootypesystem.ootype import Instance +from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong +from pypy.rpython.ootypesystem import ootype from pypy.translator.cli.option import getoption -from pypy.translator.cli import cts +from pypy.translator.cli.cts import CTS from pypy.translator.cli.opcodes import opcodes from pypy.translator.cli.metavm import InstructionList, Generator from pypy.translator.cli.node import Node @@ -20,6 +21,7 @@ class Function(Node, Generator): def __init__(self, db, graph, name = None, is_method = False, is_entrypoint = False): self.db = db + self.cts = CTS(db) self.graph = graph self.name = name or graph.name self.is_method = is_method @@ -40,7 +42,7 @@ def render(self, ilasm): self.ilasm = ilasm graph = self.graph - returntype, returnvar = cts.llvar_to_cts(graph.getreturnvar()) + returntype, returnvar = self.cts.llvar_to_cts(graph.getreturnvar()) if self.is_method: args = self.args[1:] # self is implicit @@ -88,7 +90,7 @@ continue # see above assert issubclass(link.exitcase, Exception) - cts_exc = cts.pyexception_to_cts(link.exitcase) + cts_exc = self.cts.pyexception_to_cts(link.exitcase) self.ilasm.begin_catch(cts_exc) target = link.target @@ -180,13 +182,13 @@ for v in mix: is_var = isinstance(v, flowmodel.Variable) if id(v) not in seen and is_var and v.name not in args and v.concretetype is not Void: - locals.append(cts.llvar_to_cts(v)) + locals.append(self.cts.llvar_to_cts(v)) seen[id(v)] = True self.locals = locals def _set_args(self): - self.args = map(cts.llvar_to_cts, self.graph.getargs()) + self.args = map(self.cts.llvar_to_cts, self.graph.getargs()) self.argset = set([argname for argtype, argname in self.args]) def _get_block_name(self, block): @@ -198,12 +200,17 @@ if isinstance(arg, flowmodel.Variable): lltype = arg.concretetype elif isinstance(arg, flowmodel.Constant): + lltype = arg.value - if isinstance(lltype, Instance): + if isinstance(lltype, ootype._view) and isinstance(lltype._inst, ootype._instance): + lltype = lltype._inst._TYPE + + if isinstance(lltype, ootype.Instance): self.db.classes.add(lltype) + def _render_op(self, op): instr_list = opcodes.get(op.opname, None) if instr_list is not None: @@ -218,7 +225,7 @@ def field_name(self, obj, field): class_name = self.class_name(obj) - field_type = cts.lltype_to_cts(obj._field_type(field)) + field_type = self.cts.lltype_to_cts(obj._field_type(field)) return '%s %s::%s' % (field_type, class_name, field) def ctor_name(self, ooinstance): @@ -227,11 +234,11 @@ # following methods belongs to the Generator interface def function_signature(self, graph): - return cts.graph_to_signature(graph, False) + return self.cts.graph_to_signature(graph, False) def method_signature(self, graph, class_name, name): full_name = '%s::%s' % (class_name, name) - return cts.graph_to_signature(graph, True, full_name) + return self.cts.graph_to_signature(graph, True, full_name) def class_name(self, ooinstance): return ooinstance._name @@ -271,12 +278,29 @@ self.ilasm.opcode('ldloc', repr(v.name)) elif isinstance(v, flowmodel.Constant): - if v.concretetype != Void: - iltype, ilvalue = cts.llconst_to_ilasm(v) - self.ilasm.opcode('ldc.' + iltype, ilvalue) + self._load_const(v) else: assert False + def _load_const(self, const): + type_ = const.concretetype + if type_ is Void: + pass + elif type_ is Bool: + self.ilasm.opcode('ldc.i4', str(int(const.value))) + elif type_ is Float: + self.ilasm.opcode('ldc.r8', repr(const.value)) + elif type_ in (Signed, Unsigned): + self.ilasm.opcode('ldc.i4', str(const.value)) + elif type_ in (SignedLongLong, UnsignedLongLong): + self.ilasm.opcode('ldc.i8', str(const.value)) + else: + name = self.db.record_const(const) + cts_type = self.cts.lltype_to_cts(type_) + self.ilasm.opcode('ldsfld %s Pypy.Constants::%s' % (cts_type, name)) + #assert False, 'Unknown constant %s' % const + + def store(self, v): if isinstance(v, flowmodel.Variable): if v.concretetype is not Void: Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Fri Apr 7 21:28:15 2006 @@ -41,9 +41,9 @@ return obj.compute() def bar(x, y): - obj = Derived(x) + obj = Base(x) return obj.compute() - #return foo(Base, x) + foo(Derived, y) +# return foo(Base, x) + foo(Derived, y) Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Fri Apr 7 21:28:15 2006 @@ -8,11 +8,13 @@ from pypy.translator.cli.option import getoption from pypy.translator.cli.gencli import GenCli from pypy.translator.cli.node import Node -from pypy.translator.cli.cts import graph_to_signature -from pypy.translator.cli.cts import llvar_to_cts +from pypy.translator.cli.cts import CTS +from pypy.translator.cli.database import LowLevelDatabase FLOAT_PRECISION = 8 +cts = CTS(LowLevelDatabase()) # this is a hack! + def check(func, annotation, args): mono = compile_function(func, annotation) res1 = func(*args) @@ -43,14 +45,14 @@ ilasm.opcode('ldarg.0') ilasm.opcode('ldc.i4.%d' % i) ilasm.opcode('ldelem.ref') - arg_type, arg_var = llvar_to_cts(arg) + arg_type, arg_var = cts.llvar_to_cts(arg) ilasm.call('%s class [mscorlib]System.Convert::%s(string)' % (arg_type, self.__convert_method(arg_type))) - ilasm.call(graph_to_signature(self.graph)) + ilasm.call(cts.graph_to_signature(self.graph)) # print the result using the appropriate WriteLine overload - ret_type, ret_var = llvar_to_cts(self.graph.getreturnvar()) + ret_type, ret_var = cts.llvar_to_cts(self.graph.getreturnvar()) ilasm.call('void class [mscorlib]System.Console::WriteLine(%s)' % ret_type) ilasm.opcode('ret') ilasm.end_function() @@ -139,7 +141,7 @@ retval = mono.wait() assert retval == 0, stderr - ret_type, ret_var = llvar_to_cts(self.graph.getreturnvar()) + ret_type, ret_var = cts.llvar_to_cts(self.graph.getreturnvar()) if 'int' in ret_type: return int(stdout) elif ret_type == 'float64': From cfbolz at codespeak.net Fri Apr 7 23:09:21 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 23:09:21 +0200 (CEST) Subject: [pypy-svn] r25529 - pypy/dist/pypy/translator/c Message-ID: <20060407210921.D6AF910298@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 23:09:21 2006 New Revision: 25529 Modified: pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/genc.py Log: don't actually not do source splitting. also _really_ free the memory after the source was generated Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Apr 7 23:09:21 2006 @@ -113,6 +113,7 @@ def implementation_end(self): self.lltypes = None + self.blocknum = None self.graph = self.oldgraph del self.oldgraph Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Fri Apr 7 23:09:21 2006 @@ -320,7 +320,7 @@ def gen_readable_parts_of_source(self, f): if py.std.sys.platform != "win32": - split_criteria_big = py.std.sys.maxint + split_criteria_big = SPLIT_CRITERIA * 4 else: split_criteria_big = SPLIT_CRITERIA if self.one_source_file: From cfbolz at codespeak.net Fri Apr 7 23:49:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 7 Apr 2006 23:49:28 +0200 (CEST) Subject: [pypy-svn] r25530 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060407214928.4ACB8100DD@code0.codespeak.net> Author: cfbolz Date: Fri Apr 7 23:49:27 2006 New Revision: 25530 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/backendopt/test/test_canraise.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: move find_backedges etc to support Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Fri Apr 7 23:49:27 2006 @@ -3,6 +3,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.translator.simplify import get_graph from pypy.rpython.rmodel import inputconst +from pypy.translator.backendopt import support class CreationPoint(object): def __init__(self, creation_method="?"): @@ -360,59 +361,6 @@ return False return True -def find_backedges(graph): - """finds the backedges in the flow graph""" - scheduled = [graph.startblock] - seen = {} - backedges = [] - while scheduled: - current = scheduled.pop() - seen[current] = True - for link in current.exits: - if link.target in seen: - backedges.append(link) - else: - scheduled.append(link.target) - return backedges - -def compute_reachability(graph): - reachable = {} - for block in graph.iterblocks(): - reach = {} - scheduled = [block] - while scheduled: - current = scheduled.pop() - for link in current.exits: - if link.target in reachable: - reach = setunion(reach, reachable[link.target]) - continue - if link.target not in reach: - reach[link.target] = True - reachable[block] = reach - return reachable - -def find_loop_blocks(graph): - """find the blocks in a graph that are part of a loop""" - loop = {} - reachable = compute_reachability(graph) - for backedge in find_backedges(graph): - start = backedge.target - end = backedge.prevblock - loop[start] = start - loop[end] = start - scheduled = [start] - seen = {} - while scheduled: - current = scheduled.pop() - connects = end in reachable[current] - seen[current] = True - if connects: - loop[current] = start - for link in current.exits: - if link.target not in seen: - scheduled.append(link.target) - return loop - def malloc_to_stack(t): aib = AbstractDataFlowInterpreter(t) for graph in t.graphs: @@ -420,7 +368,7 @@ aib.schedule_function(graph) aib.complete() for graph in t.graphs: - loop_blocks = find_loop_blocks(graph) + loop_blocks = support.find_loop_blocks(graph) for block in graph.iterblocks(): for op in block.operations: if op.opname == 'malloc': Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Fri Apr 7 23:49:27 2006 @@ -3,6 +3,7 @@ from pypy.translator.simplify import get_graph from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log +from pypy.annotation.model import setunion from pypy.translator.unsimplify import split_block, copyvar, insert_empty_block from pypy.objspace.flow.model import Constant, Variable, SpaceOperation, c_last_exception from pypy.rpython.lltypesystem import lltype @@ -114,6 +115,59 @@ calls[graph][called_graph] = block return calls +def find_backedges(graph): + """finds the backedges in the flow graph""" + scheduled = [graph.startblock] + seen = {} + backedges = [] + while scheduled: + current = scheduled.pop() + seen[current] = True + for link in current.exits: + if link.target in seen: + backedges.append(link) + else: + scheduled.append(link.target) + return backedges + +def compute_reachability(graph): + reachable = {} + for block in graph.iterblocks(): + reach = {} + scheduled = [block] + while scheduled: + current = scheduled.pop() + for link in current.exits: + if link.target in reachable: + reach = setunion(reach, reachable[link.target]) + continue + if link.target not in reach: + reach[link.target] = True + reachable[block] = reach + return reachable + +def find_loop_blocks(graph): + """find the blocks in a graph that are part of a loop""" + loop = {} + reachable = compute_reachability(graph) + for backedge in find_backedges(graph): + start = backedge.target + end = backedge.prevblock + loop[start] = start + loop[end] = start + scheduled = [start] + seen = {} + while scheduled: + current = scheduled.pop() + connects = end in reachable[current] + seen[current] = True + if connects: + loop[current] = start + for link in current.exits: + if link.target not in seen: + scheduled.append(link.target) + return loop + def md5digest(translator): import md5 m = md5.new() Modified: pypy/dist/pypy/translator/backendopt/test/test_canraise.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_canraise.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_canraise.py Fri Apr 7 23:49:27 2006 @@ -22,6 +22,7 @@ assert not result def test_can_raise_recursive(): + from pypy.translator.transform import insert_ll_stackcheck def g(x): return f(x) @@ -30,8 +31,9 @@ return g(x - 1) return 1 t, ra = translate(f, [int]) + insert_ll_stackcheck(t) ggraph = graphof(t, g) - result = ra.can_raise(ggraph.startblock.operations[1]) + result = ra.can_raise(ggraph.startblock.operations[-1]) assert result # due to stack check every recursive function can raise def test_can_raise_exception(): Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Fri Apr 7 23:49:27 2006 @@ -1,6 +1,6 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter, malloc_to_stack -from pypy.translator.backendopt.escape import find_backedges, find_loop_blocks +from pypy.translator.backendopt.support import find_backedges, find_loop_blocks from pypy.rpython.llinterp import LLInterpreter from pypy.rpython.objectmodel import instantiate From stephan at codespeak.net Sat Apr 8 00:14:41 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Sat, 8 Apr 2006 00:14:41 +0200 (CEST) Subject: [pypy-svn] r25531 - in pypy/dist/pypy: annotation objspace/std objspace/std/test Message-ID: <20060407221441.7D037100E0@code0.codespeak.net> Author: stephan Date: Sat Apr 8 00:14:38 2006 New Revision: 25531 Modified: pypy/dist/pypy/annotation/classdef.py pypy/dist/pypy/objspace/std/complexobject.py pypy/dist/pypy/objspace/std/complextype.py pypy/dist/pypy/objspace/std/model.py pypy/dist/pypy/objspace/std/objspace.py pypy/dist/pypy/objspace/std/setobject.py pypy/dist/pypy/objspace/std/settype.py pypy/dist/pypy/objspace/std/test/test_complexobject.py Log: complex numbers are now considered working: tests are passed and pypy-c can be created. set implementation is nearly done. To test set/frozenset, enable WITHSET in objspace/std/model.py Modified: pypy/dist/pypy/annotation/classdef.py ============================================================================== --- pypy/dist/pypy/annotation/classdef.py (original) +++ pypy/dist/pypy/annotation/classdef.py Sat Apr 8 00:14:38 2006 @@ -277,6 +277,12 @@ def generalize_attr(self, attr, s_value=None): # if the attribute exists in a superclass, generalize there, # as imposed by invariant (I) + #start debug + #if self.name.endswith('W_Root') and attr == 'setdata': + # print 'NAME:',self.name + # import pdb + # pdb.set_trace() + #stop debug for clsdef in self.getmro(): if attr in clsdef.attrs: clsdef._generalize_attr(attr, s_value) Modified: pypy/dist/pypy/objspace/std/complexobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/complexobject.py (original) +++ pypy/dist/pypy/objspace/std/complexobject.py Sat Apr 8 00:14:38 2006 @@ -254,6 +254,11 @@ def coerce__Complex_Complex(space, w_complex1, w_complex2): return space.newtuple([w_complex1, w_complex2]) +def complex_conjugate__Complex(space, w_self): + #w_real = space.call_function(space.w_float,space.wrap(w_self.realval)) + #w_imag = space.call_function(space.w_float,space.wrap(-w_self.imagval)) + return space.newcomplex(w_self.realval,-w_self.imagval) + app = gateway.applevel(""" import math def possint(f): @@ -281,4 +286,5 @@ repr__Complex = app.interphook('repr__Complex') str__Complex = app.interphook('str__Complex') -register_all(vars()) +from pypy.objspace.std import complextype +register_all(vars(), complextype) Modified: pypy/dist/pypy/objspace/std/complextype.py ============================================================================== --- pypy/dist/pypy/objspace/std/complextype.py (original) +++ pypy/dist/pypy/objspace/std/complextype.py Sat Apr 8 00:14:38 2006 @@ -1,14 +1,20 @@ from pypy.interpreter.error import OperationError from pypy.interpreter import gateway from pypy.objspace.std.strutil import interp_string_to_float, ParseStringError +from pypy.objspace.std.objspace import register_all from pypy.objspace.std.noneobject import W_NoneObject from pypy.objspace.std.stdtypedef import GetSetProperty, StdTypeDef, newmethod +from pypy.objspace.std.stdtypedef import StdObjSpaceMultiMethod # ERRORCODES ERR_WRONG_SECOND = "complex() can't take second arg if first is a string" ERR_MALFORMED = "complex() arg is a malformed string" +complex_conjugate = StdObjSpaceMultiMethod('conjugate', 1) + +register_all(vars(),globals()) + def _split_complex(s): slen = len(s) if slen == 0: @@ -122,10 +128,10 @@ raise OperationError(space.w_TypeError, space.wrap(ERR_WRONG_SECOND)) # if arguments can be cast to a float, do it try: - w_real = cast_float(space,w_real) + w_real = space.call_function(space.w_float,w_real) except:pass try: - w_imag = cast_float(space,w_imag) + w_imag = space.call_function(space.w_float,w_imag) except:pass # test for '__complex__' attribute and get result of @@ -175,18 +181,9 @@ return cnum else: return None - -def cast_float(num): - return float(num) """, filename=__file__) extract_complex = app.interphook('extract_complex') -cast_float = app.interphook('cast_float') - -def descr_conjugate(space, w_self): - assert space.is_true(space.isinstance(w_self, space.w_complex)) - from pypy.objspace.std.complexobject import W_ComplexObject - return W_ComplexObject(space,w_self.realval, -w_self.imagval) def complexwprop(name): def fget(space, w_obj): @@ -205,5 +202,6 @@ __new__ = newmethod(descr__new__), real = complexwprop('realval'), imag = complexwprop('imagval'), - conjugate = newmethod(descr_conjugate) ) + +complex_typedef.registermethods(globals()) Modified: pypy/dist/pypy/objspace/std/model.py ============================================================================== --- pypy/dist/pypy/objspace/std/model.py (original) +++ pypy/dist/pypy/objspace/std/model.py Sat Apr 8 00:14:38 2006 @@ -8,7 +8,6 @@ import pypy.interpreter.pycode import pypy.interpreter.special -WITHCOMPLEX = False WITHSET = False class StdTypeModel: @@ -21,8 +20,7 @@ from pypy.objspace.std.booltype import bool_typedef from pypy.objspace.std.inttype import int_typedef from pypy.objspace.std.floattype import float_typedef - if WITHCOMPLEX: - from pypy.objspace.std.complextype import complex_typedef + from pypy.objspace.std.complextype import complex_typedef if WITHSET: from pypy.objspace.std.settype import set_typedef from pypy.objspace.std.settype import frozenset_typedef @@ -48,8 +46,7 @@ from pypy.objspace.std import boolobject from pypy.objspace.std import intobject from pypy.objspace.std import floatobject - if WITHCOMPLEX: - from pypy.objspace.std import complexobject + from pypy.objspace.std import complexobject if WITHSET: from pypy.objspace.std import setobject from pypy.objspace.std import tupleobject @@ -90,11 +87,11 @@ pypy.interpreter.pycode.PyCode: [], pypy.interpreter.special.Ellipsis: [], } - if WITHCOMPLEX: - self.typeorder[complexobject.W_ComplexObject] = [] + self.typeorder[complexobject.W_ComplexObject] = [] if WITHSET: self.typeorder[setobject.W_SetObject] = [] self.typeorder[setobject.W_FrozensetObject] = [] + self.typeorder[setobject.W_SetIterObject] = [] for type in self.typeorder: self.typeorder[type].append((type, None)) @@ -111,36 +108,25 @@ (intobject.W_IntObject, boolobject.delegate_Bool2Int), (longobject.W_LongObject, longobject.delegate_Bool2Long), (floatobject.W_FloatObject, floatobject.delegate_Bool2Float), - #(complexobject.W_ComplexObject, complexobject.delegate_Bool2Complex), + (complexobject.W_ComplexObject, complexobject.delegate_Bool2Complex), ] self.typeorder[intobject.W_IntObject] += [ (longobject.W_LongObject, longobject.delegate_Int2Long), (floatobject.W_FloatObject, floatobject.delegate_Int2Float), - #(complexobject.W_ComplexObject, complexobject.delegate_Int2Complex), + (complexobject.W_ComplexObject, complexobject.delegate_Int2Complex), ] self.typeorder[longobject.W_LongObject] += [ (floatobject.W_FloatObject, floatobject.delegate_Long2Float), - #(complexobject.W_ComplexObject, complexobject.delegate_Long2Complex), + (complexobject.W_ComplexObject, + complexobject.delegate_Long2Complex), ] self.typeorder[floatobject.W_FloatObject] += [ - #(complexobject.W_ComplexObject, complexobject.delegate_Float2Complex), + (complexobject.W_ComplexObject, + complexobject.delegate_Float2Complex), ] self.typeorder[stringobject.W_StringObject] += [ (unicodeobject.W_UnicodeObject, unicodeobject.delegate_String2Unicode), ] - if WITHCOMPLEX: - self.typeorder[boolobject.W_BoolObject] += [ - (complexobject.W_ComplexObject, complexobject.delegate_Bool2Complex), - ] - self.typeorder[intobject.W_IntObject] += [ - (complexobject.W_ComplexObject, complexobject.delegate_Int2Complex), - ] - self.typeorder[longobject.W_LongObject] += [ - (complexobject.W_ComplexObject, complexobject.delegate_Long2Complex), - ] - self.typeorder[floatobject.W_FloatObject] += [ - (complexobject.W_ComplexObject, complexobject.delegate_Float2Complex), - ] # put W_Root everywhere self.typeorder[W_Root] = [] Modified: pypy/dist/pypy/objspace/std/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/std/objspace.py (original) +++ pypy/dist/pypy/objspace/std/objspace.py Sat Apr 8 00:14:38 2006 @@ -7,7 +7,7 @@ from pypy.tool.cache import Cache from pypy.tool.sourcetools import func_with_new_name from pypy.objspace.std.model import W_Object, UnwrapError -from pypy.objspace.std.model import WITHCOMPLEX, WITHSET +from pypy.objspace.std.model import WITHSET from pypy.objspace.std.model import W_ANY, StdObjSpaceMultiMethod, StdTypeModel from pypy.objspace.std.multimethod import FailedToImplement from pypy.objspace.descroperation import DescrOperation @@ -292,21 +292,7 @@ self.wrap(x.stop), self.wrap(x.step)) if isinstance(x, complex): - if WITHCOMPLEX: - return W_ComplexObject(self, x.real, x.imag) - else: - c = self.builtin.get('complex') - return self.call_function(c, - self.wrap(x.real), - self.wrap(x.imag)) - # SD disable for native complex - #if isinstance(x, complex): - # XXX is this right? YYY no, this is wrong right now (CT) - # ZZZ hum, seems necessary for complex literals in co_consts (AR) - # c = self.builtin.get('complex') - # return self.call_function(c, - # self.wrap(x.real), - # self.wrap(x.imag)) + return W_ComplexObject(self, x.real, x.imag) if isinstance(x, set): if WITHSET: @@ -359,10 +345,8 @@ def newfloat(self, floatval): return W_FloatObject(self, floatval) - # SD needed for complex - if WITHCOMPLEX: - def newcomplex(self, realval, imagval): - return W_ComplexObject(self, realval, imagval) + def newcomplex(self, realval, imagval): + return W_ComplexObject(self, realval, imagval) if WITHSET: def newset(self, list_w): Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Sat Apr 8 00:14:38 2006 @@ -5,114 +5,422 @@ from pypy.rpython.objectmodel import r_dict from pypy.interpreter import gateway -def _set_init(w_self, space, wrappeditems): - W_Object.__init__(w_self, space) - w_self.data = data = r_dict(space.eq_w, space.hash_w) - if space.is_true(space.isinstance(wrappeditems, space.w_frozenset)): - data.update(wrappeditems.data) - elif space.is_true(space.isinstance(wrappeditems, space.w_set)): - data.update(wrappeditems.data) - else: - iterable_w = space.unpackiterable(wrappeditems) - for w_item in iterable_w: - w_self.data[w_item] = space.w_True - - -class W_SetObject(W_Object): - from pypy.objspace.std.settype import set_typedef as typedef +class W_BaseSetObject(W_Object): def __init__(w_self, space, wrappeditems): - _set_init(w_self, space, wrappeditems) + W_Object.__init__(w_self, space) + w_self.setdata = setdata = r_dict(space.eq_w, space.hash_w) + iterable_w = space.unpackiterable(wrappeditems) + for w_item in iterable_w: + w_self.setdata[w_item] = None def __repr__(w_self): """representation for debugging purposes""" - reprlist = [repr(w_item) for w_item in w_self.data.keys()] + reprlist = [repr(w_item) for w_item in w_self.setdata.keys()] return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist)) -class W_FrozensetObject(W_Object): - from pypy.objspace.std.settype import frozenset_typedef as typedef +class W_SetObject(W_BaseSetObject): + from pypy.objspace.std.settype import set_typedef as typedef - def __init__(w_self, space, wrappeditems): - _set_init(w_self, space, wrappeditems) +class W_FrozensetObject(W_BaseSetObject): + from pypy.objspace.std.settype import frozenset_typedef as typedef registerimplementation(W_SetObject) registerimplementation(W_FrozensetObject) -app = gateway.applevel(""" - def and__Set_Set(s, o): - return s.intersection(o) +class W_SetIterObject(W_Object): + from pypy.objspace.std.settype import setiter_typedef as typedef + + def __init__(w_self, space, setdata): + W_Object.__init__(w_self, space) + w_self.content = content = setdata + w_self.len = len(content) + w_self.pos = 0 + w_self.iterator = w_self.content.iterkeys() + + def next_entry(w_self): + for w_key in w_self.iterator: + return w_key + else: + return None + +registerimplementation(W_SetIterObject) + +def iter__SetIterObject(space, w_setiter): + return w_setiter + +def next__SetIterObject(space, w_setiter): + content = w_setiter.content + if content is not None: + if w_setiter.len != len(content): + w_setiter.len = -1 # Make this error state sticky + raise OperationError(space.w_RuntimeError, + space.wrap("dictionary changed size during iteration")) + # look for the next entry + w_result = w_setiter.next_entry() + if w_result is not None: + w_setiter.pos += 1 + return w_result + # no more entries + w_setiter.content = None + raise OperationError(space.w_StopIteration, space.w_None) + +def len__SetIterObject(space, w_setiter): + content = w_setiter.content + if content is None or w_setiter.len == -1: + return space.wrap(0) + return space.wrap(w_setiter.len - w_setiter.pos) + +# some helper functions + +def _dict_to_set(space, rpdict): + return space.newset(W_SetIterObject(space, rpdict)) + +def _dict_to_frozenset(space, rpdict): + #return space.newfrozenset(space.newtuple(rpdict.keys())) + return space.newfrozenset(W_SetIterObject(space, rpdict)) + +# helper functions for set operation on dicts + +def _union_dict(space, ldict, rdict, isupdate): + if isupdate: + ld = ldict + else: + ld = ldict.copy() + ld.update(rdict) + return ld, rdict + +def _difference_dict(space, ldict, rdict, isupdate): + if isupdate: + ld = ldict + else: + ld = ldict.copy() + del_list_w = [] + for w_key in ld.iterkeys(): + if w_key in rdict: + del_list_w.append(w_key) + for w_key in del_list_w: + del ld[w_key] + + return ld, rdict + +def _intersection_dict(space, ldict, rdict, isupdate): + if isupdate: + ld = ldict + else: + ld = ldict.copy() + del_list_w = [] + for w_key in ld.iterkeys(): + if w_key not in rdict: + del_list_w.append(w_key) + + for w_key in del_list_w: + del ld[w_key] + + return ld, rdict + + +def _symmetric_difference_dict(space, ldict, rdict, isupdate): + if isupdate: + ld = ldict + else: + ld = ldict.copy() + del_list_w = [] + add_list_w = [] + for w_key in ld.iterkeys(): + if w_key in rdict: + del_list_w.append(w_key) + + for w_key in rdict.iterkeys(): + if w_key not in ld: + add_list_w.append(w_key) + + for w_key in del_list_w: + del ld[w_key] + + for w_key in add_list_w: + ld[w_key] = None + + return ld, rdict + +#end helper functions + +def set_update__Set_Set(space, w_left, w_other): + """Update a set with the union of itself and another.""" + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _union_dict(space, ld, rd, True) + return space.w_None + +set_update__Set_Frozenset = set_update__Set_Set + +def inplace_or__Set_Set(space, w_left, w_other): + set_update__Set_Set(space, w_left, w_other) + return w_left + +inplace_or__Set_Frozenset = inplace_or__Set_Set + +def set_add__Set_ANY(space, w_left, w_other): + """Add an element to a set. + + This has no effect if the element is already present. + """ + w_left.setdata[w_other] = None + return space.w_None + +def set_copy__Set(space, w_left): + return space.newset(w_left) + +def frozenset_copy_Frozenset(space, w_left): + return space.newfrozenset(w_left) + +def set_clear__Set(space, w_left): + w_left.setdata.clear() + return space.w_None + +def set_difference__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _difference_dict(space, ld, rd, False) + return _dict_to_set(space, new_ld) + +set_difference__Set_Frozenset = set_difference__Set_Set +sub__Set_Set = set_difference__Set_Set +sub__Set_Frozenset = set_difference__Set_Set + +def frozenset_difference__Frozenset_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _difference_dict(space, ld, rd, False) + return _dict_to_frozenset(space, new_ld) + +frozenset_difference__Frozenset_Frozenset = frozenset_difference__Frozenset_Set +sub__Frozenset_Set = frozenset_difference__Frozenset_Set +sub__Frozenset_Frozenset = frozenset_difference__Frozenset_Set + + +def set_difference_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _difference_dict(space, ld, rd, True) + return space.w_None + +set_difference_update__Set_Frozenset = set_difference_update__Set_Set + +def inplace_sub__Set_Set(space, w_left, w_other): + set_difference_update__Set_Set(space, w_left, w_other) + return w_left + +inplace_sub__Set_Frozenset = inplace_sub__Set_Set + +def eq__Set_Set(space, w_left, w_other): + if space.is_w(w_left, w_other): + return space.w_True + + if len(w_left.setdata) != len(w_other.setdata): + return space.w_False + + for w_key in w_left.setdata.iterkeys(): + if w_key not in w_other.setdata: + return space.w_False + return space.w_True + +eq__Set_Frozenset = eq__Set_Set +eq__Frozenset_Frozenset = eq__Set_Set +eq__Frozenset_Set = eq__Set_Set + +def contains__Set_ANY(space, w_left, w_other): + return space.newbool(w_other in w_left.setdata) + +contains__Frozenset_ANY = contains__Set_ANY + +def set_issubset__Set_Set(space, w_left, w_other): + if space.is_w(w_left, w_other): + return space.w_True + + ld, rd = w_left.setdata, w_other.setdata + if len(ld) > len(rd): + return space.w_False + + for w_key in ld: + if w_key not in rd: + return space.w_False + return space.w_True + +set_issubset__Set_Frozenset = set_issubset__Set_Set +frozenset_issubset__Frozenset_Set = set_issubset__Set_Set +frozenset_issubset__Frozenset_Frozenset = set_issubset__Set_Set + +le__Set_Set = set_issubset__Set_Set +le__Set_Frozenset = set_issubset__Set_Set +le__Frozenset_Frozenset = set_issubset__Set_Set + +def set_issuperset__Set_Set(space, w_left, w_other): + if space.is_w(w_left, w_other): + return space.w_True + + ld, rd = w_left.setdata, w_other.setdata + if len(ld) < len(rd): + return space.w_False + + for w_key in rd: + if w_key not in ld: + return space.w_False + return space.w_True + +set_issuperset__Set_Frozenset = set_issuperset__Set_Set +frozenset_issuperset__Frozenset_Set = set_issuperset__Set_Set +frozenset_issuperset__Frozenset_Frozenset = set_issuperset__Set_Set + +ge__Set_Set = set_issuperset__Set_Set +ge__Set_Frozenset = set_issuperset__Set_Set +ge__Frozenset_Frozenset = set_issuperset__Set_Set + +def set_discard__Set_ANY(space, w_left, w_item): + if w_item in w_left.setdata: + del w_left.setdata[w_item] + +def set_remove__Set_ANY(space, w_left, w_item): + try: + del w_left.setdata[w_item] + except KeyError: + raise OperationError(space.w_KeyError, + space.call_method(w_item,'__repr__')) + +def hash__Set(space, w_left): + raise OperationError(space.w_TypeError, + space.wrap('set objects are unhashable')) + +def set_pop__Set(space, w_left): + if len(w_left.setdata) == 0: + raise OperationError(space.w_KeyError, + space.wrap('pop from an empty set')) + w_keys = w_left.setdata.keys() + w_value = w_keys[0] + del w_left.setdata[w_value] + + return w_value + +def set_intersection__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _intersection_dict(space, ld, rd, False) + return _dict_to_set(space, new_ld) + +set_intersection__Set_Frozenset = set_intersection__Set_Set +and__Set_Set = set_intersection__Set_Set +and__Set_Frozenset = set_intersection__Set_Set + +def frozenset_intersection__Frozenset_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _intersection_dict(space, ld, rd, False) + return _dict_to_frozenset(space, new_ld) + +frozenset_intersection__Frozenset_Frozenset = \ + frozenset_intersection__Frozenset_Set +and__Frozenset_Set = frozenset_intersection__Frozenset_Set +and__Frozenset_Frozenset = frozenset_intersection__Frozenset_Set + +def set_intersection_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _intersection_dict(space, ld, rd, True) + return space.w_None + +set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set + +def inplace_and__Set_Set(space, w_left, w_other): + set_intersection_update__Set_Set(space, w_left, w_other) + return w_left + +inplace_and__Set_Frozenset = inplace_and__Set_Set + +def set_symmetric_difference__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) + return _dict_to_set(space, new_ld) + +set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set +xor__Set_Set = set_symmetric_difference__Set_Set +xor__Set_Frozenset = set_symmetric_difference__Set_Set + +def frozenset_symmetric_difference__Frozenset_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) + return _dict_to_frozenset(space, new_ld) + +frozenset_symmetric_difference__Frozenset_Frozenset = \ + frozenset_symmetric_difference__Frozenset_Set +xor__Frozenset_Set = frozenset_symmetric_difference__Frozenset_Set +xor__Frozenset_Frozenset = frozenset_symmetric_difference__Frozenset_Set + +def set_symmetric_difference_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _symmetric_difference_dict(space, ld, rd, True) + return space.w_None + +set_symmetric_difference_update__Set_Frozenset = \ + set_symmetric_difference_update__Set_Set + +def inplace_xor__Set_Set(space, w_left, w_other): + set_symmetric_difference_update__Set_Set(space, w_left, w_other) + return w_left + +inplace_xor__Set_Frozenset = inplace_xor__Set_Set + +def set_union__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _union_dict(space, ld, rd, False) + return _dict_to_set(space, new_ld) + +set_union__Set_Frozenset = set_union__Set_Set +or__Set_Set = set_union__Set_Set +or__Set_Frozenset = set_union__Set_Set + +def frozenset_union__Frozenset_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _union_dict(space, ld, rd, False) + return _dict_to_frozenset(space, new_ld) + +frozenset_union__Frozenset_Frozenset = frozenset_union__Frozenset_Set +or__Frozenset_Set = frozenset_union__Frozenset_Set +or__Frozenset_Frozenset = frozenset_union__Frozenset_Set + +def len__Set(space, w_left): + return space.newint(len(w_left.setdata)) + +len__Frozenset = len__Set + +def iter__Set(space, w_left): + return W_SetIterObject(space, w_left.setdata) +iter__Frozenset = iter__Set + +def cmp__Set_Set(space, w_left, w_other): + raise OperationError(space.w_TypeError, + space.wrap('cannot compare sets using cmp()')) + +cmp__Set_Frozenset = cmp__Set_Set +cmp__Frozenset_Frozenset = cmp__Set_Set +cmp__Frozenset_Set = cmp__Set_Set + +app = gateway.applevel(""" def ne__Set_Set(s, o): return not s == o - def ge__Set_Set(s, o): - return s.issuperset(o) - def gt__Set_Set(s, o): return s != o and s.issuperset(o) - def le__Set_Set(s, o): - return s.issubset(o) - def lt__Set_Set(s, o): return s != o and s.issubset(o) - def cmp__Set_Set(s, o): - raise TypeError('cannot compare sets using cmp()') - - def or__Set_Set(s, o): - return s.union(o) - - def xor__Set_Set(s, o): - return s.symmetric_difference(o) - def repr__Set(s): return 'set(%s)' % [x for x in s] def repr__Frozenset(s): return 'frozenset(%s)' % [x for x in s] - def sub__Set_Set(s, o): - return s.difference(o) - - def isub__Set_Set(s, o): - s.difference_update(o) - return s - - def ior__Set_Set(s, o): - s.update(o) - return s - - def iand__Set_Set(s, o): - s.intersection_update(o) - return s - - def ixor__Set_Set(s, o): - s.symmetric_difference_update(o) - return s - """, filename=__file__) -and__Set_Set = app.interphook("and__Set_Set") -and__Set_Frozenset = and__Set_Set -and__Frozenset_Set = and__Set_Set -and__Frozenset_Frozenset = and__Set_Set - ne__Set_Set = app.interphook("ne__Set_Set") ne__Set_Frozenset = ne__Set_Set ne__Frozenset_Set = ne__Set_Set ne__Frozenset_Frozenset = ne__Set_Set -ge__Set_Set = app.interphook("ge__Set_Set") -ge__Set_Frozenset = ge__Set_Set -ge__Frozenset_Set = ge__Set_Set -ge__Frozenset_Frozenset = ge__Set_Set - -le__Set_Set = app.interphook("le__Set_Set") -le__Set_Frozenset = le__Set_Set -le__Frozenset_Set = le__Set_Set -le__Frozenset_Frozenset = le__Set_Set - gt__Set_Set = app.interphook("gt__Set_Set") gt__Set_Frozenset = gt__Set_Set gt__Frozenset_Set = gt__Set_Set @@ -123,39 +431,8 @@ lt__Frozenset_Set = lt__Set_Set lt__Frozenset_Frozenset = lt__Set_Set -cmp__Set_Set = app.interphook("cmp__Set_Set") -cmp__Set_Frozenset = cmp__Set_Set -cmp__Frozenset_Frozenset = cmp__Set_Set -cmp__Frozenset_Set = cmp__Set_Set - -or__Set_Set = app.interphook("or__Set_Set") -or__Set_Frozenset = or__Set_Set -or__Frozenset_Set = or__Set_Set -or__Frozenset_Frozenset = or__Set_Set - -xor__Set_Set = app.interphook("xor__Set_Set") -xor__Set_Frozenset = xor__Set_Set -xor__Frozenset_Set = xor__Set_Set -xor__Frozenset_Frozenset = xor__Set_Set - repr__Set = app.interphook('repr__Set') repr__Frozenset = app.interphook('repr__Frozenset') -sub__Set_Set = app.interphook('sub__Set_Set') -sub__Set_Frozenset = sub__Set_Set -sub__Frozenset_Set = sub__Set_Set -sub__Frozenset_Frozenset = sub__Set_Set - -inplace_sub__Set_Set = app.interphook('isub__Set_Set') -inplace_sub__Set_Frozenset = inplace_sub__Set_Set - -inplace_or__Set_Set = app.interphook('ior__Set_Set') -inplace_or__Set_Frozenset = inplace_or__Set_Set - -inplace_and__Set_Set = app.interphook('iand__Set_Set') -inplace_and__Set_Frozenset = inplace_and__Set_Set - -inplace_xor__Set_Set = app.interphook('ixor__Set_Set') -inplace_xor__Set_Frozenset = inplace_xor__Set_Set - -register_all(vars()) +from pypy.objspace.std import settype +register_all(vars(), settype) Modified: pypy/dist/pypy/objspace/std/settype.py ============================================================================== --- pypy/dist/pypy/objspace/std/settype.py (original) +++ pypy/dist/pypy/objspace/std/settype.py Sat Apr 8 00:14:38 2006 @@ -1,11 +1,38 @@ from pypy.interpreter.error import OperationError from pypy.objspace.std.objspace import register_all from pypy.objspace.std.stdtypedef import StdTypeDef, newmethod -from pypy.objspace.std.stdtypedef import StdObjSpaceMultiMethod +from pypy.objspace.std.stdtypedef import StdObjSpaceMultiMethod as SMM from pypy.interpreter.gateway import NoneNotWrapped from pypy.interpreter import gateway from pypy.objspace.std.model import WITHSET +set_add = SMM('add', 2) +set_clear = SMM('clear', 1) +set_copy = SMM('copy', 1) +set_difference = SMM('difference', 2) +set_difference_update = SMM('difference_update', 2) +set_discard = SMM('discard', 2) +set_intersection = SMM('intersection', 2) +set_intersection_update = SMM('intersection_update', 2) +set_issubset = SMM('issubset', 2) +set_issuperset = SMM('issuperset', 2) +set_pop = SMM('pop', 1) +set_remove = SMM('remove', 2) +set_symmetric_difference = SMM('symmetric_difference', 2) +set_symmetric_difference_update = SMM('symmetric_difference_update', 2) +set_union = SMM('union', 2) +set_update = SMM('update', 2) + +frozenset_copy = SMM('copy', 1) +frozenset_difference = SMM('difference', 2) +frozenset_intersection = SMM('intersection', 2) +frozenset_issubset = SMM('issubset', 2) +frozenset_issuperset = SMM('issuperset', 2) +frozenset_symmetric_difference = SMM('symmetric_difference', 2) +frozenset_union = SMM('union', 2) + +register_all(vars(), globals()) + def descr__set__new__(space, w_settype, w_iterable=NoneNotWrapped): from pypy.objspace.std.setobject import W_SetObject if w_iterable is None: @@ -30,281 +57,22 @@ return w_obj -# some helper functions - -def _extract_data_dict(space, w_left, w_right): - assert (space.is_true(space.isinstance(w_left, space.w_set)) or - space.is_true(space.isinstance(w_left, space.w_frozenset))) - if not (space.is_true(space.isinstance(w_right, space.w_set)) or - space.is_true(space.isinstance(w_right, space.w_frozenset))): - w_right = space.newset(w_right) - - return w_left.data, w_right.data - -def _dict_to_set(space, rpdict): - return space.newset(space.newtuple(rpdict.keys())) - -def _dict_to_frozenset(space, rpdict): - return space.newfrozenset(space.newtuple(rpdict.keys())) - -# helper functions for set operation on dicts - -def _union_dict(space, ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - ld.update(rdict) - return ld, rdict - -def _difference_dict(space, ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - del_list_w = [] - for w_key in ld.iterkeys(): - if w_key in rdict: - del_list_w.append(w_key) - for w_key in del_list_w: - del ld[w_key] - - return ld, rdict - -def _intersection_dict(space, ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - del_list_w = [] - for w_key in ld.iterkeys(): - if w_key not in rdict: - del_list_w.append(w_key) - - for w_key in del_list_w: - del ld[w_key] - - return ld, rdict - - -def _symmetric_difference_dict(space, ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - del_list_w = [] - add_list_w = [] - for w_key in ld.iterkeys(): - if w_key in rdict: - del_list_w.append(w_key) - - for w_key in rdict.iterkeys(): - if w_key not in ld: - add_list_w.append(w_key) - - for w_key in del_list_w: - del ld[w_key] - - for w_key in add_list_w: - ld[w_key] = space.w_True - - return ld, rdict - -def descr_update(space, w_self, w_iterable): - """Update a set with the union of itself and another.""" - ld, rd = _extract_data_dict(space, w_self, w_iterable) - new_ld, rd = _union_dict(space, ld, rd, True) - return space.w_None - -def descr_add(space, w_self, w_other): - """Add an element to a set. - - This has no effect if the element is already present. - """ - - w_self.data[w_other] = space.w_True - -def descr_copy_s(space, w_self): - return space.newset(w_self) - -def descr_copy_fs(space, w_self): - return space.newfrozenset(w_self) - -def descr_clear(space, w_self): - w_self.data.clear() - -def descr_difference_s(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _difference_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) - -def descr_difference_fs(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _difference_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - - -def descr_difference_update(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _difference_dict(space, ld, rd, True) - return space.w_None - -def descr__set__eq__(space, w_self, w_other): - if space.is_w(w_self, w_other): - return space.w_True - - if len(w_self.data) != len(w_other.data): - return space.w_False - - for w_key in w_self.data.iterkeys(): - if w_key not in w_other.data: - return space.w_False - return space.w_True - -def descr__set__contains__(space, w_self, w_other): - return space.newbool(w_other in w_self.data) - -def descr_issubset(space, w_self, w_other): - if space.is_w(w_self, w_other): - return space.w_True - - if len(w_self.data) > len(w_other.data): - return space.w_False - - for w_key in w_self.data: - if w_key not in w_other.data: - return space.w_False - return space.w_True - -def descr_issuperset(space, w_self, w_other): - if space.is_w(w_self, w_other): - return space.w_True - - if len(w_self.data) < len(w_other.data): - return space.w_False - - for w_key in w_other.data: - if w_key not in w_self.data: - return space.w_False - return space.w_True - -def descr_discard(space, w_self, w_item): - if w_item in w_self.data: - del w_self.data[w_item] - -def descr_remove(space, w_self, w_item): - try: - del w_self.data[w_item] - except KeyError: - raise OperationError(space.w_KeyError, - space.call_method(w_item,'__repr__')) - -def descr__set__hash__(space, w_self): - raise OperationError(space.w_TypeError, - space.wrap('set objects are unhashable')) - -def descr_pop(space, w_self): - if len(w_self.data) == 0: - raise OperationError(space.w_KeyError, - space.wrap('pop from an empty set')) - w_keys = w_self.data.keys() - w_value = w_keys[0] - del w_self.data[w_value] - - return w_value - -def descr_intersection_s(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _intersection_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) - -def descr_intersection_fs(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _intersection_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -def descr_intersection_update(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _intersection_dict(space, ld, rd, True) - return space.w_None - -def descr_symmetric_difference_s(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) - -def descr_symmetric_difference_fs(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -def descr_symmetric_difference_update(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _symmetric_difference_dict(space, ld, rd, True) - return space.w_None - -def descr_union_s(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _union_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) - -def descr_union_fs(space, w_self, w_other): - ld, rd = _extract_data_dict(space, w_self, w_other) - new_ld, rd = _union_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -def descr__set__len__(space, w_self): - return space.newint(len(w_self.data)) - -def descr__set__iter__(space, w_self): - from pypy.objspace.std import iterobject - return iterobject.W_SeqIterObject(space, - space.newtuple(w_self.data.keys())) - set_typedef = StdTypeDef("set", __doc__ = """set(iterable) --> set object Build an unordered collection.""", __new__ = newmethod(descr__set__new__), - __eq__ = newmethod(descr__set__eq__), - __contains__ = newmethod(descr__set__contains__), - __len__ = newmethod(descr__set__len__), - __iter__ = newmethod(descr__set__iter__), - __hash__ = newmethod(descr__set__hash__), - add = newmethod(descr_add), - clear = newmethod(descr_clear), - copy = newmethod(descr_copy_s), - difference = newmethod(descr_difference_s), - difference_update = newmethod(descr_difference_update), - discard = newmethod(descr_discard), - intersection = newmethod(descr_intersection_s), - intersection_update = newmethod(descr_intersection_update), - issubset = newmethod(descr_issubset), - issuperset = newmethod(descr_issuperset), - pop = newmethod(descr_pop), - remove = newmethod(descr_remove), - symmetric_difference = newmethod(descr_symmetric_difference_s), - symmetric_difference_update = newmethod(descr_symmetric_difference_update), - union = newmethod(descr_union_s), - update = newmethod(descr_update), ) -#set_typedef.registermethods(globals()) +set_typedef.registermethods(globals()) frozenset_typedef = StdTypeDef("frozenset", __doc__ = """frozenset(iterable) --> frozenset object Build an immutable unordered collection.""", __new__ = newmethod(descr__frozenset__new__), - __eq__ = newmethod(descr__set__eq__), - __contains__ = newmethod(descr__set__contains__), - __len__ = newmethod(descr__set__len__), - __iter__ = newmethod(descr__set__iter__), - copy = newmethod(descr_copy_fs), - difference = newmethod(descr_difference_fs), - intersection = newmethod(descr_intersection_fs), - issubset = newmethod(descr_issubset), - issuperset = newmethod(descr_issuperset), - symmetric_difference = newmethod(descr_symmetric_difference_fs), - union = newmethod(descr_union_fs), ) + +frozenset_typedef.registermethods(globals()) + +setiter_typedef = StdTypeDef("setiterator") Modified: pypy/dist/pypy/objspace/std/test/test_complexobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/test/test_complexobject.py (original) +++ pypy/dist/pypy/objspace/std/test/test_complexobject.py Sat Apr 8 00:14:38 2006 @@ -1,8 +1,5 @@ import autopath import py -from pypy.objspace.std.model import WITHCOMPLEX -if not WITHCOMPLEX: - py.test.skip("only works if WITHCOMPLEX is enabled") from pypy.objspace.std import complexobject as cobj from pypy.objspace.std import complextype as cobjtype from pypy.objspace.std.objspace import FailedToImplement From cfbolz at codespeak.net Sat Apr 8 10:14:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 10:14:49 +0200 (CEST) Subject: [pypy-svn] r25532 - pypy/dist/pypy/translator/backendopt Message-ID: <20060408081449.29A9D100EA@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 10:14:47 2006 New Revision: 25532 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: (pedronis, cfbolz): increase the weight of edges that go back in a loop because they are much more likely Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 10:14:47 2006 @@ -9,7 +9,7 @@ from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void, Ptr from pypy.rpython import rmodel from pypy.tool.algo import sparsemat -from pypy.translator.backendopt.support import log, split_block_with_keepalive, generate_keepalive +from pypy.translator.backendopt.support import log, split_block_with_keepalive, generate_keepalive, find_backedges, find_loop_blocks BASE_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -426,6 +426,7 @@ 'keepalive': 0, 'direct_call': 2, # guess 'indirect_call': 2, # guess +# 'malloc': 2, 'yield_current_frame_to_caller': sys.maxint, # XXX bit extreme } @@ -444,18 +445,38 @@ for block in graph.iterblocks(): blockmap[block] = len(blocks) blocks.append(block) + backedges = dict.fromkeys(find_backedges(graph), True) + loops = find_loop_blocks(graph) M = sparsemat.SparseMatrix(len(blocks)) vector = [] for i, block in enumerate(blocks): vector.append(block_weight(block)) M[i, i] = 1 if block.exits: - f = 1.0 / len(block.exits) + if block not in loops: + current_loop_start = None + else: + current_loop_start = loops[block] + loop_exits = [] + for link in block.exits: + if (link.target in loops and + loops[link.target] is current_loop_start): + loop_exits.append(link) + if len(loop_exits) and len(loop_exits) < len(block.exits): + f = 0.3 / (len(block.exits) - len(loop_exits)) + b = 0.7 / len(loop_exits) + else: + b = f = 1.0 / len(block.exits) for link in block.exits: - M[i, blockmap[link.target]] -= f + if (link.target in loops and + loops[link.target] is current_loop_start): + M[i, blockmap[link.target]] -= b + else: + M[i, blockmap[link.target]] -= f try: Solution = M.solve(vector) except ValueError: + raise return sys.maxint else: res = Solution[blockmap[graph.startblock]] From hpk at codespeak.net Sat Apr 8 10:32:07 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 10:32:07 +0200 (CEST) Subject: [pypy-svn] r25533 - pypy/branch/stacklesscfg/pypy/rpython Message-ID: <20060408083207.EBFB610279@code0.codespeak.net> Author: hpk Date: Sat Apr 8 10:32:07 2006 New Revision: 25533 Modified: pypy/branch/stacklesscfg/pypy/rpython/rtyper.py Log: factor out converting of links Modified: pypy/branch/stacklesscfg/pypy/rpython/rtyper.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/rpython/rtyper.py (original) +++ pypy/branch/stacklesscfg/pypy/rpython/rtyper.py Sat Apr 8 10:32:07 2006 @@ -333,34 +333,36 @@ # consider it as a link source instead self.insert_link_conversions(extrablock) + def _convert_link(self, block, link): + if link.exitcase is not None: + if isinstance(block.exitswitch, Variable): + r_case = self.bindingrepr(block.exitswitch) + else: + assert block.exitswitch == c_last_exception + r_case = rclass.get_type_repr(self) + link.llexitcase = r_case.convert_const(link.exitcase) + else: + link.llexitcase = None + + a = link.last_exception + if isinstance(a, Variable): + a.concretetype = self.exceptiondata.lltype_of_exception_type + elif isinstance(a, Constant): + link.last_exception = inputconst( + self.exceptiondata.r_exception_type, a.value) + + a = link.last_exc_value + if isinstance(a, Variable): + a.concretetype = self.exceptiondata.lltype_of_exception_value + elif isinstance(a, Constant): + link.last_exc_value = inputconst( + self.exceptiondata.r_exception_value, a.value) + def insert_link_conversions(self, block, skip=0): # insert the needed conversions on the links can_insert_here = block.exitswitch is None and len(block.exits) == 1 for link in block.exits[skip:]: - if link.exitcase is not None: - if isinstance(block.exitswitch, Variable): - r_case = self.bindingrepr(block.exitswitch) - else: - assert block.exitswitch == c_last_exception - r_case = rclass.get_type_repr(self) - link.llexitcase = r_case.convert_const(link.exitcase) - else: - link.llexitcase = None - - a = link.last_exception - if isinstance(a, Variable): - a.concretetype = self.exceptiondata.lltype_of_exception_type - elif isinstance(a, Constant): - link.last_exception = inputconst( - self.exceptiondata.r_exception_type, a.value) - - a = link.last_exc_value - if isinstance(a, Variable): - a.concretetype = self.exceptiondata.lltype_of_exception_value - elif isinstance(a, Constant): - link.last_exc_value = inputconst( - self.exceptiondata.r_exception_value, a.value) - + self._convert_link(block, link) inputargs_reprs = self.setup_block_entry(link.target) newops = self.make_new_lloplist(block) newlinkargs = {} From hpk at codespeak.net Sat Apr 8 10:33:15 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 10:33:15 +0200 (CEST) Subject: [pypy-svn] r25534 - in pypy/branch/stacklesscfg/pypy/translator/stackless: . test Message-ID: <20060408083315.6497B10287@code0.codespeak.net> Author: hpk Date: Sat Apr 8 10:33:14 2006 New Revision: 25534 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: checkgraph() on the transformed graphs works (but something is still not quite right) Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py Sat Apr 8 10:33:14 2006 @@ -41,10 +41,8 @@ st = StacklessTransfomer(t) st.transform_graph(graphof(t, example)) st.transform_graph(graphof(t, g)) - st.transform_graph(graphof(t, check)) - #for graph in t.graphs: - #checkgraph(graph) # XXX - #st.transform_graph(graph) + for graph in t.graphs: + checkgraph(graph) # XXX example_graph = graphof(t, example) if conftest.option.view: t.view() Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 10:33:14 2006 @@ -7,6 +7,7 @@ from pypy.annotation import model as annmodel from pypy.rpython.annlowlevel import MixLevelHelperAnnotator from pypy.translator.stackless import code +from pypy.rpython.rclass import getinstancerepr STORAGE_TYPES = [llmemory.Address, lltype.Signed, @@ -141,14 +142,16 @@ var_unwind_exception = varoftype(evalue) - save_block = self.generate_save_block(link.args, var_unwind_exception) + save_block = self.generate_save_block( + link.args, var_unwind_exception) newlink = model.Link(link.args + [var_unwind_exception], save_block, code.UnwindException) - r_case = rclass.get_type_repr(self.translator.rtyper) - newlink.llexitcase = r_case.convert_const(newlink.exitcase) block.exitswitch = model.c_last_exception + newlink.last_exception = model.Constant(code.UnwindException) + newlink.last_exc_value = var_unwind_exception block.recloseblock(link, newlink) # exits.append(newlink) + self.translator.rtyper._convert_link(block, newlink) # ARGH ... block = after_block @@ -161,8 +164,6 @@ etype = edata.lltype_of_exception_type evalue = edata.lltype_of_exception_value inputargs = [copyvar(self.translator, v) for v in varstosave] - var_unwind_exception0 = copyvar(self.translator, var_unwind_exception) - from pypy.rpython.rclass import getinstancerepr var_unwind_exception = varoftype(getinstancerepr(self.translator.rtyper, self.translator.annotator.bookkeeper.getuniqueclassdef( code.UnwindException)).lowleveltype) @@ -202,7 +203,8 @@ [var_unwind_exception, model.Constant("frame", lltype.Void)], varoftype(lltype.Void))) - save_state_block.closeblock(model.Link([varoftype(etype), varoftype(evalue)], + save_state_block.closeblock(model.Link([model.Constant(code.UnwindException), + var_unwind_exception], self.curr_graph.exceptblock)) return save_state_block From cfbolz at codespeak.net Sat Apr 8 10:33:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 10:33:28 +0200 (CEST) Subject: [pypy-svn] r25535 - pypy/dist/pypy/translator/backendopt Message-ID: <20060408083328.C1DAD102A2@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 10:33:27 2006 New Revision: 25535 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: wua! bad me Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 10:33:27 2006 @@ -476,7 +476,6 @@ try: Solution = M.solve(vector) except ValueError: - raise return sys.maxint else: res = Solution[blockmap[graph.startblock]] From hpk at codespeak.net Sat Apr 8 12:14:28 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 12:14:28 +0200 (CEST) Subject: [pypy-svn] r25536 - in pypy/branch/stacklesscfg/pypy/translator/stackless: . test Message-ID: <20060408101428.A75971021C@code0.codespeak.net> Author: hpk Date: Sat Apr 8 12:14:27 2006 New Revision: 25536 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: (hpk, mwh) intermediate checkins with some bugs fixed but C not compiling ... stay tuned Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/test/test_transform.py Sat Apr 8 12:14:27 2006 @@ -1,8 +1,13 @@ +import py +import os from pypy.translator.stackless.transform import StacklessTransfomer +from pypy.translator.c.genc import CStandaloneBuilder from pypy.rpython.memory.gctransform import varoftype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.translator import TranslationContext, graphof from pypy.objspace.flow.model import checkgraph +from pypy.annotation import model as annmodel +from pypy.annotation.listdef import ListDef from pypy import conftest ## def test_frame_types(): @@ -32,18 +37,32 @@ raise UnwindException # XXX or so def g(x): check(x) - return 2 + return x + 1 def example(x): - return g(x) + x + 1 + return g(x) + 1 + res = run_stackless_function(example, example, g) + assert res == "6" + +def run_stackless_function(fn, *stacklessfuncs): + def entry_point(argv): + os.write(1, str(fn(len(argv)))+'\n') + return 0 + + s_list_of_strings = annmodel.SomeList(ListDef(None, annmodel.SomeString())) + s_list_of_strings.listdef.resize() t = TranslationContext() - t.buildannotator().build_types(example, [int]) + t.buildannotator().build_types(entry_point, [s_list_of_strings]) t.buildrtyper().specialize() + st = StacklessTransfomer(t) - st.transform_graph(graphof(t, example)) - st.transform_graph(graphof(t, g)) - for graph in t.graphs: - checkgraph(graph) # XXX - example_graph = graphof(t, example) + for func in stacklessfuncs: + graph = graphof(t, func) + st.transform_graph(graph) + checkgraph(graph) if conftest.option.view: t.view() - + + cbuilder = CStandaloneBuilder(t, entry_point) + cbuilder.generate_source() + cbuilder.compile() + return cbuilder.cmdexec('') Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 12:14:27 2006 @@ -1,5 +1,5 @@ from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rpython import rarithmetic, rclass +from pypy.rpython import rarithmetic, rclass, rmodel from pypy.translator.backendopt import support from pypy.objspace.flow import model from pypy.rpython.memory.gctransform import varoftype @@ -120,7 +120,7 @@ assert self.curr_graph is None self.curr_graph = graph - for block in graph.iterblocks(): + for block in list(graph.iterblocks()): self.transform_block(block) if self.resume_points: XXX @@ -141,17 +141,18 @@ link = block.exits[0] var_unwind_exception = varoftype(evalue) - + + args = [v for v in link.args if v is not op.result] save_block = self.generate_save_block( - link.args, var_unwind_exception) + args, var_unwind_exception) - newlink = model.Link(link.args + [var_unwind_exception], + newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) block.exitswitch = model.c_last_exception newlink.last_exception = model.Constant(code.UnwindException) newlink.last_exc_value = var_unwind_exception block.recloseblock(link, newlink) # exits.append(newlink) - self.translator.rtyper._convert_link(block, newlink) + self._convertlink(block, newlink) # ARGH ... block = after_block @@ -159,20 +160,34 @@ else: i += 1 + def _convertlink(self, block, link): + rtyper = self.translator.rtyper + if link.exitcase is not None: + if isinstance(block.exitswitch, model.Variable): + r_case = rtyper.bindingrepr(block.exitswitch) + else: + assert block.exitswitch == model.c_last_exception + r_case = rclass.get_type_repr(rtyper) + link.llexitcase = r_case.convert_const(link.exitcase) + else: + link.llexitcase = None + def generate_save_block(self, varstosave, var_unwind_exception): - edata = self.translator.rtyper.getexceptiondata() + rtyper = self.translator.rtyper + edata = rtyper.getexceptiondata() etype = edata.lltype_of_exception_type evalue = edata.lltype_of_exception_value inputargs = [copyvar(self.translator, v) for v in varstosave] - var_unwind_exception = varoftype(getinstancerepr(self.translator.rtyper, + unwind_exception_type = getinstancerepr(rtyper, self.translator.annotator.bookkeeper.getuniqueclassdef( - code.UnwindException)).lowleveltype) + code.UnwindException)).lowleveltype + var_unwind_exception = copyvar(self.translator, var_unwind_exception) fields = [] for i, v in enumerate(varstosave): if v.concretetype is not lltype.Void: fields.append(('field_%d'%(i,), v.concretetype)) - frame_type = lltype.Struct("S", + frame_type = lltype.GcStruct("S", ('header', STATE_HEADER), *fields) @@ -191,24 +206,30 @@ ## state.header.state = XXX ## u.frame = state.header header_var = varoftype(lltype.Ptr(STATE_HEADER)) + var_exc = varoftype(unwind_exception_type) + saveops.append(model.SpaceOperation("cast_pointer", [var_unwind_exception], + var_exc)) + saveops.append(model.SpaceOperation("cast_pointer", [frame_state_var], header_var)) var_unwind_exception_frame = varoftype(lltype.Ptr(STATE_HEADER)) - saveops.append(model.SpaceOperation("getfield", - [var_unwind_exception, model.Constant("frame", lltype.Void)], - var_unwind_exception_frame)) - saveops.append(model.SpaceOperation("setfield", - [header_var, model.Constant("f_back", lltype.Void), var_unwind_exception_frame], - varoftype(lltype.Void))) - saveops.append(model.SpaceOperation("setfield", - [var_unwind_exception, model.Constant("frame", lltype.Void)], - varoftype(lltype.Void))) - - save_state_block.closeblock(model.Link([model.Constant(code.UnwindException), - var_unwind_exception], + # XXX + saveops.append(model.SpaceOperation( + "getfield", [var_exc, model.Constant("frame_bottom", lltype.Void)], + var_unwind_exception_frame)) + saveops.append(model.SpaceOperation( + "setfield", [header_var, model.Constant("f_back", lltype.Void), + var_unwind_exception_frame], + varoftype(lltype.Void))) + saveops.append(model.SpaceOperation( + "setfield", [var_exc, model.Constant("frame", lltype.Void), header_var], + varoftype(lltype.Void))) + + s_exc = self.translator.annotator.bookkeeper.immutablevalue(code.UnwindException) + r_exc = rtyper.getrepr(s_exc) + c_unwindexception = rmodel.inputconst(r_exc, code.UnwindException) + save_state_block.closeblock(model.Link([c_unwindexception, var_exc], self.curr_graph.exceptblock)) - return save_state_block - def generate_saveops(self, frame_state_var, varstosave): From mwh at codespeak.net Sat Apr 8 13:13:13 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 13:13:13 +0200 (CEST) Subject: [pypy-svn] r25537 - pypy/branch/stacklesscfg/pypy/translator/stackless Message-ID: <20060408111313.D7BCE1026C@code0.codespeak.net> Author: mwh Date: Sat Apr 8 13:13:13 2006 New Revision: 25537 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: (hpk, mwh) whack whack whack until it compiles. it segfaults, but what did you expect? Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 13:13:13 2006 @@ -149,10 +149,10 @@ newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) block.exitswitch = model.c_last_exception - newlink.last_exception = model.Constant(code.UnwindException) + newlink.last_exception = model.Constant(code.UnwindException, etype) newlink.last_exc_value = var_unwind_exception block.recloseblock(link, newlink) # exits.append(newlink) - self._convertlink(block, newlink) + self.translator.rtyper._convert_link(block, newlink) # ARGH ... block = after_block @@ -202,33 +202,38 @@ saveops.extend(self.generate_saveops(frame_state_var, inputargs)) -## state.header.f_back = u.frame -## state.header.state = XXX -## u.frame = state.header - header_var = varoftype(lltype.Ptr(STATE_HEADER)) var_exc = varoftype(unwind_exception_type) saveops.append(model.SpaceOperation("cast_pointer", [var_unwind_exception], var_exc)) + +## header = state.header +## header.state = XXX +## exc.frame_bottom.f_back = state.header +## exc.frame_bottom = state.header + + var_header = varoftype(lltype.Ptr(STATE_HEADER)) - saveops.append(model.SpaceOperation("cast_pointer", [frame_state_var], header_var)) - var_unwind_exception_frame = varoftype(lltype.Ptr(STATE_HEADER)) - # XXX + # XXX should this be getsubstruct? + saveops.append(model.SpaceOperation("cast_pointer", [frame_state_var], var_header)) + + var_exc_frame_bottom = varoftype(lltype.Ptr(STATE_HEADER)) saveops.append(model.SpaceOperation( - "getfield", [var_exc, model.Constant("frame_bottom", lltype.Void)], - var_unwind_exception_frame)) + "getfield", [var_exc, model.Constant("inst_frame_bottom", lltype.Void)], + var_exc_frame_bottom)) + saveops.append(model.SpaceOperation( - "setfield", [header_var, model.Constant("f_back", lltype.Void), - var_unwind_exception_frame], + "setfield", [var_exc_frame_bottom, model.Constant("f_back", lltype.Void), + var_header], varoftype(lltype.Void))) saveops.append(model.SpaceOperation( - "setfield", [var_exc, model.Constant("frame", lltype.Void), header_var], + "setfield", [var_exc, model.Constant("inst_frame_bottom", lltype.Void), var_header], varoftype(lltype.Void))) - s_exc = self.translator.annotator.bookkeeper.immutablevalue(code.UnwindException) - r_exc = rtyper.getrepr(s_exc) - c_unwindexception = rmodel.inputconst(r_exc, code.UnwindException) + type_repr = rclass.get_type_repr(rtyper) + c_unwindexception = model.Constant(type_repr.convert_const(code.UnwindException), type_repr.lowleveltype) save_state_block.closeblock(model.Link([c_unwindexception, var_exc], self.curr_graph.exceptblock)) + self.translator.rtyper._convert_link(save_state_block, save_state_block.exits[0]) return save_state_block From mwh at codespeak.net Sat Apr 8 13:43:40 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 13:43:40 +0200 (CEST) Subject: [pypy-svn] r25538 - pypy/branch/stacklesscfg/pypy/translator/stackless Message-ID: <20060408114340.7BD4810102@code0.codespeak.net> Author: mwh Date: Sat Apr 8 13:43:39 2006 New Revision: 25538 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/code.py pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: (hpk, mwh) use a helper function to attach the frame state to the UnwindException. this makes the segfaults go away, but code that tries to unwind will crash out with an RPythonException, as we don't run the slp_main_loop yet. Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/code.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/code.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/code.py Sat Apr 8 13:43:39 2006 @@ -79,3 +79,10 @@ raise global_state.exception +def add_frame_state(u, frame_state): + if not u.frame_top: + u.frame_top = u.frame_bottom = frame_state + else: + u.frame_bottom.f_back = frame_state + u.frame_bottom = frame_state + Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 13:43:39 2006 @@ -77,6 +77,11 @@ self.translator = translator edata = translator.rtyper.getexceptiondata() + bk = translator.annotator.bookkeeper + + self.unwind_exception_type = getinstancerepr( + self.translator.rtyper, + bk.getuniqueclassdef(code.UnwindException)).lowleveltype self.frametypes = {} self.curr_graph = None @@ -91,6 +96,20 @@ graph=slp_main_loop_graph), SLP_MAIN_LOOP_TYPE) + annotations = [annmodel.SomeInstance(bk.getuniqueclassdef(code.UnwindException)), + annmodel.SomePtr(lltype.Ptr(STATE_HEADER))] + + add_frame_state_graph = mixlevelannotator.getgraph( + code.add_frame_state, + annotations, l2a(lltype.Void)) + ADD_FRAME_STATE_TYPE = lltype.FuncType( + [self.unwind_exception_type, lltype.Ptr(STATE_HEADER)], + lltype.Void) + self.add_frame_state_ptr = model.Constant(lltype.functionptr( + ADD_FRAME_STATE_TYPE, "add_frame_state", + graph=add_frame_state_graph), + lltype.Ptr(ADD_FRAME_STATE_TYPE)) + mixlevelannotator.finish() def frame_type_for_vars(self, vars): @@ -178,9 +197,6 @@ etype = edata.lltype_of_exception_type evalue = edata.lltype_of_exception_value inputargs = [copyvar(self.translator, v) for v in varstosave] - unwind_exception_type = getinstancerepr(rtyper, - self.translator.annotator.bookkeeper.getuniqueclassdef( - code.UnwindException)).lowleveltype var_unwind_exception = copyvar(self.translator, var_unwind_exception) fields = [] @@ -202,31 +218,18 @@ saveops.extend(self.generate_saveops(frame_state_var, inputargs)) - var_exc = varoftype(unwind_exception_type) + var_exc = varoftype(self.unwind_exception_type) saveops.append(model.SpaceOperation("cast_pointer", [var_unwind_exception], var_exc)) - -## header = state.header -## header.state = XXX -## exc.frame_bottom.f_back = state.header -## exc.frame_bottom = state.header var_header = varoftype(lltype.Ptr(STATE_HEADER)) # XXX should this be getsubstruct? saveops.append(model.SpaceOperation("cast_pointer", [frame_state_var], var_header)) - - var_exc_frame_bottom = varoftype(lltype.Ptr(STATE_HEADER)) - saveops.append(model.SpaceOperation( - "getfield", [var_exc, model.Constant("inst_frame_bottom", lltype.Void)], - var_exc_frame_bottom)) - - saveops.append(model.SpaceOperation( - "setfield", [var_exc_frame_bottom, model.Constant("f_back", lltype.Void), - var_header], - varoftype(lltype.Void))) + saveops.append(model.SpaceOperation( - "setfield", [var_exc, model.Constant("inst_frame_bottom", lltype.Void), var_header], + "direct_call", + [self.add_frame_state_ptr, var_exc, var_header], varoftype(lltype.Void))) type_repr = rclass.get_type_repr(rtyper) From hpk at codespeak.net Sat Apr 8 13:54:19 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 13:54:19 +0200 (CEST) Subject: [pypy-svn] r25539 - pypy/branch/stacklesscfg/pypy/translator/stackless Message-ID: <20060408115419.BF22010274@code0.codespeak.net> Author: hpk Date: Sat Apr 8 13:54:19 2006 New Revision: 25539 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: (mwh, hpk) remove unused code, prune varstosave early Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 13:54:19 2006 @@ -179,19 +179,8 @@ else: i += 1 - def _convertlink(self, block, link): - rtyper = self.translator.rtyper - if link.exitcase is not None: - if isinstance(block.exitswitch, model.Variable): - r_case = rtyper.bindingrepr(block.exitswitch) - else: - assert block.exitswitch == model.c_last_exception - r_case = rclass.get_type_repr(rtyper) - link.llexitcase = r_case.convert_const(link.exitcase) - else: - link.llexitcase = None - def generate_save_block(self, varstosave, var_unwind_exception): + varstosave = [v for v in varstosave if v.concretetype is not lltype.Void] rtyper = self.translator.rtyper edata = rtyper.getexceptiondata() etype = edata.lltype_of_exception_type @@ -201,8 +190,9 @@ fields = [] for i, v in enumerate(varstosave): - if v.concretetype is not lltype.Void: - fields.append(('field_%d'%(i,), v.concretetype)) + assert v.concretetype is not lltype.Void + fields.append(('field_%d'%(i,), v.concretetype)) + frame_type = lltype.GcStruct("S", ('header', STATE_HEADER), *fields) From cfbolz at codespeak.net Sat Apr 8 13:56:16 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 13:56:16 +0200 (CEST) Subject: [pypy-svn] r25540 - in pypy/dist/pypy/rpython: . lltypesystem memory test Message-ID: <20060408115616.ADB0310275@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 13:56:15 2006 New Revision: 25540 Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py pypy/dist/pypy/rpython/lltypesystem/rclass.py pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/normalizecalls.py pypy/dist/pypy/rpython/test/test_normalizecalls.py Log: (pedronis, cfbolz): fix the minid, maxid computation so that the ComputedIntSymbolic subclasses have proper comparisons defined. Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Sat Apr 8 13:56:15 2006 @@ -78,6 +78,7 @@ 'direct_call': LLOp(canraise=(Exception,)), 'indirect_call': LLOp(canraise=(Exception,)), + 'safe_call': LLOp(), # __________ numeric operations __________ Modified: pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rbuiltin.py Sat Apr 8 13:56:15 2006 @@ -32,7 +32,7 @@ if isinstance(v_cls, Constant): cls = v_cls.value # XXX re-implement the following optimization - #if cls.subclassrange_max == cls.subclassrange_min + 1: + #if cls.subclassrange_max == cls.subclassrange_min: # # a class with no subclass # return hop.gendirectcall(rclass.ll_isinstance_exact, v_obj, v_cls) #else: Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Sat Apr 8 13:56:15 2006 @@ -267,7 +267,7 @@ if isinstance(v_cls2, Constant): cls2 = v_cls2.value # XXX re-implement the following optimization -## if cls2.subclassrange_max == cls2.subclassrange_min + 1: +## if cls2.subclassrange_max == cls2.subclassrange_min: ## # a class with no subclass ## return hop.genop('ptr_eq', [v_cls1, v_cls2], resulttype=Bool) ## else: @@ -708,10 +708,10 @@ return cast_pointer(OBJECTPTR, obj).typeptr def ll_issubclass(subcls, cls): - return cls.subclassrange_min <= subcls.subclassrange_min < cls.subclassrange_max + return cls.subclassrange_min <= subcls.subclassrange_min <= cls.subclassrange_max def ll_issubclass_const(subcls, minid, maxid): - return minid <= subcls.subclassrange_min < maxid + return minid <= subcls.subclassrange_min <= maxid def ll_isinstance(obj, cls): # obj should be cast to OBJECT or NONGCOBJECT Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Sat Apr 8 13:56:15 2006 @@ -4,6 +4,7 @@ from pypy.rpython.lltypesystem import llmemory from pypy.rpython.lltypesystem import lltype from pypy.rpython.memory.lltypelayout import convert_offset_to_int +from pypy.rpython.objectmodel import ComputedIntSymbolic class address(object): #XXX should be _address! @@ -50,10 +51,12 @@ return simulator.getstruct(fmt, self.intaddress) def _store(self, fmt, *values): - # XXX annoyance: suddenly an OffsetOf changes into a Signed?! + # XXX annoyance: suddenly a Symbolic changes into a Signed?! from pypy.rpython.memory.lltypelayout import convert_offset_to_int if len(values) == 1 and isinstance(values[0], llmemory.AddressOffset): values = [convert_offset_to_int(values[0])] + elif len(values) == 1 and isinstance(values[0], ComputedIntSymbolic): + values = [values[0].compute_fn()] simulator.setstruct(fmt, self.intaddress, *values) def __nonzero__(self): Modified: pypy/dist/pypy/rpython/normalizecalls.py ============================================================================== --- pypy/dist/pypy/rpython/normalizecalls.py (original) +++ pypy/dist/pypy/rpython/normalizecalls.py Sat Apr 8 13:56:15 2006 @@ -275,42 +275,117 @@ # ____________________________________________________________ class MinIdSymbolic(ComputedIntSymbolic): - def __init__(self, classdef): + def __init__(self, classdef, rootid): self.classdef = classdef + if classdef is None: + self.parent = None + elif classdef.basedef is None: + self.parent = rootid + else: + self.parent = classdef.basedef.minid + if self.parent: + self.parent.children.append(self) + self.children = [] + if rootid is None: + self.rootid = self + else: + self.rootid = rootid + def compute_fn(self): if self.classdef.minid is self: compute_inheritance_ids(self.classdef.bookkeeper) return self.classdef.minid + def __eq__(self, other): + if isinstance(other, MinIdSymbolic): + return self is other + elif isinstance(other, MaxIdSymbolic): + return False + raise NotImplementedError + + def __ne__(self, other): + return not (self == other) + + def __le__(self, other): + if isinstance(other, MinIdSymbolic): + common_classdef = self.classdef.commonbase(other.classdef) + if common_classdef is None: + baseid = self.rootid + else: + baseid = common_classdef.minid + if baseid is self: + return True + if baseid is other: + return False + current_self = self + while current_self.parent is not baseid: + current_self = current_self.parent + current_other = other + while current_other.parent is not baseid: + current_other = current_other.parent + selfindex = baseid.children.index(current_self) + otherindex = baseid.children.index(current_other) + return selfindex <= otherindex + elif isinstance(other, MaxIdSymbolic): + rightmost = other.minid + while rightmost.children: + rightmost = rightmost.children[-1] + return self <= rightmost + raise NotImplementedError + + def compute_inheritance_ids(self, id_=0): + if self.classdef is not None: + self.classdef.minid = id_ + maxid = id_ + for child in self.children: + maxid = child.compute_inheritance_ids(maxid + 1) + if self.classdef is not None: + self.classdef.maxid = maxid + return maxid + class MaxIdSymbolic(ComputedIntSymbolic): - def __init__(self, classdef): - self.classdef = classdef + def __init__(self, minid): + self.minid = minid + def compute_fn(self): - if self.classdef.maxid is self: - compute_inheritance_ids(self.classdef.bookkeeper) - return self.classdef.maxid + if self.minid.classdef.minid is self.minid: + compute_inheritance_ids(self.minid.classdef.bookkeeper) + return self.minid.classdef.maxid def compute_inheritance_ids(bookkeeper): - def assign_id(classdef, nextid): - assert isinstance(classdef.minid, MinIdSymbolic) - assert isinstance(classdef.maxid, MaxIdSymbolic) - classdef.minid = nextid - nextid += 1 - for subclass in classdef.subdefs: - nextid = assign_id(subclass, nextid) - classdef.maxid = nextid - return classdef.maxid - id_ = 0 - for classdef in bookkeeper.classdefs: - if classdef.basedef is None: - id_ = assign_id(classdef, id_) + bookkeeper.annotator.rootid.compute_inheritance_ids() +# def assign_id(classdef, nextid): +# assert isinstance(classdef.minid, MinIdSymbolic) +# assert isinstance(classdef.maxid, MaxIdSymbolic) +# classdef.minid = nextid +# nextid += 1 +# for subclass in classdef.subdefs: +# nextid = assign_id(subclass, nextid) +# classdef.maxid = nextid +# return classdef.maxid +# id_ = 0 +# for classdef in bookkeeper.classdefs: +# if classdef.basedef is None: +# id_ = assign_id(classdef, id_) + def assign_inheritance_ids(annotator): - for classdef in annotator.bookkeeper.classdefs: + if hasattr(annotator, 'rootid'): + rootid = annotator.rootid + else: + rootid = MinIdSymbolic(None, None) + annotator.rootid = rootid + def assign_id(classdef): if not hasattr(classdef, 'minid'): - classdef.minid = MinIdSymbolic(classdef) + classdef.minid = MinIdSymbolic(classdef, rootid) if not hasattr(classdef, 'maxid'): - classdef.maxid = MaxIdSymbolic(classdef) + classdef.maxid = MaxIdSymbolic(classdef.minid) + for subclass in classdef.subdefs: + assign_id(subclass) + for classdef in annotator.bookkeeper.classdefs: + + if classdef.basedef is None: + assign_id(classdef) # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/test/test_normalizecalls.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_normalizecalls.py (original) +++ pypy/dist/pypy/rpython/test/test_normalizecalls.py Sat Apr 8 13:56:15 2006 @@ -164,7 +164,7 @@ class TestNormalizeAfterTheFact(TestNormalize): - def rtype(self, fn, argtypes, resulttype): + def rtype(self, fn, argtypes, resulttype, checkfunction=None): t = TranslationContext() a = t.buildannotator() a.build_types(prefn, [int]) @@ -180,6 +180,10 @@ graph = annhelper.getgraph(fn, [a.typeannotation(argtype) for argtype in argtypes], s_result) annhelper.finish() + t.checkgraphs() + + if checkfunction is not None: + checkfunction(t) # sanity check prefn llinterp = LLInterpreter(typer) @@ -187,8 +191,7 @@ assert res == 100 res = llinterp.eval_graph(graphof(t, prefn), [2]) assert res == 201 - - t.checkgraphs() + return t def test_mix_after_recursion(self): @@ -214,6 +217,8 @@ annhelper.finish() def test_add_more_subclasses(self): + from pypy.rpython import rclass + from pypy.rpython.lltypesystem.rclass import ll_issubclass class Sub3(PBase): def newmethod(self): return 3 @@ -221,7 +226,19 @@ x = Sub3() return x.newmethod() - translator = self.rtype(dummyfn, [int], int) + def checkfunction(translator): + # make sure that there is a sensible comparison defined on the + # symbolics + bk = translator.annotator.bookkeeper + rtyper = translator.rtyper + base_classdef = bk.getuniqueclassdef(PBase) + base_vtable = rclass.getclassrepr(rtyper, base_classdef).getruntime() + sub3_classdef = bk.getuniqueclassdef(Sub3) + sub3_vtable = rclass.getclassrepr(rtyper, sub3_classdef).getruntime() + assert ll_issubclass(sub3_vtable, base_vtable) + assert not ll_issubclass(base_vtable, sub3_vtable) + + translator = self.rtype(dummyfn, [int], int, checkfunction) base_graph = graphof(translator, PBase.fn.im_func) sub1_graph = graphof(translator, PSub1.fn.im_func) sub2_graph = graphof(translator, PSub2.fn.im_func) @@ -232,5 +249,4 @@ assert sub2_graph.getreturnvar().concretetype == lltype.Signed assert sub3_graph.getreturnvar().concretetype == lltype.Signed assert dummyfn_graph.getreturnvar().concretetype == lltype.Signed - - + From cfbolz at codespeak.net Sat Apr 8 14:17:15 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 14:17:15 +0200 (CEST) Subject: [pypy-svn] r25541 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060408121715.CF83C10278@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 14:17:15 2006 New Revision: 25541 Modified: pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/test/test_inline.py Log: improve the weights of some operations. for some definition of improve Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 14:17:15 2006 @@ -424,15 +424,17 @@ OP_WEIGHTS = {'same_as': 0, 'cast_pointer': 0, 'keepalive': 0, - 'direct_call': 2, # guess - 'indirect_call': 2, # guess -# 'malloc': 2, + 'malloc': 2, 'yield_current_frame_to_caller': sys.maxint, # XXX bit extreme } def block_weight(block, weights=OP_WEIGHTS): total = 0 for op in block.operations: + if op.opname == "direct_call": + total += 1.5 + len(op.args) / 2 + elif op.opname == "indirect_call": + total += 2 + len(op.args) / 2 total += weights.get(op.opname, 1) if block.exitswitch is not None: total += 1 @@ -543,7 +545,8 @@ break # finished heappop(fiboheap) - log.inlining('%7.2f %50s' % (weight, graph.name)) + if callers[graph]: + log.inlining('%7.2f %50s' % (weight, graph.name)) for parentgraph in callers[graph]: if parentgraph == graph: continue Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Sat Apr 8 14:17:15 2006 @@ -410,7 +410,7 @@ t = TranslationContext() graph = t.buildflowgraph(f) res = measure_median_execution_cost(graph) - assert res == 19 + assert round(res, 5) == round(32.333333333, 5) def test_indirect_call_with_exception(): class MyExc(Exception): From cfbolz at codespeak.net Sat Apr 8 14:22:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 14:22:52 +0200 (CEST) Subject: [pypy-svn] r25542 - pypy/dist/pypy/translator/backendopt Message-ID: <20060408122252.C3CAE10289@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 14:22:51 2006 New Revision: 25542 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: (pedronis, cfbolz): trying to make the order of inlining even more deterministic Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 14:22:51 2006 @@ -527,26 +527,27 @@ for graph1, graph2 in static_callers(translator, ignore_primitives=True): callers.setdefault(graph2, {})[graph1] = True callees.setdefault(graph1, {})[graph2] = True - fiboheap = [(0.0, graph) for graph in callers] + heap = [(0.0, -len(callers[graph]), graph) for graph in callers] valid_weight = {} couldnt_inline = {} lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() - while fiboheap: - weight, graph = fiboheap[0] + while heap: + weight, _, graph = heap[0] if not valid_weight.get(graph): weight = inlining_heuristic(graph, callers.get(graph), callees.get(graph)) #print ' + cost %7.2f %50s' % (weight, graph.name) - heapreplace(fiboheap, (weight, graph)) + heapreplace(heap, (weight, -len(callers[graph]), graph)) valid_weight[graph] = True continue if weight >= threshold: break # finished - heappop(fiboheap) + heappop(heap) if callers[graph]: log.inlining('%7.2f %50s' % (weight, graph.name)) + print callers[graph] for parentgraph in callers[graph]: if parentgraph == graph: continue @@ -557,6 +558,7 @@ except CannotInline: couldnt_inline[graph] = True res = CannotInline + print "fail" if res is True: # the parentgraph should now contain all calls that were # done by 'graph' @@ -568,5 +570,5 @@ # been modified. Maybe now we can inline it into further # parents? del couldnt_inline[parentgraph] - heappush(fiboheap, (0.0, parentgraph)) + heappush(heap, (0.0, -len(callers[parentgraph]), parentgraph)) valid_weight[parentgraph] = False From cfbolz at codespeak.net Sat Apr 8 14:25:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 14:25:28 +0200 (CEST) Subject: [pypy-svn] r25543 - pypy/dist/pypy/translator/backendopt Message-ID: <20060408122528.F20DD10291@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 14:25:27 2006 New Revision: 25543 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: remove debug print Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 14:25:27 2006 @@ -558,7 +558,6 @@ except CannotInline: couldnt_inline[graph] = True res = CannotInline - print "fail" if res is True: # the parentgraph should now contain all calls that were # done by 'graph' From cfbolz at codespeak.net Sat Apr 8 14:25:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 14:25:52 +0200 (CEST) Subject: [pypy-svn] r25544 - pypy/dist/pypy/translator/backendopt Message-ID: <20060408122552.54DCC102A3@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 14:25:51 2006 New Revision: 25544 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: remove another debug print :-( Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 14:25:51 2006 @@ -547,7 +547,6 @@ heappop(heap) if callers[graph]: log.inlining('%7.2f %50s' % (weight, graph.name)) - print callers[graph] for parentgraph in callers[graph]: if parentgraph == graph: continue From hpk at codespeak.net Sat Apr 8 16:29:40 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 16:29:40 +0200 (CEST) Subject: [pypy-svn] r25548 - in pypy/branch/stacklesscfg/pypy/translator: . backendopt llvm/backendopt stackless Message-ID: <20060408142940.C14A41024C@code0.codespeak.net> Author: hpk Date: Sat Apr 8 16:29:38 2006 New Revision: 25548 Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py pypy/branch/stacklesscfg/pypy/translator/llvm/backendopt/exception.py pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py pypy/branch/stacklesscfg/pypy/translator/unsimplify.py Log: slighg refactoring: split_block_*() now returns the "splitlink" and not the afterblock anymore. Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py Sat Apr 8 16:29:38 2006 @@ -1,7 +1,7 @@ import sys from pypy.translator.simplify import join_blocks, cleanup_graph from pypy.translator.simplify import get_graph -from pypy.translator.unsimplify import copyvar, split_block +from pypy.translator.unsimplify import copyvar from pypy.objspace.flow.model import Variable, Constant, Block, Link from pypy.objspace.flow.model import SpaceOperation, c_last_exception from pypy.objspace.flow.model import traverse, mkentrymap, checkgraph @@ -9,7 +9,8 @@ from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void from pypy.rpython import rmodel from pypy.tool.algo import sparsemat -from pypy.translator.backendopt.support import log, split_block_with_keepalive, generate_keepalive +from pypy.translator.backendopt.support import log, split_block_with_keepalive +from pypy.translator.backendopt.support import generate_keepalive BASE_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -339,8 +340,9 @@ def do_inline(self, block, index_operation): - afterblock = split_block_with_keepalive( + splitlink = split_block_with_keepalive( self.translator, self.graph, block, index_operation) + afterblock = splitlink.target # these variables have to be passed along all the links in the inlined # graph because the original function needs them in the blocks after # the inlined function @@ -350,8 +352,7 @@ if isinstance(arg, Variable)] assert afterblock.operations[0] is self.op #vars that need to be passed through the blocks of the inlined function - linktoinlined = block.exits[0] - assert linktoinlined.target is afterblock + linktoinlined = splitlink copiedstartblock = self.copy_block(self.graph_to_inline.startblock) copiedstartblock.isstartblock = False #find args passed to startblock of inlined function Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py Sat Apr 8 16:29:38 2006 @@ -62,7 +62,8 @@ def split_block_with_keepalive(translator, graph, block, index_operation, keep_alive_op_args=True): - afterblock = split_block(translator, graph, block, index_operation) + splitlink = split_block(translator, graph, block, index_operation) + afterblock = splitlink.target conservative_keepalives = needs_conservative_livevar_calculation(block) if conservative_keepalives: keep_alive_vars = [var for var in block.getvariables() @@ -71,10 +72,10 @@ # alive by something else. but this is sometimes hard to know for i, var in enumerate(keep_alive_vars): try: - index = block.exits[0].args.index(var) + index = splitlink.args.index(var) newvar = afterblock.inputargs[index] except ValueError: - block.exits[0].args.append(var) + splitlink.args.append(var) newvar = copyvar(translator, var) afterblock.inputargs.append(newvar) keep_alive_vars[i] = newvar @@ -92,7 +93,7 @@ betweenblock.operations = generate_keepalive(fresh_vars) else: afterblock.operations.extend(generate_keepalive(keep_alive_vars)) - return afterblock + return splitlink def md5digest(translator): import md5 Modified: pypy/branch/stacklesscfg/pypy/translator/llvm/backendopt/exception.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/llvm/backendopt/exception.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/llvm/backendopt/exception.py Sat Apr 8 16:29:38 2006 @@ -32,7 +32,7 @@ continue n_calls_patched += 1 - afterblock = split_block(translator, graph, block, i+1) + split_block(translator, graph, block, i+1) res = Variable() res.concretetype = Bool Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 16:29:38 2006 @@ -156,9 +156,8 @@ while i < len(block.operations): op = block.operations[i] if op.opname in ('direct_call', 'indirect_call'): - after_block = support.split_block_with_keepalive(self.translator, self.curr_graph, block, i+1) - link = block.exits[0] - + link = support.split_block_with_keepalive(self.translator, + self.curr_graph, block, i+1) var_unwind_exception = varoftype(evalue) args = [v for v in link.args if v is not op.result] @@ -174,7 +173,7 @@ self.translator.rtyper._convert_link(block, newlink) # ARGH ... - block = after_block + block = link.target i = 0 else: i += 1 Modified: pypy/branch/stacklesscfg/pypy/translator/unsimplify.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/unsimplify.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/unsimplify.py Sat Apr 8 16:29:38 2006 @@ -36,9 +36,10 @@ return newblock def split_block(translator, graph, block, index): - """split a block in two, inserting a proper link between the new - blocks. if you call this after rtyping, you WILL need to worry - about keepalives.""" + """return a link where prevblock is the block leading up but excluding the + index'th operation and target is a new block with the neccessary variables + passed on. NOTE: if you call this after rtyping, you WILL need to worry + about keepalives, you may use backendopt.support.split_block_with_keepalive.""" assert 0 <= index <= len(block.operations) if block.exitswitch == c_last_exception: assert index < len(block.operations) @@ -85,7 +86,7 @@ block.exitswitch = None block.exc_handler = False #checkgraph(graph) - return newblock + return link def remove_direct_loops(translator, graph): """This is useful for code generators: it ensures that no link has From pedronis at codespeak.net Sat Apr 8 16:38:44 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 8 Apr 2006 16:38:44 +0200 (CEST) Subject: [pypy-svn] r25549 - pypy/dist/pypy/translator/c/test Message-ID: <20060408143844.DEC9F10251@code0.codespeak.net> Author: pedronis Date: Sat Apr 8 16:38:43 2006 New Revision: 25549 Modified: pypy/dist/pypy/translator/c/test/test_stackless.py Log: oops, insert stack checks. Modified: pypy/dist/pypy/translator/c/test/test_stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_stackless.py (original) +++ pypy/dist/pypy/translator/c/test/test_stackless.py Sat Apr 8 16:38:43 2006 @@ -25,6 +25,9 @@ t.buildannotator().build_types(entry_point, [s_list_of_strings]) t.buildrtyper().specialize() + from pypy.translator.transform import insert_ll_stackcheck + insert_ll_stackcheck(t) + cbuilder = CStandaloneBuilder(t, entry_point, gcpolicy=self.gcpolicy) cbuilder.stackless = True #cbuilder.use_stackless_transformation = True From hpk at codespeak.net Sat Apr 8 17:32:31 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 17:32:31 +0200 (CEST) Subject: [pypy-svn] r25555 - in pypy/dist/pypy/translator: . backendopt llvm/backendopt Message-ID: <20060408153231.248961025C@code0.codespeak.net> Author: hpk Date: Sat Apr 8 17:32:31 2006 New Revision: 25555 Modified: pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/llvm/backendopt/exception.py pypy/dist/pypy/translator/unsimplify.py Log: merge r25548 of stacklesscfg branch (split_block returns splitlink instead of afterblock) Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Apr 8 17:32:31 2006 @@ -1,7 +1,7 @@ import sys from pypy.translator.simplify import join_blocks, cleanup_graph from pypy.translator.simplify import get_graph -from pypy.translator.unsimplify import copyvar, split_block +from pypy.translator.unsimplify import copyvar from pypy.objspace.flow.model import Variable, Constant, Block, Link from pypy.objspace.flow.model import SpaceOperation, c_last_exception from pypy.objspace.flow.model import traverse, mkentrymap, checkgraph @@ -9,7 +9,8 @@ from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void, Ptr from pypy.rpython import rmodel from pypy.tool.algo import sparsemat -from pypy.translator.backendopt.support import log, split_block_with_keepalive, generate_keepalive, find_backedges, find_loop_blocks +from pypy.translator.backendopt.support import log, split_block_with_keepalive +from pypy.translator.backendopt.support import generate_keepalive, find_backedges, find_loop_blocks BASE_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -349,8 +350,9 @@ def do_inline(self, block, index_operation): - afterblock = split_block_with_keepalive( + splitlink = split_block_with_keepalive( self.translator, self.graph, block, index_operation) + afterblock = splitlink.target # these variables have to be passed along all the links in the inlined # graph because the original function needs them in the blocks after # the inlined function @@ -360,8 +362,7 @@ if isinstance(arg, Variable)] assert afterblock.operations[0] is self.op #vars that need to be passed through the blocks of the inlined function - linktoinlined = block.exits[0] - assert linktoinlined.target is afterblock + linktoinlined = splitlink copiedstartblock = self.copy_block(self.graph_to_inline.startblock) copiedstartblock.isstartblock = False #find args passed to startblock of inlined function Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Sat Apr 8 17:32:31 2006 @@ -64,7 +64,8 @@ def split_block_with_keepalive(translator, graph, block, index_operation, keep_alive_op_args=True): - afterblock = split_block(translator, graph, block, index_operation) + splitlink = split_block(translator, graph, block, index_operation) + afterblock = splitlink.target conservative_keepalives = needs_conservative_livevar_calculation(block) if conservative_keepalives: keep_alive_vars = [var for var in block.getvariables() @@ -73,10 +74,10 @@ # alive by something else. but this is sometimes hard to know for i, var in enumerate(keep_alive_vars): try: - index = block.exits[0].args.index(var) + index = splitlink.args.index(var) newvar = afterblock.inputargs[index] except ValueError: - block.exits[0].args.append(var) + splitlink.args.append(var) newvar = copyvar(translator, var) afterblock.inputargs.append(newvar) keep_alive_vars[i] = newvar @@ -94,7 +95,7 @@ betweenblock.operations = generate_keepalive(fresh_vars) else: afterblock.operations.extend(generate_keepalive(keep_alive_vars)) - return afterblock + return splitlink def calculate_call_graph(translator): calls = {} Modified: pypy/dist/pypy/translator/llvm/backendopt/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/backendopt/exception.py (original) +++ pypy/dist/pypy/translator/llvm/backendopt/exception.py Sat Apr 8 17:32:31 2006 @@ -32,7 +32,7 @@ continue n_calls_patched += 1 - afterblock = split_block(translator, graph, block, i+1) + split_block(translator, graph, block, i+1) res = Variable() res.concretetype = Bool Modified: pypy/dist/pypy/translator/unsimplify.py ============================================================================== --- pypy/dist/pypy/translator/unsimplify.py (original) +++ pypy/dist/pypy/translator/unsimplify.py Sat Apr 8 17:32:31 2006 @@ -36,9 +36,10 @@ return newblock def split_block(translator, graph, block, index): - """split a block in two, inserting a proper link between the new - blocks. if you call this after rtyping, you WILL need to worry - about keepalives.""" + """return a link where prevblock is the block leading up but excluding the + index'th operation and target is a new block with the neccessary variables + passed on. NOTE: if you call this after rtyping, you WILL need to worry + about keepalives, you may use backendopt.support.split_block_with_keepalive.""" assert 0 <= index <= len(block.operations) if block.exitswitch == c_last_exception: assert index < len(block.operations) @@ -85,7 +86,7 @@ block.exitswitch = None block.exc_handler = False #checkgraph(graph) - return newblock + return link def remove_direct_loops(translator, graph): """This is useful for code generators: it ensures that no link has From hpk at codespeak.net Sat Apr 8 17:33:53 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 17:33:53 +0200 (CEST) Subject: [pypy-svn] r25556 - pypy/dist/pypy/rpython Message-ID: <20060408153353.312511022D@code0.codespeak.net> Author: hpk Date: Sat Apr 8 17:33:48 2006 New Revision: 25556 Modified: pypy/dist/pypy/rpython/rtyper.py Log: merge r25533 of stacklesscfg branch (factor out of link conversion) Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Sat Apr 8 17:33:48 2006 @@ -339,34 +339,36 @@ # consider it as a link source instead self.insert_link_conversions(extrablock) + def _convert_link(self, block, link): + if link.exitcase is not None: + if isinstance(block.exitswitch, Variable): + r_case = self.bindingrepr(block.exitswitch) + else: + assert block.exitswitch == c_last_exception + r_case = rclass.get_type_repr(self) + link.llexitcase = r_case.convert_const(link.exitcase) + else: + link.llexitcase = None + + a = link.last_exception + if isinstance(a, Variable): + a.concretetype = self.exceptiondata.lltype_of_exception_type + elif isinstance(a, Constant): + link.last_exception = inputconst( + self.exceptiondata.r_exception_type, a.value) + + a = link.last_exc_value + if isinstance(a, Variable): + a.concretetype = self.exceptiondata.lltype_of_exception_value + elif isinstance(a, Constant): + link.last_exc_value = inputconst( + self.exceptiondata.r_exception_value, a.value) + def insert_link_conversions(self, block, skip=0): # insert the needed conversions on the links can_insert_here = block.exitswitch is None and len(block.exits) == 1 for link in block.exits[skip:]: - if link.exitcase is not None: - if isinstance(block.exitswitch, Variable): - r_case = self.bindingrepr(block.exitswitch) - else: - assert block.exitswitch == c_last_exception - r_case = rclass.get_type_repr(self) - link.llexitcase = r_case.convert_const(link.exitcase) - else: - link.llexitcase = None - - a = link.last_exception - if isinstance(a, Variable): - a.concretetype = self.exceptiondata.lltype_of_exception_type - elif isinstance(a, Constant): - link.last_exception = inputconst( - self.exceptiondata.r_exception_type, a.value) - - a = link.last_exc_value - if isinstance(a, Variable): - a.concretetype = self.exceptiondata.lltype_of_exception_value - elif isinstance(a, Constant): - link.last_exc_value = inputconst( - self.exceptiondata.r_exception_value, a.value) - + self._convert_link(block, link) inputargs_reprs = self.setup_block_entry(link.target) newops = self.make_new_lloplist(block) newlinkargs = {} From mwh at codespeak.net Sat Apr 8 17:39:50 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 17:39:50 +0200 (CEST) Subject: [pypy-svn] r25557 - pypy/branch/stacklesscfg/pypy/translator/stackless Message-ID: <20060408153950.C76CA10232@code0.codespeak.net> Author: mwh Date: Sat Apr 8 17:39:49 2006 New Revision: 25557 Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Log: correct a type Modified: pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/stackless/transform.py Sat Apr 8 17:39:49 2006 @@ -94,7 +94,7 @@ self.slp_main_loop_type_ptr = model.Constant(lltype.functionptr( SLP_MAIN_LOOP_TYPE, "slp_main_loop", graph=slp_main_loop_graph), - SLP_MAIN_LOOP_TYPE) + lltype.Ptr(SLP_MAIN_LOOP_TYPE)) annotations = [annmodel.SomeInstance(bk.getuniqueclassdef(code.UnwindException)), annmodel.SomePtr(lltype.Ptr(STATE_HEADER))] From mwh at codespeak.net Sat Apr 8 17:49:51 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 17:49:51 +0200 (CEST) Subject: [pypy-svn] r25559 - pypy/branch/stacklesscfg/pypy/rpython Message-ID: <20060408154951.2561B1026E@code0.codespeak.net> Author: mwh Date: Sat Apr 8 17:49:50 2006 New Revision: 25559 Modified: pypy/branch/stacklesscfg/pypy/rpython/rtyper.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/rpython/rtyper.py revisions 25473 to 25557: ------------------------------------------------------------------------ r25556 | hpk | 2006-04-08 17:33:48 +0200 (Sat, 08 Apr 2006) | 2 lines merge r25533 of stacklesscfg branch (factor out of link conversion) ------------------------------------------------------------------------ r25495 | cfbolz | 2006-04-07 15:18:05 +0200 (Fri, 07 Apr 2006) | 6 lines (pedronis, cfbolz): introduced an optimizations remove_duplicate_casts to reduce the size of the live variable set. For this we needed to make the exception heuristic in inlining cleverer. ------------------------------------------------------------------------ Modified: pypy/branch/stacklesscfg/pypy/rpython/rtyper.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/rpython/rtyper.py (original) +++ pypy/branch/stacklesscfg/pypy/rpython/rtyper.py Sat Apr 8 17:49:50 2006 @@ -102,6 +102,12 @@ def getexceptiondata(self): return self.exceptiondata # built at the end of specialize() + def lltype_to_classdef_mapping(self): + result = {} + for (classdef, _), repr in self.instance_reprs.iteritems(): + result[repr.lowleveltype] = classdef + return result + def makekey(self, s_obj): return pair(self.type_system, s_obj).rtyper_makekey(self) From mwh at codespeak.net Sat Apr 8 17:49:54 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 17:49:54 +0200 (CEST) Subject: [pypy-svn] r25560 - pypy/branch/stacklesscfg/pypy/translator/backendopt Message-ID: <20060408154954.E782B1026E@code0.codespeak.net> Author: mwh Date: Sat Apr 8 17:49:52 2006 New Revision: 25560 Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/backendopt/inline.py revisions 25473 to 25557: ------------------------------------------------------------------------ r25555 | hpk | 2006-04-08 17:32:31 +0200 (Sat, 08 Apr 2006) | 2 lines merge r25548 of stacklesscfg branch (split_block returns splitlink instead of afterblock) ------------------------------------------------------------------------ r25544 | cfbolz | 2006-04-08 14:25:51 +0200 (Sat, 08 Apr 2006) | 2 lines remove another debug print :-( ------------------------------------------------------------------------ r25543 | cfbolz | 2006-04-08 14:25:27 +0200 (Sat, 08 Apr 2006) | 2 lines remove debug print ------------------------------------------------------------------------ r25542 | cfbolz | 2006-04-08 14:22:51 +0200 (Sat, 08 Apr 2006) | 3 lines (pedronis, cfbolz): trying to make the order of inlining even more deterministic ------------------------------------------------------------------------ r25541 | cfbolz | 2006-04-08 14:17:15 +0200 (Sat, 08 Apr 2006) | 2 lines improve the weights of some operations. for some definition of improve ------------------------------------------------------------------------ r25535 | cfbolz | 2006-04-08 10:33:27 +0200 (Sat, 08 Apr 2006) | 2 lines wua! bad me ------------------------------------------------------------------------ r25532 | cfbolz | 2006-04-08 10:14:47 +0200 (Sat, 08 Apr 2006) | 5 lines (pedronis, cfbolz): increase the weight of edges that go back in a loop because they are much more likely ------------------------------------------------------------------------ r25495 | cfbolz | 2006-04-07 15:18:05 +0200 (Fri, 07 Apr 2006) | 6 lines (pedronis, cfbolz): introduced an optimizations remove_duplicate_casts to reduce the size of the live variable set. For this we needed to make the exception heuristic in inlining cleverer. ------------------------------------------------------------------------ Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py Sat Apr 8 17:49:52 2006 @@ -6,11 +6,11 @@ from pypy.objspace.flow.model import SpaceOperation, c_last_exception from pypy.objspace.flow.model import traverse, mkentrymap, checkgraph from pypy.annotation import model as annmodel -from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void +from pypy.rpython.lltypesystem.lltype import Bool, typeOf, Void, Ptr from pypy.rpython import rmodel from pypy.tool.algo import sparsemat from pypy.translator.backendopt.support import log, split_block_with_keepalive -from pypy.translator.backendopt.support import generate_keepalive +from pypy.translator.backendopt.support import generate_keepalive, find_backedges, find_loop_blocks BASE_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -68,25 +68,28 @@ except StopIteration: return False -def inline_function(translator, inline_func, graph): - inliner = Inliner(translator, graph, inline_func) +def inline_function(translator, inline_func, graph, lltype_to_classdef): + inliner = Inliner(translator, graph, inline_func, lltype_to_classdef) return inliner.inline_all() +def simple_inline_function(translator, inline_func, graph): + inliner = Inliner(translator, graph, inline_func, + translator.rtyper.lltype_to_classdef_mapping()) + return inliner.inline_all() + + def _find_exception_type(block): #XXX slightly brittle: find the exception type for simple cases #(e.g. if you do only raise XXXError) by doing pattern matching - ops = [op for op in block.operations if op.opname != 'keepalive'] - if (len(ops) < 6 or - ops[-6].opname != "malloc" or ops[-5].opname != "cast_pointer" or - ops[-4].opname != "setfield" or ops[-3].opname != "cast_pointer" or - ops[-2].opname != "getfield" or ops[-1].opname != "cast_pointer" or - len(block.exits) != 1 or block.exits[0].args[0] != ops[-2].result or - block.exitswitch is not None or - block.exits[0].args[1] != ops[-1].result or - not isinstance(ops[-4].args[1], Constant) or - ops[-4].args[1].value != "typeptr"): + currvar = block.exits[0].args[1] + for op in block.operations[::-1]: + if op.opname in ("same_as", "cast_pointer") and op.result is currvar: + currvar = op.args[0] + elif op.opname == "malloc" and op.result is currvar: + break + else: return None, None - return ops[-4].args[2].value, block.exits[0] + return Ptr(op.args[0].value), block.exits[0] def does_raise_directly(graph, raise_analyzer): """ this function checks, whether graph contains operations which can raise @@ -102,7 +105,8 @@ return False class BaseInliner(object): - def __init__(self, translator, graph, inline_guarded_calls=False, + def __init__(self, translator, graph, lltype_to_classdef, + inline_guarded_calls=False, inline_guarded_calls_no_matter_what=False, raise_analyzer=None): self.translator = translator self.graph = graph @@ -115,6 +119,7 @@ self.raise_analyzer = raise_analyzer else: self.raise_analyzer = None + self.lltype_to_classdef = lltype_to_classdef def inline_all(self): count = 0 @@ -275,22 +280,27 @@ def rewire_exceptblock_with_guard(self, afterblock, copiedexceptblock): # this rewiring does not always succeed. in the cases where it doesn't # there will be generic code inserted + from pypy.rpython.lltypesystem import rclass exc_match = self.translator.rtyper.getexceptiondata().fn_exception_match for link in self.entrymap[self.graph_to_inline.exceptblock]: copiedblock = self.copy_block(link.prevblock) - eclass, copiedlink = _find_exception_type(copiedblock) + VALUE, copiedlink = _find_exception_type(copiedblock) #print copiedblock.operations - if eclass is None: + if VALUE is None or VALUE not in self.lltype_to_classdef: continue - etype = copiedlink.args[0] - evalue = copiedlink.args[1] + classdef = self.lltype_to_classdef[VALUE] + rtyper = self.translator.rtyper + classrepr = rclass.getclassrepr(rtyper, classdef) + vtable = classrepr.getruntime() + var_etype = copiedlink.args[0] + var_evalue = copiedlink.args[1] for exceptionlink in afterblock.exits[1:]: - if exc_match(eclass, exceptionlink.llexitcase): + if exc_match(vtable, exceptionlink.llexitcase): passon_vars = self.passon_vars(link.prevblock) copiedblock.operations += generate_keepalive(passon_vars) copiedlink.target = exceptionlink.target linkargs = self.find_args_in_exceptional_case( - exceptionlink, link.prevblock, etype, evalue, afterblock, passon_vars) + exceptionlink, link.prevblock, var_etype, var_evalue, afterblock, passon_vars) copiedlink.args = linkargs break @@ -388,9 +398,9 @@ class Inliner(BaseInliner): - def __init__(self, translator, graph, inline_func, inline_guarded_calls=False, + def __init__(self, translator, graph, inline_func, lltype_to_classdef, inline_guarded_calls=False, inline_guarded_calls_no_matter_what=False, raise_analyzer=None): - BaseInliner.__init__(self, translator, graph, + BaseInliner.__init__(self, translator, graph, lltype_to_classdef, inline_guarded_calls, inline_guarded_calls_no_matter_what, raise_analyzer) @@ -404,21 +414,10 @@ self.block_to_index.setdefault(block, {})[i] = g class OneShotInliner(BaseInliner): - def __init__(self, translator, graph, inline_guarded_calls=False, - inline_guarded_calls_no_matter_what=False, raise_analyzer=None): - BaseInliner.__init__(self, translator, graph, - inline_guarded_calls, - inline_guarded_calls_no_matter_what, - raise_analyzer) - def search_for_calls(self, block): pass -def _inline_function(translator, graph, block, index_operation): - inline_func = block.operations[index_operation].args[0].value._obj._callable - inliner = Inliner(translator, graph, inline_func) - # ____________________________________________________________ # # Automatic inlining @@ -426,14 +425,17 @@ OP_WEIGHTS = {'same_as': 0, 'cast_pointer': 0, 'keepalive': 0, - 'direct_call': 2, # guess - 'indirect_call': 2, # guess + 'malloc': 2, 'yield_current_frame_to_caller': sys.maxint, # XXX bit extreme } def block_weight(block, weights=OP_WEIGHTS): total = 0 for op in block.operations: + if op.opname == "direct_call": + total += 1.5 + len(op.args) / 2 + elif op.opname == "indirect_call": + total += 2 + len(op.args) / 2 total += weights.get(op.opname, 1) if block.exitswitch is not None: total += 1 @@ -446,15 +448,34 @@ for block in graph.iterblocks(): blockmap[block] = len(blocks) blocks.append(block) + backedges = dict.fromkeys(find_backedges(graph), True) + loops = find_loop_blocks(graph) M = sparsemat.SparseMatrix(len(blocks)) vector = [] for i, block in enumerate(blocks): vector.append(block_weight(block)) M[i, i] = 1 if block.exits: - f = 1.0 / len(block.exits) + if block not in loops: + current_loop_start = None + else: + current_loop_start = loops[block] + loop_exits = [] for link in block.exits: - M[i, blockmap[link.target]] -= f + if (link.target in loops and + loops[link.target] is current_loop_start): + loop_exits.append(link) + if len(loop_exits) and len(loop_exits) < len(block.exits): + f = 0.3 / (len(block.exits) - len(loop_exits)) + b = 0.7 / len(loop_exits) + else: + b = f = 1.0 / len(block.exits) + for link in block.exits: + if (link.target in loops and + loops[link.target] is current_loop_start): + M[i, blockmap[link.target]] -= b + else: + M[i, blockmap[link.target]] -= f try: Solution = M.solve(vector) except ValueError: @@ -507,30 +528,33 @@ for graph1, graph2 in static_callers(translator, ignore_primitives=True): callers.setdefault(graph2, {})[graph1] = True callees.setdefault(graph1, {})[graph2] = True - fiboheap = [(0.0, graph) for graph in callers] + heap = [(0.0, -len(callers[graph]), graph) for graph in callers] valid_weight = {} couldnt_inline = {} + lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() - while fiboheap: - weight, graph = fiboheap[0] + while heap: + weight, _, graph = heap[0] if not valid_weight.get(graph): weight = inlining_heuristic(graph, callers.get(graph), callees.get(graph)) #print ' + cost %7.2f %50s' % (weight, graph.name) - heapreplace(fiboheap, (weight, graph)) + heapreplace(heap, (weight, -len(callers[graph]), graph)) valid_weight[graph] = True continue if weight >= threshold: break # finished - heappop(fiboheap) - log.inlining('%7.2f %50s' % (weight, graph.name)) + heappop(heap) + if callers[graph]: + log.inlining('%7.2f %50s' % (weight, graph.name)) for parentgraph in callers[graph]: if parentgraph == graph: continue sys.stdout.flush() try: - res = bool(inline_function(translator, graph, parentgraph)) + res = bool(inline_function(translator, graph, parentgraph, + lltype_to_classdef)) except CannotInline: couldnt_inline[graph] = True res = CannotInline @@ -545,5 +569,5 @@ # been modified. Maybe now we can inline it into further # parents? del couldnt_inline[parentgraph] - heappush(fiboheap, (0.0, parentgraph)) + heappush(heap, (0.0, -len(callers[parentgraph]), parentgraph)) valid_weight[parentgraph] = False From mwh at codespeak.net Sat Apr 8 17:49:57 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 17:49:57 +0200 (CEST) Subject: [pypy-svn] r25561 - pypy/branch/stacklesscfg/pypy/translator/backendopt Message-ID: <20060408154957.230BE102A4@code0.codespeak.net> Author: mwh Date: Sat Apr 8 17:49:56 2006 New Revision: 25561 Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py Log: merging of http://codespeak.net/svn/pypy/dist/pypy/translator/backendopt/support.py revisions 25473 to 25557: ------------------------------------------------------------------------ r25555 | hpk | 2006-04-08 17:32:31 +0200 (Sat, 08 Apr 2006) | 2 lines merge r25548 of stacklesscfg branch (split_block returns splitlink instead of afterblock) ------------------------------------------------------------------------ r25530 | cfbolz | 2006-04-07 23:49:27 +0200 (Fri, 07 Apr 2006) | 2 lines move find_backedges etc to support ------------------------------------------------------------------------ r25523 | cfbolz | 2006-04-07 20:42:39 +0200 (Fri, 07 Apr 2006) | 5 lines (pedronis, cfbolz): do the (very indeterministic) stack check insertion as late as possible, just before code generation. ------------------------------------------------------------------------ Modified: pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py ============================================================================== --- pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py (original) +++ pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py Sat Apr 8 17:49:56 2006 @@ -1,7 +1,9 @@ import py from pypy.rpython.lltypesystem import lltype +from pypy.translator.simplify import get_graph from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log +from pypy.annotation.model import setunion from pypy.translator.unsimplify import split_block, copyvar, insert_empty_block from pypy.objspace.flow.model import Constant, Variable, SpaceOperation, c_last_exception from pypy.rpython.lltypesystem import lltype @@ -95,6 +97,78 @@ afterblock.operations.extend(generate_keepalive(keep_alive_vars)) return splitlink +def calculate_call_graph(translator): + calls = {} + for graph in translator.graphs: + if getattr(getattr(graph, "func", None), "suggested_primitive", False): + continue + calls[graph] = {} + for block in graph.iterblocks(): + for op in block.operations: + if op.opname == "direct_call": + called_graph = get_graph(op.args[0], translator) + if called_graph is not None: + calls[graph][called_graph] = block + if op.opname == "indirect_call": + graphs = op.args[-1].value + if graphs is not None: + for called_graph in graphs: + calls[graph][called_graph] = block + return calls + +def find_backedges(graph): + """finds the backedges in the flow graph""" + scheduled = [graph.startblock] + seen = {} + backedges = [] + while scheduled: + current = scheduled.pop() + seen[current] = True + for link in current.exits: + if link.target in seen: + backedges.append(link) + else: + scheduled.append(link.target) + return backedges + +def compute_reachability(graph): + reachable = {} + for block in graph.iterblocks(): + reach = {} + scheduled = [block] + while scheduled: + current = scheduled.pop() + for link in current.exits: + if link.target in reachable: + reach = setunion(reach, reachable[link.target]) + continue + if link.target not in reach: + reach[link.target] = True + reachable[block] = reach + return reachable + +def find_loop_blocks(graph): + """find the blocks in a graph that are part of a loop""" + loop = {} + reachable = compute_reachability(graph) + for backedge in find_backedges(graph): + start = backedge.target + end = backedge.prevblock + loop[start] = start + loop[end] = start + scheduled = [start] + seen = {} + while scheduled: + current = scheduled.pop() + connects = end in reachable[current] + seen[current] = True + if connects: + loop[current] = start + for link in current.exits: + if link.target not in seen: + scheduled.append(link.target) + return loop + def md5digest(translator): import md5 m = md5.new() From mwh at codespeak.net Sat Apr 8 17:51:18 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 17:51:18 +0200 (CEST) Subject: [pypy-svn] r25562 - in pypy/dist/pypy: rpython translator/backendopt translator/c translator/c/test translator/stackless Message-ID: <20060408155118.2772610299@code0.codespeak.net> Author: mwh Date: Sat Apr 8 17:51:17 2006 New Revision: 25562 Added: pypy/dist/pypy/rpython/rtyper.py - copied unchanged from r25557, pypy/branch/stacklesscfg/pypy/rpython/rtyper.py pypy/dist/pypy/translator/backendopt/inline.py - copied unchanged from r25557, pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py - copied unchanged from r25557, pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/stackless/ - copied from r25557, pypy/branch/stacklesscfg/pypy/translator/stackless/ Removed: pypy/dist/pypy/translator/c/stacklesstransform.py pypy/dist/pypy/translator/c/test/test_stacklesstransform.py Log: merge the stacklesscfg branch -- we don't actually seem to be being that disruptive so far From mwh at codespeak.net Sat Apr 8 18:02:29 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 18:02:29 +0200 (CEST) Subject: [pypy-svn] r25563 - in pypy/dist/pypy: rpython translator/backendopt Message-ID: <20060408160229.2B6D7102A6@code0.codespeak.net> Author: mwh Date: Sat Apr 8 18:02:28 2006 New Revision: 25563 Removed: pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py Log: delete files in preparation for fixing bodged merge. From mwh at codespeak.net Sat Apr 8 18:02:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 18:02:53 +0200 (CEST) Subject: [pypy-svn] r25564 - in pypy/dist/pypy: rpython translator/backendopt Message-ID: <20060408160253.D9143102B2@code0.codespeak.net> Author: mwh Date: Sat Apr 8 18:02:52 2006 New Revision: 25564 Added: pypy/dist/pypy/rpython/rtyper.py - copied unchanged from r25563, pypy/branch/stacklesscfg/pypy/rpython/rtyper.py pypy/dist/pypy/translator/backendopt/inline.py - copied unchanged from r25563, pypy/branch/stacklesscfg/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py - copied unchanged from r25563, pypy/branch/stacklesscfg/pypy/translator/backendopt/support.py Log: fix bodged merge From hpk at codespeak.net Sat Apr 8 18:07:09 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 18:07:09 +0200 (CEST) Subject: [pypy-svn] r25565 - pypy/dist/pypy/translator/c Message-ID: <20060408160709.6491310100@code0.codespeak.net> Author: hpk Date: Sat Apr 8 18:07:09 2006 New Revision: 25565 Modified: pypy/dist/pypy/translator/c/exceptiontransform.py Log: refix for changed split_block (dogded merge repair) Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Sat Apr 8 18:07:09 2006 @@ -152,8 +152,9 @@ if not self.raise_analyzer.can_raise(op): continue - afterblock = support.split_block_with_keepalive( + splitlink = support.split_block_with_keepalive( self.translator, graph, block, i+1, False) + afterblock = splitlink.target if lastblock is block: lastblock = afterblock From hpk at codespeak.net Sat Apr 8 18:19:52 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 18:19:52 +0200 (CEST) Subject: [pypy-svn] r25566 - pypy/dist/pypy/translator/stackless Message-ID: <20060408161952.4F45F10109@code0.codespeak.net> Author: hpk Date: Sat Apr 8 18:19:51 2006 New Revision: 25566 Modified: pypy/dist/pypy/translator/stackless/transform.py Log: (mwh, hpk) some more support for actually getting to issue the resume handling code at the start of a block ... Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sat Apr 8 18:19:51 2006 @@ -71,6 +71,11 @@ ## return retval + x + 1 +class ResumePoint: + def __init__(self, var_result, varstoload, targetblock): + self.var_result = var_result + self.varstoload = varstoload + self.targetblock = targetblock class StacklessTransfomer(object): def __init__(self, translator): @@ -141,11 +146,16 @@ for block in list(graph.iterblocks()): self.transform_block(block) + if self.resume_points: - XXX + self.insert_resume_handling(graph) self.curr_graph = None + def insert_resume_handling(self, graph): + #graph.startblock.isstartblock = False + pass + def transform_block(self, block): i = 0 @@ -160,10 +170,13 @@ self.curr_graph, block, i+1) var_unwind_exception = varoftype(evalue) - args = [v for v in link.args if v is not op.result] + args = [v for v in link.args + if v is not op.result and v.concretetype is not lltype.Void] save_block = self.generate_save_block( args, var_unwind_exception) + self.resume_points.append(ResumePoint(op.result, args, link.target)) + newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) block.exitswitch = model.c_last_exception @@ -179,7 +192,6 @@ i += 1 def generate_save_block(self, varstosave, var_unwind_exception): - varstosave = [v for v in varstosave if v.concretetype is not lltype.Void] rtyper = self.translator.rtyper edata = rtyper.getexceptiondata() etype = edata.lltype_of_exception_type From hpk at codespeak.net Sat Apr 8 18:46:26 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 18:46:26 +0200 (CEST) Subject: [pypy-svn] r25567 - pypy/branch/stacklesscfg Message-ID: <20060408164626.6D1E8101CE@code0.codespeak.net> Author: hpk Date: Sat Apr 8 18:46:18 2006 New Revision: 25567 Removed: pypy/branch/stacklesscfg/ Log: remove the stacklesscfg branch (it will probably be revived later) From antocuni at codespeak.net Sat Apr 8 18:56:55 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 8 Apr 2006 18:56:55 +0200 (CEST) Subject: [pypy-svn] r25568 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060408165655.1213310104@code0.codespeak.net> Author: antocuni Date: Sat Apr 8 18:56:50 2006 New Revision: 25568 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: Added the _lookup_field method to _instance class. The method body is similar to _lookup. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sat Apr 8 18:56:50 2006 @@ -132,6 +132,17 @@ _check_field = _field_type + def _lookup_field(self, name): + field = self._fields.get(name) + + if field is None and self._superclass is not None: + return self._superclass._lookup_field(name) + + try: + return self, field[0] + except TypeError: + return self, None + def _lookup(self, meth_name): meth = self._methods.get(meth_name) @@ -455,6 +466,9 @@ callb, checked_args = self._checkargs(args) return callb(*checked_args) + def __repr__(self): + return 'sm %s' % self._name + class _meth(_callable): def __init__(self, METHOD, **attrs): From mwh at codespeak.net Sat Apr 8 18:59:55 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 18:59:55 +0200 (CEST) Subject: [pypy-svn] r25569 - pypy/dist/pypy/translator/stackless Message-ID: <20060408165955.DE42110244@code0.codespeak.net> Author: mwh Date: Sat Apr 8 18:59:43 2006 New Revision: 25569 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/transform.py Log: (hpk, mwh) a start at inserting resume blocks. something get concretetyped as pyobject, but given the cheating we're doing this will hopefully go away when the cheating does. Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sat Apr 8 18:59:43 2006 @@ -85,4 +85,6 @@ else: u.frame_bottom.f_back = frame_state u.frame_bottom = frame_state - + +def resume_state(): + return global_state.restart_substate Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sat Apr 8 18:59:43 2006 @@ -115,6 +115,14 @@ graph=add_frame_state_graph), lltype.Ptr(ADD_FRAME_STATE_TYPE)) + RESUME_STATE_TYPE = lltype.FuncType([], lltype.Signed) + resume_state_graph = mixlevelannotator.getgraph( + code.resume_state, [], annmodel.SomeInteger()) + self.resume_state_ptr = model.Constant(lltype.functionptr( + RESUME_STATE_TYPE, "resume_state", + graph=resume_state_graph), + lltype.Ptr(RESUME_STATE_TYPE)) + mixlevelannotator.finish() def frame_type_for_vars(self, vars): @@ -153,8 +161,28 @@ self.curr_graph = None def insert_resume_handling(self, graph): - #graph.startblock.isstartblock = False - pass + old_start_block = graph.startblock + newinputargs = [copyvar(self.translator, v) for v in old_start_block.inputargs] + new_start_block = model.Block(newinputargs) + var_resume_state = varoftype(lltype.Signed) + new_start_block.operations.append( + model.SpaceOperation("direct_call", [self.resume_state_ptr], var_resume_state)) + not_resuming_link = model.Link(newinputargs, old_start_block, 0) + resuming_links = [] + for i, resume_point in enumerate(self.resume_points): + newblock = model.Block([]) + args = [] + for arg in resume_point.targetblock.inputargs: + args.append(model.Constant(arg.concretetype._example(), arg.concretetype)) + newblock.closeblock(model.Link(args, resume_point.targetblock)) + + resuming_links.append(model.Link([], newblock, i+1)) + new_start_block.exitswitch = var_resume_state + new_start_block.closeblock(not_resuming_link, *resuming_links) + + old_start_block.isstartblock = False + new_start_block.isstartblock = True + graph.startblock = new_start_block def transform_block(self, block): i = 0 From antocuni at codespeak.net Sat Apr 8 19:07:56 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 8 Apr 2006 19:07:56 +0200 (CEST) Subject: [pypy-svn] r25573 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060408170756.9DD7F1025F@code0.codespeak.net> Author: antocuni Date: Sat Apr 8 19:07:41 2006 New Revision: 25573 Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/ilgenerator.py pypy/dist/pypy/translator/cli/test/compile.py Log: Added support for the 'meta' field in the Object class. The constants used for setting the 'meta' field just after the creation of the object are instantiated at startup time and stored as static fields in the 'pypy.runtime.Constant' class. Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Sat Apr 8 19:07:41 2006 @@ -8,19 +8,25 @@ self.cts = CTS(db) self.classdef = classdef self.namespace, self.name = self.cts.split_class_name(classdef._name) - #0/0 + + def is_root(classdef): + return classdef._superclass is None + is_root = staticmethod(is_root) def get_name(self): return self.name def get_base_class(self): base_class = self.classdef._superclass - if base_class is None: + if self.is_root(base_class): return '[mscorlib]System.Object' else: return base_class._name def render(self, ilasm): + if self.is_root(self.classdef): + return + self.ilasm = ilasm if self.namespace: ilasm.begin_namespace(self.namespace) Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Sat Apr 8 19:07:41 2006 @@ -23,7 +23,7 @@ UnsignedLongLong: 'unsigned int64', Bool: 'bool', Float: 'float64', - Class: '[mscorlib]System.Type', # TODO: check this + Class: 'class [mscorlib]System.Type', # TODO: check this } _pyexception_to_cts = { Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Sat Apr 8 19:07:41 2006 @@ -1,8 +1,12 @@ +from pypy.rpython.ootypesystem import ootype + try: set except NameError: from sets import Set as set +CONST_NAMESPACE = 'pypy.runtime' +CONST_CLASS = 'Constants' class LowLevelDatabase(object): def __init__(self): @@ -18,7 +22,66 @@ def function_name(self, graph): return self.functions.get(graph, None) - def record_const(self, const): - name = '__XXX__CONST__NAME__XXX__' # TODO + def record_const(self, value): + const = AbstractConst.make(value) + name = const.get_name(len(self.consts)) self.consts[const] = name - return name + return '%s.%s::%s' % (CONST_NAMESPACE, CONST_CLASS, name) + + def gen_constants(self, ilasm): + ilasm.begin_namespace(CONST_NAMESPACE) + ilasm.begin_class(CONST_CLASS) + + # render field definitions + for const, name in self.consts.iteritems(): + ilasm.field(name, const.get_type(), static=True) + + # initialize fields + ilasm.begin_function('.cctor', [], 'void', False, 'static', + 'specialname', 'rtspecialname', 'default') + for const, name in self.consts.iteritems(): + const.init(ilasm) + type_ = const.get_type() + ilasm.opcode('stsfld %s %s.%s::%s' % (type_, CONST_NAMESPACE, CONST_CLASS, name)) + + ilasm.opcode('ret') + ilasm.end_function() + + ilasm.end_class() + ilasm.end_namespace() + + +class AbstractConst(object): + def make(const): + if isinstance(const, ootype._view): + const = const._inst + if isinstance(const, ootype._instance): + return InstanceConst(const) + else: + assert False, 'Unknown constant: %s' % const + make = staticmethod(make) + + def get_name(self, n): + pass + + def get_type(self): + pass + + def init(self, ilasm): + pass + +class InstanceConst(AbstractConst): + def __init__(self, obj): + self.obj = obj + + def get_name(self, n): + name = self.obj._TYPE._name.replace('.', '_') + return '%s_%d' % (name, n) + + def get_type(self): + return 'class %s' % self.obj._TYPE._name + + def init(self, ilasm): + classdef = self.obj._TYPE + ilasm.new('instance void class %s::.ctor()' % classdef._name) + # TODO: initialize fields Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Sat Apr 8 19:07:41 2006 @@ -209,8 +209,6 @@ if isinstance(lltype, ootype.Instance): self.db.classes.add(lltype) - - def _render_op(self, op): instr_list = opcodes.get(op.opname, None) if instr_list is not None: @@ -224,8 +222,11 @@ assert False, 'Unknown opcode: %s ' % op def field_name(self, obj, field): - class_name = self.class_name(obj) - field_type = self.cts.lltype_to_cts(obj._field_type(field)) + class_, type_ = obj._lookup_field(field) + assert type_ is not None, 'Cannot find the field %s in the object %s' % (field, obj) + + class_name = self.class_name(class_) + field_type = self.cts.lltype_to_cts(type_) return '%s %s::%s' % (field_type, class_name, field) def ctor_name(self, ooinstance): @@ -282,7 +283,7 @@ else: assert False - def _load_const(self, const): + def _load_const(self, const): type_ = const.concretetype if type_ is Void: pass @@ -295,9 +296,9 @@ elif type_ in (SignedLongLong, UnsignedLongLong): self.ilasm.opcode('ldc.i8', str(const.value)) else: - name = self.db.record_const(const) + name = self.db.record_const(const.value) cts_type = self.cts.lltype_to_cts(type_) - self.ilasm.opcode('ldsfld %s Pypy.Constants::%s' % (cts_type, name)) + self.ilasm.opcode('ldsfld %s %s' % (cts_type, name)) #assert False, 'Unknown constant %s' % const Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Sat Apr 8 19:07:41 2006 @@ -49,7 +49,8 @@ self.gen_entrypoint() self.find_superclasses() self.gen_classes() - self.gen_functions() + self.gen_functions() + self.db.gen_constants(self.ilasm) out.close() return self.tmpfile.strpath Modified: pypy/dist/pypy/translator/cli/ilgenerator.py ============================================================================== --- pypy/dist/pypy/translator/cli/ilgenerator.py (original) +++ pypy/dist/pypy/translator/cli/ilgenerator.py Sat Apr 8 19:07:41 2006 @@ -59,8 +59,13 @@ def end_class(self): self.code.closeblock() - def field(self, name, type_): - self.code.writeline('.field public %s %s' %(type_, name)) + def field(self, name, type_, static = False): + if static: + s = 'static' + else: + s = '' + + self.code.writeline('.field public %s %s %s' % (s, type_, name)) def begin_function(self, name, arglist, returntype, is_entrypoint = False, *args): # TODO: .maxstack Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Sat Apr 8 19:07:41 2006 @@ -1,4 +1,5 @@ #!/bin/env python +import autopath import sys import py from pypy.rpython.rarithmetic import r_int, r_uint, r_ulonglong, r_longlong, ovfcheck @@ -27,15 +28,12 @@ self.x = x def compute(self): - return self.x + return self.x + 1 class Derived(Base): def __init__(self, x): Base.__init__(self, x) - def compute(self): - return self.x+1 - def foo(cls, arg): obj = cls(arg) return obj.compute() @@ -45,12 +43,10 @@ return obj.compute() # return foo(Base, x) + foo(Derived, y) - - f = compile_function(bar, [int, int]) -try: - pass -except py.test.Item.Skipped: - print 'Test skipped' +##try: +## pass +##except py.test.Item.Skipped: +## print 'Test skipped' From mwh at codespeak.net Sat Apr 8 19:39:10 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 19:39:10 +0200 (CEST) Subject: [pypy-svn] r25575 - pypy/dist/pypy/translator/c Message-ID: <20060408173910.50AFF102AD@code0.codespeak.net> Author: mwh Date: Sat Apr 8 19:39:03 2006 New Revision: 25575 Modified: pypy/dist/pypy/translator/c/funcgen.py Log: do a run time, rather than compile time, assert in the case of a switch not having a default case. Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Sat Apr 8 19:39:03 2006 @@ -266,12 +266,13 @@ yield '\t' + op yield 'break;' - # ? Emit default case - if defaultlink is None: - raise TypeError('switches must have a default case.') + # Emit default case yield 'default:' - for op in self.gen_link(defaultlink): - yield '\t' + op + if defaultlink is None: + yield '\tassert(!"bad switch!!");' + else: + for op in self.gen_link(defaultlink): + yield '\t' + op yield '}' else: From mwh at codespeak.net Sat Apr 8 19:41:05 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 19:41:05 +0200 (CEST) Subject: [pypy-svn] r25576 - pypy/dist/pypy/translator/stackless Message-ID: <20060408174105.397E7102AE@code0.codespeak.net> Author: mwh Date: Sat Apr 8 19:40:54 2006 New Revision: 25576 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/transform.py Log: (hpk, mwh) actually generate the code to restore the state in the resumption blocks. next: the main loop? Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sat Apr 8 19:40:54 2006 @@ -87,4 +87,6 @@ u.frame_bottom = frame_state def resume_state(): + # XXX BAAAAAH! + global_state.retval_long = 0 return global_state.restart_substate Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sat Apr 8 19:40:54 2006 @@ -72,10 +72,10 @@ ## return retval + x + 1 class ResumePoint: - def __init__(self, var_result, varstoload, targetblock): + def __init__(self, var_result, link_to_resumption, frame_state_type): self.var_result = var_result - self.varstoload = varstoload - self.targetblock = targetblock + self.link_to_resumption = link_to_resumption + self.frame_state_type = frame_state_type class StacklessTransfomer(object): def __init__(self, translator): @@ -125,6 +125,12 @@ mixlevelannotator.finish() + s_global_state = bk.immutablevalue(code.global_state) + r_global_state = translator.rtyper.getrepr(s_global_state) + self.ll_global_state = model.Constant(r_global_state.convert_const(code.global_state), + r_global_state.lowleveltype) + + def frame_type_for_vars(self, vars): types = [storage_type(v.concretetype) for v in vars] counts = dict.fromkeys(range(len(STORAGE_TYPES)), 0) @@ -168,15 +174,45 @@ new_start_block.operations.append( model.SpaceOperation("direct_call", [self.resume_state_ptr], var_resume_state)) not_resuming_link = model.Link(newinputargs, old_start_block, 0) + not_resuming_link.llexitcase = 0 resuming_links = [] - for i, resume_point in enumerate(self.resume_points): + for resume_point_index, resume_point in enumerate(self.resume_points): newblock = model.Block([]) - args = [] - for arg in resume_point.targetblock.inputargs: - args.append(model.Constant(arg.concretetype._example(), arg.concretetype)) - newblock.closeblock(model.Link(args, resume_point.targetblock)) + newargs = [] + ops = [] + uncasted_frame_top = varoftype(lltype.Ptr(STATE_HEADER)) + ops.append(model.SpaceOperation( + "getfield", + [self.ll_global_state, model.Constant("inst_top", lltype.Void)], + uncasted_frame_top)) + frame_state_type = resume_point.frame_state_type + frame_top = varoftype(lltype.Ptr(frame_state_type)) + ops.append(model.SpaceOperation( + "cast_pointer", + [uncasted_frame_top], + frame_top)) + i = 0 + for arg in resume_point.link_to_resumption.args: + newarg = copyvar(self.translator, arg) + if arg is resume_point.var_result: + ops.append(model.SpaceOperation( + "getfield", + [self.ll_global_state, model.Constant("inst_retval_long", lltype.Void)], + newarg)) + else: + # frame_state_type._names[0] is 'header' + fname = model.Constant(frame_state_type._names[i+1], lltype.Void) + ops.append(model.SpaceOperation( + 'getfield', + [frame_top, fname], + newarg)) + i += 1 + newargs.append(newarg) + newblock.operations.extend(ops) + newblock.closeblock(model.Link(newargs, resume_point.link_to_resumption.target)) - resuming_links.append(model.Link([], newblock, i+1)) + resuming_links.append(model.Link([], newblock, resume_point_index+1)) + resuming_links[-1].llexitcase = resume_point_index+1 new_start_block.exitswitch = var_resume_state new_start_block.closeblock(not_resuming_link, *resuming_links) @@ -200,10 +236,10 @@ args = [v for v in link.args if v is not op.result and v.concretetype is not lltype.Void] - save_block = self.generate_save_block( + save_block, frame_state_type = self.generate_save_block( args, var_unwind_exception) - self.resume_points.append(ResumePoint(op.result, args, link.target)) + self.resume_points.append(ResumePoint(op.result, link, frame_state_type)) newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) @@ -212,7 +248,6 @@ newlink.last_exc_value = var_unwind_exception block.recloseblock(link, newlink) # exits.append(newlink) self.translator.rtyper._convert_link(block, newlink) - # ARGH ... block = link.target i = 0 @@ -266,7 +301,7 @@ save_state_block.closeblock(model.Link([c_unwindexception, var_exc], self.curr_graph.exceptblock)) self.translator.rtyper._convert_link(save_state_block, save_state_block.exits[0]) - return save_state_block + return save_state_block, frame_type def generate_saveops(self, frame_state_var, varstosave): From hpk at codespeak.net Sat Apr 8 20:02:41 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 8 Apr 2006 20:02:41 +0200 (CEST) Subject: [pypy-svn] r25577 - pypy/dist/pypy/translator/stackless Message-ID: <20060408180241.9078610206@code0.codespeak.net> Author: hpk Date: Sat Apr 8 20:02:41 2006 New Revision: 25577 Modified: pypy/dist/pypy/translator/stackless/transform.py Log: (mwh, hpk) refactor getting of fields from global state Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sat Apr 8 20:02:41 2006 @@ -166,6 +166,26 @@ self.curr_graph = None + def ops_read_global_state_field(self, targetvar, fieldname): + ops = [] + llfieldname = "inst_%s" % fieldname + llfieldtype = self.ll_global_state.value._T._flds[llfieldname] + if llfieldtype == targetvar.concretetype: + tmpvar = targetvar + else: + assert isinstance(llfieldtype, lltype.Ptr) + tmpvar = varoftype(llfieldtype) + + ops.append(model.SpaceOperation( + "getfield", + [self.ll_global_state, model.Constant(llfieldname, lltype.Void)], + tmpvar)) + if tmpvar is not targetvar: + ops.append(model.SpaceOperation( + "cast_pointer", [tmpvar], + targetvar)) + return ops + def insert_resume_handling(self, graph): old_start_block = graph.startblock newinputargs = [copyvar(self.translator, v) for v in old_start_block.inputargs] @@ -180,32 +200,19 @@ newblock = model.Block([]) newargs = [] ops = [] - uncasted_frame_top = varoftype(lltype.Ptr(STATE_HEADER)) - ops.append(model.SpaceOperation( - "getfield", - [self.ll_global_state, model.Constant("inst_top", lltype.Void)], - uncasted_frame_top)) frame_state_type = resume_point.frame_state_type frame_top = varoftype(lltype.Ptr(frame_state_type)) - ops.append(model.SpaceOperation( - "cast_pointer", - [uncasted_frame_top], - frame_top)) + ops.extend(self.ops_read_global_state_field(frame_top, "top")) i = 0 for arg in resume_point.link_to_resumption.args: newarg = copyvar(self.translator, arg) if arg is resume_point.var_result: - ops.append(model.SpaceOperation( - "getfield", - [self.ll_global_state, model.Constant("inst_retval_long", lltype.Void)], - newarg)) + ops.extend(self.ops_read_global_state_field(newarg, "retval_long")) else: # frame_state_type._names[0] is 'header' fname = model.Constant(frame_state_type._names[i+1], lltype.Void) ops.append(model.SpaceOperation( - 'getfield', - [frame_top, fname], - newarg)) + 'getfield', [frame_top, fname], newarg)) i += 1 newargs.append(newarg) newblock.operations.extend(ops) From mwh at codespeak.net Sat Apr 8 20:06:36 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 8 Apr 2006 20:06:36 +0200 (CEST) Subject: [pypy-svn] r25581 - pypy/dist/pypy/translator/stackless Message-ID: <20060408180636.83C17102AC@code0.codespeak.net> Author: mwh Date: Sat Apr 8 20:06:31 2006 New Revision: 25581 Modified: pypy/dist/pypy/translator/stackless/code.py Log: make some simple fixes, and another simple change that for moderately obscure reasons allows the removal of an "XXX BAAAAH" Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sat Apr 8 20:06:31 2006 @@ -33,21 +33,21 @@ fn2() elif signature == 'long': fn3 = llmemory.cast_adr_to_ptr(fn, long_void_func) - global_state.long_retval = fn3() + global_state.retval_long = fn3() elif signature == 'longlong': fn3 = llmemory.cast_adr_to_ptr(fn, longlong_void_func) - global_state.longlong_retval = fn3() + global_state.retval_longlong = fn3() elif signature == 'float': fn3 = llmemory.cast_adr_to_ptr(fn, float_void_func) - global_state.float_retval = fn3() + global_state.retval_double = fn3() elif signature == 'pointer': fn5 = llmemory.cast_adr_to_ptr(fn, pointer_void_func) - global_state.pointer_retval = fn5() + global_state.retval_void_p = fn5() null_address = llmemory.fakeaddress(None) def decode_state(state): - return null_address, 'void', 0 + return null_address, 'long', 0 class UnwindException(Exception): def __init__(self): @@ -87,6 +87,4 @@ u.frame_bottom = frame_state def resume_state(): - # XXX BAAAAAH! - global_state.retval_long = 0 return global_state.restart_substate From cfbolz at codespeak.net Sat Apr 8 22:51:24 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 22:51:24 +0200 (CEST) Subject: [pypy-svn] r25599 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060408205124.DB76810225@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 22:51:23 2006 New Revision: 25599 Modified: pypy/dist/pypy/rpython/memory/gc.py pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/gcwrapper.py pypy/dist/pypy/rpython/memory/support.py pypy/dist/pypy/rpython/memory/test/test_gc.py pypy/dist/pypy/rpython/memory/test/test_support.py Log: don't use global freelist. bad idea. (slightly messy to remove, though) Modified: pypy/dist/pypy/rpython/memory/gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gc.py (original) +++ pypy/dist/pypy/rpython/memory/gc.py Sat Apr 8 22:51:23 2006 @@ -1,6 +1,6 @@ from pypy.rpython.memory.lladdress import raw_malloc, raw_free, raw_memcopy from pypy.rpython.memory.lladdress import NULL, address -from pypy.rpython.memory.support import AddressLinkedList +from pypy.rpython.memory.support import get_address_linked_list from pypy.rpython.memory import lltypesimulation from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.objectmodel import free_non_gc_object @@ -29,9 +29,9 @@ class GCError(Exception): pass -def get_dummy_annotate(gc_class): +def get_dummy_annotate(gc_class, AddressLinkedList): def dummy_annotate(): - gc = gc_class() + gc = gc_class(AddressLinkedList) gc.get_roots = dummy_get_roots1 #prevent the get_roots attribute to gc.get_roots = dummy_get_roots2 #be constants a = gc.malloc(1, 2) @@ -39,28 +39,29 @@ gc.write_barrier(raw_malloc(1), raw_malloc(2), raw_malloc(1)) gc.collect() return a - b - return dummy_annotate + + def dummy_get_roots1(): + ll = AddressLinkedList() + ll.append(NULL) + ll.append(raw_malloc(10)) + ll.pop() #make the annotator see pop + return ll + + def dummy_get_roots2(): + ll = AddressLinkedList() + ll.append(raw_malloc(10)) + ll.append(NULL) + ll.pop() #make the annotator see pop + return ll + return dummy_annotate, dummy_get_roots1, dummy_get_roots2 + gc_interface = { "malloc": lltype.FuncType((lltype.Signed, lltype.Signed), lltype.Signed), "collect": lltype.FuncType((), lltype.Void), "write_barrier": lltype.FuncType((llmemory.Address, ) * 3, lltype.Void), } - -def dummy_get_roots1(): - ll = AddressLinkedList() - ll.append(NULL) - ll.append(raw_malloc(10)) - ll.pop() #make the annotator see pop - return ll - -def dummy_get_roots2(): - ll = AddressLinkedList() - ll.append(raw_malloc(10)) - ll.append(NULL) - ll.pop() #make the annotator see pop - return ll - + class GCBase(object): _alloc_flavor_ = "raw" @@ -89,7 +90,7 @@ class DummyGC(GCBase): _alloc_flavor_ = "raw" - def __init__(self, dummy=None, get_roots=None): + def __init__(self, AddressLinkedList, dummy=None, get_roots=None): self.get_roots = get_roots #self.set_query_functions(None, None, None, None, None, None, None) @@ -115,12 +116,13 @@ _size_gc_header = gc_header_one_int - def __init__(self, start_heap_size=4096, get_roots=None): + def __init__(self, AddressLinkedList, start_heap_size=4096, get_roots=None): self.bytes_malloced = 0 self.heap_size = start_heap_size #need to maintain a list of malloced objects, since we used the systems #allocator and can't walk the heap self.malloced_objects = AddressLinkedList() + self.AddressLinkedList = AddressLinkedList #self.set_query_functions(None, None, None, None, None, None, None) self.get_roots = get_roots @@ -164,7 +166,7 @@ ## print "collecting" self.bytes_malloced = 0 roots = self.get_roots() - objects = AddressLinkedList() + objects = self.AddressLinkedList() while 1: curr = roots.pop() ## print "root: ", curr @@ -209,7 +211,7 @@ i += 1 gc_info.signed[0] = gc_info.signed[0] | 1 free_non_gc_object(objects) - newmo = AddressLinkedList() + newmo = self.AddressLinkedList() curr_heap_size = 0 freed_size = 0 while 1: #sweep @@ -244,7 +246,8 @@ class SemiSpaceGC(GCBase): _alloc_flavor_ = "raw" - def __init__(self, space_size=1024*int_size, get_roots=None): + def __init__(self, AddressLinkedList, space_size=1024*int_size, + get_roots=None): self.bytes_malloced = 0 self.space_size = space_size self.tospace = raw_malloc(space_size) @@ -382,8 +385,9 @@ class DeferredRefcountingGC(GCBase): _alloc_flavor_ = "raw" - def __init__(self, max_refcount_zero=50, get_roots=None): + def __init__(self, AddressLinkedList, max_refcount_zero=50, get_roots=None): self.zero_ref_counts = AddressLinkedList() + self.AddressLinkedList = AddressLinkedList self.length_zero_ref_counts = 0 self.max_refcount_zero = max_refcount_zero #self.set_query_functions(None, None, None, None, None, None, None) @@ -407,7 +411,7 @@ else: self.collecting = True roots = self.get_roots() - roots_copy = AddressLinkedList() + roots_copy = self.AddressLinkedList() curr = roots.pop() while curr != NULL: ## print "root", root, root.address[0] @@ -416,7 +420,7 @@ roots_copy.append(curr) curr = roots.pop() roots = roots_copy - dealloc_list = AddressLinkedList() + dealloc_list = self.AddressLinkedList() self.length_zero_ref_counts = 0 while 1: candidate = self.zero_ref_counts.pop() Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Apr 8 22:51:23 2006 @@ -666,7 +666,9 @@ class FrameworkGCTransformer(BoehmGCTransformer): def __init__(self, translator): + from pypy.rpython.memory.support import get_address_linked_list super(FrameworkGCTransformer, self).__init__(translator, inline=True) + AddressLinkedList = get_address_linked_list() class GCData(object): from pypy.rpython.memory.gc import MarkSweepGC as GCClass startheapsize = 8*1024*1024 # XXX adjust @@ -747,7 +749,7 @@ stackbase = lladdress.raw_malloc(GCData.rootstacksize) gcdata.root_stack_top = stackbase gcdata.root_stack_base = stackbase - gcdata.gc = GCData.GCClass(GCData.startheapsize, StackRootIterator) + gcdata.gc = GCData.GCClass(AddressLinkedList, GCData.startheapsize, StackRootIterator) gcdata.gc.set_query_functions( q_is_varsize, q_offsets_to_gc_pointers, Modified: pypy/dist/pypy/rpython/memory/gcwrapper.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gcwrapper.py (original) +++ pypy/dist/pypy/rpython/memory/gcwrapper.py Sat Apr 8 22:51:23 2006 @@ -1,7 +1,7 @@ from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rpython.memory.support import AddressLinkedList, INT_SIZE +from pypy.rpython.memory.support import get_address_linked_list, INT_SIZE from pypy.rpython.memory.lladdress import raw_malloc, raw_free, NULL from pypy.rpython.memory import lltypelayout from pypy.rpython.memory import lltypesimulation @@ -175,10 +175,11 @@ class GcWrapper(object): def __init__(self, llinterp, flowgraphs, gc_class): self.query_types = QueryTypes() + self.AddressLinkedList = get_address_linked_list(3) # XXX there might me GCs that have headers that depend on the type # therefore we have to change the query functions to annotatable ones # later - self.gc = gc_class() + self.gc = gc_class(self.AddressLinkedList) self.gc.set_query_functions(*self.query_types.get_setup_query_functions()) fgcc = FlowGraphConstantConverter(flowgraphs, self.gc, self.query_types) fgcc.convert() @@ -251,7 +252,7 @@ def get_roots(self): self.get_roots_from_llinterp() - ll = AddressLinkedList() + ll = self.AddressLinkedList() for i, root in enumerate(self.roots): self.pseudo_root_pointers.address[i] = root._address ll.append(self.pseudo_root_pointers + INT_SIZE * i) @@ -268,15 +269,17 @@ def annotate_rtype_gc(self): # annotate and specialize functions gc_class = self.gc.__class__ + AddressLinkedList = self.AddressLinkedList def instantiate_linked_list(): return AddressLinkedList() f1, f2, f3, f4, f5, f6, f7 = self.query_types.create_query_functions() def instantiate_gc(): - gc = gc_class() + gc = gc_class(AddressLinkedList) gc.set_query_functions(f1, f2, f3, f4, f5, f6, f7) return gc - func = gc.get_dummy_annotate(self.gc.__class__) - self.gc.get_roots = gc.dummy_get_roots1 + func, dummy_get_roots1, dummy_get_roots2 = gc.get_dummy_annotate( + self.gc.__class__, self.AddressLinkedList) + self.gc.get_roots = dummy_get_roots1 a = RPythonAnnotator() a.build_types(instantiate_gc, []) a.build_types(func, []) @@ -295,7 +298,7 @@ self.gcptr = self.llinterp.eval_graph( a.bookkeeper.getdesc(instantiate_gc).cachedgraph(None)) GETROOTS_FUNCTYPE = lltype.typeOf( - getfunctionptr(a, gc.dummy_get_roots1)).TO + getfunctionptr(a, dummy_get_roots1)).TO setattr(self.gcptr, "inst_get_roots", lltypesimulation.functionptr(GETROOTS_FUNCTYPE, "get_roots", _callable=self.get_roots)) Modified: pypy/dist/pypy/rpython/memory/support.py ============================================================================== --- pypy/dist/pypy/rpython/memory/support.py (original) +++ pypy/dist/pypy/rpython/memory/support.py Sat Apr 8 22:51:23 2006 @@ -25,41 +25,42 @@ chunk.address[0] = self.free_list self.free_list = chunk -class AddressLinkedList(object): - _alloc_flavor_ = "raw" - - unused_chunks = FreeList(CHUNK_SIZE + 2) - - def __init__(self): - self.chunk = NULL - - def append(self, addr): - if addr == NULL: - return - if self.chunk == NULL or self.chunk.signed[1] == CHUNK_SIZE: - new = AddressLinkedList.unused_chunks.get() - new.address[0] = self.chunk - new.signed[1] = 0 - self.chunk = new - used_chunks = self.chunk.signed[1] - self.chunk.signed[1] += 1 - self.chunk.address[used_chunks + 2] = addr +def get_address_linked_list(chunk_size=CHUNK_SIZE): + unused_chunks = FreeList(chunk_size + 2) + class AddressLinkedList(object): + _alloc_flavor_ = "raw" - def pop(self): - used_chunks = self.chunk.signed[1] - if used_chunks == 0: - old = self.chunk - previous = old.address[0] - if previous == NULL: - return NULL - self.chunk = previous - AddressLinkedList.unused_chunks.put(old) + def __init__(self): + self.chunk = NULL + + def append(self, addr): + if addr == NULL: + return + if self.chunk == NULL or self.chunk.signed[1] == chunk_size: + new = unused_chunks.get() + new.address[0] = self.chunk + new.signed[1] = 0 + self.chunk = new + used_chunks = self.chunk.signed[1] + self.chunk.signed[1] += 1 + self.chunk.address[used_chunks + 2] = addr + + def pop(self): used_chunks = self.chunk.signed[1] - result = self.chunk.address[used_chunks + 1] - self.chunk.address[used_chunks + 1] = NULL - self.chunk.signed[1] = used_chunks - 1 - return result + if used_chunks == 0: + old = self.chunk + previous = old.address[0] + if previous == NULL: + return NULL + self.chunk = previous + unused_chunks.put(old) + used_chunks = self.chunk.signed[1] + result = self.chunk.address[used_chunks + 1] + self.chunk.address[used_chunks + 1] = NULL + self.chunk.signed[1] = used_chunks - 1 + return result - def free(self): - while self.pop() != NULL: - pass + def free(self): + while self.pop() != NULL: + pass + return AddressLinkedList Modified: pypy/dist/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gc.py Sat Apr 8 22:51:23 2006 @@ -6,7 +6,7 @@ from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.memory.gc import GCError, MarkSweepGC, SemiSpaceGC from pypy.rpython.memory.gc import DeferredRefcountingGC, DummyGC -from pypy.rpython.memory.support import AddressLinkedList, INT_SIZE, CHUNK_SIZE, FreeList +from pypy.rpython.memory.support import INT_SIZE, CHUNK_SIZE from pypy.rpython.memory import support from pypy.rpython.memory.lladdress import raw_malloc, raw_free, NULL from pypy.rpython.memory.simulator import MemorySimulatorError @@ -29,11 +29,9 @@ py.log.setconsumer("llinterp frame", stdout_ignore_ll_functions) py.log.setconsumer("llinterp operation", None) gclltype.prepare_graphs_and_create_gc = gclltype.create_gc - support.CHUNK_SIZE = 1 def teardown_module(mod): gclltype.prepare_graphs_and_create_gc = gclltype.create_no_gc - support.CHUNK_SIZE = CHUNK_SIZE class TestMarkSweepGC(object): def setup_class(cls): @@ -100,7 +98,6 @@ class TestMarkSweepGCRunningOnLLinterp(TestMarkSweepGC): def setup_class(cls): - AddressLinkedList.unused_chunks = FreeList(3) # 'leaks' but well cls.prep_old = gclltype.prepare_graphs_and_create_gc gclltype.prepare_graphs_and_create_gc = gclltype.create_gc_run_on_llinterp def teardown_class(cls): @@ -115,7 +112,6 @@ class TestSemiSpaceGCRunningOnLLinterp(TestMarkSweepGC): def setup_class(cls): - AddressLinkedList.unused_chunks = FreeList(3) # 'leaks' but well cls.prep_old = gclltype.prepare_graphs_and_create_gc gclltype.prepare_graphs_and_create_gc = gclltype.create_gc_run_on_llinterp gclltype.use_gc = SemiSpaceGC @@ -135,7 +131,6 @@ class TestDeferredRefcountingGCRunningOnLLinterp(TestMarkSweepGC): def setup_class(cls): - AddressLinkedList.unused_chunks = FreeList(3) # 'leaks' but well cls.prep_old = gclltype.prepare_graphs_and_create_gc gclltype.prepare_graphs_and_create_gc = gclltype.create_gc_run_on_llinterp gclltype.use_gc = DeferredRefcountingGC @@ -154,7 +149,6 @@ class TestDummyGCRunningOnLLinterp(TestMarkSweepGC): def setup_class(cls): - AddressLinkedList.unused_chunks = FreeList(3) # 'leaks' but well cls.prep_old = gclltype.prepare_graphs_and_create_gc gclltype.prepare_graphs_and_create_gc = gclltype.create_gc_run_on_llinterp gclltype.use_gc = DummyGC Modified: pypy/dist/pypy/rpython/memory/test/test_support.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_support.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_support.py Sat Apr 8 22:51:23 2006 @@ -1,5 +1,5 @@ from pypy.rpython.objectmodel import free_non_gc_object -from pypy.rpython.memory.support import AddressLinkedList, FreeList, CHUNK_SIZE +from pypy.rpython.memory.support import get_address_linked_list, FreeList from pypy.rpython.memory import support from pypy.rpython.memory.lladdress import raw_malloc, raw_free, NULL from pypy.rpython.memory.test.test_llinterpsim import interpret @@ -7,6 +7,7 @@ class TestAddressLinkedList(object): def test_simple_access(self): + AddressLinkedList = get_address_linked_list() addr = raw_malloc(100) ll = AddressLinkedList() ll.append(addr) @@ -32,6 +33,7 @@ raw_free(addr) def test_big_access(self): + AddressLinkedList = get_address_linked_list() addr = raw_malloc(1) ll = AddressLinkedList() for i in range(3000): @@ -51,6 +53,7 @@ raw_free(addr) def test_linked_list_annotate(): + AddressLinkedList = get_address_linked_list() def f(): addr = raw_malloc(100) ll = AddressLinkedList() @@ -91,7 +94,6 @@ ## a.translator.specialize() ## a.translator.view() assert f() - # grumpf: make sure that we don't get a polluted class attribute - AddressLinkedList.unused_chunks = FreeList(CHUNK_SIZE + 2) + AddressLinkedList = get_address_linked_list() res = interpret(f, []) assert res From cfbolz at codespeak.net Sat Apr 8 23:17:31 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 23:17:31 +0200 (CEST) Subject: [pypy-svn] r25601 - pypy/dist/pypy/translator/backendopt Message-ID: <20060408211731.45E521024B@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 23:17:30 2006 New Revision: 25601 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: rename aib (?!) to adi Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Sat Apr 8 23:17:30 2006 @@ -362,11 +362,11 @@ return True def malloc_to_stack(t): - aib = AbstractDataFlowInterpreter(t) + adi = AbstractDataFlowInterpreter(t) for graph in t.graphs: - if graph.startblock not in aib.flown_blocks: - aib.schedule_function(graph) - aib.complete() + if graph.startblock not in adi.flown_blocks: + adi.schedule_function(graph) + adi.complete() for graph in t.graphs: loop_blocks = support.find_loop_blocks(graph) for block in graph.iterblocks(): @@ -380,7 +380,7 @@ continue except (ValueError, AttributeError), e: pass - varstate = aib.getstate(op.result) + varstate = adi.getstate(op.result) assert len(varstate.creation_points) == 1 crep = varstate.creation_points.keys()[0] if not crep.escapes: @@ -390,3 +390,4 @@ op.args.insert(0, inputconst(lltype.Void, 'stack')) else: print "%s in %s is a non-escaping malloc in a loop" % (op, graph.name) + From cfbolz at codespeak.net Sat Apr 8 23:51:55 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 8 Apr 2006 23:51:55 +0200 (CEST) Subject: [pypy-svn] r25602 - pypy/dist/pypy/doc/discussion Message-ID: <20060408215155.95ED810254@code0.codespeak.net> Author: cfbolz Date: Sat Apr 8 23:51:53 2006 New Revision: 25602 Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt Log: update the swamp-map Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/translation-swamp.txt (original) +++ pypy/dist/pypy/doc/discussion/translation-swamp.txt Sat Apr 8 23:51:53 2006 @@ -7,12 +7,12 @@ * nondeterminism of insert stackcheck: - * (TESTING) move it after inlining - * avoid inserting it multiple times in the same block + * (DONE) move it after inlining + * (DONE) avoid inserting it multiple times in the same block * experiment with different heuristics: - * weight backedges more + * weigh backedges more * consider size of outer function * consider number of arguments @@ -22,6 +22,13 @@ malloc removal possibilities (using escape analysis) * move the inlining of gc helpers just before emitting the code. - throw the graph away + throw the graph away (TESTING) + + * for gcc: use just one implement file (TRIED: turns out to be a bad idea, + because gcc uses too much ram) + +things to improve the framework gc +================================== + + * find out whether a function can collect - * for gcc: use just one implement file (TESTING) From pedronis at codespeak.net Sat Apr 8 23:54:49 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 8 Apr 2006 23:54:49 +0200 (CEST) Subject: [pypy-svn] r25603 - pypy/dist/pypy/doc/discussion Message-ID: <20060408215449.3D21A102A8@code0.codespeak.net> Author: pedronis Date: Sat Apr 8 23:54:48 2006 New Revision: 25603 Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt Log: (cfbolz, pedronis) update status of work/issues. Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/translation-swamp.txt (original) +++ pypy/dist/pypy/doc/discussion/translation-swamp.txt Sat Apr 8 23:54:48 2006 @@ -5,27 +5,23 @@ * understand nondeterminism after rtyping - * nondeterminism of insert stackcheck: - - * (DONE) move it after inlining - * (DONE) avoid inserting it multiple times in the same block - * experiment with different heuristics: - * weigh backedges more + * weigh backedges more (TESTING) * consider size of outer function - * consider number of arguments + * consider number of arguments (TESTING) - * find a more deterministic inlining order (number of calls?) + * find a more deterministic inlining order (TESTING using number of callers) * experiment with using a base inlining threshold and then drive inlining by malloc removal possibilities (using escape analysis) * move the inlining of gc helpers just before emitting the code. - throw the graph away (TESTING) + throw the graph away (TESTING, need to do a new framework translation) * for gcc: use just one implement file (TRIED: turns out to be a bad idea, - because gcc uses too much ram) + because gcc uses too much ram). Need to experiment more now that + inlining should at least be more deterministic! things to improve the framework gc ================================== From mwh at codespeak.net Sun Apr 9 10:54:14 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 10:54:14 +0200 (CEST) Subject: [pypy-svn] r25607 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060409085414.6546610131@code0.codespeak.net> Author: mwh Date: Sun Apr 9 10:54:12 2006 New Revision: 25607 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py Log: (hpk, mwh, pedronis consulting) run the stackless main loop after the entry point has unwound. cope with a small annotator boggle, see comments in test_transform. Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sun Apr 9 10:54:12 2006 @@ -60,8 +60,8 @@ def slp_main_loop(): currentframe = global_state.top - while currentframe is not None: - nextframe = currentframe.f_back + while currentframe: + global_state.top = nextframe = currentframe.f_back framestate = currentframe.state fn, signature, global_state.restart_substate = decode_state(framestate) try: Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Sun Apr 9 10:54:12 2006 @@ -30,6 +30,8 @@ ## assert s2_1 is s2_2 +from pypy.translator.stackless import code + def test_simple_transform(): from pypy.translator.stackless.code import UnwindException def check(x): @@ -45,13 +47,30 @@ def run_stackless_function(fn, *stacklessfuncs): def entry_point(argv): - os.write(1, str(fn(len(argv)))+'\n') + try: + r = fn(len(argv)) + except code.UnwindException, u: + code.global_state.top = u.frame_top + code.slp_main_loop() + r = code.global_state.retval_long + os.write(1, str(r)+'\n') return 0 s_list_of_strings = annmodel.SomeList(ListDef(None, annmodel.SomeString())) s_list_of_strings.listdef.resize() t = TranslationContext() - t.buildannotator().build_types(entry_point, [s_list_of_strings]) + annotator = t.buildannotator() + bk = annotator.bookkeeper + # we want to make sure that the annotator knows what + # code.UnwindException looks like very early on, because otherwise + # it can get mutated during the annotation of the low level + # helpers which can cause slp_main_loop to get re-annotated after + # it is rtyped. which is bad. + unwind_def = bk.getuniqueclassdef(code.UnwindException) + unwind_def.generalize_attr('frame_top', annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) + unwind_def.generalize_attr('frame_bottom', annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) + + annotator.build_types(entry_point, [s_list_of_strings]) t.buildrtyper().specialize() st = StacklessTransfomer(t) Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 10:54:12 2006 @@ -93,14 +93,6 @@ mixlevelannotator = MixLevelHelperAnnotator(translator.rtyper) l2a = annmodel.lltype_to_annotation - slp_main_loop_graph = mixlevelannotator.getgraph( - code.slp_main_loop, [], l2a(lltype.Void)) - SLP_MAIN_LOOP_TYPE = lltype.FuncType([], lltype.Void) - self.slp_main_loop_type_ptr = model.Constant(lltype.functionptr( - SLP_MAIN_LOOP_TYPE, "slp_main_loop", - graph=slp_main_loop_graph), - lltype.Ptr(SLP_MAIN_LOOP_TYPE)) - annotations = [annmodel.SomeInstance(bk.getuniqueclassdef(code.UnwindException)), annmodel.SomePtr(lltype.Ptr(STATE_HEADER))] From mwh at codespeak.net Sun Apr 9 11:12:19 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 11:12:19 +0200 (CEST) Subject: [pypy-svn] r25608 - pypy/dist/pypy/translator/c Message-ID: <20060409091219.E1866101CB@code0.codespeak.net> Author: mwh Date: Sun Apr 9 11:12:16 2006 New Revision: 25608 Modified: pypy/dist/pypy/translator/c/primitive.py Log: remove nonsensical assert Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Sun Apr 9 11:12:16 2006 @@ -98,7 +98,6 @@ else: return db.get(value.ob) else: - assert value.offset is not None if isinstance(typeOf(value.ob), ContainerType): base = db.getcontainernode(value.ob).ptrname else: From hpk at codespeak.net Sun Apr 9 11:24:07 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sun, 9 Apr 2006 11:24:07 +0200 (CEST) Subject: [pypy-svn] r25609 - pypy/dist/pypy/translator/stackless Message-ID: <20060409092407.71348101E9@code0.codespeak.net> Author: hpk Date: Sun Apr 9 11:24:06 2006 New Revision: 25609 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/transform.py Log: (mwh, hpk) some cleanups and moving closer to having the main loop run, resuming/saving state for functions returning signed ... Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sun Apr 9 11:24:06 2006 @@ -3,11 +3,16 @@ STATE_HEADER = lltype.Struct('state_header', ('f_back', lltype.Ptr(lltype.ForwardReference())), - ('state', lltype.Signed)) + ('restartstate', lltype.Signed), + ('function', llmemory.Address), +) STATE_HEADER.f_back.TO.become(STATE_HEADER) null_state = lltype.nullptr(STATE_HEADER) +def decode_state(currentframe): + return currentframe.function, "long", currentframe.restartstate + class StacklessData: def __init__(self): self.top = null_state @@ -46,8 +51,6 @@ null_address = llmemory.fakeaddress(None) -def decode_state(state): - return null_address, 'long', 0 class UnwindException(Exception): def __init__(self): @@ -62,8 +65,7 @@ while currentframe: global_state.top = nextframe = currentframe.f_back - framestate = currentframe.state - fn, signature, global_state.restart_substate = decode_state(framestate) + fn, signature, global_state.restart_substate = decode_state(currentframe) try: call_function(fn, signature) except UnwindException, u: #XXX annotation support needed Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 11:24:06 2006 @@ -9,6 +9,8 @@ from pypy.translator.stackless import code from pypy.rpython.rclass import getinstancerepr +from pypy.translator.stackless.code import STATE_HEADER, null_state + STORAGE_TYPES = [llmemory.Address, lltype.Signed, lltype.Float, @@ -34,15 +36,6 @@ else: raise Exception("don't know about %r" % (T,)) - - -STATE_HEADER = lltype.Struct('state_header', - ('f_back', lltype.Ptr(lltype.ForwardReference())), - ('state', lltype.Signed)) -STATE_HEADER.f_back.TO.become(STATE_HEADER) - -null_state = lltype.nullptr(STATE_HEADER) - ## def func(x): ## return g() + x + 1 @@ -295,6 +288,18 @@ [self.add_frame_state_ptr, var_exc, var_header], varoftype(lltype.Void))) + saveops.append(model.SpaceOperation( + "setfield", [var_header, model.Constant("restartstate", lltype.Void), + model.Constant(len(self.resume_points)+1, lltype.Signed)], + varoftype(lltype.Void))) + # XXX add returntypes + FUNCTYPE = lltype.FuncType([], lltype.Signed) + funcptr = lltype.functionptr(FUNCTYPE, "", graph=self.curr_graph) + #saveops.append(model.SpaceOperation( + # "setfield", [var_header, model.Constant("function", lltype.Void), + # model.Constant(llmemory.fakeaddress(funcptr), llmemory.Address)], + # varoftype(lltype.Void))) + type_repr = rclass.get_type_repr(rtyper) c_unwindexception = model.Constant(type_repr.convert_const(code.UnwindException), type_repr.lowleveltype) save_state_block.closeblock(model.Link([c_unwindexception, var_exc], From cfbolz at codespeak.net Sun Apr 9 12:08:20 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 9 Apr 2006 12:08:20 +0200 (CEST) Subject: [pypy-svn] r25610 - in pypy/dist/pypy: objspace/flow translator/backendopt translator/backendopt/test Message-ID: <20060409100820.D7C38101EC@code0.codespeak.net> Author: cfbolz Date: Sun Apr 9 12:08:15 2006 New Revision: 25610 Added: pypy/dist/pypy/translator/backendopt/mallocprediction.py pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py Modified: pypy/dist/pypy/objspace/flow/model.py pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/inline.py Log: try to find out calls that are interesting to inline because doing that will probably enable malloc removal. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Sun Apr 9 12:08:15 2006 @@ -98,6 +98,11 @@ seen[block] = True stack += block.exits[::-1] + def iterblockops(self): + for block in self.iterblocks(): + for op in block.operations: + yield block, op + def show(self): from pypy.translator.tool.graphpage import SingleGraphPage SingleGraphPage(self).display() Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Sun Apr 9 12:08:15 2006 @@ -369,25 +369,25 @@ adi.complete() for graph in t.graphs: loop_blocks = support.find_loop_blocks(graph) - for block in graph.iterblocks(): - for op in block.operations: - if op.opname == 'malloc': - STRUCT = op.args[0].value - # must not remove mallocs of structures that have a RTTI with a destructor - try: - destr_ptr = lltype.getRuntimeTypeInfo(STRUCT)._obj.destructor_funcptr - if destr_ptr: - continue - except (ValueError, AttributeError), e: - pass - varstate = adi.getstate(op.result) - assert len(varstate.creation_points) == 1 - crep = varstate.creation_points.keys()[0] - if not crep.escapes: - if block not in loop_blocks: - print "moving object from heap to stack %s in %s" % (op, graph.name) - op.opname = 'flavored_malloc' - op.args.insert(0, inputconst(lltype.Void, 'stack')) - else: - print "%s in %s is a non-escaping malloc in a loop" % (op, graph.name) + for block, op in graph.iterblockops(): + if op.opname == 'malloc': + STRUCT = op.args[0].value + # must not remove mallocs of structures that have a RTTI with a destructor + try: + destr_ptr = lltype.getRuntimeTypeInfo(STRUCT)._obj.destructor_funcptr + if destr_ptr: + continue + except (ValueError, AttributeError), e: + pass + varstate = adi.getstate(op.result) + assert len(varstate.creation_points) == 1 + crep = varstate.creation_points.keys()[0] + if not crep.escapes: + if block not in loop_blocks: + print "moving object from heap to stack %s in %s" % (op, graph.name) + op.opname = 'flavored_malloc' + op.args.insert(0, inputconst(lltype.Void, 'stack')) + else: + print "%s in %s is a non-escaping malloc in a loop" % (op, graph.name) + Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Apr 9 12:08:15 2006 @@ -520,12 +520,14 @@ return result -def auto_inlining(translator, threshold=1): +def auto_inlining(translator, threshold=1, callgraph=None): from heapq import heappush, heappop, heapreplace threshold *= BASE_INLINE_THRESHOLD callers = {} # {graph: {graphs-that-call-it}} callees = {} # {graph: {graphs-that-it-calls}} - for graph1, graph2 in static_callers(translator, ignore_primitives=True): + if callgraph is None: + callgraph = static_callers(translator, ignore_primitives=True) + for graph1, graph2 in callgraph: callers.setdefault(graph2, {})[graph1] = True callees.setdefault(graph1, {})[graph2] = True heap = [(0.0, -len(callers[graph]), graph) for graph in callers] Added: pypy/dist/pypy/translator/backendopt/mallocprediction.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/backendopt/mallocprediction.py Sun Apr 9 12:08:15 2006 @@ -0,0 +1,116 @@ +from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter +from pypy.translator.backendopt.malloc import remove_simple_mallocs +from pypy.translator.backendopt.inline import auto_inlining +from pypy.rpython.lltypesystem import lltype +from pypy.translator.simplify import get_graph + +def find_malloc_creps(graph, adi, translator): + # mapping from malloc creation point to graphs that it flows into + malloc_creps = {} + # find all mallocs that don't escape + for block, op in graph.iterblockops(): + if op.opname == 'malloc': + STRUCT = op.args[0].value + # must not remove mallocs of structures that have a RTTI with a destructor + try: + destr_ptr = lltype.getRuntimeTypeInfo( + STRUCT)._obj.destructor_funcptr + if destr_ptr: + continue + except (ValueError, AttributeError), e: + pass + varstate = adi.getstate(op.result) + assert len(varstate.creation_points) == 1 + crep = varstate.creation_points.keys()[0] + if not crep.escapes: + malloc_creps[crep] = {} + return malloc_creps + +def find_calls_where_creps_go(interesting_creps, graph, adi, + translator, seen): + print "find_calls_where_creps_go", interesting_creps, graph.name + print seen + # drop creps that are merged with another creation point + for block in graph.iterblocks(): + for var in block.getvariables(): + varstate = adi.getstate(var) + if varstate is None: + continue + for crep in varstate.creation_points: + if crep in interesting_creps: + if len(varstate.creation_points) != 1: + del interesting_creps[crep] + break + + # drop creps that are passed into an indirect_call + for block, op in graph.iterblockops(): + if not interesting_creps: + return + if op.opname == "indirect_call": + for var in op.args[:-1]: + varstate = adi.getstate(var) + for crep in varstate.creation_points: + if crep in interesting_creps: + del interesting_creps[crep] + elif op.opname == "direct_call": + print op, interesting_creps + called_graph = get_graph(op.args[0], translator) + if called_graph is None: + del interesting_creps[crep] + print "graph not found" + continue + interesting = {} + for i, var in enumerate(op.args[1:]): + print i, var, + varstate = adi.getstate(var) + if varstate is None: + print "no varstate" + continue + if len(varstate.creation_points) == 1: + crep = varstate.creation_points.keys()[0] + if crep not in interesting_creps: + print "not interesting" + continue + if (called_graph, i) in seen: + seen[(called_graph, i)][graph] = True + print "seen already" + else: + print "taking", crep + seen[(called_graph, i)] = {graph: True} + arg = called_graph.startblock.inputargs[i] + argstate = adi.getstate(arg) + argcrep = [c for c in argstate.creation_points + if c.creation_method == "arg"][0] + interesting[argcrep] = True + print interesting + if interesting: + find_calls_where_creps_go(interesting, called_graph, + adi, translator, seen) + return interesting_creps + +def find_malloc_removal_candidates(t): + adi = AbstractDataFlowInterpreter(t) + for graph in t.graphs: + if graph.startblock not in adi.flown_blocks: + adi.schedule_function(graph) + adi.complete() + caller_candidates = {} + seen = {} + for graph in t.graphs: + creps = find_malloc_creps(graph, adi, t) + print "malloc creps", creps + if creps: + find_calls_where_creps_go(creps, graph, adi, t, seen) + if creps: + caller_candidates[graph] = True + callgraph = [] + for (called_graph, i), callers in seen.iteritems(): + for caller in callers: + callgraph.append((caller, called_graph)) + return callgraph, caller_candidates + +def inline_and_remove(t, threshold=1): + callgraph, caller_candidates = find_malloc_removal_candidates(t) + auto_inlining(t, threshold, callgraph) + for graph in caller_candidates: + remove_simple_mallocs(graph) Added: pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py Sun Apr 9 12:08:15 2006 @@ -0,0 +1,86 @@ +import py +from pypy.translator.translator import TranslationContext +from pypy.translator.backendopt.inline import inline_function +from pypy.translator.backendopt.all import backend_optimizations +from pypy.translator.translator import TranslationContext, graphof +from pypy.rpython.llinterp import LLInterpreter +from pypy.objspace.flow.model import checkgraph, flatten, Block +from pypy.conftest import option + +from pypy.translator.backendopt.mallocprediction import * + +def compile(fn, signature): + t = TranslationContext() + t.buildannotator().build_types(fn, signature) + t.buildrtyper().specialize() + graph = graphof(t, fn) + if option.view: + t.view() + return t, graph + + +def check_inlining(t, graph, args, result): + callgraph, caller_candidates = find_malloc_removal_candidates(t) + nice_callgraph = {} + for caller, callee in callgraph: + nice_callgraph.setdefault(caller, {})[callee] = True + inline_and_remove(t) + if option.view: + t.view() + interp = LLInterpreter(t.rtyper) + res = interp.eval_graph(graph, args) + assert res == result + return nice_callgraph, caller_candidates + +def test_fn(): + class A: + pass + class B(A): + pass + def g(a, b, i): + a.b = b + b.i = i + return a.b.i + def h(x): + return x + 42 + def fn(i): + a = A() + b = B() + x = h(i) + return g(a, b, x) + t, graph = compile(fn, [int]) + callgraph, caller_candidates = check_inlining(t, graph, [0], 42) + assert caller_candidates == {graph: True} + assert len(callgraph) == 1 + ggraph = graphof(t, g) + assert callgraph[graph] == {ggraph: True} + +def test_multiple_calls(): + class A: + pass + class B(A): + pass + def g2(b, i): + b.i = h(i) + def g1(a, b, i): + a.b = b + g2(b, h(i)) + return a.b.i + def h(x): + return x + 42 + def fn(i): + a = A() + b = B() + x = h(i) + return g1(a, b, x) + t, graph = compile(fn, [int]) + callgraph, caller_candidates = check_inlining(t, graph, [0], 3 * 42) + print callgraph + assert caller_candidates == {graph: True} + assert len(callgraph) == 1 + g1graph = graphof(t, g1) + g2graph = graphof(t, g2) + assert callgraph[graph] == {g1graph: True} + callgraph, caller_candidates = check_inlining(t, graph, [0], 3 * 42) + assert callgraph[graph] == {g2graph: True} + From cfbolz at codespeak.net Sun Apr 9 12:10:29 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 9 Apr 2006 12:10:29 +0200 (CEST) Subject: [pypy-svn] r25611 - pypy/dist/pypy/translator/backendopt Message-ID: <20060409101029.EBAD0101ED@code0.codespeak.net> Author: cfbolz Date: Sun Apr 9 12:10:29 2006 New Revision: 25611 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: remove this strange thing. it made the bytecode dispatching explode Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Apr 9 12:10:29 2006 @@ -491,12 +491,8 @@ count += block_weight(block) return count -def inlining_heuristic(graph, callers=None, callees=None): +def inlining_heuristic(graph): # XXX ponderation factors? - factor = 1 - if callers is not None: - if len(callers) == 1: - factor = 0.3 return (0.9999 * measure_median_execution_cost(graph) + static_instruction_count(graph)) * factor From cfbolz at codespeak.net Sun Apr 9 12:11:02 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 9 Apr 2006 12:11:02 +0200 (CEST) Subject: [pypy-svn] r25612 - pypy/dist/pypy/translator/backendopt Message-ID: <20060409101102.9852D101F4@code0.codespeak.net> Author: cfbolz Date: Sun Apr 9 12:11:01 2006 New Revision: 25612 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: this belongs to the last checkin: I forgot to safe the file :-( Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Apr 9 12:11:01 2006 @@ -534,7 +534,7 @@ while heap: weight, _, graph = heap[0] if not valid_weight.get(graph): - weight = inlining_heuristic(graph, callers.get(graph), callees.get(graph)) + weight = inlining_heuristic(graph) #print ' + cost %7.2f %50s' % (weight, graph.name) heapreplace(heap, (weight, -len(callers[graph]), graph)) valid_weight[graph] = True From arigo at codespeak.net Sun Apr 9 12:25:07 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 12:25:07 +0200 (CEST) Subject: [pypy-svn] r25613 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060409102507.8E17B101F1@code0.codespeak.net> Author: arigo Date: Sun Apr 9 12:25:06 2006 New Revision: 25613 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: (arre, arigo) Conversion from Array to Array(nolength=True), as well as from FixedSizeArray to Array(nolength=True). Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Sun Apr 9 12:25:06 2006 @@ -612,6 +612,24 @@ raise TypeError, "can only cast pointers to other pointers" return ptr._cast_to(PTRTYPE) +def cast_array_pointer(PTRTYPE, ptr): + CURTYPE = typeOf(ptr) + if not isinstance(CURTYPE, Ptr) or not isinstance(PTRTYPE, Ptr): + raise TypeError, "can only cast pointers to other pointers" + if (not isinstance(PTRTYPE.TO, Array) or + not PTRTYPE.TO._hints.get('nolength', False)): + raise TypeError, "can only convert to nolength Arrays" + if isinstance(CURTYPE.TO, Array): + if CURTYPE.TO._hints.get('nolength', False): + raise TypeError, "cannot convert from a nolength Array" + elif isinstance(CURTYPE.TO, FixedSizeArray): + pass + else: + raise TypeError, "can only convert from an array" + if CURTYPE.TO.OF != PTRTYPE.TO.OF: + raise TypeError, "pointers to arrays of different item types" + return ptr._cast_array_to(PTRTYPE) + def _expose(val, solid=False): """XXX A nice docstring here""" T = typeOf(val) @@ -692,7 +710,8 @@ def _setobj(self, pointing_to, solid=False): if pointing_to is None: obj0 = None - elif solid or isinstance(self._T, (GC_CONTAINER, FuncType)): + elif (solid or isinstance(self._T, (GC_CONTAINER, FuncType)) + or isinstance(pointing_to, _nolengtharray)): obj0 = pointing_to else: self._set_weak(True) @@ -761,20 +780,15 @@ field_name)) def __getitem__(self, i): # ! can only return basic or ptr ! - if isinstance(self._T, Array): - if not (0 <= i < len(self._obj.items)): + if isinstance(self._T, (Array, FixedSizeArray)): + if not (0 <= i < self._obj.getlength()): raise IndexError("array index out of bounds") - o = self._obj.items[i] - return _expose(o, self._solid) - if isinstance(self._T, FixedSizeArray): - if not (0 <= i < self._T.length): - raise IndexError("fixed-size array index out of bounds") - o = getattr(self._obj, 'item%d' % i) + o = self._obj.getitem(i) return _expose(o, self._solid) raise TypeError("%r instance is not an array" % (self._T,)) def __setitem__(self, i, val): - if isinstance(self._T, Array): + if isinstance(self._T, (Array, FixedSizeArray)): T1 = self._T.OF if isinstance(T1, ContainerType): raise TypeError("cannot directly assign to container array items") @@ -783,34 +797,18 @@ raise TypeError("%r items:\n" "expect %r\n" " got %r" % (self._T, T1, T2)) - if not (0 <= i < len(self._obj.items)): + if not (0 <= i < self._obj.getlength()): raise IndexError("array index out of bounds") - self._obj.items[i] = val - return - if isinstance(self._T, FixedSizeArray): - T1 = self._T.OF - if isinstance(T1, ContainerType): - raise TypeError("cannot directly assign to container array items") - T2 = typeOf(val) - if T2 != T1: - raise TypeError("%r items:\n" - "expect %r\n" - " got %r" % (self._T, T1, T2)) - if not (0 <= i < self._T.length): - raise IndexError("fixed-size array index out of bounds") - setattr(self._obj, 'item%d' % i, val) + self._obj.setitem(i, val) return raise TypeError("%r instance is not an array" % (self._T,)) def __len__(self): - if isinstance(self._T, Array): + if isinstance(self._T, (Array, FixedSizeArray)): if self._T._hints.get('nolength', False): raise TypeError("%r instance has no length attribute" % (self._T,)) - - return len(self._obj.items) - if isinstance(self._T, FixedSizeArray): - return self._T.length + return self._obj.getlength() raise TypeError("%r instance is not an array" % (self._T,)) def __repr__(self): @@ -862,7 +860,10 @@ if PARENTTYPE != PTRTYPE.TO: raise TypeError("widening %r inside %r instead of %r" % (CURTYPE, PARENTTYPE, PTRTYPE.TO)) return _ptr(PTRTYPE, struc, solid=self._solid) - + + def _cast_array_to(self, PTRTYPE): + return _ptr(PTRTYPE, _nolengtharray(PTRTYPE.TO, self._obj)) + def _cast_to_int(self): obj = self._obj while obj._parentstructure(): @@ -977,6 +978,18 @@ __setstate__ = setstate_with_slots + def getlength(self): # for FixedSizeArray kind of structs + assert isinstance(self._TYPE, FixedSizeArray) + return self._TYPE.length + + def getitem(self, index): # for FixedSizeArray kind of structs + assert isinstance(self._TYPE, FixedSizeArray) + return getattr(self, 'item%d' % index) + + def setitem(self, index, value): # for FixedSizeArray kind of structs + assert isinstance(self._TYPE, FixedSizeArray) + setattr(self, 'item%d' % index, value) + class _array(_parentable): _kind = "array" @@ -1010,6 +1023,41 @@ return 'array [ %s ]' % (', '.join([self._str_item(item) for item in self.items]),) + def getlength(self): + return len(self.items) + + def getitem(self, index): + return self.items[index] + + def setitem(self, index, value): + self.items[index] = value + +class _nolengtharray(object): + _kind = "array" + + __slots__ = ('_TYPE', 'wr_parentarray') + + def __init__(self, TYPE, parentarray): + self._TYPE = TYPE + self.wr_parentarray = pickleable_weakref(parentarray) + + def getparentarray(self): + parentarray = self.wr_parentarray() + if parentarray is None: + raise RuntimeError("accessing already garbage collected %r" + % (self._T,)) + return parentarray + + def getlength(self): + # for debugging only + return self.getparentarray().getlength() + + def getitem(self, index): + return self.getparentarray().getitem(index) + + def setitem(self, index, value): + return self.getparentarray().setitem(index, value) + assert not '__dict__' in dir(_array) assert not '__dict__' in dir(_struct) Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Sun Apr 9 12:25:06 2006 @@ -519,6 +519,50 @@ a = malloc(A, 10) py.test.raises(TypeError, len, a) +def test_cast_length_to_no_length(): + A = Array(Char, hints={'nolength': True}) + S = GcStruct("str", ("hash", Signed), + ("chars", Array(Char))) + s = malloc(S, 5) + s.chars[0] = 'h' + s.chars[1] = 'e' + s.chars[2] = 'l' + s.chars[3] = 'l' + s.chars[4] = 'o' + a = cast_array_pointer(Ptr(A), s.chars) + assert a[0] == 'h' + assert a[1] == 'e' + assert a[2] == 'l' + assert a[3] == 'l' + assert a[4] == 'o' + py.test.raises(TypeError, len, a) + a[1] = 'E' + assert s.chars[1] == 'E' + s.chars[2] = 'L' + assert a[2] == 'L' + +def test_cast_fixed_to_no_length(): + A = Array(Char, hints={'nolength': True}) + S = GcStruct("str", ("hash", Signed), + ("chars", FixedSizeArray(Char, 5))) + s = malloc(S) + s.chars[0] = 'h' + s.chars[1] = 'e' + s.chars[2] = 'l' + s.chars[3] = 'l' + s.chars[4] = 'o' + a = cast_array_pointer(Ptr(A), s.chars) + assert a[0] == 'h' + assert a[1] == 'e' + assert a[2] == 'l' + assert a[3] == 'l' + assert a[4] == 'o' + py.test.raises(TypeError, len, a) + a[1] = 'E' + assert s.chars[1] == 'E' + s.chars[2] = 'L' + assert a[2] == 'L' + def test_dissect_ll_instance(): assert list(dissect_ll_instance(1)) == [(Signed, 1)] GcS = GcStruct("S", ('x', Signed)) From cfbolz at codespeak.net Sun Apr 9 13:03:10 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 9 Apr 2006 13:03:10 +0200 (CEST) Subject: [pypy-svn] r25614 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060409110310.8850A101FB@code0.codespeak.net> Author: cfbolz Date: Sun Apr 9 13:03:08 2006 New Revision: 25614 Modified: pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/mallocprediction.py pypy/dist/pypy/translator/backendopt/test/test_inline.py pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py Log: (pedronis, cfbolz): a clever inliner that tries to inline stuff more if it might lead to the removal of mallocs Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Sun Apr 9 13:03:08 2006 @@ -8,6 +8,7 @@ from pypy.translator.backendopt.merge_if_blocks import merge_if_blocks from pypy.translator import simplify from pypy.translator.backendopt.escape import malloc_to_stack +from pypy.translator.backendopt.mallocprediction import clever_inlining_and_malloc_removal from pypy.translator.backendopt.support import log PRINT_STATISTICS = False @@ -18,7 +19,8 @@ ssa_form=True, merge_if_blocks_to_switch=True, propagate=False, - heap2stack=False): + heap2stack=False, + clever_malloc_removal=False): if PRINT_STATISTICS: print "before optimizations:" @@ -42,33 +44,40 @@ if propagate: propagate_all(translator) - # inline functions in each other - if inline_threshold: - auto_inlining(translator, inline_threshold) - for graph in translator.graphs: - removenoops.remove_superfluous_keep_alive(graph) - removenoops.remove_duplicate_casts(graph, translator) - - if PRINT_STATISTICS: - print "after inlining:" - print_statistics(translator.graphs[0], translator) - - # vaporize mallocs - if mallocs: - tot = 0 - for graph in translator.graphs: - count = remove_simple_mallocs(graph) - if count: - # remove typical leftovers from malloc removal - removenoops.remove_same_as(graph) - simplify.eliminate_empty_blocks(graph) - simplify.transform_dead_op_vars(graph, translator) - tot += count - log.malloc("removed %d simple mallocs in total" % tot) - - if PRINT_STATISTICS: - print "after malloc removal:" - print_statistics(translator.graphs[0], translator) + if not clever_malloc_removal: + # inline functions in each other + if inline_threshold: + auto_inlining(translator, inline_threshold) + for graph in translator.graphs: + removenoops.remove_superfluous_keep_alive(graph) + removenoops.remove_duplicate_casts(graph, translator) + + if PRINT_STATISTICS: + print "after inlining:" + print_statistics(translator.graphs[0], translator) + + # vaporize mallocs + if mallocs: + tot = 0 + for graph in translator.graphs: + count = remove_simple_mallocs(graph) + if count: + # remove typical leftovers from malloc removal + removenoops.remove_same_as(graph) + simplify.eliminate_empty_blocks(graph) + simplify.transform_dead_op_vars(graph, translator) + tot += count + log.malloc("removed %d simple mallocs in total" % tot) + + if PRINT_STATISTICS: + print "after malloc removal:" + print_statistics(translator.graphs[0], translator) + else: + clever_inlining_and_malloc_removal(translator) + + if PRINT_STATISTICS: + print "after clever inlining and malloc removal" + print_statistics(translator.graphs[0], translator) if propagate: propagate_all(translator) Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Sun Apr 9 13:03:08 2006 @@ -131,7 +131,7 @@ return resultstate, args def flow_block(self, block, graph): - print "flowing in block %s of function %s" % (block, graph.name) + #print "flowing in block %s of function %s" % (block, graph.name) self.flown_blocks[block] = True if block is graph.returnblock: if isonheap(block.inputargs[0]): Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Apr 9 13:03:08 2006 @@ -494,7 +494,7 @@ def inlining_heuristic(graph): # XXX ponderation factors? return (0.9999 * measure_median_execution_cost(graph) + - static_instruction_count(graph)) * factor + static_instruction_count(graph)) def static_callers(translator, ignore_primitives=False): @@ -516,9 +516,10 @@ return result -def auto_inlining(translator, threshold=1, callgraph=None): +def auto_inlining(translator, multiplier=1, callgraph=None, + threshold=BASE_INLINE_THRESHOLD): from heapq import heappush, heappop, heapreplace - threshold *= BASE_INLINE_THRESHOLD + threshold = threshold * multiplier callers = {} # {graph: {graphs-that-call-it}} callees = {} # {graph: {graphs-that-it-calls}} if callgraph is None: @@ -530,6 +531,7 @@ valid_weight = {} couldnt_inline = {} lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() + count = 0 while heap: weight, _, graph = heap[0] @@ -557,6 +559,7 @@ couldnt_inline[graph] = True res = CannotInline if res is True: + count += 1 # the parentgraph should now contain all calls that were # done by 'graph' for graph2 in callees.get(graph, {}): @@ -569,3 +572,4 @@ del couldnt_inline[parentgraph] heappush(heap, (0.0, -len(callers[parentgraph]), parentgraph)) valid_weight[parentgraph] = False + return count Modified: pypy/dist/pypy/translator/backendopt/mallocprediction.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/mallocprediction.py (original) +++ pypy/dist/pypy/translator/backendopt/mallocprediction.py Sun Apr 9 13:03:08 2006 @@ -1,8 +1,13 @@ from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter from pypy.translator.backendopt.malloc import remove_simple_mallocs from pypy.translator.backendopt.inline import auto_inlining +from pypy.translator.backendopt import removenoops from pypy.rpython.lltypesystem import lltype -from pypy.translator.simplify import get_graph +from pypy.translator import simplify +from pypy.translator.backendopt.support import log + +SMALL_THRESHOLD = 15 +BIG_THRESHOLD = 50 def find_malloc_creps(graph, adi, translator): # mapping from malloc creation point to graphs that it flows into @@ -28,8 +33,8 @@ def find_calls_where_creps_go(interesting_creps, graph, adi, translator, seen): - print "find_calls_where_creps_go", interesting_creps, graph.name - print seen + #print "find_calls_where_creps_go", interesting_creps, graph.name + #print seen # drop creps that are merged with another creation point for block in graph.iterblocks(): for var in block.getvariables(): @@ -53,36 +58,36 @@ if crep in interesting_creps: del interesting_creps[crep] elif op.opname == "direct_call": - print op, interesting_creps - called_graph = get_graph(op.args[0], translator) - if called_graph is None: - del interesting_creps[crep] - print "graph not found" - continue + #print op, interesting_creps + called_graph = simplify.get_graph(op.args[0], translator) interesting = {} for i, var in enumerate(op.args[1:]): - print i, var, + #print i, var, varstate = adi.getstate(var) if varstate is None: - print "no varstate" + #print "no varstate" continue if len(varstate.creation_points) == 1: crep = varstate.creation_points.keys()[0] if crep not in interesting_creps: - print "not interesting" + #print "not interesting" + continue + if called_graph is None: + del interesting_creps[crep] + #print "graph not found" continue if (called_graph, i) in seen: seen[(called_graph, i)][graph] = True - print "seen already" + #print "seen already" else: - print "taking", crep + #print "taking", crep seen[(called_graph, i)] = {graph: True} arg = called_graph.startblock.inputargs[i] argstate = adi.getstate(arg) argcrep = [c for c in argstate.creation_points if c.creation_method == "arg"][0] interesting[argcrep] = True - print interesting + #print interesting if interesting: find_calls_where_creps_go(interesting, called_graph, adi, translator, seen) @@ -98,7 +103,7 @@ seen = {} for graph in t.graphs: creps = find_malloc_creps(graph, adi, t) - print "malloc creps", creps + #print "malloc creps", creps if creps: find_calls_where_creps_go(creps, graph, adi, t, seen) if creps: @@ -109,8 +114,46 @@ callgraph.append((caller, called_graph)) return callgraph, caller_candidates -def inline_and_remove(t, threshold=1): +def inline_and_remove(t, threshold=BIG_THRESHOLD): callgraph, caller_candidates = find_malloc_removal_candidates(t) - auto_inlining(t, threshold, callgraph) - for graph in caller_candidates: - remove_simple_mallocs(graph) + log.inlineandremove("found %s malloc removal candidates" % + len(caller_candidates)) + if callgraph: + count = auto_inlining(t, callgraph=callgraph, threshold=threshold) + if not count: + return False + count = remove_mallocs(t, caller_candidates.keys()) + return count + else: + return False + +def remove_mallocs(translator, graphs=None): + tot = 0 + if graphs is None: + graphs = translator.graphs + for graph in translator.graphs: + count = remove_simple_mallocs(graph) + if count: + # remove typical leftovers from malloc removal + removenoops.remove_same_as(graph) + simplify.eliminate_empty_blocks(graph) + simplify.transform_dead_op_vars(graph, translator) + tot += count + log.malloc("removed %d simple mallocs in total" % tot) + return tot + +def clever_inlining_and_malloc_removal(translator): + count = remove_mallocs(translator) + auto_inlining(translator, threshold=SMALL_THRESHOLD) + count += remove_mallocs(translator) + while 1: + newcount = inline_and_remove(translator) + if not newcount: + break + count += newcount + log.inlineandremove("removed %s mallocs in total" % count) + for graph in translator.graphs: + removenoops.remove_superfluous_keep_alive(graph) + removenoops.remove_duplicate_casts(graph, translator) + return count + Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Sun Apr 9 13:03:08 2006 @@ -69,16 +69,16 @@ return interp.eval_graph(graphof(t, entry), args) return eval_func -def check_auto_inlining(func, sig, threshold=None): +def check_auto_inlining(func, sig, multiplier=None): t = translate(func, sig) if option.view: t.view() # inline! sanity_check(t) # also check before inlining (so we don't blame it) - if threshold is None: + if multiplier is None: auto_inlining(t) else: - auto_inlining(t, threshold=threshold) + auto_inlining(t, multiplier=multiplier) sanity_check(t) if option.view: t.view() @@ -229,7 +229,7 @@ return 3 return 1 - eval_func, _ = check_auto_inlining(g, [int], threshold=10) + eval_func, _ = check_auto_inlining(g, [int], multiplier=10) result = eval_func([0]) assert result == 2 result = eval_func([1]) @@ -329,7 +329,7 @@ return g(n) except OverflowError: return -1 - eval_func, t = check_auto_inlining(f, [int], threshold=10) + eval_func, t = check_auto_inlining(f, [int], multiplier=10) f_graph = graphof(t, f) assert len(collect_called_graphs(f_graph, t)) == 0 Modified: pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py Sun Apr 9 13:03:08 2006 @@ -9,7 +9,7 @@ from pypy.translator.backendopt.mallocprediction import * -def compile(fn, signature): +def rtype(fn, signature): t = TranslationContext() t.buildannotator().build_types(fn, signature) t.buildrtyper().specialize() @@ -48,7 +48,7 @@ b = B() x = h(i) return g(a, b, x) - t, graph = compile(fn, [int]) + t, graph = rtype(fn, [int]) callgraph, caller_candidates = check_inlining(t, graph, [0], 42) assert caller_candidates == {graph: True} assert len(callgraph) == 1 @@ -73,7 +73,7 @@ b = B() x = h(i) return g1(a, b, x) - t, graph = compile(fn, [int]) + t, graph = rtype(fn, [int]) callgraph, caller_candidates = check_inlining(t, graph, [0], 3 * 42) print callgraph assert caller_candidates == {graph: True} @@ -83,4 +83,17 @@ assert callgraph[graph] == {g1graph: True} callgraph, caller_candidates = check_inlining(t, graph, [0], 3 * 42) assert callgraph[graph] == {g2graph: True} - + +def test_pystone(): + from pypy.translator.goal.targetrpystonex import make_target_definition + entrypoint, _, _ = make_target_definition(10) + # does not crash + t, graph = rtype(entrypoint, [int]) + total = clever_inlining_and_malloc_removal(t) + assert total == 12 + +def test_richards(): + from pypy.translator.goal.richards import entry_point + t, graph = rtype(entry_point, [int]) + total = clever_inlining_and_malloc_removal(t) + assert total == 11 From cfbolz at codespeak.net Sun Apr 9 13:18:07 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 9 Apr 2006 13:18:07 +0200 (CEST) Subject: [pypy-svn] r25615 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060409111807.1C3BA101FC@code0.codespeak.net> Author: cfbolz Date: Sun Apr 9 13:18:06 2006 New Revision: 25615 Modified: pypy/dist/pypy/translator/backendopt/mallocprediction.py pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py Log: make malloc prediction work for indirect_calls Modified: pypy/dist/pypy/translator/backendopt/mallocprediction.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/mallocprediction.py (original) +++ pypy/dist/pypy/translator/backendopt/mallocprediction.py Sun Apr 9 13:18:06 2006 @@ -54,6 +54,8 @@ if op.opname == "indirect_call": for var in op.args[:-1]: varstate = adi.getstate(var) + if varstate is None: + continue for crep in varstate.creation_points: if crep in interesting_creps: del interesting_creps[crep] Modified: pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py Sun Apr 9 13:18:06 2006 @@ -84,6 +84,38 @@ callgraph, caller_candidates = check_inlining(t, graph, [0], 3 * 42) assert callgraph[graph] == {g2graph: True} +def test_indirect_call(): + class A(object): + pass + def f1(a, i): + return a.x + def f2(a, i): + return a.x + 1 + def g1(a, i): + return a + def g2(a, i): + return None + def f(i): + a1 = A() + a2 = A() + a1.x = 1 + a2.x = 2 + if i: + f = f1 + g = g1 + else: + f = f2 + g = g2 + x = f(a1, 0) + a0 = g(a2, 1) + if a0 is not None: + return 43 + else: + return 42 + t, graph = rtype(f, [int]) + callgraph, caller_candidate = check_inlining(t, graph, [0], 42) + assert caller_candidate == {} + def test_pystone(): from pypy.translator.goal.targetrpystonex import make_target_definition entrypoint, _, _ = make_target_definition(10) From ale at codespeak.net Sun Apr 9 13:26:16 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 9 Apr 2006 13:26:16 +0200 (CEST) Subject: [pypy-svn] r25616 - in pypy/dist/pypy/translator/microbench/pybench: . package Message-ID: <20060409112616.E9D2F101FF@code0.codespeak.net> Author: ale Date: Sun Apr 9 13:26:13 2006 New Revision: 25616 Added: pypy/dist/pypy/translator/microbench/pybench/ pypy/dist/pypy/translator/microbench/pybench/Arithmetic.py pypy/dist/pypy/translator/microbench/pybench/Calls.py pypy/dist/pypy/translator/microbench/pybench/CommandLine.py pypy/dist/pypy/translator/microbench/pybench/Constructs.py pypy/dist/pypy/translator/microbench/pybench/Dict.py pypy/dist/pypy/translator/microbench/pybench/Exceptions.py pypy/dist/pypy/translator/microbench/pybench/Imports.py pypy/dist/pypy/translator/microbench/pybench/Instances.py pypy/dist/pypy/translator/microbench/pybench/Lists.py pypy/dist/pypy/translator/microbench/pybench/Lookups.py pypy/dist/pypy/translator/microbench/pybench/Numbers.py pypy/dist/pypy/translator/microbench/pybench/README pypy/dist/pypy/translator/microbench/pybench/Setup.py pypy/dist/pypy/translator/microbench/pybench/Strings.py pypy/dist/pypy/translator/microbench/pybench/Tuples.py pypy/dist/pypy/translator/microbench/pybench/Unicode.py pypy/dist/pypy/translator/microbench/pybench/package/ pypy/dist/pypy/translator/microbench/pybench/package/__init__.py pypy/dist/pypy/translator/microbench/pybench/package/submodule.py pypy/dist/pypy/translator/microbench/pybench/platform.py (contents, props changed) pypy/dist/pypy/translator/microbench/pybench/pybench.py (contents, props changed) pypy/dist/pypy/translator/microbench/pybench/python2.4.2.bench Log: Added pybench benchmarking. It doesn't contain a license file, but I have retained the original README file which mentions the original author : Marc-Andre Lemburg Added: pypy/dist/pypy/translator/microbench/pybench/Arithmetic.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Arithmetic.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,778 @@ +from pybench import Test + +class SimpleIntegerArithmetic(Test): + + version = 0.3 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SimpleFloatArithmetic(Test): + + version = 0.3 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 100000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2.1 + b = 3.3332 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SimpleIntFloatArithmetic(Test): + + version = 0.3 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + b = 3 + c = 3.14159 + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class SimpleLongArithmetic(Test): + + version = 0.3 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 30000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2220001L + b = 100001L + c = 30005L + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SimpleComplexArithmetic(Test): + + version = 0.3 + operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) + rounds = 40000 + + def test(self): + + for i in xrange(self.rounds): + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + a = 2 + 3j + b = 2.5 + 4.5j + c = 1.2 + 6.2j + + c = a + b + c = b + c + c = c + a + c = a + b + c = b + c + + c = c - a + c = a - b + c = b - c + c = c - a + c = b - c + + c = a / b + c = b / a + c = c / b + + c = a * b + c = b * a + c = c * b + + c = a / b + c = b / a + c = c / b + + def calibrate(self): + + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/Calls.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Calls.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,410 @@ +from pybench import Test + +class PythonFunctionCalls(Test): + + version = 0.3 + operations = 5*(1+4+4+2) + rounds = 60000 + + def test(self): + + global f,f1,g,h + + # define functions + def f(): + pass + + def f1(x): + pass + + def g(a,b,c): + return a,b,c + + def h(a,b,c,d=1,e=2,f=3): + return d,e,f + + # do calls + for i in xrange(self.rounds): + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + f() + f1(i) + f1(i) + f1(i) + f1(i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + g(i,i,i) + h(i,i,3,i,i) + h(i,i,i,2,i,3) + + def calibrate(self): + + global f,f1,g,h + + # define functions + def f(): + pass + + def f1(x): + pass + + def g(a,b,c): + return a,b,c + + def h(a,b,c,d=1,e=2,f=3): + return d,e,f + + # do calls + for i in xrange(self.rounds): + pass + +### + +class BuiltinFunctionCalls(Test): + + version = 0.4 + operations = 5*(2+5+5+5) + rounds = 30000 + + def test(self): + + # localize functions + f0 = globals + f1 = hash + f2 = cmp + f3 = range + + # do calls + for i in xrange(self.rounds): + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + f0() + f0() + f1(i) + f1(i) + f1(i) + f1(i) + f1(i) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f2(1,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + f3(1,3,2) + + def calibrate(self): + + # localize functions + f0 = dir + f1 = hash + f2 = range + f3 = range + + # do calls + for i in xrange(self.rounds): + pass + +### + +class PythonMethodCalls(Test): + + version = 0.3 + operations = 5*(6 + 5 + 4) + rounds = 20000 + + def test(self): + + class c: + + x = 2 + s = 'string' + + def f(self): + + return self.x + + def j(self,a,b): + + self.y = a + self.t = b + return self.y + + def k(self,a,b,c=3): + + self.y = a + self.s = b + self.t = c + + o = c() + + for i in xrange(self.rounds): + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + o.f() + o.f() + o.f() + o.f() + o.f() + o.f() + o.j(i,i) + o.j(i,i) + o.j(i,2) + o.j(i,2) + o.j(2,2) + o.k(i,i) + o.k(i,2) + o.k(i,2,3) + o.k(i,i,c=4) + + def calibrate(self): + + class c: + + x = 2 + s = 'string' + + def f(self): + + return self.x + + def j(self,a,b): + + self.y = a + self.t = b + + def k(self,a,b,c=3): + + self.y = a + self.s = b + self.t = c + + o = c + + for i in xrange(self.rounds): + pass + +### + +class Recursion(Test): + + version = 0.3 + operations = 5 + rounds = 50000 + + def test(self): + + global f + + def f(x): + + if x > 1: + return f(x-1) + return 1 + + for i in xrange(self.rounds): + f(10) + f(10) + f(10) + f(10) + f(10) + + def calibrate(self): + + global f + + def f(x): + + if x > 0: + return f(x-1) + return 1 + + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/CommandLine.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/CommandLine.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,573 @@ +""" CommandLine - Get and parse command line options + + NOTE: This still is very much work in progress !!! + + Different version are likely to be incompatible. + + TODO: + + ? Incorporate the changes made by (see Inbox) + ? Add number range option using srange() + + Copyright (c) 1997-2001, Marc-Andre Lemburg; mailto:mal at lemburg.com + Copyright (c) 2000-2001, eGenix.com Software GmbH; mailto:info at egenix.com + See the documentation for further information on copyrights, + or contact the author. All Rights Reserved. +""" + +__version__ = '1.0' + +import sys,getopt,string,glob,os,traceback,re + +### Helpers + +def _getopt_flags(options): + + """ Convert the option list to a getopt flag string and long opt + list + + """ + s = [] + l = [] + for o in options: + if o.prefix == '-': + # short option + s.append(o.name) + if o.takes_argument: + s.append(':') + else: + # long option + if o.takes_argument: + l.append(o.name+'=') + else: + l.append(o.name) + return string.join(s,''),l + +def invisible_input(prompt='>>> '): + + """ Get raw input from a terminal without echoing the characters to + the terminal, e.g. for password queries. + + """ + import getpass + entry = getpass.getpass(prompt) + if entry is None: + raise KeyboardInterrupt + return entry + +def option_dict(options): + + """ Return a dictionary mapping option names to Option instances. + """ + d = {} + for option in options: + d[option.name] = option + return d + +# Alias +getpasswd = invisible_input + +_integerRE = re.compile('\s*(-?\d+)\s*$') +_integerRangeRE = re.compile('\s*(-?\d+)\s*-\s*(-?\d+)\s*$') + +def srange(s, + + split=string.split,integer=_integerRE, + integerRange=_integerRangeRE): + + """ Converts a textual representation of integer numbers and ranges + to a Python list. + + Supported formats: 2,3,4,2-10,-1 - -3, 5 - -2 + + Values are appended to the created list in the order specified + in the string. + + """ + l = [] + append = l.append + for entry in split(s,','): + m = integer.match(entry) + if m: + append(int(m.groups()[0])) + continue + m = integerRange.match(entry) + if m: + start,end = map(int,m.groups()) + l[len(l):] = range(start,end+1) + return l + +### Option classes + +class Option: + + """ Option base class. Takes no argument. + + """ + default = None + helptext = '' + prefix = '-' + takes_argument = 0 + has_default = 0 + tab = 15 + + def __init__(self,name,help=None): + + if not name[:1] == '-': + raise TypeError,'option names must start with "-"' + if name[1:2] == '-': + self.prefix = '--' + self.name = name[2:] + else: + self.name = name[1:] + if help: + self.help = help + + def __str__(self): + + o = self + name = o.prefix + o.name + if o.takes_argument: + name = name + ' arg' + if len(name) > self.tab: + name = name + '\n' + ' ' * (self.tab + 1 + len(o.prefix)) + else: + name = '%-*s ' % (self.tab, name) + description = o.help + if o.has_default: + description = description + ' (%s)' % o.default + return '%s %s' % (name, description) + +class ArgumentOption(Option): + + """ Option that takes an argument. + + An optional default argument can be given. + + """ + def __init__(self,name,help=None,default=None): + + # Basemethod + Option.__init__(self,name,help) + + if default is not None: + self.default = default + self.has_default = 1 + self.takes_argument = 1 + +class SwitchOption(Option): + + """ Options that can be on or off. Has an optional default value. + + """ + def __init__(self,name,help=None,default=None): + + # Basemethod + Option.__init__(self,name,help) + + if default is not None: + self.default = default + self.has_default = 1 + +### Application baseclass + +class Application: + + """ Command line application interface with builtin argument + parsing. + + """ + # Options the program accepts (Option instances) + options = [] + + # Standard settings; these are appended to options in __init__ + preset_options = [SwitchOption('-v','generate verbose output'), + SwitchOption('-h','show this help text'), + SwitchOption('--help','show this help text'), + SwitchOption('--debug','enable debugging'), + SwitchOption('--copyright','show copyright'), + SwitchOption('--examples','show examples of usage')] + + # The help layout looks like this: + # [header] - defaults to '' + # + # [synopsis] - formatted as ' %s' % self.synopsis + # + # options: + # [options] - formatted from self.options + # + # [version] - formatted as 'Version:\n %s' % self.version, if given + # + # [about] - defaults to '' + # + # Note: all fields that do not behave as template are formatted + # using the instances dictionary as substitution namespace, + # e.g. %(name)s will be replaced by the applications name. + # + + # Header (default to program name) + header = '' + + # Name (defaults to program name) + name = '' + + # Synopsis (%(name)s is replaced by the program name) + synopsis = '%(name)s [option] files...' + + # Version (optional) + version = '' + + # General information printed after the possible options (optional) + about = '' + + # Examples of usage to show when the --examples option is given (optional) + examples = '' + + # Copyright to show + copyright = ( + 'Copyright (c) 1997-2001, Marc-Andre Lemburg; mailto:mal at lemburg.com\n' + 'Copyright (c) 2000-2001, eGenix.com Software GmbH; mailto:info at egenix.com\n' + 'See the documentation for further information on copyrights,\n' + 'or contact the author. All Rights Reserved.\n' + '*** UNAUTHORIZED COPYING, USAGE or DISTRIBUTION PROHIBITED. ***' + ) + + # Apply file globbing ? + globbing = 1 + + # Generate debug output ? + debug = 0 + + # Generate verbose output ? + verbose = 0 + + # Instance variables: + values = None # Dictionary of passed options (or default values) + # indexed by the options name, e.g. '-h' + files = None # List of passed filenames + + def __init__(self,argv=None): + + # Setup application specs + if argv is None: + argv = sys.argv + self.filename = os.path.split(argv[0])[1] + if not self.name: + self.name = os.path.split(self.filename)[1] + else: + self.name = self.name + if not self.header: + self.header = self.name + else: + self.header = self.header + + # Init .arguments list + self.arguments = argv[1:] + + # Setup Option mapping + self.option_map = option_dict(self.options) + + # Append preset options + for option in self.preset_options: + if not self.option_map.has_key(option.name): + self.add_option(option) + + # Init .files list + self.files = [] + + # Start Application + try: + # Process startup + rc = self.startup() + if rc is not None: + raise SystemExit,rc + + # Parse command line + rc = self.parse() + if rc is not None: + raise SystemExit,rc + + # Start application + rc = self.main() + if rc is None: + rc = 0 + + except SystemExit,rc: + pass + + except KeyboardInterrupt: + print + print '* User Break' + rc = 1 + + except: + print + print '* Internal Error' + if self.debug: + print + traceback.print_exc(20) + rc = 1 + + raise SystemExit,rc + + def add_option(self, option): + + """ Add a new Option instance to the Application dynamically. + + Note that this has to be done *before* .parse() is being + executed. + + """ + self.options.append(option) + self.option_map[option.name] = option + + def startup(self): + + """ Set user defined instance variables. + + If this method returns anything other than None, the + process is terminated with the return value as exit code. + + """ + return None + + def exit(self, rc=0): + + """ Exit the program. + + rc is used as exit code and passed back to the calling + program. It defaults to 0 which usually means: OK. + + """ + raise SystemExit, rc + + def parse(self): + + """ Parse the command line and fill in self.values and self.files. + + After having parsed the options, the remaining command line + arguments are interpreted as files and passed to .handle_files() + for processing. + + As final step the option handlers are called in the order + of the options given on the command line. + + """ + # Parse arguments + self.values = values = {} + for o in self.options: + if o.has_default: + values[o.prefix+o.name] = o.default + else: + values[o.prefix+o.name] = 0 + flags,lflags = _getopt_flags(self.options) + try: + optlist,files = getopt.getopt(self.arguments,flags,lflags) + if self.globbing: + l = [] + for f in files: + gf = glob.glob(f) + if not gf: + l.append(f) + else: + l[len(l):] = gf + files = l + self.optionlist = optlist + self.files = files + self.files + except getopt.error,why: + self.help(why) + sys.exit(1) + + # Call file handler + rc = self.handle_files(self.files) + if rc is not None: + sys.exit(rc) + + # Call option handlers + for optionname, value in optlist: + + # Try to convert value to integer + try: + value = string.atoi(value) + except ValueError: + pass + + # Find handler and call it (or count the number of option + # instances on the command line) + handlername = 'handle' + string.replace(optionname, '-', '_') + try: + handler = getattr(self, handlername) + except AttributeError: + if value == '': + # count the number of occurances + if values.has_key(optionname): + values[optionname] = values[optionname] + 1 + else: + values[optionname] = 1 + else: + values[optionname] = value + else: + rc = handler(value) + if rc is not None: + raise SystemExit, rc + + # Apply final file check (for backward compatibility) + rc = self.check_files(self.files) + if rc is not None: + sys.exit(rc) + + def check_files(self,filelist): + + """ Apply some user defined checks on the files given in filelist. + + This may modify filelist in place. A typical application + is checking that at least n files are given. + + If this method returns anything other than None, the + process is terminated with the return value as exit code. + + """ + return None + + def help(self,note=''): + + self.print_header() + if self.synopsis: + print 'Synopsis:' + # To remain backward compatible: + try: + synopsis = self.synopsis % self.name + except (NameError, KeyError, TypeError): + synopsis = self.synopsis % self.__dict__ + print ' ' + synopsis + print + self.print_options() + if self.version: + print 'Version:' + print ' %s' % self.version + print + if self.about: + print string.strip(self.about % self.__dict__) + print + if note: + print '-'*72 + print 'Note:',note + print + + def notice(self,note): + + print '-'*72 + print 'Note:',note + print '-'*72 + print + + def print_header(self): + + print '-'*72 + print self.header % self.__dict__ + print '-'*72 + print + + def print_options(self): + + options = self.options + print 'Options and default settings:' + if not options: + print ' None' + return + long = filter(lambda x: x.prefix == '--', options) + short = filter(lambda x: x.prefix == '-', options) + items = short + long + for o in options: + print ' ',o + print + + # + # Example handlers: + # + # If a handler returns anything other than None, processing stops + # and the return value is passed to sys.exit() as argument. + # + + # File handler + def handle_files(self,files): + + """ This may process the files list in place. + """ + return None + + # Short option handler + def handle_h(self,arg): + + self.help() + return 0 + + def handle_v(self, value): + + """ Turn on verbose output. + """ + self.verbose = 1 + + # Handlers for long options have two underscores in their name + def handle__help(self,arg): + + self.help() + return 0 + + def handle__debug(self,arg): + + self.debug = 1 + + def handle__copyright(self,arg): + + self.print_header() + print string.strip(self.copyright % self.__dict__) + print + return 0 + + def handle__examples(self,arg): + + self.print_header() + if self.examples: + print 'Examples:' + print + print string.strip(self.examples % self.__dict__) + print + else: + print 'No examples available.' + print + return 0 + + def main(self): + + """ Override this method as program entry point. + + The return value is passed to sys.exit() as argument. If + it is None, 0 is assumed (meaning OK). Unhandled + exceptions are reported with exit status code 1 (see + __init__ for further details). + + """ + return None + +# Alias +CommandLine = Application + +def _test(): + + class MyApplication(Application): + header = 'Test Application' + version = __version__ + options = [Option('-v','verbose')] + + def handle_v(self,arg): + print 'VERBOSE, Yeah !' + + cmd = MyApplication() + if not cmd.values['-h']: + cmd.help() + print 'files:',cmd.files + print 'Bye...' + +if __name__ == '__main__': + _test() Added: pypy/dist/pypy/translator/microbench/pybench/Constructs.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Constructs.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,565 @@ +from pybench import Test + +class IfThenElse(Test): + + version = 0.31 + operations = 30*3 # hard to say... + rounds = 150000 + + def test(self): + + a,b,c = 1,2,3 + for i in xrange(self.rounds): + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + if a == 1: + if b == 2: + if c != 3: + c = 3 + b = 3 + else: + c = 2 + elif b == 3: + b = 2 + a = 2 + elif a == 2: + a = 3 + else: + a = 1 + + def calibrate(self): + + a,b,c = 1,2,3 + for i in xrange(self.rounds): + pass + +class NestedForLoops(Test): + + version = 0.3 + operations = 1000*10*5 + rounds = 150 + + def test(self): + + l1 = range(1000) + l2 = range(10) + l3 = range(5) + for i in xrange(self.rounds): + for i in l1: + for j in l2: + for k in l3: + pass + + def calibrate(self): + + l1 = range(1000) + l2 = range(10) + l3 = range(5) + for i in xrange(self.rounds): + pass + +class ForLoops(Test): + + version = 0.1 + operations = 5 * 5 + rounds = 8000 + + def test(self): + + l1 = range(100) + for i in xrange(self.rounds): + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + for i in l1: + pass + + def calibrate(self): + + l1 = range(1000) + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/Dict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Dict.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,503 @@ +from pybench import Test + +class DictCreation(Test): + + version = 0.3 + operations = 5*(5 + 5) + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + d1 = {} + d2 = {} + d3 = {} + d4 = {} + d5 = {} + + d1 = {1:2,3:4,5:6} + d2 = {2:3,4:5,6:7} + d3 = {3:4,5:6,7:8} + d4 = {4:5,6:7,8:9} + d5 = {6:7,8:9,10:11} + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class DictWithStringKeys(Test): + + version = 0.1 + operations = 5*(6 + 6) + rounds = 200000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + d['abc'] = 1 + d['def'] = 2 + d['ghi'] = 3 + d['jkl'] = 4 + d['mno'] = 5 + d['pqr'] = 6 + + d['abc'] + d['def'] + d['ghi'] + d['jkl'] + d['mno'] + d['pqr'] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + +class DictWithFloatKeys(Test): + + version = 0.1 + operations = 5*(6 + 6) + rounds = 200000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + d[1.234] = 1 + d[2.345] = 2 + d[3.456] = 3 + d[4.567] = 4 + d[5.678] = 5 + d[6.789] = 6 + + d[1.234] + d[2.345] + d[3.456] + d[4.567] + d[5.678] + d[6.789] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + +class DictWithIntegerKeys(Test): + + version = 0.1 + operations = 5*(6 + 6) + rounds = 200000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + d[1] = 1 + d[2] = 2 + d[3] = 3 + d[4] = 4 + d[5] = 5 + d[6] = 6 + + d[1] + d[2] + d[3] + d[4] + d[5] + d[6] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + +class SimpleDictManipulation(Test): + + version = 0.3 + operations = 5*(6 + 6 + 6 + 6) + rounds = 50000 + + def test(self): + + d = {} + + for i in xrange(self.rounds): + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + d.has_key(0) + d.has_key(2) + d.has_key(4) + d.has_key(6) + d.has_key(8) + d.has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + d.has_key(0) + d.has_key(2) + d.has_key(4) + d.has_key(6) + d.has_key(8) + d.has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + d.has_key(0) + d.has_key(2) + d.has_key(4) + d.has_key(6) + d.has_key(8) + d.has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + d.has_key(0) + d.has_key(2) + d.has_key(4) + d.has_key(6) + d.has_key(8) + d.has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + d[0] = 3 + d[1] = 4 + d[2] = 5 + d[3] = 3 + d[4] = 4 + d[5] = 5 + + x = d[0] + x = d[1] + x = d[2] + x = d[3] + x = d[4] + x = d[5] + + d.has_key(0) + d.has_key(2) + d.has_key(4) + d.has_key(6) + d.has_key(8) + d.has_key(10) + + del d[0] + del d[1] + del d[2] + del d[3] + del d[4] + del d[5] + + def calibrate(self): + + d = {} + + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/Exceptions.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Exceptions.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,681 @@ +from pybench import Test + +class TryRaiseExcept(Test): + + version = 0.1 + operations = 2 + 3 + rounds = 60000 + + def test(self): + + error = ValueError + + for i in xrange(self.rounds): + try: + raise error + except: + pass + try: + raise error + except: + pass + try: + raise error,"something" + except: + pass + try: + raise error,"something" + except: + pass + try: + raise error,"something" + except: + pass + + def calibrate(self): + + error = ValueError + + for i in xrange(self.rounds): + pass + + +class TryExcept(Test): + + version = 0.1 + operations = 15 * 10 + rounds = 200000 + + def test(self): + + for i in xrange(self.rounds): + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + try: + pass + except: + pass + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + Added: pypy/dist/pypy/translator/microbench/pybench/Imports.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Imports.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,139 @@ +from pybench import Test + +# First imports: +import os +import package.submodule + +class SecondImport(Test): + + version = 0.1 + operations = 5 * 5 + rounds = 20000 + + def test(self): + + for i in xrange(self.rounds): + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + import os + import os + import os + import os + import os + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class SecondPackageImport(Test): + + version = 0.1 + operations = 5 * 5 + rounds = 20000 + + def test(self): + + for i in xrange(self.rounds): + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + import package + import package + import package + import package + import package + + def calibrate(self): + + for i in xrange(self.rounds): + pass + +class SecondSubmoduleImport(Test): + + version = 0.1 + operations = 5 * 5 + rounds = 20000 + + def test(self): + + for i in xrange(self.rounds): + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + import package.submodule + import package.submodule + import package.submodule + import package.submodule + import package.submodule + + def calibrate(self): + + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/Instances.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Instances.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,68 @@ +from pybench import Test + +class CreateInstances(Test): + + version = 0.2 + operations = 3 + 7 + 4 + rounds = 60000 + + def test(self): + + class c: + pass + + class d: + def __init__(self,a,b,c): + self.a = a + self.b = b + self.c = c + + class e: + def __init__(self,a,b,c=4): + self.a = a + self.b = b + self.c = c + self.d = a + self.e = b + self.f = c + + for i in xrange(self.rounds): + o = c() + o1 = c() + o2 = c() + p = d(i,i,3) + p1 = d(i,i,3) + p2 = d(i,3,3) + p3 = d(3,i,3) + p4 = d(i,i,i) + p5 = d(3,i,3) + p6 = d(i,i,i) + q = e(i,i,3) + q1 = e(i,i,3) + q2 = e(i,i,3) + q3 = e(i,i) + + def calibrate(self): + + class c: + pass + + class d: + def __init__(self,a,b,c): + self.a = a + self.b = b + self.c = c + + class e: + def __init__(self,a,b,c=4): + self.a = a + self.b = b + self.c = c + self.d = a + self.e = b + self.f = c + + for i in xrange(self.rounds): + pass + + Added: pypy/dist/pypy/translator/microbench/pybench/Lists.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Lists.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,292 @@ +from pybench import Test + +class SimpleListManipulation(Test): + + version = 0.2 + operations = 5* (6 + 6 + 6) + rounds = 60000 + + def test(self): + + l = [] + + for i in xrange(self.rounds): + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + x = l[0] + x = l[1] + x = l[2] + x = l[3] + x = l[4] + x = l[5] + + if len(l) > 10000: + # cut down the size + l = [] + + def calibrate(self): + + l = [] + + for i in xrange(self.rounds): + pass + +class ListSlicing(Test): + + version = 0.4 + operations = 25*(3+1+2+1) + rounds = 400 + + def test(self): + + n = range(100) + r = range(25) + + for i in xrange(self.rounds): + + l = range(100) + + for j in r: + + m = l[50:] + m = l[:25] + m = l[50:55] + l[:3] = n + m = l[:-1] + m = l[1:] + l[-1:] = n + + def calibrate(self): + + n = range(100) + r = range(25) + + for i in xrange(self.rounds): + + l = range(100) + + for j in r: + pass + +class SmallLists(Test): + + version = 0.3 + operations = 5*(1+ 6 + 6 + 3 + 1) + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + l = [] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + l = [] + + l.append(2) + l.append(3) + l.append(4) + l.append(2) + l.append(3) + l.append(4) + + l[0] = 3 + l[1] = 4 + l[2] = 5 + l[3] = 3 + l[4] = 4 + l[5] = 5 + + l[:3] = [1,2,3] + m = l[:-1] + m = l[1:] + + l[-1:] = [4,5,6] + + def calibrate(self): + + for i in xrange(self.rounds): + l = [] + Added: pypy/dist/pypy/translator/microbench/pybench/Lookups.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Lookups.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,946 @@ +from pybench import Test + +class SpecialClassAttribute(Test): + + version = 0.3 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + + for i in xrange(self.rounds): + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + c.__a = 2 + c.__b = 3 + c.__c = 4 + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + x = c.__a + x = c.__b + x = c.__c + + def calibrate(self): + + class c: + pass + + for i in xrange(self.rounds): + pass + +class NormalClassAttribute(Test): + + version = 0.3 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + + for i in xrange(self.rounds): + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + c.a = 2 + c.b = 3 + c.c = 4 + + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + x = c.a + x = c.b + x = c.c + + def calibrate(self): + + class c: + pass + + for i in xrange(self.rounds): + pass + +class SpecialInstanceAttribute(Test): + + version = 0.3 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + o.__a__ = 2 + o.__b__ = 3 + o.__c__ = 4 + + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + x = o.__a__ + x = o.__b__ + x = o.__c__ + + def calibrate(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + pass + +class NormalInstanceAttribute(Test): + + version = 0.3 + operations = 5*(12 + 12) + rounds = 100000 + + def test(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + o.a = 2 + o.b = 3 + o.c = 4 + + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + x = o.a + x = o.b + x = o.c + + def calibrate(self): + + class c: + pass + o = c() + + for i in xrange(self.rounds): + pass + +class BuiltinMethodLookup(Test): + + version = 0.3 + operations = 5*(3*5 + 3*5) + rounds = 70000 + + def test(self): + + l = [] + d = {} + + for i in xrange(self.rounds): + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + l.append + l.append + l.append + l.append + l.append + + l.insert + l.insert + l.insert + l.insert + l.insert + + l.sort + l.sort + l.sort + l.sort + l.sort + + d.has_key + d.has_key + d.has_key + d.has_key + d.has_key + + d.items + d.items + d.items + d.items + d.items + + d.get + d.get + d.get + d.get + d.get + + def calibrate(self): + + l = [] + d = {} + + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/Numbers.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Numbers.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,784 @@ +from pybench import Test + +class CompareIntegers(Test): + + version = 0.1 + operations = 30 * 5 + rounds = 120000 + + def test(self): + + for i in xrange(self.rounds): + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + 2 < 3 + 2 > 3 + 2 == 3 + 2 > 3 + 2 < 3 + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class CompareFloats(Test): + + version = 0.1 + operations = 30 * 5 + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + 2.1 < 3.31 + 2.1 > 3.31 + 2.1 == 3.31 + 2.1 > 3.31 + 2.1 < 3.31 + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class CompareFloatsIntegers(Test): + + version = 0.1 + operations = 30 * 5 + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + 2.1 < 4 + 2.1 > 4 + 2.1 == 4 + 2.1 > 4 + 2.1 < 4 + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class CompareLongs(Test): + + version = 0.1 + operations = 30 * 5 + rounds = 60000 + + def test(self): + + for i in xrange(self.rounds): + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + 1234567890L < 3456789012345L + 1234567890L > 3456789012345L + 1234567890L == 3456789012345L + 1234567890L > 3456789012345L + 1234567890L < 3456789012345L + + def calibrate(self): + + for i in xrange(self.rounds): + pass Added: pypy/dist/pypy/translator/microbench/pybench/README ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/README Sun Apr 9 13:26:13 2006 @@ -0,0 +1,76 @@ +PYBENCH - A Python Benchmark Suite +================================== + +Micro-Manual +------------ + +Run 'pybench.py -h' to see the help screen. +Run 'pybench.py' to just let the benchmark suite do it's thing and +'pybench.py -f ' to have it store the results in a file too. + +Change list +----------- + +0.5 -> 0.6: + added more tests and fixed some others + added overhead output + switched from default warp 10 to warp 20 +0.5: + initial release +0.4: + pre-release + +Sample comparism output +--------------------------------- + +projects/pybench> pybench.py -s mxp15-O-1 -c p151-O-1 +PYBENCH 0.6 + +Benchmark: mxp15-O-1 (rounds=10, warp=20) + +Tests: per run per oper. diff * +------------------------------------------------------------------------ + BuiltinFunctionCalls: 496.55 ms 3.89 us -25.77% + BuiltinMethodLookup: 634.70 ms 1.21 us -28.20% + ConcatStrings: 531.45 ms 3.54 us -7.49% + CreateInstances: 687.15 ms 16.36 us -21.10% + CreateStringsWithConcat: 531.15 ms 2.66 us -15.74% + DictCreation: 657.05 ms 4.38 us -17.14% + ForLoops: 684.10 ms 68.41 us -41.58% + IfThenElse: 720.75 ms 1.07 us -24.18% + ListSlicing: 547.90 ms 156.54 us -1.92% + NestedForLoops: 448.55 ms 1.28 us -35.98% + NormalClassAttribute: 790.20 ms 1.32 us -7.80% + NormalInstanceAttribute: 572.10 ms 0.95 us -35.66% + PythonFunctionCalls: 649.00 ms 3.93 us -13.90% + PythonMethodCalls: 570.00 ms 7.60 us -15.00% + Recursion: 505.55 ms 40.44 us -15.54% + SecondImport: 469.00 ms 18.76 us -3.82% + SecondPackageImport: 483.00 ms 19.32 us -5.40% + SecondSubmoduleImport: 613.05 ms 24.52 us -5.34% + SimpleComplexArithmetic: 540.10 ms 2.46 us -18.32% + SimpleDictManipulation: 505.60 ms 1.69 us -32.73% + SimpleFloatArithmetic: 552.05 ms 1.00 us -29.24% + SimpleIntFloatArithmetic: 509.15 ms 0.77 us -32.00% + SimpleIntegerArithmetic: 509.15 ms 0.77 us -32.10% + SimpleListManipulation: 461.10 ms 1.71 us -36.03% + SimpleLongArithmetic: 448.50 ms 2.72 us -22.26% + SmallLists: 730.55 ms 2.86 us -27.39% + SmallTuples: 668.10 ms 2.78 us -23.16% + SpecialClassAttribute: 696.25 ms 1.16 us -20.26% + SpecialInstanceAttribute: 682.20 ms 1.14 us -37.42% + StringSlicing: 481.60 ms 2.75 us -14.29% + TryExcept: 920.40 ms 0.61 us -6.39% + TryRaiseExcept: 561.05 ms 37.40 us -13.38% + TupleSlicing: 594.65 ms 5.66 us -6.38% +------------------------------------------------------------------------ + Average round time: 21895.00 ms -22.49% + +*) measured against: p151-O-1 (rounds=10, warp=20) + +Note: if you're interested in receiving the patch that turns +Python 1.5 into mxPython1.5 (mxp15 was done with it), drop me line +and I'll send you the URL for the patch. + +Have fun, +Marc-Andre Lemburg, mailto:mal at lemburg.com Added: pypy/dist/pypy/translator/microbench/pybench/Setup.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Setup.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,30 @@ +#!python + +# Setup file for pybench +# +# This file has to import all tests to be run; it is executed as +# Python source file, so you can do all kinds of manipulations here +# rather than having to edit the tests themselves. +# + +# Defaults +Number_of_rounds = 10 +Warp_factor = 20 + +# Import tests +from Arithmetic import * +from Calls import * +from Constructs import * +from Lookups import * +from Instances import * +from Lists import * +from Tuples import * +from Dict import * +from Exceptions import * +from Imports import * +from Strings import * +from Numbers import * +try: + from Unicode import * +except (ImportError, SyntaxError): + pass Added: pypy/dist/pypy/translator/microbench/pybench/Strings.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Strings.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,564 @@ +from pybench import Test +from string import join + +class ConcatStrings(Test): + + version = 0.1 + operations = 10 * 5 + rounds = 60000 + + def test(self): + + # Make sure the strings are *not* interned + s = join(map(str,range(100))) + t = join(map(str,range(1,101))) + + for i in xrange(self.rounds): + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + def calibrate(self): + + s = join(map(str,range(100))) + t = join(map(str,range(1,101))) + + for i in xrange(self.rounds): + pass + + +class CompareStrings(Test): + + version = 0.2 + operations = 10 * 5 + rounds = 200000 + + def test(self): + + # Make sure the strings are *not* interned + s = join(map(str,range(10))) + t = join(map(str,range(10))) + "abc" + + for i in xrange(self.rounds): + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + def calibrate(self): + + s = join(map(str,range(10))) + t = join(map(str,range(10))) + "abc" + + for i in xrange(self.rounds): + pass + + +class CompareInternedStrings(Test): + + version = 0.1 + operations = 10 * 5 + rounds = 200000 + + def test(self): + + # Make sure the strings *are* interned + s = intern(join(map(str,range(10)))) + t = s + + for i in xrange(self.rounds): + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + t == s + t == s + t >= s + t > s + t < s + + def calibrate(self): + + s = intern(join(map(str,range(10)))) + t = s + + for i in xrange(self.rounds): + pass + + +class CreateStringsWithConcat(Test): + + version = 0.1 + operations = 10 * 5 + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + s = 'om' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + s = s + 'xax' + s = s + 'xbx' + s = s + 'xcx' + s = s + 'xdx' + s = s + 'xex' + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class StringSlicing(Test): + + version = 0.1 + operations = 5 * 7 + rounds = 100000 + + def test(self): + + s = join(map(str,range(100))) + + for i in xrange(self.rounds): + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + def calibrate(self): + + s = join(map(str,range(100))) + + for i in xrange(self.rounds): + pass + +### String methods + +if hasattr('', 'lower'): + + class StringMappings(Test): + + version = 0.1 + operations = 3 * (5 + 4 + 2 + 1) + rounds = 70000 + + def test(self): + + s = join(map(chr,range(20)),'') + t = join(map(chr,range(50)),'') + u = join(map(chr,range(100)),'') + v = join(map(chr,range(256)),'') + + for i in xrange(self.rounds): + + s.lower() + s.lower() + s.lower() + s.lower() + s.lower() + + s.upper() + s.upper() + s.upper() + s.upper() + s.upper() + + s.title() + s.title() + s.title() + s.title() + s.title() + + t.lower() + t.lower() + t.lower() + t.lower() + + t.upper() + t.upper() + t.upper() + t.upper() + + t.title() + t.title() + t.title() + t.title() + + u.lower() + u.lower() + + u.upper() + u.upper() + + u.title() + u.title() + + v.lower() + + v.upper() + + v.title() + + def calibrate(self): + + s = join(map(chr,range(20)),'') + t = join(map(chr,range(50)),'') + u = join(map(chr,range(100)),'') + v = join(map(chr,range(256)),'') + + for i in xrange(self.rounds): + pass + + class StringPredicates(Test): + + version = 0.1 + operations = 10 * 7 + rounds = 80000 + + def test(self): + + data = ('abc', '123', ' ', '???', '?'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdigit() + s.islower() + s.isspace() + s.istitle() + s.isupper() + + def calibrate(self): + + data = ('abc', '123', ' ', '\u1234\u2345\u3456', '\uFFFF'*10) + data = ('abc', '123', ' ', '???', '?'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + + Added: pypy/dist/pypy/translator/microbench/pybench/Tuples.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Tuples.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,365 @@ +from pybench import Test + +class TupleSlicing(Test): + + version = 0.31 + operations = 3 * 25 * 10 * 7 + rounds = 400 + + def test(self): + + r = range(25) + + for i in xrange(self.rounds): + + t = tuple(range(100)) + + for j in r: + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + m = t[50:] + m = t[:25] + m = t[50:55] + m = t[:-1] + m = t[1:] + m = t[-10:] + m = t[:10] + + def calibrate(self): + + r = range(25) + + for i in xrange(self.rounds): + + t = tuple(range(100)) + + for j in r: + + pass + +class SmallTuples(Test): + + version = 0.3 + operations = 5*(1 + 3 + 6 + 2) + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + t = (1,2,3,4,5,6) + + a,b,c,d,e,f = t + a,b,c,d,e,f = t + a,b,c,d,e,f = t + + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + a,b,c = t[:3] + + l = list(t) + t = tuple(l) + + def calibrate(self): + + for i in xrange(self.rounds): + pass + Added: pypy/dist/pypy/translator/microbench/pybench/Unicode.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/Unicode.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,542 @@ +try: + unicode +except NameError: + raise ImportError + +from pybench import Test +from string import join + +class ConcatUnicode(Test): + + version = 0.1 + operations = 10 * 5 + rounds = 60000 + + def test(self): + + # Make sure the strings are *not* interned + s = unicode(join(map(str,range(100)))) + t = unicode(join(map(str,range(1,101)))) + + for i in xrange(self.rounds): + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + t + s + t + s + t + s + t + s + t + s + + def calibrate(self): + + s = unicode(join(map(str,range(100)))) + t = unicode(join(map(str,range(1,101)))) + + for i in xrange(self.rounds): + pass + + +class CompareUnicode(Test): + + version = 0.1 + operations = 10 * 5 + rounds = 150000 + + def test(self): + + # Make sure the strings are *not* interned + s = unicode(join(map(str,range(10)))) + t = unicode(join(map(str,range(10))) + "abc") + + for i in xrange(self.rounds): + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + t < s + t > s + t == s + t > s + t < s + + def calibrate(self): + + s = unicode(join(map(str,range(10)))) + t = unicode(join(map(str,range(10))) + "abc") + + for i in xrange(self.rounds): + pass + + +class CreateUnicodeWithConcat(Test): + + version = 0.1 + operations = 10 * 5 + rounds = 80000 + + def test(self): + + for i in xrange(self.rounds): + s = u'om' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + s = s + u'xax' + s = s + u'xbx' + s = s + u'xcx' + s = s + u'xdx' + s = s + u'xex' + + def calibrate(self): + + for i in xrange(self.rounds): + pass + + +class UnicodeSlicing(Test): + + version = 0.1 + operations = 5 * 7 + rounds = 100000 + + def test(self): + + s = unicode(join(map(str,range(100)))) + + for i in xrange(self.rounds): + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + s[50:] + s[:25] + s[50:55] + s[-1:] + s[:1] + s[2:] + s[11:-11] + + def calibrate(self): + + s = unicode(join(map(str,range(100)))) + + for i in xrange(self.rounds): + pass + +### String methods + +class UnicodeMappings(Test): + + version = 0.1 + operations = 3 * (5 + 4 + 2 + 1) + rounds = 10000 + + def test(self): + + s = join(map(unichr,range(20)),'') + t = join(map(unichr,range(100)),'') + u = join(map(unichr,range(500)),'') + v = join(map(unichr,range(1000)),'') + + for i in xrange(self.rounds): + + s.lower() + s.lower() + s.lower() + s.lower() + s.lower() + + s.upper() + s.upper() + s.upper() + s.upper() + s.upper() + + s.title() + s.title() + s.title() + s.title() + s.title() + + t.lower() + t.lower() + t.lower() + t.lower() + + t.upper() + t.upper() + t.upper() + t.upper() + + t.title() + t.title() + t.title() + t.title() + + u.lower() + u.lower() + + u.upper() + u.upper() + + u.title() + u.title() + + v.lower() + + v.upper() + + v.title() + + def calibrate(self): + + s = join(map(unichr,range(20)),'') + t = join(map(unichr,range(100)),'') + u = join(map(unichr,range(500)),'') + v = join(map(unichr,range(1000)),'') + + for i in xrange(self.rounds): + pass + +class UnicodePredicates(Test): + + version = 0.1 + operations = 5 * 9 + rounds = 100000 + + def test(self): + + data = (u'abc', u'123', u' ', u'\u1234\u2345\u3456', u'\uFFFF'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + s.isalnum() + s.isalpha() + s.isdecimal() + s.isdigit() + s.islower() + s.isnumeric() + s.isspace() + s.istitle() + s.isupper() + + def calibrate(self): + + data = (u'abc', u'123', u' ', u'\u1234\u2345\u3456', u'\uFFFF'*10) + len_data = len(data) + + for i in xrange(self.rounds): + s = data[i % len_data] + +try: + import unicodedata +except ImportError: + pass +else: + class UnicodeProperties(Test): + + version = 0.1 + operations = 5 * 8 + rounds = 100000 + + def test(self): + + data = (u'a', u'1', u' ', u'\u1234', u'\uFFFF') + len_data = len(data) + digit = unicodedata.digit + numeric = unicodedata.numeric + decimal = unicodedata.decimal + category = unicodedata.category + bidirectional = unicodedata.bidirectional + decomposition = unicodedata.decomposition + mirrored = unicodedata.mirrored + combining = unicodedata.combining + + for i in xrange(self.rounds): + + c = data[i % len_data] + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + digit(c, None) + numeric(c, None) + decimal(c, None) + category(c) + bidirectional(c) + decomposition(c) + mirrored(c) + combining(c) + + def calibrate(self): + + data = (u'a', u'1', u' ', u'\u1234', u'\uFFFF') + len_data = len(data) + digit = unicodedata.digit + numeric = unicodedata.numeric + decimal = unicodedata.decimal + category = unicodedata.category + bidirectional = unicodedata.bidirectional + decomposition = unicodedata.decomposition + mirrored = unicodedata.mirrored + combining = unicodedata.combining + + for i in xrange(self.rounds): + + c = data[i % len_data] Added: pypy/dist/pypy/translator/microbench/pybench/package/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/microbench/pybench/package/submodule.py ============================================================================== Added: pypy/dist/pypy/translator/microbench/pybench/platform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/platform.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,1217 @@ +#!/usr/local/bin/python + +""" This module tries to retrieve as much platform identifying data as + possible. It makes this information available via function APIs. + + If called from the command line, it prints the platform + information concatenated as single string to stdout. The output + format is useable as part of a filename. + + Note that this module is a fast moving target. I plan to release + version 1.0 as the final version. + + Still needed: + ? more support for WinCE + ? support for MS-DOS (PythonDX ?) + ? support for Amiga and other still unsupported platforms running Python + ? support for additional Linux distributions + + Many thanks to all those who helped adding platform specific + checks (in no particular order): + + Charles G Waldman, David Arnold, Gordon McMillan, Ben Darnell, + Jeff Bauer, Cliff Crawford, Ivan Van Laningham, Josef + Betancourt, Randall Hopper, Karl Putland, John Farrell, Greg + Andruk, Just van Rossum, Thomas Heller, Mark R. Levinson, Mark + Hammond, Bill Tutt, Hans Nowak, Uwe Zessin (OpenVMS support), + Colin Kong, Trent Mick + + History: + 0.8.0 - added sys.version parser and various new access + APIs (python_version(), python_compiler(), etc.) + 0.7.2 - fixed architecture() to use sizeof(pointer) where available + 0.7.1 - added support for Caldera OpenLinux + 0.7.0 - some fixes for WinCE; untabified the source file + 0.6.2 - support for OpenVMS - requires version 1.5.2-V006 or higher and + vms_lib.getsyi() configured + 0.6.1 - added code to prevent 'uname -p' on platforms which are + known not to support it + 0.6.0 - fixed win32_ver() to hopefully work on Win95,98,NT and Win2k; + did some cleanup of the interfaces - some APIs have changed + 0.5.5 - fixed another type in the MacOS code... should have + used more coffee today ;-) + 0.5.4 - fixed a few typos in the MacOS code + 0.5.3 - added experimental MacOS support; added better popen() + workarounds in _syscmd_ver() -- still not 100% elegant + though + 0.5.2 - fixed uname() to return '' instead of 'unkown' in all + return values (the system uname command tends to return + 'unkown' instead of just leaving the field emtpy) + 0.5.1 - included code for slackware dist; added exception handlers + to cover up situations where platforms don't have os.popen + (e.g. Mac) or fail on socket.gethostname(); fixed libc + detection RE + 0.5.0 - changed the API names referring to system commands to *syscmd*; + added java_ver(); made syscmd_ver() a private + API (was system_ver() in previous versions) -- use uname() + instead; extended the win32_ver() to also return processor + type information + 0.4.0 - added win32_ver() and modified the platform() output for WinXX + 0.3.4 - fixed a bug in _follow_symlinks() + 0.3.3 - fixed popen() and "file" command invokation bugs + 0.3.2 - added architecture() API and support for it in platform() + 0.3.1 - fixed syscmd_ver() RE to support Windows NT + 0.3.0 - added system alias support + 0.2.3 - removed 'wince' again... oh well. + 0.2.2 - added 'wince' to syscmd_ver() supported platforms + 0.2.1 - added cache logic and changed the platform string format + 0.2.0 - changed the API to use functions instead of module globals + since some action take too long to be run on module import + 0.1.0 - first release + + You can always get the latest version of this module at: + + http://www.lemburg.com/python/platform.py + + If that URL should fail, try contacting the author. + + ---------------------------------------------------------------------- + + Copyright (c) 1999-2000, Marc-Andre Lemburg; mailto:mal at lemburg.com + Copyright (c) 2000-2001, eGenix.com Software GmbH; mailto:info at egenix.com + + Permission to use, copy, modify, and distribute this software and its + documentation for any purpose and without fee or royalty is hereby granted, + provided that the above copyright notice appear in all copies and that + both that copyright notice and this permission notice appear in + supporting documentation or portions thereof, including modifications, + that you make. + + THE AUTHOR MARC-ANDRE LEMBURG DISCLAIMS ALL WARRANTIES WITH REGARD TO + THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND + FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, + INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING + FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, + NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION + WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ! + +""" + +__version__ = '0.8.0' + +import sys,string,os,re + +### Platform specific APIs + +def libc_ver(executable=sys.executable,lib='',version='', + + chunksize=2048, + libc_search=re.compile('(__libc_init)' + '|' + '(GLIBC_([0-9.]+))' + '|' + '(libc(_\w+)?\.so(?:\.(\d[0-9.]*))?)' + ) + ): + + """ Tries to determine the libc version against which the + file executable (defaults to the Python interpreter) is linked. + + Returns a tuple of strings (lib,version) which default to the + given parameters in case the lookup fails. + + Note that the function has intimate knowledge of how different + libc versions add symbols to the executable is probably only + useable for executables compiled using gcc. + + The file is read and scanned in chunks of chunksize bytes. + + """ + f = open(executable,'rb') + binary = f.read(chunksize) + pos = 0 + while 1: + m = libc_search.search(binary,pos) + if not m: + binary = f.read(chunksize) + if not binary: + break + pos = 0 + continue + libcinit,glibc,glibcversion,so,threads,soversion = m.groups() + if libcinit and not lib: + lib = 'libc' + elif glibc: + if lib != 'glibc': + lib = 'glibc' + version = glibcversion + elif glibcversion > version: + version = glibcversion + elif so: + if lib != 'glibc': + lib = 'libc' + if soversion > version: + version = soversion + if threads and version[-len(threads):] != threads: + version = version + threads + pos = m.end() + f.close() + return lib,version + +def _dist_try_harder(distname,version,id): + + """ Tries some special tricks to get the distribution + information in case the default method fails. + + Currently supports older SuSE Linux, Caldera OpenLinux and + Slackware Linux distributions. + + """ + if os.path.exists('/var/adm/inst-log/info'): + # SuSE Linux stores distribution information in that file + info = open('/var/adm/inst-log/info').readlines() + distname = 'SuSE' + for line in info: + tv = string.split(line) + if len(tv) == 2: + tag,value = tv + else: + continue + if tag == 'MIN_DIST_VERSION': + version = string.strip(value) + elif tag == 'DIST_IDENT': + values = string.split(value,'-') + id = values[2] + return distname,version,id + + if os.path.exists('/etc/.installed'): + # Caldera OpenLinux has some infos in that file (thanks to Colin Kong) + info = open('/etc/.installed').readlines() + for line in info: + pkg = string.split(line,'-') + if len(pkg) >= 2 and pkg[0] == 'OpenLinux': + # XXX does Caldera support non Intel platforms ? If yes, + # where can we find the needed id ? + return 'OpenLinux',pkg[1],id + + if os.path.isdir('/usr/lib/setup'): + # Check for slackware verson tag file (thanks to Greg Andruk) + verfiles = os.listdir('/usr/lib/setup') + for n in range(len(verfiles)-1, -1, -1): + if verfiles[n][:14] != 'slack-version-': + del verfiles[n] + if verfiles: + verfiles.sort() + distname = 'slackware' + version = verfiles[-1][14:] + return distname,version,id + + return distname,version,id + +def dist(distname='',version='',id='', + + supported_dists=('SuSE','debian','redhat','mandrake'), + release_filename=re.compile('(\w+)[-_](release|version)'), + release_version=re.compile('([\d.]+)[^(]*(?:\((.+)\))?')): + + """ Tries to determine the name of the OS distribution name + + The function first looks for a distribution release file in + /etc and then reverts to _dist_try_harder() in case no + suitable files are found. + + Returns a tuple distname,version,id which default to the + args given as parameters. + + """ + try: + etc = os.listdir('/etc') + except os.error: + # Probably not a Unix system + return distname,version,id + for file in etc: + m = release_filename.match(file) + if m: + _distname,dummy = m.groups() + if _distname in supported_dists: + distname = _distname + break + else: + return _dist_try_harder(distname,version,id) + f = open('/etc/'+file,'r') + firstline = f.readline() + f.close() + m = release_version.search(firstline) + if m: + _version,_id = m.groups() + if _version: + version = _version + if _id: + id = _id + else: + # Unkown format... take the first two words + l = string.split(string.strip(firstline)) + if l: + version = l[0] + if len(l) > 1: + id = l[1] + return distname,version,id + +class _popen: + + """ Fairly portable (alternative) popen implementation. + + This is mostly needed in case os.popen() is not available, or + doesn't work as advertised, e.g. in Win9X GUI programs like + PythonWin or IDLE. + + XXX Writing to the pipe is currently not supported. + + """ + tmpfile = '' + pipe = None + bufsize = None + mode = 'r' + + def __init__(self,cmd,mode='r',bufsize=None): + + if mode != 'r': + raise ValueError,'popen()-emulation only supports read mode' + import tempfile + self.tmpfile = tmpfile = tempfile.mktemp() + os.system(cmd + ' > %s' % tmpfile) + self.pipe = open(tmpfile,'rb') + self.bufsize = bufsize + self.mode = mode + + def read(self): + + return self.pipe.read() + + def readlines(self): + + if self.bufsize is not None: + return self.pipe.readlines() + + def close(self, + + remove=os.unlink,error=os.error): + + if self.pipe: + rc = self.pipe.close() + else: + rc = 255 + if self.tmpfile: + try: + remove(self.tmpfile) + except error: + pass + return rc + + # Alias + __del__ = close + +def popen(cmd, mode='r', bufsize=None): + + """ Portable popen() interface. + """ + # Find a working popen implementation preferring win32pipe.popen + # over os.popen over _popen + popen = None + if os.environ.get('OS','') == 'Windows_NT': + # On NT win32pipe should work; on Win9x it hangs due to bugs + # in the MS C lib (see MS KnowledgeBase article Q150956) + try: + import win32pipe + except ImportError: + pass + else: + popen = win32pipe.popen + if popen is None: + if hasattr(os,'popen'): + popen = os.popen + # Check whether it works... it doesn't in GUI programs + # on Windows platforms + if sys.platform == 'win32': # XXX Others too ? + try: + popen('') + except os.error: + popen = _popen + else: + popen = _popen + if bufsize is None: + return popen(cmd,mode) + else: + return popen(cmd,mode,bufsize) + +def _norm_version(version,build=''): + + """ Normalize the version and build strings and return a sinlge + vesion string using the format major.minor.build (or patchlevel). + """ + l = string.split(version,'.') + if build: + l.append(build) + try: + ints = map(int,l) + except ValueError: + strings = l + else: + strings = map(str,ints) + version = string.join(strings[:3],'.') + return version + +def _syscmd_ver(system='',release='',version='', + + supported_platforms=('win32','win16','dos','os2'), + ver_output=re.compile('(?:([\w ]+) ([\w.]+) ' + '.*' + 'Version ([\d.]+))')): + + """ Tries to figure out the OS version used and returns + a tuple (system,release,version). + + It uses the "ver" shell command for this which is known + to exists on Windows, DOS and OS/2. XXX Others too ? + + In case this fails, the given parameters are used as + defaults. + + """ + if sys.platform not in supported_platforms: + return system,release,version + + # Try some common cmd strings + for cmd in ('ver','command /c ver','cmd /c ver'): + try: + pipe = popen(cmd) + info = pipe.read() + if pipe.close(): + raise os.error,'command failed' + # XXX How can I supress shell errors from being written + # to stderr ? + except os.error,why: + #print 'Command %s failed: %s' % (cmd,why) + continue + except IOError,why: + #print 'Command %s failed: %s' % (cmd,why) + continue + else: + break + else: + return system,release,version + + # Parse the output + info = string.strip(info) + m = ver_output.match(info) + if m: + system,release,version = m.groups() + # Strip trailing dots from version and release + if release[-1] == '.': + release = release[:-1] + if version[-1] == '.': + version = version[:-1] + # Normalize the version and build strings (eliminating additional + # zeros) + version = _norm_version(version) + return system,release,version + +def _win32_getvalue(key,name,default=''): + + """ Read a value for name from the registry key. + + In case this fails, default is returned. + + """ + from win32api import RegQueryValueEx + try: + return RegQueryValueEx(key,name) + except: + return default + +def win32_ver(release='',version='',csd='',ptype=''): + + """ Get additional version information from the Windows Registry + and return a tuple (version,csd,ptype) referring to version + number, CSD level and OS type (multi/single + processor). + + As a hint: ptype returns 'Uniprocessor Free' on single + processor NT machines and 'Multiprocessor Free' on multi + processor machines. The 'Free' refers to the OS version being + free of debugging code. It could also state 'Checked' which + means the OS version uses debugging code, i.e. code that + checks arguments, ranges, etc. (Thomas Heller). + + Note: this functions only works if Mark Hammond's win32 + package is installed and obviously only runs on Win32 + compatible platforms. + + XXX Is there any way to find out the processor type on WinXX ? + + XXX Is win32 available on Windows CE ? + + Adapted from code posted by Karl Putland to comp.lang.python. + + """ + # Import the needed APIs + try: + import win32api + except ImportError: + return release,version,csd,ptype + from win32api import RegQueryValueEx,RegOpenKeyEx,RegCloseKey,GetVersionEx + from win32con import HKEY_LOCAL_MACHINE,VER_PLATFORM_WIN32_NT,\ + VER_PLATFORM_WIN32_WINDOWS + + # Find out the registry key and some general version infos + maj,min,buildno,plat,csd = GetVersionEx() + version = '%i.%i.%i' % (maj,min,buildno & 0xFFFF) + if csd[:13] == 'Service Pack ': + csd = 'SP' + csd[13:] + if plat == VER_PLATFORM_WIN32_WINDOWS: + regkey = 'SOFTWARE\\Microsoft\\Windows\\CurrentVersion' + # Try to guess the release name + if maj == 4: + if min == 0: + release = '95' + else: + release = '98' + elif maj == 5: + release = '2000' + elif plat == VER_PLATFORM_WIN32_NT: + regkey = 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion' + if maj <= 4: + release = 'NT' + elif maj == 5: + release = '2000' + else: + if not release: + # E.g. Win3.1 with win32s + release = '%i.%i' % (maj,min) + return release,version,csd,ptype + + # Open the registry key + try: + keyCurVer = RegOpenKeyEx(HKEY_LOCAL_MACHINE,regkey) + # Get a value to make sure the key exists... + RegQueryValueEx(keyCurVer,'SystemRoot') + except: + return release,version,csd,ptype + + # Parse values + #subversion = _win32_getvalue(keyCurVer, + # 'SubVersionNumber', + # ('',1))[0] + #if subversion: + # release = release + subversion # 95a, 95b, etc. + build = _win32_getvalue(keyCurVer, + 'CurrentBuildNumber', + ('',1))[0] + ptype = _win32_getvalue(keyCurVer, + 'CurrentType', + (ptype,1))[0] + + # Normalize version + version = _norm_version(version,build) + + # Close key + RegCloseKey(keyCurVer) + return release,version,csd,ptype + +def _mac_ver_lookup(selectors,default=None): + + from gestalt import gestalt + l = [] + append = l.append + for selector in selectors: + try: + append(gestalt(selector)) + except RuntimeError: + append(default) + return l + +def _bcd2str(bcd): + + return hex(bcd)[2:] + +def mac_ver(release='',versioninfo=('','',''),machine=''): + + """ Get MacOS version information and return it as tuple (release, + versioninfo, machine) with versioninfo being a tuple (version, + dev_stage, non_release_version). + + Entries which cannot be determined are set to ''. All tuple + entries are strings. + + Thanks to Mark R. Levinson for mailing documentation links and + code examples for this function. Documentation for the + gestalt() API is available online at: + + http://www.rgaros.nl/gestalt/ + + """ + # Check whether the version info module is available + try: + import gestalt + except ImportError: + return release,versioninfo,machine + # Get the infos + sysv,sysu,sysa = _mac_ver_lookup(('sysv','sysu','sysa')) + # Decode the infos + if sysv: + major = (sysv & 0xFF00) >> 8 + minor = (sysv & 0x00F0) >> 4 + patch = (sysv & 0x000F) + release = '%s.%i.%i' % (_bcd2str(major),minor,patch) + if sysu: + major = (sysu & 0xFF000000) >> 24 + minor = (sysu & 0x00F00000) >> 20 + bugfix = (sysu & 0x000F0000) >> 16 + stage = (sysu & 0x0000FF00) >> 8 + nonrel = (sysu & 0x000000FF) + version = '%s.%i.%i' % (_bcd2str(major),minor,bugfix) + nonrel = _bcd2str(nonrel) + stage = {0x20:'development', + 0x40:'alpha', + 0x60:'beta', + 0x80:'final'}.get(stage,'') + versioninfo = (version,stage,nonrel) + if sysa: + machine = {0x1: '68k', + 0x2: 'PowerPC'}.get(sysa,'') + return release,versioninfo,machine + +def _java_getprop(self,name,default): + + from java.lang import System + try: + return System.getProperty(name) + except: + return default + +def java_ver(release='',vendor='',vminfo=('','',''),osinfo=('','','')): + + """ Version interface for JPython. + + Returns a tuple (release,vendor,vminfo,osinfo) with vminfo being + a tuple (vm_name,vm_release,vm_vendor) and osinfo being a + tuple (os_name,os_version,os_arch). + + Values which cannot be determined are set to the defaults + given as parameters (which all default to ''). + + """ + # Import the needed APIs + try: + import java.lang + except ImportError: + return release,vendor,vminfo,osinfo + + vendor = _java_getprop('java.vendor',vendor) + release = _java_getprop('java.version',release) + vm_name,vm_release,vm_vendor = vminfo + vm_name = _java_getprop('java.vm.name',vm_name) + vm_vendor = _java_getprop('java.vm.vendor',vm_vendor) + vm_release = _java_getprop('java.vm.version',vm_release) + vminfo = vm_name,vm_release,vm_vendor + os_name,os_version,os_arch = osinfo + os_arch = _java_getprop('java.os.arch',os_arch) + os_name = _java_getprop('java.os.name',os_name) + os_version = _java_getprop('java.os.version',os_version) + osinfo = os_name,os_version,os_arch + + return release,vendor,vminfo,osinfo + +### System name aliasing + +def system_alias(system,release,version): + + """ Returns (system,release,version) aliased to common + marketing names used for some systems. + + It also does some reordering of the information in some cases + where it would otherwise cause confusion. + + """ + if system == 'Rhapsody': + # Apple's BSD derivative + # XXX How can we determine the marketing release number ? + return 'MacOS X Server',system+release,version + + elif system == 'SunOS': + # Sun's OS + if release < '5': + # These releases use the old name SunOS + return system,release,version + # Modify release (marketing release = SunOS release - 3) + l = string.split(release,'.') + if l: + try: + major = int(l[0]) + except ValueError: + pass + else: + major = major - 3 + l[0] = str(major) + release = string.join(l,'.') + if release < '6': + system = 'Solaris' + else: + # XXX Whatever the new SunOS marketing name is... + system = 'Solaris' + + elif system == 'IRIX64': + # IRIX reports IRIX64 on platforms with 64-bit support; yet it + # is really a version and not a different platform, since 32-bit + # apps are also supported.. + system = 'IRIX' + if version: + version = version + ' (64bit)' + else: + version = '64bit' + + elif system in ('win32','win16'): + # In case one of the other tricks + system = 'Windows' + + return system,release,version + +### Various internal helpers + +def _platform(*args): + + """ Helper to format the platform string in a filename + compatible format e.g. "system-version-machine". + """ + # Format the platform string + platform = string.join( + map(string.strip, + filter(len,args)), + '-') + + # Cleanup some possible filename obstacles... + replace = string.replace + platform = replace(platform,' ','_') + platform = replace(platform,'/','-') + platform = replace(platform,'\\','-') + platform = replace(platform,':','-') + platform = replace(platform,';','-') + platform = replace(platform,'"','-') + platform = replace(platform,'(','-') + platform = replace(platform,')','-') + + # No need to report 'unkown' information... + platform = replace(platform,'unknown','') + + # Fold '--'s and remove trailing '-' + while 1: + cleaned = replace(platform,'--','-') + if cleaned == platform: + break + platform = cleaned + while platform[-1] == '-': + platform = platform[:-1] + + return platform + +def _node(default=''): + + """ Helper to determine the node name of this machine. + """ + try: + import socket + except ImportError: + # No sockets... + return default + try: + return socket.gethostname() + except socket.error: + # Still not working... + return default + +# os.path.abspath is new in Python 1.5.2: +if not hasattr(os.path,'abspath'): + + def _abspath(path, + + isabs=os.path.isabs,join=os.path.join,getcwd=os.getcwd, + normpath=os.path.normpath): + + if not isabs(path): + path = join(getcwd(), path) + return normpath(path) + +else: + + _abspath = os.path.abspath + +def _follow_symlinks(filepath): + + """ In case filepath is a symlink, follow it until a + real file is reached. + """ + filepath = _abspath(filepath) + while os.path.islink(filepath): + filepath = os.path.normpath( + os.path.join(filepath,os.readlink(filepath))) + return filepath + +def _syscmd_uname(option,default=''): + + """ Interface to the system's uname command. + """ + if sys.platform in ('dos','win32','win16','os2'): + # XXX Others too ? + return default + try: + f = os.popen('uname %s 2> /dev/null' % option) + except (AttributeError,os.error): + return default + output = string.strip(f.read()) + rc = f.close() + if not output or rc: + return default + else: + return output + +def _syscmd_file(target,default=''): + + """ Interface to the system's file command. + + The function uses the -b option of the file command to have it + ommit the filename in its output and if possible the -L option + to have the command follow symlinks. It returns default in + case the command should fail. + + """ + target = _follow_symlinks(target) + try: + f = os.popen('file %s 2> /dev/null' % target) + except (AttributeError,os.error): + return default + output = string.strip(f.read()) + rc = f.close() + if not output or rc: + return default + else: + return output + +### Information about the used architecture + +# Default values for architecture; non-empty strings override the +# defaults given as parameters +_default_architecture = { + 'win32': ('','WindowsPE'), + 'win16': ('','Windows'), + 'dos': ('','MSDOS'), +} + +def architecture(executable=sys.executable,bits='',linkage='', + + split=re.compile('[\s,]').split): + + """ Queries the given executable (defaults to the Python interpreter + binary) for various architecture informations. + + Returns a tuple (bits,linkage) which contain information about + the bit architecture and the linkage format used for the + executable. Both values are returned as strings. + + Values that cannot be determined are returned as given by the + parameter presets. If bits is given as '', the sizeof(pointer) + (or sizeof(long) on Python version < 1.5.2) is used as + indicator for the supported pointer size. + + The function relies on the system's "file" command to do the + actual work. This is available on most if not all Unix + platforms. On some non-Unix platforms and then only if the + executable points to the Python interpreter defaults from + _default_architecture are used. + + """ + # Use the sizeof(pointer) as default number of bits if nothing + # else is given as default. + if not bits: + import struct + try: + size = struct.calcsize('P') + except struct.error: + # Older installations can only query longs + size = struct.calcsize('l') + bits = str(size*8) + 'bit' + + # Get data from the 'file' system command + output = _syscmd_file(executable,'') + + if not output and \ + executable == sys.executable: + # "file" command did not return anything; we'll try to provide + # some sensible defaults then... + if _default_architecture.has_key(sys.platform): + b,l = _default_architecture[sys.platform] + if b: + bits = b + if l: + linkage = l + return bits,linkage + + # Split the output into a list of strings omitting the filename + fileout = split(output)[1:] + + if 'executable' not in fileout: + # Format not supported + return bits,linkage + + # Bits + if '32-bit' in fileout: + bits = '32bit' + elif 'N32' in fileout: + # On Irix only + bits = 'n32bit' + elif '64-bit' in fileout: + bits = '64bit' + + # Linkage + if 'ELF' in fileout: + linkage = 'ELF' + elif 'PE' in fileout: + # E.g. Windows uses this format + if 'Windows' in fileout: + linkage = 'WindowsPE' + else: + linkage = 'PE' + elif 'COFF' in fileout: + linkage = 'COFF' + elif 'MS-DOS' in fileout: + linkage = 'MSDOS' + else: + # XXX the A.OUT format also falls under this class... + pass + + return bits,linkage + +### Portable uname() interface + +_uname_cache = None + +def uname(): + + """ Fairly portable uname interface. Returns a tuple + of strings (system,node,release,version,machine,processor) + identifying the underlying platform. + + Note that unlike the os.uname function this also returns + possible processor information as additional tuple entry. + + Entries which cannot be determined are set to ''. + + """ + global _uname_cache + + if _uname_cache is not None: + return _uname_cache + + # Get some infos from the builtin os.uname API... + try: + system,node,release,version,machine = os.uname() + + except AttributeError: + # Hmm, no uname... we'll have to poke around the system then. + system = sys.platform + release = '' + version = '' + node = _node() + machine = '' + processor = '' + use_syscmd_ver = 1 + + # Try win32_ver() on win32 platforms + if system == 'win32': + release,version,csd,ptype = win32_ver() + if release and version: + use_syscmd_ver = 0 + + # Try the 'ver' system command available on some + # platforms + if use_syscmd_ver: + system,release,version = _syscmd_ver(system) + + # In case we still don't know anything useful, we'll try to + # help ourselves + if system in ('win32','win16'): + if not version: + if system == 'win32': + version = '32bit' + else: + version = '16bit' + system = 'Windows' + + elif system[:4] == 'java': + release,vendor,vminfo,osinfo = java_ver() + system = 'Java' + version = string.join(vminfo,', ') + if not version: + version = vendor + + elif os.name == 'mac': + release,(version,stage,nonrel),machine = mac_ver() + system = 'MacOS' + + else: + # System specific extensions + if system == 'OpenVMS': + # OpenVMS seems to have release and version mixed up + if not release or release == '0': + release = version + version = '' + # Get processor information + try: + import vms_lib + except ImportError: + pass + else: + csid, cpu_number = vms_lib.getsyi('SYI$_CPU',0) + if (cpu_number >= 128): + processor = 'Alpha' + else: + processor = 'VAX' + else: + # Get processor information from the uname system command + processor = _syscmd_uname('-p','') + + # 'unkown' is not really any useful as information; we'll convert + # it to '' which is more portable + if system == 'unknown': + system = '' + if node == 'unknown': + node = '' + if release == 'unknown': + release = '' + if version == 'unknown': + version = '' + if machine == 'unknown': + machine = '' + if processor == 'unknown': + processor = '' + _uname_cache = system,node,release,version,machine,processor + return _uname_cache + +### Direct interfaces to some of the uname() return values + +def system(): + + """ Returns the system/OS name, e.g. 'Linux', 'Windows' or 'Java'. + + An empty string is returned if the value cannot be determined. + + """ + return uname()[0] + +def node(): + + """ Returns the computer's network name (may not be fully qualified !) + + An empty string is returned if the value cannot be determined. + + """ + return uname()[1] + +def release(): + + """ Returns the system's release, e.g. '2.2.0' or 'NT' + + An empty string is returned if the value cannot be determined. + + """ + return uname()[2] + +def version(): + + """ Returns the system's release version, e.g. '#3 on degas' + + An empty string is returned if the value cannot be determined. + + """ + return uname()[3] + +def machine(): + + """ Returns the machine type, e.g. 'i386' + + An empty string is returned if the value cannot be determined. + + """ + return uname()[4] + +def processor(): + + """ Returns the (true) processor name, e.g. 'amdk6' + + An empty string is returned if the value cannot be + determined. Note that many platforms do not provide this + information or simply return the same value as for machine(), + e.g. NetBSD does this. + + """ + return uname()[5] + +### Various APIs for extracting information from sys.version + +_sys_version_parser = re.compile('([\w.+]+)\s*' + '\(#(\d+),\s*([\w ]+),\s*([\w :]+)\)\s*' + '\[([^\]]+)\]?') +_sys_version_cache = None + +def _sys_version(): + + """ Returns a parsed version of Python's sys.version as tuple + (version, buildno, builddate, compiler) referring to the Python + version, build number, build date/time as string and the compiler + identification string. + + Note that unlike the Python sys.version, the returned value + for the Python version will always include the patchlevel (it + defaults to '.0'). + + """ + global _sys_version_cache + import sys, re, time + + if _sys_version_cache is not None: + return _sys_version_cache + version, buildno, builddate, buildtime, compiler = \ + _sys_version_parser.match(sys.version).groups() + buildno = int(buildno) + builddate = builddate + ' ' + buildtime + l = string.split(version, '.') + if len(l) == 2: + l.append('0') + version = string.join(l, '.') + _sys_version_cache = (version, buildno, builddate, compiler) + return _sys_version_cache + +def python_version(): + + """ Returns the Python version as string 'major.minor.patchlevel' + + Note that unlike the Python sys.version, the returned value + will always include the patchlevel (it defaults to 0). + + """ + return _sys_version()[0] + +def python_build(): + + """ Returns a tuple (buildno, buildate) stating the Python + build number and date as strings. + + """ + return _sys_version()[1:3] + +def python_compiler(): + + """ Returns a string identifying the compiler used for compiling + Python. + + """ + return _sys_version()[3] + +### The Opus Magnum of platform strings :-) + +_platform_cache = None +_platform_aliased_cache = None + +def platform(aliased=0, terse=0): + + """ Returns a single string identifying the underlying platform + with as much useful information as possible (but no more :). + + The output is intended to be human readable rather than + machine parseable. It may look different on different + platforms and this is intended. + + If "aliased" is true, the function will use aliases for + various platforms that report system names which differ from + their common names, e.g. SunOS will be reported as + Solaris. The system_alias() function is used to implement + this. + + Setting terse to true causes the function to return only the + absolute minimum information needed to identify the platform. + + """ + global _platform_cache,_platform_aliased_cache + + if not aliased and (_platform_cache is not None): + return _platform_cache + elif _platform_aliased_cache is not None: + return _platform_aliased_cache + + # Get uname information and then apply platform specific cosmetics + # to it... + system,node,release,version,machine,processor = uname() + if machine == processor: + processor = '' + if aliased: + system,release,version = system_alias(system,release,version) + + if system == 'Windows': + # MS platforms + rel,vers,csd,ptype = win32_ver(version) + if terse: + platform = _platform(system,release) + else: + platform = _platform(system,release,version,csd) + + elif system in ('Linux',): + # Linux based systems + distname,distversion,distid = dist('') + if distname and not terse: + platform = _platform(system,release,machine,processor, + 'with', + distname,distversion,distid) + else: + # If the distribution name is unknown check for libc vs. glibc + libcname,libcversion = libc_ver(sys.executable) + platform = _platform(system,release,machine,processor, + 'with', + libcname+libcversion) + elif system == 'Java': + # Java platforms + r,v,vminfo,(os_name,os_version,os_arch) = java_ver() + if terse: + platform = _platform(system,release,version) + else: + platform = _platform(system,release,version, + 'on', + os_name,os_version,os_arch) + + elif system == 'MacOS': + # MacOS platforms + if terse: + platform = _platform(system,release) + else: + platform = _platform(system,release,machine) + + else: + # Generic handler + if terse: + platform = _platform(system,release) + else: + bits,linkage = architecture(sys.executable) + platform = _platform(system,release,machine,processor,bits,linkage) + + if aliased: + _platform_aliased_cache = platform + elif terse: + pass + else: + _platform_cache = platform + return platform + +if __name__ == '__main__': + # Default is to print the aliased verbose platform string + terse = ('terse' in sys.argv or '--terse' in sys.argv) + aliased = (not 'nonaliased' in sys.argv and not '--nonaliased' in sys.argv) + print platform(aliased,terse) + sys.exit(0) Added: pypy/dist/pypy/translator/microbench/pybench/pybench.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/pybench.py Sun Apr 9 13:26:13 2006 @@ -0,0 +1,390 @@ +#!/usr/local/bin/python -O + +""" A Python Benchmark Suite +""" +__copyright__="""\ + Copyright (c), 1997-2001, Marc-Andre Lemburg (mal at lemburg.com) + + All Rights Reserved. + + Permission to use, copy, modify, and distribute this software and + its documentation for any purpose and without fee or royalty is + hereby granted, provided that the above copyright notice appear in + all copies and that both that copyright notice and this permission + notice appear in supporting documentation or portions thereof, + including modifications, that you make. + + THE AUTHOR MARC-ANDRE LEMBURG DISCLAIMS ALL WARRANTIES WITH REGARD + TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE + FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN + AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING + OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS + SOFTWARE ! + +""" + +__version__ = '1.0' + +# +# NOTE: Use xrange for all test loops unless you want to face +# a 20MB process ! +# +# All tests should have rounds set to values so that a run() +# takes between 20-50 seconds. This is to get fairly good +# clock() values. You can use option -w to speedup the tests +# by a fixed integer factor (the "warp factor"). +# + +import sys,time,operator +from CommandLine import * + +try: + import cPickle + pickle = cPickle +except ImportError: + import pickle + +### Test baseclass + +class Test: + + """ All test must have this class as baseclass. It provides + the necessary interface to the benchmark machinery. + + The tests must set .rounds to a value high enough to let the + test run between 20-50 seconds. This is needed because + clock()-timing only gives rather inaccurate values (on Linux, + for example, it is accurate to a few hundreths of a + second). If you don't want to wait that long, use a warp + factor larger than 1. + + It is also important to set the .operations variable to a + value representing the number of "virtual operations" done per + call of .run(). + + If you change a test in some way, don't forget to increase + it's version number. + + """ + operations = 1 # number of operations done per test()-call + rounds = 100000 # number of rounds per run + is_a_test = 1 # identifier + last_timing = (0,0,0) # last timing (real,run,calibration) + warp = 1 # warp factor this test uses + cruns = 20 # number of calibration runs + overhead = None # list of calibration timings + + version = 1.0 # version number of this test + + def __init__(self,warp=1): + + if warp > 1: + self.rounds = self.rounds / warp + self.warp = warp + self.times = [] + self.overhead = [] + # We want these to be in the instance dict, so that pickle + # saves them + self.version = self.version + self.operations = self.operations + self.rounds = self.rounds + + def run(self): + + """ Run the test in two phases: first calibrate, then + do the actual test. Be careful to keep the calibration + timing low w/r to the test timing. + """ + test = self.test + calibrate = self.calibrate + clock = time.clock + cruns = self.cruns + # first calibrate + offset = 0.0 + for i in range(cruns): + t = clock() + calibrate() + t = clock() - t + offset = offset + t + offset = offset / cruns + # now the real thing + t = clock() + test() + t = clock() - t + self.last_timing = (t-offset,t,offset) + self.times.append(t-offset) + + def calibrate(self): + + """ Run the test, doing self.rounds loops, but without + the actual test target in place. + """ + return + + def test(self): + + """ Run the test, doing self.rounds loop + """ + # do some tests + return + + def stat(self): + + """ Returns two value: average time per run and average per + operation. + """ + runs = len(self.times) + if runs == 0: + return 0,0 + totaltime = reduce(operator.add,self.times,0.0) + avg = totaltime / float(runs) + op_avg = totaltime / float(runs * self.rounds * self.operations) + if self.overhead: + totaloverhead = reduce(operator.add,self.overhead,0.0) + ov_avg = totaloverhead / float(runs) + else: + # use self.last_timing - not too accurate + ov_avg = self.last_timing[2] + return avg,op_avg,ov_avg + +### load Setup +import Setup + +### Benchmark base class + +class Benchmark: + + name = '?' # Name of the benchmark + rounds = 1 # Number of rounds to run + warp = 1 # Warp factor + roundtime = 0 # Average round time + version = None # Benchmark version number (see __init__) + + def __init__(self): + + self.tests = {} + self.version = 0.31 + + def load_tests(self,setupmod,warp=1): + + self.warp = warp + tests = self.tests + print 'Searching for tests...' + setupmod.__dict__.values() + for c in setupmod.__dict__.values(): + if hasattr(c,'is_a_test') and c.__name__ != 'Test': + tests[c.__name__] = c(warp) + l = tests.keys() + l.sort() + for t in l: + print ' ',t + print + + def run(self): + + tests = self.tests.items() + tests.sort() + clock = time.clock + print 'Running %i round(s) of the suite: ' % self.rounds + print + roundtime = clock() + for i in range(self.rounds): + print ' Round %-25i real abs overhead' % (i+1) + for j in range(len(tests)): + name,t = tests[j] + print '%30s:' % name, + t.run() + print ' %.3fr %.3fa %.3fo' % t.last_timing + print ' ----------------------' + print ' Average round time: %.3f seconds' % \ + ((clock() - roundtime)/(i+1)) + print + self.roundtime = (clock() - roundtime) / self.rounds + print + + def print_stat(self, compare_to=None, hidenoise=0): + + if not compare_to: + print '%-30s per run per oper. overhead' % 'Tests:' + print '-'*72 + tests = self.tests.items() + tests.sort() + for name,t in tests: + avg,op_avg,ov_avg = t.stat() + print '%30s: %10.2f ms %7.2f us %7.2f ms' % \ + (name,avg*1000.0,op_avg*1000000.0,ov_avg*1000.0) + print '-'*72 + print '%30s: %10.2f ms' % \ + ('Average round time',self.roundtime * 1000.0) + + else: + print '%-30s per run per oper. diff *)' % \ + 'Tests:' + print '-'*72 + tests = self.tests.items() + tests.sort() + compatible = 1 + for name,t in tests: + avg,op_avg,ov_avg = t.stat() + try: + other = compare_to.tests[name] + except KeyError: + other = None + if other and other.version == t.version and \ + other.operations == t.operations: + avg1,op_avg1,ov_avg1 = other.stat() + qop_avg = (op_avg/op_avg1-1.0)*100.0 + if hidenoise and abs(qop_avg) < 10: + qop_avg = '' + else: + qop_avg = '%+7.2f%%' % qop_avg + else: + qavg,qop_avg = 'n/a', 'n/a' + compatible = 0 + print '%30s: %10.2f ms %7.2f us %8s' % \ + (name,avg*1000.0,op_avg*1000000.0,qop_avg) + print '-'*72 + if compatible and compare_to.roundtime > 0 and \ + compare_to.version == self.version: + print '%30s: %10.2f ms %+7.2f%%' % \ + ('Average round time',self.roundtime * 1000.0, + ((self.roundtime*self.warp)/ + (compare_to.roundtime*compare_to.warp)-1.0)*100.0) + else: + print '%30s: %10.2f ms n/a' % \ + ('Average round time',self.roundtime * 1000.0) + print + print '*) measured against: %s (rounds=%i, warp=%i)' % \ + (compare_to.name,compare_to.rounds,compare_to.warp) + print + +def print_machine(): + + import platform + print 'Machine Details:' + print ' Platform ID: %s' % platform.platform() + # There's a bug in Python 2.2b1+... + if 1 or sys.version[:6] != '2.2b1+': + print ' Python: %s' % platform.python_version() + print ' Compiler: %s' % platform.python_compiler() + buildno, buildate = platform.python_build() + print ' Build: %s (#%i)' % (buildate, buildno) + +class PyBenchCmdline(Application): + + header = ("PYBENCH - a benchmark test suite for Python " + "interpreters/compilers.") + + version = __version__ + + options = [ArgumentOption('-n','number of rounds',Setup.Number_of_rounds), + ArgumentOption('-f','save benchmark to file arg',''), + ArgumentOption('-c','compare benchmark with the one in file arg',''), + ArgumentOption('-s','show benchmark in file arg, then exit',''), + ArgumentOption('-w','set warp factor to arg',Setup.Warp_factor), + SwitchOption('-d','hide noise in compares', 0), + SwitchOption('--no-gc','disable garbage collection', 0), + ] + + about = """\ +The normal operation is to run the suite and display the +results. Use -f to save them for later reuse or comparisms. + +Examples: + +python1.5 pybench.py -w 100 -f p15 +python1.4 pybench.py -w 100 -f p14 +python pybench.py -s p15 -c p14 +""" + copyright = __copyright__ + + def main(self): + + rounds = self.values['-n'] + reportfile = self.values['-f'] + show_bench = self.values['-s'] + compare_to = self.values['-c'] + hidenoise = self.values['-d'] + warp = self.values['-w'] + nogc = self.values['--no-gc'] + + # Switch off GC + if nogc: + try: + import gc + except ImportError: + nogc = 0 + else: + if self.values['--no-gc']: + gc.disable() + + print 'PYBENCH',__version__ + print + + if not compare_to: + print_machine() + print + + if compare_to: + try: + f = open(compare_to,'rb') + bench = pickle.load(f) + bench.name = compare_to + f.close() + compare_to = bench + except IOError: + print '* Error opening/reading file',compare_to + compare_to = None + + if show_bench: + try: + f = open(show_bench,'rb') + bench = pickle.load(f) + bench.name = show_bench + f.close() + print 'Benchmark: %s (rounds=%i, warp=%i)' % \ + (bench.name,bench.rounds,bench.warp) + print + bench.print_stat(compare_to, hidenoise) + except IOError: + print '* Error opening/reading file',show_bench + print + return + + if reportfile: + if nogc: + print 'Benchmark: %s (rounds=%i, warp=%i, no GC)' % \ + (reportfile,rounds,warp) + else: + print 'Benchmark: %s (rounds=%i, warp=%i)' % \ + (reportfile,rounds,warp) + print + + # Create benchmark object + bench = Benchmark() + bench.rounds = rounds + bench.load_tests(Setup,warp) + try: + bench.run() + except KeyboardInterrupt: + print + print '*** KeyboardInterrupt -- Aborting' + print + return + bench.print_stat(compare_to) + # ring bell + sys.stderr.write('\007') + + if reportfile: + try: + f = open(reportfile,'wb') + bench.name = reportfile + pickle.dump(bench,f) + f.close() + except IOError: + print '* Error opening/writing reportfile' + +if __name__ == '__main__': + PyBenchCmdline() Added: pypy/dist/pypy/translator/microbench/pybench/python2.4.2.bench ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/pybench/python2.4.2.bench Sun Apr 9 13:26:13 2006 @@ -0,0 +1,1711 @@ +(i__main__ +Benchmark +p1 +(dp2 +S'tests' +p3 +(dp4 +S'SpecialInstanceAttribute' +p5 +(iLookups +SpecialInstanceAttribute +p6 +(dp7 +S'operations' +p8 +I120 +sS'warp' +p9 +I20 +sS'times' +p10 +(lp11 +F0.20949999999999996 +aF0.18950000000000128 +aF0.18949999999999764 +aF0.19899999999999932 +aF0.18949999999999764 +aF0.18949999999999784 +aF0.18949999999999784 +aF0.20950000000000096 +aF0.19950000000000295 +aF0.19949999999998838 +asS'version' +p12 +F0.29999999999999999 +sS'overhead' +p13 +(lp14 +sS'rounds' +p15 +I5000 +sS'last_timing' +p16 +(F0.19949999999998838 +F0.19999999999998863 +F0.00050000000000025577 +tp17 +sbsS'PythonFunctionCalls' +p18 +(iCalls +PythonFunctionCalls +p19 +(dp20 +g8 +I55 +sg9 +I20 +sg10 +(lp21 +F0.10949999999999986 +aF0.10949999999999944 +aF0.11000000000000298 +aF0.10949999999999935 +aF0.10999999999999943 +aF0.11949999999999719 +aF0.10949999999999918 +aF0.10949999999999918 +aF0.10949999999999953 +aF0.10999999999999943 +asg12 +F0.29999999999999999 +sg13 +(lp22 +sg15 +I3000 +sg16 +(F0.10999999999999943 +F0.10999999999999943 +F0 +tp23 +sbsS'CompareFloatsIntegers' +p24 +(iNumbers +CompareFloatsIntegers +p25 +(dp26 +g8 +I150 +sg9 +I20 +sg10 +(lp27 +F0.13 +aF0.13999999999999968 +aF0.12950000000000078 +aF0.12949999999999892 +aF0.12949999999999892 +aF0.12950000000000267 +aF0.1294999999999952 +aF0.13000000000000256 +aF0.13950000000000068 +aF0.12950000000000267 +asg12 +F0.10000000000000001 +sg13 +(lp28 +sg15 +I3000 +sg16 +(F0.12950000000000267 +F0.13000000000000256 +F0.00049999999999990048 +tp29 +sbsS'PythonMethodCalls' +p30 +(iCalls +PythonMethodCalls +p31 +(dp32 +g8 +I75 +sg9 +I20 +sg10 +(lp33 +F0.12000000000000011 +aF0.12000000000000099 +aF0.10999999999999943 +aF0.10999999999999943 +aF0.12000000000000099 +aF0.11999999999999744 +aF0.10999999999999943 +aF0.10999999999999943 +aF0.10999999999999943 +aF0.10949999999999953 +asg12 +F0.29999999999999999 +sg13 +(lp34 +sg15 +I1000 +sg16 +(F0.10949999999999953 +F0.10999999999999943 +F0.00049999999999990048 +tp35 +sbsS'SecondSubmoduleImport' +p36 +(iImports +SecondSubmoduleImport +p37 +(dp38 +g8 +I25 +sg9 +I20 +sg10 +(lp39 +F0.080000000000000071 +aF0.079500000000000084 +aF0.080000000000001847 +aF0.080000000000001847 +aF0.069999999999996732 +aF0.079999999999998295 +aF0.079999999999998295 +aF0.079999999999998295 +aF0.079999999999998295 +aF0.079999999999998295 +asg12 +F0.10000000000000001 +sg13 +(lp40 +sg15 +I1000 +sg16 +(F0.079999999999998295 +F0.079999999999998295 +F0 +tp41 +sbsS'DictWithIntegerKeys' +p42 +(iDict +DictWithIntegerKeys +p43 +(dp44 +g8 +I60 +sg9 +I20 +sg10 +(lp45 +F0.12899999999999989 +aF0.12899999999999903 +aF0.12899999999999903 +aF0.12900000000000258 +aF0.1390000000000006 +aF0.12850000000000286 +aF0.13900000000000076 +aF0.12899999999999529 +aF0.12899999999999529 +aF0.13900000000000076 +asg12 +F0.10000000000000001 +sg13 +(lp46 +sg15 +I10000 +sg16 +(F0.13900000000000076 +F0.14000000000000057 +F0.00099999999999980096 +tp47 +sbsS'UnicodeSlicing' +p48 +(iUnicode +UnicodeSlicing +p49 +(dp50 +g8 +I35 +sg9 +I20 +sg10 +(lp51 +F0.10950000000000033 +aF0.11950000000000101 +aF0.10949999999999935 +aF0.10949999999999953 +aF0.10949999999999918 +aF0.10950000000000663 +aF0.10949999999999953 +aF0.10899999999999962 +aF0.10849999999999937 +aF0.10949999999999989 +asg12 +F0.10000000000000001 +sg13 +(lp52 +sg15 +I5000 +sg16 +(F0.10949999999999989 +F0.10999999999999943 +F0.0004999999999995453 +tp53 +sbsS'NestedForLoops' +p54 +(iConstructs +NestedForLoops +p55 +(dp56 +g8 +I50000 +sg9 +I20 +sg10 +(lp57 +F0.060000000000000053 +aF0.060000000000000497 +aF0.059500000000000511 +aF0.060000000000002274 +aF0.060000000000002274 +aF0.060000000000002274 +aF0.060000000000002274 +aF0.059999999999995168 +aF0.059999999999995168 +aF0.070000000000000284 +asg12 +F0.29999999999999999 +sg13 +(lp58 +sg15 +I7 +sg16 +(F0.070000000000000284 +F0.070000000000000284 +F0 +tp59 +sbsS'UnicodePredicates' +p60 +(iUnicode +UnicodePredicates +p61 +(dp62 +g8 +I45 +sg9 +I20 +sg10 +(lp63 +F0.1180000000000001 +aF0.12800000000000084 +aF0.13800000000000043 +aF0.11799999999999748 +aF0.12799999999999551 +aF0.12799999999999551 +aF0.11800000000000459 +aF0.12800000000000261 +aF0.12799999999999551 +aF0.12800000000001005 +asg12 +F0.10000000000000001 +sg13 +(lp64 +sg15 +I5000 +sg16 +(F0.12800000000001005 +F0.13000000000000966 +F0.0019999999999996019 +tp65 +sbsS'ForLoops' +p66 +(iConstructs +ForLoops +p67 +(dp68 +g8 +I25 +sg9 +I20 +sg10 +(lp69 +F0.070000000000000284 +aF0.069500000000000298 +aF0.070000000000000284 +aF0.070000000000000284 +aF0.070000000000000284 +aF0.079499999999998044 +aF0.069500000000000034 +aF0.070000000000000284 +aF0.070000000000000284 +aF0.070000000000000284 +asg12 +F0.10000000000000001 +sg13 +(lp70 +sg15 +I400 +sg16 +(F0.070000000000000284 +F0.070000000000000284 +F0 +tp71 +sbsS'ConcatStrings' +p72 +(iStrings +ConcatStrings +p73 +(dp74 +g8 +I50 +sg9 +I20 +sg10 +(lp75 +F0.099499999999999866 +aF0.099500000000000546 +aF0.099500000000001435 +aF0.099499999999997785 +aF0.088999999999999885 +aF0.098999999999994162 +aF0.088999999999996152 +aF0.10949999999999953 +aF0.099500000000001518 +aF0.099500000000001518 +asg12 +F0.10000000000000001 +sg13 +(lp76 +sg15 +I3000 +sg16 +(F0.099500000000001518 +F0.10000000000000142 +F0.00049999999999990048 +tp77 +sbsS'SecondPackageImport' +p78 +(iImports +SecondPackageImport +p79 +(dp80 +g8 +I25 +sg9 +I20 +sg10 +(lp81 +F0.059500000000000039 +aF0.060000000000000497 +aF0.059499999999998644 +aF0.059499999999998644 +aF0.059999999999998721 +aF0.060000000000002274 +aF0.060000000000002274 +aF0.05950000000000237 +aF0.069500000000000034 +aF0.070000000000000284 +asg12 +F0.10000000000000001 +sg13 +(lp82 +sg15 +I1000 +sg16 +(F0.070000000000000284 +F0.070000000000000284 +F0 +tp83 +sbsS'CompareIntegers' +p84 +(iNumbers +CompareIntegers +p85 +(dp86 +g8 +I150 +sg9 +I20 +sg10 +(lp87 +F0.089499999999999968 +aF0.08950000000000076 +aF0.099499999999999658 +aF0.089499999999999955 +aF0.089499999999999774 +aF0.089499999999996402 +aF0.089499999999996055 +aF0.089499999999996055 +aF0.099500000000001518 +aF0.099500000000001518 +asg12 +F0.10000000000000001 +sg13 +(lp88 +sg15 +I6000 +sg16 +(F0.099500000000001518 +F0.10000000000000142 +F0.00049999999999990048 +tp89 +sbsS'CreateStringsWithConcat' +p90 +(iStrings +CreateStringsWithConcat +p91 +(dp92 +g8 +I50 +sg9 +I20 +sg10 +(lp93 +F0.049500000000000044 +aF0.060000000000000497 +aF0.049500000000000724 +aF0.049500000000000634 +aF0.049500000000000807 +aF0.04950000000000436 +aF0.049499999999997255 +aF0.059999999999995168 +aF0.060000000000002274 +aF0.04950000000000436 +asg12 +F0.10000000000000001 +sg13 +(lp94 +sg15 +I4000 +sg16 +(F0.04950000000000436 +F0.050000000000004263 +F0.00049999999999990048 +tp95 +sbsS'SmallLists' +p96 +(iLists +SmallLists +p97 +(dp98 +g8 +I85 +sg9 +I20 +sg10 +(lp99 +F0.11949999999999919 +aF0.11950000000000101 +aF0.11899999999999747 +aF0.11949999999999736 +aF0.11950000000000109 +aF0.11899999999999729 +aF0.11900000000000474 +aF0.11949999999999754 +aF0.11949999999999754 +aF0.11949999999999719 +asg12 +F0.29999999999999999 +sg13 +(lp100 +sg15 +I3000 +sg16 +(F0.11949999999999719 +F0.11999999999999744 +F0.00050000000000025577 +tp101 +sbsS'CompareUnicode' +p102 +(iUnicode +CompareUnicode +p103 +(dp104 +g8 +I50 +sg9 +I20 +sg10 +(lp105 +F0.1095000000000001 +aF0.11950000000000008 +aF0.10899999999999946 +aF0.10899999999999946 +aF0.10949999999999935 +aF0.099000000000001614 +aF0.099000000000001614 +aF0.10949999999999918 +aF0.10899999999999928 +aF0.10900000000000673 +asg12 +F0.10000000000000001 +sg13 +(lp106 +sg15 +I7500 +sg16 +(F0.10900000000000673 +F0.11000000000000654 +F0.00099999999999980096 +tp107 +sbsS'SimpleListManipulation' +p108 +(iLists +SimpleListManipulation +p109 +(dp110 +g8 +I90 +sg9 +I20 +sg10 +(lp111 +F0.069500000000000298 +aF0.079500000000000084 +aF0.05950000000000237 +aF0.05950000000000237 +aF0.070000000000000284 +aF0.069500000000000034 +aF0.059499999999994911 +aF0.059499999999994911 +aF0.05950000000000237 +aF0.070000000000000284 +asg12 +F0.20000000000000001 +sg13 +(lp112 +sg15 +I3000 +sg16 +(F0.070000000000000284 +F0.070000000000000284 +F0 +tp113 +sbsS'DictWithFloatKeys' +p114 +(iDict +DictWithFloatKeys +p115 +(dp116 +g8 +I60 +sg9 +I20 +sg10 +(lp117 +F0.22899999999999998 +aF0.23900000000000024 +aF0.22900000000000045 +aF0.22900000000000045 +aF0.22900000000000045 +aF0.22900000000000417 +aF0.22850000000000392 +aF0.22899999999999671 +aF0.22899999999999671 +aF0.22900000000000417 +asg12 +F0.10000000000000001 +sg13 +(lp118 +sg15 +I10000 +sg16 +(F0.22900000000000417 +F0.23000000000000398 +F0.00099999999999980096 +tp119 +sbsS'TryExcept' +p120 +(iExceptions +TryExcept +p121 +(dp122 +g8 +I150 +sg9 +I20 +sg10 +(lp123 +F0.17900000000000063 +aF0.17899999999999974 +aF0.17899999999999974 +aF0.17899999999999974 +aF0.18899999999999756 +aF0.16900000000000154 +aF0.17899999999999991 +aF0.17899999999999991 +aF0.17899999999999991 +aF0.1789999999999928 +asg12 +F0.10000000000000001 +sg13 +(lp124 +sg15 +I10000 +sg16 +(F0.1789999999999928 +F0.17999999999999261 +F0.00099999999999980096 +tp125 +sbsS'SimpleDictManipulation' +p126 +(iDict +SimpleDictManipulation +p127 +(dp128 +g8 +I120 +sg9 +I20 +sg10 +(lp129 +F0.089500000000000315 +aF0.089999999999999858 +aF0.088999999999999885 +aF0.079499999999998211 +aF0.089999999999999858 +aF0.090000000000003411 +aF0.090000000000003411 +aF0.079499999999998391 +aF0.079499999999998391 +aF0.079499999999998044 +asg12 +F0.29999999999999999 +sg13 +(lp130 +sg15 +I2500 +sg16 +(F0.079499999999998044 +F0.079999999999998295 +F0.00050000000000025577 +tp131 +sbsS'TryRaiseExcept' +p132 +(iExceptions +TryRaiseExcept +p133 +(dp134 +g8 +I5 +sg9 +I20 +sg10 +(lp135 +F0.099499999999999658 +aF0.10000000000000142 +aF0.089499999999999955 +aF0.099499999999997785 +aF0.10000000000000142 +aF0.10000000000000142 +aF0.089499999999996055 +aF0.089500000000003507 +aF0.089500000000003507 +aF0.10999999999999943 +asg12 +F0.10000000000000001 +sg13 +(lp136 +sg15 +I3000 +sg16 +(F0.10999999999999943 +F0.10999999999999943 +F0 +tp137 +sbsS'IfThenElse' +p138 +(iConstructs +IfThenElse +p139 +(dp140 +g8 +I90 +sg9 +I20 +sg10 +(lp141 +F0.068999999999999839 +aF0.069500000000000298 +aF0.069000000000000311 +aF0.069000000000000311 +aF0.079499999999998211 +aF0.069500000000000381 +aF0.069500000000000381 +aF0.069000000000000131 +aF0.069000000000000131 +aF0.068999999999993025 +asg12 +F0.31 +sg13 +(lp142 +sg15 +I7500 +sg16 +(F0.068999999999993025 +F0.069999999999993179 +F0.0010000000000001564 +tp143 +sbsS'NormalInstanceAttribute' +p144 +(iLookups +NormalInstanceAttribute +p145 +(dp146 +g8 +I120 +sg9 +I20 +sg10 +(lp147 +F0.14949999999999991 +aF0.14950000000000035 +aF0.15900000000000017 +aF0.15900000000000017 +aF0.15950000000000006 +aF0.14949999999999869 +aF0.14949999999999869 +aF0.1594999999999967 +aF0.14949999999999833 +aF0.15950000000000381 +asg12 +F0.29999999999999999 +sg13 +(lp148 +sg15 +I5000 +sg16 +(F0.15950000000000381 +F0.16000000000000369 +F0.00049999999999990048 +tp149 +sbsS'BuiltinFunctionCalls' +p150 +(iCalls +BuiltinFunctionCalls +p151 +(dp152 +g8 +I85 +sg9 +I20 +sg10 +(lp153 +F0.089999999999999969 +aF0.10000000000000053 +aF0.080000000000000071 +aF0.079500000000001944 +aF0.10000000000000142 +aF0.079499999999998391 +aF0.079499999999998391 +aF0.089999999999996305 +aF0.079999999999998295 +aF0.079999999999998295 +asg12 +F0.40000000000000002 +sg13 +(lp154 +sg15 +I1500 +sg16 +(F0.079999999999998295 +F0.079999999999998295 +F0 +tp155 +sbsS'UnicodeMappings' +p156 +(iUnicode +UnicodeMappings +p157 +(dp158 +g8 +I36 +sg9 +I20 +sg10 +(lp159 +F0.21849999999999975 +aF0.22900000000000045 +aF0.21899999999999889 +aF0.20850000000000096 +aF0.2189999999999987 +aF0.22900000000000417 +aF0.21849999999999881 +aF0.20800000000000091 +aF0.2085000000000008 +aF0.21899999999999836 +asg12 +F0.10000000000000001 +sg13 +(lp160 +sg15 +I500 +sg16 +(F0.21899999999999836 +F0.21999999999999886 +F0.0010000000000005115 +tp161 +sbsS'CompareFloats' +p162 +(iNumbers +CompareFloats +p163 +(dp164 +g8 +I150 +sg9 +I20 +sg10 +(lp165 +F0.089499999999999968 +aF0.08949999999999983 +aF0.099999999999999645 +aF0.089999999999999858 +aF0.089499999999999955 +aF0.089500000000003507 +aF0.089499999999996402 +aF0.089499999999996402 +aF0.090000000000003411 +aF0.090000000000003411 +asg12 +F0.10000000000000001 +sg13 +(lp166 +sg15 +I3000 +sg16 +(F0.090000000000003411 +F0.090000000000003411 +F0 +tp167 +sbsS'StringMappings' +p168 +(iStrings +StringMappings +p169 +(dp170 +g8 +I36 +sg9 +I20 +sg10 +(lp171 +F0.34949999999999959 +aF0.35949999999999943 +aF0.33899999999999986 +aF0.34950000000000137 +aF0.34900000000000142 +aF0.34900000000000125 +aF0.34949999999999404 +aF0.34949999999999404 +aF0.34950000000000153 +aF0.34949999999999404 +asg12 +F0.10000000000000001 +sg13 +(lp172 +sg15 +I3500 +sg16 +(F0.34949999999999404 +F0.34999999999999432 +F0.00050000000000025577 +tp173 +sbsS'Recursion' +p174 +(iCalls +Recursion +p175 +(dp176 +g8 +I5 +sg9 +I20 +sg10 +(lp177 +F0.069499999999999854 +aF0.080000000000000071 +aF0.069500000000000201 +aF0.069500000000000201 +aF0.079999999999998295 +aF0.070000000000000284 +aF0.070000000000000284 +aF0.069500000000000381 +aF0.069500000000000034 +aF0.0800000000000054 +asg12 +F0.29999999999999999 +sg13 +(lp178 +sg15 +I2500 +sg16 +(F0.0800000000000054 +F0.0800000000000054 +F0 +tp179 +sbsS'SimpleLongArithmetic' +p180 +(iArithmetic +SimpleLongArithmetic +p181 +(dp182 +g8 +I110 +sg9 +I20 +sg10 +(lp183 +F0.079500000000000084 +aF0.080000000000000071 +aF0.079500000000001944 +aF0.079999999999998295 +aF0.080000000000001847 +aF0.069500000000000381 +aF0.069500000000000034 +aF0.0800000000000054 +aF0.079999999999998295 +aF0.079999999999998295 +asg12 +F0.29999999999999999 +sg13 +(lp184 +sg15 +I1500 +sg16 +(F0.079999999999998295 +F0.079999999999998295 +F0 +tp185 +sbsS'UnicodeProperties' +p186 +(iUnicode +UnicodeProperties +p187 +(dp188 +g8 +I40 +sg9 +I20 +sg10 +(lp189 +F0.13799999999999968 +aF0.12799999999999895 +aF0.12799999999999906 +aF0.13800000000000043 +aF0.12799999999999515 +aF0.12799999999999515 +aF0.13800000000000062 +aF0.12800000000000261 +aF0.12799999999999515 +aF0.12800000000001005 +asg12 +F0.10000000000000001 +sg13 +(lp190 +sg15 +I5000 +sg16 +(F0.12800000000001005 +F0.13000000000000966 +F0.0019999999999996019 +tp191 +sbsS'SimpleIntFloatArithmetic' +p192 +(iArithmetic +SimpleIntFloatArithmetic +p193 +(dp194 +g8 +I110 +sg9 +I20 +sg10 +(lp195 +F0.08949999999999983 +aF0.089499999999999871 +aF0.088999999999999885 +aF0.089000000000003437 +aF0.088999999999999885 +aF0.089500000000003507 +aF0.089500000000003507 +aF0.088999999999996152 +aF0.088999999999996152 +aF0.089500000000003507 +asg12 +F0.29999999999999999 +sg13 +(lp196 +sg15 +I6000 +sg16 +(F0.089500000000003507 +F0.090000000000003411 +F0.00049999999999990048 +tp197 +sbsS'CreateUnicodeWithConcat' +p198 +(iUnicode +CreateUnicodeWithConcat +p199 +(dp200 +g8 +I50 +sg9 +I20 +sg10 +(lp201 +F0.15949999999999992 +aF0.14949999999999858 +aF0.14949999999999858 +aF0.14950000000000224 +aF0.1494999999999985 +aF0.14949999999999869 +aF0.14949999999999833 +aF0.14949999999999833 +aF0.17000000000000171 +aF0.14949999999999869 +asg12 +F0.10000000000000001 +sg13 +(lp202 +sg15 +I4000 +sg16 +(F0.14949999999999869 +F0.14999999999999858 +F0.00049999999999990048 +tp203 +sbsS'ListSlicing' +p204 +(iLists +ListSlicing +p205 +(dp206 +g8 +I175 +sg9 +I20 +sg10 +(lp207 +F0.080000000000000071 +aF0.069500000000000298 +aF0.060000000000000497 +aF0.059999999999998721 +aF0.059999999999998721 +aF0.05950000000000237 +aF0.069500000000000381 +aF0.05950000000000237 +aF0.079999999999998295 +aF0.060000000000002274 +asg12 +F0.40000000000000002 +sg13 +(lp208 +sg15 +I20 +sg16 +(F0.060000000000002274 +F0.060000000000002274 +F0 +tp209 +sbsS'CompareLongs' +p210 +(iNumbers +CompareLongs +p211 +(dp212 +g8 +I150 +sg9 +I20 +sg10 +(lp213 +F0.099500000000000088 +aF0.089499999999999871 +aF0.089499999999999871 +aF0.079500000000001944 +aF0.089999999999999858 +aF0.090000000000003411 +aF0.090000000000003411 +aF0.089500000000003507 +aF0.089500000000003507 +aF0.089499999999996055 +asg12 +F0.10000000000000001 +sg13 +(lp214 +sg15 +I3000 +sg16 +(F0.089499999999996055 +F0.089999999999996305 +F0.00050000000000025577 +tp215 +sbsS'SimpleIntegerArithmetic' +p216 +(iArithmetic +SimpleIntegerArithmetic +p217 +(dp218 +g8 +I110 +sg9 +I20 +sg10 +(lp219 +F0.089499999999999871 +aF0.089499999999999871 +aF0.089499999999999774 +aF0.099500000000001518 +aF0.099499999999997785 +aF0.089499999999996402 +aF0.089499999999996402 +aF0.089499999999996402 +aF0.089499999999996055 +aF0.088999999999996152 +asg12 +F0.29999999999999999 +sg13 +(lp220 +sg15 +I6000 +sg16 +(F0.088999999999996152 +F0.089999999999996305 +F0.0010000000000001564 +tp221 +sbsS'SpecialClassAttribute' +p222 +(iLookups +SpecialClassAttribute +p223 +(dp224 +g8 +I120 +sg9 +I20 +sg10 +(lp225 +F0.14949999999999944 +aF0.14950000000000035 +aF0.15950000000000025 +aF0.1494999999999985 +aF0.15950000000000025 +aF0.15950000000000381 +aF0.15950000000000381 +aF0.14949999999999869 +aF0.14949999999999869 +aF0.15949999999999634 +asg12 +F0.29999999999999999 +sg13 +(lp226 +sg15 +I5000 +sg16 +(F0.15949999999999634 +F0.15999999999999659 +F0.00050000000000025577 +tp227 +sbsS'SecondImport' +p228 +(iImports +SecondImport +p229 +(dp230 +g8 +I25 +sg9 +I20 +sg10 +(lp231 +F0.060000000000000053 +aF0.059999999999998721 +aF0.059999999999998721 +aF0.059999999999998721 +aF0.060000000000002274 +aF0.059499999999994911 +aF0.05950000000000237 +aF0.060000000000002274 +aF0.059999999999995168 +aF0.059999999999995168 +asg12 +F0.10000000000000001 +sg13 +(lp232 +sg15 +I1000 +sg16 +(F0.059999999999995168 +F0.059999999999995168 +F0 +tp233 +sbsS'TupleSlicing' +p234 +(iTuples +TupleSlicing +p235 +(dp236 +g8 +I5250 +sg9 +I20 +sg10 +(lp237 +F0.099499999999999617 +aF0.10999999999999943 +aF0.099499999999997785 +aF0.099500000000001337 +aF0.10999999999999943 +aF0.10999999999999943 +aF0.099499999999994065 +aF0.099500000000001518 +aF0.10999999999999943 +aF0.10999999999999943 +asg12 +F0.31 +sg13 +(lp238 +sg15 +I20 +sg16 +(F0.10999999999999943 +F0.10999999999999943 +F0 +tp239 +sbsS'SimpleFloatArithmetic' +p240 +(iArithmetic +SimpleFloatArithmetic +p241 +(dp242 +g8 +I110 +sg9 +I20 +sg10 +(lp243 +F0.089499999999999871 +aF0.099499999999999561 +aF0.089499999999999774 +aF0.099499999999997785 +aF0.099500000000001518 +aF0.099500000000001518 +aF0.099500000000001518 +aF0.088999999999996152 +aF0.099500000000001518 +aF0.10949999999999953 +asg12 +F0.29999999999999999 +sg13 +(lp244 +sg15 +I5000 +sg16 +(F0.10949999999999953 +F0.10999999999999943 +F0.00049999999999990048 +tp245 +sbsS'StringPredicates' +p246 +(iStrings +StringPredicates +p247 +(dp248 +g8 +I70 +sg9 +I20 +sg10 +(lp249 +F0.1884999999999995 +aF0.17850000000000152 +aF0.18799999999999759 +aF0.18850000000000139 +aF0.17849999999999966 +aF0.17799999999999977 +aF0.17800000000000687 +aF0.18849999999999767 +aF0.19849999999999568 +aF0.18850000000001188 +asg12 +F0.10000000000000001 +sg13 +(lp250 +sg15 +I4000 +sg16 +(F0.18850000000001188 +F0.19000000000001194 +F0.0015000000000000568 +tp251 +sbsS'BuiltinMethodLookup' +p252 +(iLookups +BuiltinMethodLookup +p253 +(dp254 +g8 +I150 +sg9 +I20 +sg10 +(lp255 +F0.16000000000000003 +aF0.15999999999999925 +aF0.15950000000000014 +aF0.15950000000000025 +aF0.16000000000000014 +aF0.14949999999999833 +aF0.14950000000000579 +aF0.16000000000000369 +aF0.15949999999999634 +aF0.15949999999999634 +asg12 +F0.29999999999999999 +sg13 +(lp256 +sg15 +I3500 +sg16 +(F0.15949999999999634 +F0.15999999999999659 +F0.00050000000000025577 +tp257 +sbsS'SimpleComplexArithmetic' +p258 +(iArithmetic +SimpleComplexArithmetic +p259 +(dp260 +g8 +I110 +sg9 +I20 +sg10 +(lp261 +F0.12999999999999989 +aF0.099499999999999658 +aF0.099999999999997868 +aF0.099999999999997868 +aF0.099500000000001337 +aF0.099500000000001518 +aF0.10949999999999953 +aF0.10000000000000142 +aF0.10000000000000142 +aF0.10000000000000142 +asg12 +F0.29999999999999999 +sg13 +(lp262 +sg15 +I2000 +sg16 +(F0.10000000000000142 +F0.10000000000000142 +F0 +tp263 +sbsS'ConcatUnicode' +p264 +(iUnicode +ConcatUnicode +p265 +(dp266 +g8 +I50 +sg9 +I20 +sg10 +(lp267 +F0.11950000000000011 +aF0.10949999999999944 +aF0.10949999999999944 +aF0.10949999999999935 +aF0.10899999999999946 +aF0.10899999999999928 +aF0.10899999999999928 +aF0.10949999999999953 +aF0.10949999999999953 +aF0.10949999999999953 +asg12 +F0.10000000000000001 +sg13 +(lp268 +sg15 +I3000 +sg16 +(F0.10949999999999953 +F0.10999999999999943 +F0.00049999999999990048 +tp269 +sbsS'CompareInternedStrings' +p270 +(iStrings +CompareInternedStrings +p271 +(dp272 +g8 +I50 +sg9 +I20 +sg10 +(lp273 +F0.078999999999999959 +aF0.079000000000000098 +aF0.079000000000000098 +aF0.078500000000001791 +aF0.088999999999999885 +aF0.088999999999996152 +aF0.098999999999994162 +aF0.078999999999998141 +aF0.078999999999998141 +aF0.078999999999998488 +asg12 +F0.10000000000000001 +sg13 +(lp274 +sg15 +I10000 +sg16 +(F0.078999999999998488 +F0.079999999999998295 +F0.00099999999999980096 +tp275 +sbsS'CompareStrings' +p276 +(iStrings +CompareStrings +p277 +(dp278 +g8 +I50 +sg9 +I20 +sg10 +(lp279 +F0.10899999999999987 +aF0.1089999999999994 +aF0.099000000000001448 +aF0.099000000000001448 +aF0.10899999999999946 +aF0.10899999999999928 +aF0.10899999999999928 +aF0.10899999999999962 +aF0.099000000000001614 +aF0.10899999999999928 +asg12 +F0.20000000000000001 +sg13 +(lp280 +sg15 +I10000 +sg16 +(F0.10899999999999928 +F0.10999999999999943 +F0.0010000000000001564 +tp281 +sbsS'SmallTuples' +p282 +(iTuples +SmallTuples +p283 +(dp284 +g8 +I60 +sg9 +I20 +sg10 +(lp285 +F0.1094999999999994 +aF0.10949999999999944 +aF0.11000000000000298 +aF0.10949999999999935 +aF0.10950000000000308 +aF0.10999999999999943 +aF0.10949999999999953 +aF0.10949999999999918 +aF0.10950000000000663 +aF0.10949999999999953 +asg12 +F0.29999999999999999 +sg13 +(lp286 +sg15 +I4000 +sg16 +(F0.10949999999999953 +F0.10999999999999943 +F0.00049999999999990048 +tp287 +sbsS'NormalClassAttribute' +p288 +(iLookups +NormalClassAttribute +p289 +(dp290 +g8 +I120 +sg9 +I20 +sg10 +(lp291 +F0.16949999999999993 +aF0.16949999999999993 +aF0.15950000000000025 +aF0.15950000000000025 +aF0.16950000000000182 +aF0.16950000000000182 +aF0.16950000000000182 +aF0.16950000000000146 +aF0.16949999999999435 +aF0.16950000000000182 +asg12 +F0.29999999999999999 +sg13 +(lp292 +sg15 +I5000 +sg16 +(F0.16950000000000182 +F0.17000000000000171 +F0.00049999999999990048 +tp293 +sbsS'CreateInstances' +p294 +(iInstances +CreateInstances +p295 +(dp296 +g8 +I14 +sg9 +I20 +sg10 +(lp297 +F0.1399999999999999 +aF0.14000000000000057 +aF0.12900000000000081 +aF0.13949999999999693 +aF0.12949999999999892 +aF0.14000000000000057 +aF0.14000000000000057 +aF0.13950000000000032 +aF0.12950000000000267 +aF0.12950000000000267 +asg12 +F0.20000000000000001 +sg13 +(lp298 +sg15 +I3000 +sg16 +(F0.12950000000000267 +F0.13000000000000256 +F0.00049999999999990048 +tp299 +sbsS'DictWithStringKeys' +p300 +(iDict +DictWithStringKeys +p301 +(dp302 +g8 +I60 +sg9 +I20 +sg10 +(lp303 +F0.10899999999999987 +aF0.098499999999999588 +aF0.11899999999999915 +aF0.10899999999999946 +aF0.10899999999999946 +aF0.098999999999994162 +aF0.098999999999994162 +aF0.10899999999999962 +aF0.10899999999999928 +aF0.10899999999999928 +asg12 +F0.10000000000000001 +sg13 +(lp304 +sg15 +I10000 +sg16 +(F0.10899999999999928 +F0.10999999999999943 +F0.0010000000000001564 +tp305 +sbsS'DictCreation' +p306 +(iDict +DictCreation +p307 +(dp308 +g8 +I50 +sg9 +I20 +sg10 +(lp309 +F0.09000000000000008 +aF0.079499999999999987 +aF0.089499999999999774 +aF0.089499999999999955 +aF0.089499999999999774 +aF0.079500000000005497 +aF0.079499999999998391 +aF0.089999999999996305 +aF0.089999999999996305 +aF0.089500000000003507 +asg12 +F0.29999999999999999 +sg13 +(lp310 +sg15 +I3000 +sg16 +(F0.089500000000003507 +F0.090000000000003411 +F0.00049999999999990048 +tp311 +sbsS'StringSlicing' +p312 +(iStrings +StringSlicing +p313 +(dp314 +g8 +I35 +sg9 +I20 +sg10 +(lp315 +F0.099499999999999617 +aF0.099499999999999658 +aF0.089000000000003437 +aF0.099500000000001518 +aF0.098999999999997895 +aF0.099000000000001614 +aF0.10949999999999953 +aF0.099500000000001171 +aF0.099500000000001171 +aF0.099500000000008984 +asg12 +F0.10000000000000001 +sg13 +(lp316 +sg15 +I5000 +sg16 +(F0.099500000000008984 +F0.10000000000000853 +F0.0004999999999995453 +tp317 +sbssS'name' +p318 +S'python2.4.2.bench' +p319 +sS'roundtime' +p320 +F6.5750000000000002 +sg9 +I20 +sg12 +F0.31 +sg15 +I10 +sb. \ No newline at end of file From ale at codespeak.net Sun Apr 9 13:28:34 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 9 Apr 2006 13:28:34 +0200 (CEST) Subject: [pypy-svn] r25617 - pypy/dist/pypy/translator/microbench/pybench Message-ID: <20060409112834.843CC10201@code0.codespeak.net> Author: ale Date: Sun Apr 9 13:28:32 2006 New Revision: 25617 Modified: pypy/dist/pypy/translator/microbench/pybench/pybench.py Log: Added the beginings of a html presentation of the benchmark result Modified: pypy/dist/pypy/translator/microbench/pybench/pybench.py ============================================================================== --- pypy/dist/pypy/translator/microbench/pybench/pybench.py (original) +++ pypy/dist/pypy/translator/microbench/pybench/pybench.py Sun Apr 9 13:28:32 2006 @@ -260,6 +260,73 @@ (compare_to.name,compare_to.rounds,compare_to.warp) print + def html_stat(self, compare_to=None, hidenoise=0): + + from py.xml import html + + if not compare_to: + table = html.table(html.thead( + html.tr([ html.th(x, align='left', mochi_format = y) + for (x,y) in [('Tests','str'), ('per run','float'), + ('per oper.', 'float'), ('overhead', 'float')])), + id = "sortable_table") + + tests = self.tests.items() + tests.sort() + for name,t in tests: + avg,op_avg,ov_avg = t.stat() + table.append(html.tr( html.td(name), + html.td((avg*1000.0), + html.td(op_avg*1000000.0), + html.td(ov_avg*1000.0) + ) + table.append(html.tr('Average round time %s' % (self.roundtime * 1000.0)) + return table + else: + table = html.table(html.thead( + html.tr([ html.th(x, align='left', mochi_format = y) + for (x,y) in [('Tests','str'), ('per run','float'), + ('per oper.', 'float'), ('diff', 'float')])), + id = "sortable_table") + tests = self.tests.items() + tests.sort() + compatible = 1 + for name,t in tests: + avg,op_avg,ov_avg = t.stat() + try: + other = compare_to.tests[name] + except KeyError: + other = None + if other and other.version == t.version and \ + other.operations == t.operations: + avg1,op_avg1,ov_avg1 = other.stat() + qop_avg = (op_avg/op_avg1-1.0)*100.0 + if hidenoise and abs(qop_avg) < 10: + qop_avg = '' + else: + qop_avg = '%+7.2f%%' % qop_avg + else: + qavg,qop_avg = 'n/a', 'n/a' + compatible = 0 + table.append(html.tr( html.td(name), + html.td((avg*1000.0), + html.td(op_avg*1000000.0), + html.td(qop_avg) + ) + if compatible and compare_to.roundtime > 0 and \ + compare_to.version == self.version: + table.append(html.tr( + html.td('Average round time'), + html.td(self.roundtime * 1000.0), + html.td(''), + html.td('%+7.2f%%'% (((self.roundtime*self.warp)/ + (compare_to.roundtime*compare_to.warp)-1.0)*100.0))) + else: + table.append(html.tr( + html.td('Average round time'), + html.td(self.roundtime * 1000.0))) + return table + def print_machine(): import platform From hpk at codespeak.net Sun Apr 9 13:35:10 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sun, 9 Apr 2006 13:35:10 +0200 (CEST) Subject: [pypy-svn] r25618 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060409113510.81EF710207@code0.codespeak.net> Author: hpk Date: Sun Apr 9 13:35:10 2006 New Revision: 25618 Modified: pypy/dist/pypy/translator/stackless/ (props changed) pypy/dist/pypy/translator/stackless/test/ (props changed) pypy/dist/pypy/translator/stackless/transform.py Log: progress ... (thanks armin) now we segfault at least + fixeol Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 13:35:10 2006 @@ -293,12 +293,12 @@ model.Constant(len(self.resume_points)+1, lltype.Signed)], varoftype(lltype.Void))) # XXX add returntypes - FUNCTYPE = lltype.FuncType([], lltype.Signed) - funcptr = lltype.functionptr(FUNCTYPE, "", graph=self.curr_graph) - #saveops.append(model.SpaceOperation( - # "setfield", [var_header, model.Constant("function", lltype.Void), - # model.Constant(llmemory.fakeaddress(funcptr), llmemory.Address)], - # varoftype(lltype.Void))) + + funcptr = rtyper.type_system.getcallable(self.curr_graph) + saveops.append(model.SpaceOperation( + "setfield", [var_header, model.Constant("function", lltype.Void), + model.Constant(llmemory.fakeaddress(funcptr), llmemory.Address)], + varoftype(lltype.Void))) type_repr = rclass.get_type_repr(rtyper) c_unwindexception = model.Constant(type_repr.convert_const(code.UnwindException), type_repr.lowleveltype) From mwh at codespeak.net Sun Apr 9 13:39:02 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 13:39:02 +0200 (CEST) Subject: [pypy-svn] r25619 - in pypy/extradoc/talk/accu2006: . accu-2006.key accu-2006.key/thumbs Message-ID: <20060409113902.34E0310212@code0.codespeak.net> Author: mwh Date: Sun Apr 9 13:38:39 2006 New Revision: 25619 Modified: pypy/extradoc/talk/accu2006/accu-2006.key/index.apxl.gz pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/mt0-4.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st1-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st12-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st2.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-2.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-3.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-4.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-5.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st4.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st5.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7.tiff pypy/extradoc/talk/accu2006/accu-2006.pdf Log: another round of revisions to my ACCU slides after feedback from the leysin sprinters. the "translation aspects" slide is not quite done, everything else should be good now. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/index.apxl.gz ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/mt0-4.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st1-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st12-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st2.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-2.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-3.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-4.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-5.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st4.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st5.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.pdf ============================================================================== Binary files. No diff available. From mwh at codespeak.net Sun Apr 9 14:05:57 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 14:05:57 +0200 (CEST) Subject: [pypy-svn] r25620 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060409120557.5B57810210@code0.codespeak.net> Author: mwh Date: Sun Apr 9 14:05:56 2006 New Revision: 25620 Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py Log: (hpk, mwh) small fixes to get the compiled binary to not segfault (a cast_pointer becomes a getsubstruct and use boehm for now to avoid a gc issue) also some commented-out code to run the code we generate on top of llinterp, which crashes in probably the "same" way as the compiled version does with refcounting Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Sun Apr 9 14:05:56 2006 @@ -2,6 +2,7 @@ import os from pypy.translator.stackless.transform import StacklessTransfomer from pypy.translator.c.genc import CStandaloneBuilder +from pypy.translator.c import gc from pypy.rpython.memory.gctransform import varoftype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.translator import TranslationContext, graphof @@ -43,7 +44,7 @@ def example(x): return g(x) + 1 res = run_stackless_function(example, example, g) - assert res == "6" + assert res.strip() == "3" def run_stackless_function(fn, *stacklessfuncs): def entry_point(argv): @@ -81,7 +82,14 @@ if conftest.option.view: t.view() - cbuilder = CStandaloneBuilder(t, entry_point) +## r_list_of_strings = t.rtyper.getrepr(s_list_of_strings) +## ll_list = r_list_of_strings.convert_const(['']) +## from pypy.rpython.llinterp import LLInterpreter +## interp = LLInterpreter(t.rtyper) +## interp.eval_graph(graphof(t, entry_point), [ll_list]) +## return + + cbuilder = CStandaloneBuilder(t, entry_point, gcpolicy=gc.BoehmGcPolicy) cbuilder.generate_source() cbuilder.compile() return cbuilder.cmdexec('') Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 14:05:56 2006 @@ -281,7 +281,7 @@ var_header = varoftype(lltype.Ptr(STATE_HEADER)) # XXX should this be getsubstruct? - saveops.append(model.SpaceOperation("cast_pointer", [frame_state_var], var_header)) + saveops.append(model.SpaceOperation("getsubstruct", [frame_state_var, model.Constant("header", lltype.Void)], var_header)) saveops.append(model.SpaceOperation( "direct_call", @@ -301,8 +301,12 @@ varoftype(lltype.Void))) type_repr = rclass.get_type_repr(rtyper) - c_unwindexception = model.Constant(type_repr.convert_const(code.UnwindException), type_repr.lowleveltype) - save_state_block.closeblock(model.Link([c_unwindexception, var_exc], + c_unwindexception = model.Constant(type_repr.convert_const(code.UnwindException), etype) + if not hasattr(self.curr_graph.exceptblock.inputargs[0], 'concretetype'): + self.curr_graph.exceptblock.inputargs[0].concretetype = etype + if not hasattr(self.curr_graph.exceptblock.inputargs[1], 'concretetype'): + self.curr_graph.exceptblock.inputargs[1].concretetype = evalue + save_state_block.closeblock(model.Link([c_unwindexception, var_unwind_exception], self.curr_graph.exceptblock)) self.translator.rtyper._convert_link(save_state_block, save_state_block.exits[0]) return save_state_block, frame_type From antocuni at codespeak.net Sun Apr 9 14:18:34 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sun, 9 Apr 2006 14:18:34 +0200 (CEST) Subject: [pypy-svn] r25621 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060409121834.6EEA510220@code0.codespeak.net> Author: antocuni Date: Sun Apr 9 14:18:17 2006 New Revision: 25621 Added: pypy/dist/pypy/translator/cli/test/autopath.py (contents, props changed) Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py pypy/dist/pypy/translator/cli/test/test_oo.py Log: The algorithm for rendering pending nodes has been refactored. Now every node (such as Function or Class) can enqueue new nodes that it depends on, instead of having static dependencies as it was before. Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Sun Apr 9 14:18:17 2006 @@ -1,5 +1,4 @@ from pypy.translator.cli.node import Node -from pypy.translator.cli.function import Function from pypy.translator.cli.cts import CTS class Class(Node): @@ -9,6 +8,15 @@ self.classdef = classdef self.namespace, self.name = self.cts.split_class_name(classdef._name) + if not self.is_root(classdef): + self.db.pending_class(classdef._superclass) + + def __hash__(self): + return hash(self.classdef) + + def __eq__(self, other): + return self.classdef == other.classdef + def is_root(classdef): return classdef._superclass is None is_root = staticmethod(is_root) @@ -26,7 +34,10 @@ def render(self, ilasm): if self.is_root(self.classdef): return - + + if self.db.class_name(self.classdef) is not None: + return # already rendered + self.ilasm = ilasm if self.namespace: ilasm.begin_namespace(self.namespace) @@ -38,9 +49,10 @@ # TODO: should the .ctor set the default values? self._ctor() + # lazy import to avoid circular dependencies + import pypy.translator.cli.function as function for m_name, m_meth in self.classdef._methods.iteritems(): - # TODO: handle class methods and attributes - f = Function(self.db, m_meth.graph, m_name, is_method = True) + f = function.Function(self.db, m_meth.graph, m_name, is_method = True) f.render(ilasm) ilasm.end_class() @@ -48,10 +60,12 @@ if self.namespace: ilasm.end_namespace() + self.db.record_class(self.classdef, self.name) + + def _ctor(self): self.ilasm.begin_function('.ctor', [], 'void', False, 'specialname', 'rtspecialname', 'instance') self.ilasm.opcode('ldarg.0') self.ilasm.call('instance void %s::.ctor()' % self.get_base_class()) self.ilasm.opcode('ret') self.ilasm.end_function() - Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Sun Apr 9 14:18:17 2006 @@ -49,7 +49,7 @@ def lltype_to_cts(self, t): if isinstance(t, Instance): name = t._name - self.db.classes.add(t) + self.db.pending_class(t) return 'class %s' % name return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Sun Apr 9 14:18:17 2006 @@ -1,3 +1,6 @@ +from pypy.translator.cli.cts import CTS +from pypy.translator.cli.function import Function +from pypy.translator.cli.class_ import Class from pypy.rpython.ootypesystem import ootype try: @@ -10,20 +13,38 @@ class LowLevelDatabase(object): def __init__(self): - self.classes = set() - self.pending_graphs = [] + self._pending_nodes = set() + self._rendered_nodes = set() + self.classes = {} # classdef --> class_name self.functions = {} # graph --> function_name self.methods = {} # graph --> method_name self.consts = {} # value --> const_name + def pending_function(self, graph): + self.pending_node(Function(self, graph)) + + def pending_class(self, classdef): + self.pending_node(Class(self, classdef)) + + def pending_node(self, node): + if node in self._pending_nodes or node in self._rendered_nodes: + return + self._pending_nodes.add(node) + def record_function(self, graph, name): self.functions[graph] = name + def record_class(self, classdef, name): + self.classes[classdef] = name + def function_name(self, graph): return self.functions.get(graph, None) + def class_name(self, classdef): + return self.classes.get(classdef, None) + def record_const(self, value): - const = AbstractConst.make(value) + const = AbstractConst.make(self, value) name = const.get_name(len(self.consts)) self.consts[const] = name return '%s.%s::%s' % (CONST_NAMESPACE, CONST_CLASS, name) @@ -52,11 +73,11 @@ class AbstractConst(object): - def make(const): + def make(db, const): if isinstance(const, ootype._view): const = const._inst if isinstance(const, ootype._instance): - return InstanceConst(const) + return InstanceConst(db, const) else: assert False, 'Unknown constant: %s' % const make = staticmethod(make) @@ -71,7 +92,8 @@ pass class InstanceConst(AbstractConst): - def __init__(self, obj): + def __init__(self, db, obj): + self.cts = CTS(db) self.obj = obj def get_name(self, n): @@ -79,9 +101,16 @@ return '%s_%d' % (name, n) def get_type(self): - return 'class %s' % self.obj._TYPE._name + return self.cts.lltype_to_cts(self.obj._TYPE) + #return 'class %s' % self.obj._TYPE._name def init(self, ilasm): classdef = self.obj._TYPE ilasm.new('instance void class %s::.ctor()' % classdef._name) - # TODO: initialize fields + while classdef is not None: + for name, (type_, value) in classdef._fields.iteritems(): + if isinstance(type_, ootype.StaticMethod): + continue + elif type_ is ootype.Class: + continue + classdef = classdef._superclass Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Sun Apr 9 14:18:17 2006 @@ -12,6 +12,7 @@ from pypy.translator.cli.opcodes import opcodes from pypy.translator.cli.metavm import InstructionList, Generator from pypy.translator.cli.node import Node +from pypy.translator.cli.class_ import Class from pypy.tool.ansi_print import ansi_log import py @@ -33,6 +34,12 @@ def get_name(self): return self.name + def __hash__(self): + return hash(self.graph) + + def __eq__(self, other): + return self.graph == other.graph + def _is_return_block(self, block): return (not block.exits) and len(block.inputargs) == 1 @@ -40,6 +47,9 @@ return (not block.exits) and len(block.inputargs) == 2 def render(self, ilasm): + if self.db.function_name(self.graph) is not None and not self.is_method: + return # already rendered + self.ilasm = ilasm graph = self.graph returntype, returnvar = self.cts.llvar_to_cts(graph.getreturnvar()) @@ -67,7 +77,7 @@ self.ilasm.begin_try() for op in block.operations: - self._search_for_classes(op) + #self._search_for_classes(op) self._render_op(op) if self._is_raise_block(block): @@ -200,14 +210,13 @@ if isinstance(arg, flowmodel.Variable): lltype = arg.concretetype elif isinstance(arg, flowmodel.Constant): - lltype = arg.value if isinstance(lltype, ootype._view) and isinstance(lltype._inst, ootype._instance): lltype = lltype._inst._TYPE if isinstance(lltype, ootype.Instance): - self.db.classes.add(lltype) + self.db.pending_class(lltype) def _render_op(self, op): instr_list = opcodes.get(op.opname, None) @@ -248,7 +257,7 @@ self.ilasm.opcode(instr, *args) def call(self, graph, func_name): - self.db.pending_graphs.append(graph) + self.db.pending_function(graph) self.ilasm.call(func_name) def new(self, obj): Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Sun Apr 9 14:18:17 2006 @@ -47,41 +47,22 @@ # methods are rendered twice, once within the class and once # as an external function. Fix this. self.gen_entrypoint() - self.find_superclasses() - self.gen_classes() - self.gen_functions() + self.gen_pendings() self.db.gen_constants(self.ilasm) + self.gen_pendings() out.close() return self.tmpfile.strpath def gen_entrypoint(self): if self.entrypoint: - self.db.pending_graphs += self.entrypoint.render(self.ilasm) + self.entrypoint.db = self.db + self.db.pending_node(self.entrypoint) else: - self.db.pending_graphs.append(self.translator.graphs[0]) + self.db.pending_function(self.translator.graphs[0]) - self.gen_functions() + def gen_pendings(self): + while self.db._pending_nodes: + node = self.db._pending_nodes.pop() + node.render(self.ilasm) - def gen_functions(self): - while self.db.pending_graphs: - graph = self.db.pending_graphs.pop() - if self.db.function_name(graph) is None: - f = Function(self.db, graph) - f.render(self.ilasm) - def find_superclasses(self): - classes = set() - pendings = self.db.classes - - while pendings: - classdef = pendings.pop() - if classdef not in classes and classdef is not None: - classes.add(classdef) - pendings.add(classdef._superclass) - - self.db.classes = classes - - def gen_classes(self): - for classdef in self.db.classes: - c = Class(self.db, classdef) - c.render(self.ilasm) Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Sun Apr 9 14:18:17 2006 @@ -84,8 +84,8 @@ class _SetField(MicroInstruction): def render(self, generator, op): this, field, value = op.args - if field.value == 'meta': - return # TODO +## if field.value == 'meta': +## return # TODO generator.load(this) generator.load(value) Added: pypy/dist/pypy/translator/cli/test/autopath.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/test/autopath.py Sun Apr 9 14:18:17 2006 @@ -0,0 +1,112 @@ +""" +self cloning, automatic path configuration + +copy this into any subdirectory of pypy from which scripts need +to be run, typically all of the test subdirs. +The idea is that any such script simply issues + + import autopath + +and this will make sure that the parent directory containing "pypy" +is in sys.path. + +If you modify the master "autopath.py" version (in pypy/tool/autopath.py) +you can directly run it which will copy itself on all autopath.py files +it finds under the pypy root directory. + +This module always provides these attributes: + + pypydir pypy root directory path + this_dir directory where this autopath.py resides + +""" + + +def __dirinfo(part): + """ return (partdir, this_dir) and insert parent of partdir + into sys.path. If the parent directories don't have the part + an EnvironmentError is raised.""" + + import sys, os + try: + head = this_dir = os.path.realpath(os.path.dirname(__file__)) + except NameError: + head = this_dir = os.path.realpath(os.path.dirname(sys.argv[0])) + + while head: + partdir = head + head, tail = os.path.split(head) + if tail == part: + break + else: + raise EnvironmentError, "'%s' missing in '%r'" % (partdir, this_dir) + + pypy_root = os.path.join(head, '') + try: + sys.path.remove(head) + except ValueError: + pass + sys.path.insert(0, head) + + munged = {} + for name, mod in sys.modules.items(): + fn = getattr(mod, '__file__', None) + if '.' in name or not isinstance(fn, str): + continue + newname = os.path.splitext(os.path.basename(fn))[0] + if not newname.startswith(part + '.'): + continue + path = os.path.join(os.path.dirname(os.path.realpath(fn)), '') + if path.startswith(pypy_root) and newname != part: + modpaths = os.path.normpath(path[len(pypy_root):]).split(os.sep) + if newname != '__init__': + modpaths.append(newname) + modpath = '.'.join(modpaths) + if modpath not in sys.modules: + munged[modpath] = mod + + for name, mod in munged.iteritems(): + if name not in sys.modules: + sys.modules[name] = mod + if '.' in name: + prename = name[:name.rfind('.')] + postname = name[len(prename)+1:] + if prename not in sys.modules: + __import__(prename) + if not hasattr(sys.modules[prename], postname): + setattr(sys.modules[prename], postname, mod) + + return partdir, this_dir + +def __clone(): + """ clone master version of autopath.py into all subdirs """ + from os.path import join, walk + if not this_dir.endswith(join('pypy','tool')): + raise EnvironmentError("can only clone master version " + "'%s'" % join(pypydir, 'tool',_myname)) + + + def sync_walker(arg, dirname, fnames): + if _myname in fnames: + fn = join(dirname, _myname) + f = open(fn, 'rwb+') + try: + if f.read() == arg: + print "checkok", fn + else: + print "syncing", fn + f = open(fn, 'w') + f.write(arg) + finally: + f.close() + s = open(join(pypydir, 'tool', _myname), 'rb').read() + walk(pypydir, sync_walker, s) + +_myname = 'autopath.py' + +# set guaranteed attributes + +pypydir, this_dir = __dirinfo('pypy') + +if __name__ == '__main__': + __clone() Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Sun Apr 9 14:18:17 2006 @@ -28,25 +28,24 @@ self.x = x def compute(self): - return self.x + 1 + return self.x class Derived(Base): def __init__(self, x): - Base.__init__(self, x) + Base.__init__(self, x+42) -def foo(cls, arg): - obj = cls(arg) - return obj.compute() +def foo(cls, x): + return cls(x).compute() def bar(x, y): - obj = Base(x) - return obj.compute() + b = Derived(x) + return b.compute() # return foo(Base, x) + foo(Derived, y) f = compile_function(bar, [int, int]) -##try: -## pass -##except py.test.Item.Skipped: -## print 'Test skipped' +try: + check(bar, f, 42, 13) +except py.test.Item.Skipped: + print 'Test skipped' Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Sun Apr 9 14:18:17 2006 @@ -7,6 +7,7 @@ from pypy.translator.translator import TranslationContext from pypy.translator.cli.option import getoption from pypy.translator.cli.gencli import GenCli +from pypy.translator.cli.function import Function from pypy.translator.cli.node import Node from pypy.translator.cli.cts import CTS from pypy.translator.cli.database import LowLevelDatabase @@ -33,6 +34,7 @@ def __init__(self, graph_to_call): self.graph = graph_to_call + self.db = None def get_name(self): return 'main' @@ -56,7 +58,7 @@ ilasm.call('void class [mscorlib]System.Console::WriteLine(%s)' % ret_type) ilasm.opcode('ret') ilasm.end_function() - return [self.graph] + self.db.pending_function(self.graph) def __convert_method(self, arg_type): _conv = { @@ -126,8 +128,10 @@ return None self.__check_helper("ilasm") - retval = subprocess.call(["ilasm", tmpfile], stdout=subprocess.PIPE) - assert retval == 0, 'ilasm failed to assemble %s (%s)' % (self.graph.name, tmpfile) + ilasm = subprocess.Popen(["ilasm", tmpfile], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = ilasm.communicate() + retval = ilasm.wait() + assert retval == 0, 'ilasm failed to assemble %s (%s):\n%s' % (self.graph.name, tmpfile, stdout) return tmpfile.replace('.il', '.exe') def __call__(self, *args): Modified: pypy/dist/pypy/translator/cli/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cli/test/test_oo.py Sun Apr 9 14:18:17 2006 @@ -9,6 +9,8 @@ class MyClass: + INCREMENT = 1 + def __init__(self, x, y): self.x = x self.y = y @@ -23,7 +25,16 @@ return x*y static_meth = staticmethod(static_meth) +## def class_meth(cls, x, y): +## return x*y + cls.INCREMENT +## class_meth = classmethod(class_meth) + + def class_attribute(self): + return self.x + self.INCREMENT + class MyDerivedClass(MyClass): + INCREMENT = 2 + def __init__(self, x, y): MyClass.__init__(self, x, y) @@ -56,3 +67,15 @@ return base.static_meth(x,y) + derived.static_meth(x, y)\ + MyClass.static_meth(x, y) + MyDerivedClass.static_meth(x, y) +def oo_class_attribute(x, y): + base = MyClass(x, y) + derived = MyDerivedClass(x, y) + return base.class_attribute() + derived.class_attribute() + +##def oo_class_meth(x, y): +## return MyClass.class_meth(x, y) + MyDerivedClass.class_meth(x, y) + +if __name__ == '__main__': + from pypy.translator.cli import conftest + conftest.option.wd = True + check(oo_liskov, [int, int], (42, 13)) From hpk at codespeak.net Sun Apr 9 14:20:32 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sun, 9 Apr 2006 14:20:32 +0200 (CEST) Subject: [pypy-svn] r25622 - pypy/dist/pypy/translator/stackless Message-ID: <20060409122032.44D8110223@code0.codespeak.net> Author: hpk Date: Sun Apr 9 14:20:31 2006 New Revision: 25622 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/transform.py Log: (mwh, hpk) fix of one issue but the assertion still fails ... Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sun Apr 9 14:20:31 2006 @@ -89,4 +89,15 @@ u.frame_bottom = frame_state def resume_state(): - return global_state.restart_substate + x = global_state.restart_substate + global_state.restart_substate = 0 + return x + +# XXX would like to be able to say +#def resume_header(): +# x = global_state.restart_substate +# if x: +# top = global_state.top +# global_state.top = None +# XXX and then insert the rtyped graph of this into functions + Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 14:20:31 2006 @@ -188,6 +188,10 @@ frame_state_type = resume_point.frame_state_type frame_top = varoftype(lltype.Ptr(frame_state_type)) ops.extend(self.ops_read_global_state_field(frame_top, "top")) + ops.append(model.SpaceOperation( + "setfield", [self.ll_global_state, model.Constant("inst_top", lltype.Void), + model.Constant(null_state, lltype.typeOf(null_state))], + varoftype(lltype.Void))) i = 0 for arg in resume_point.link_to_resumption.args: newarg = copyvar(self.translator, arg) From mwh at codespeak.net Sun Apr 9 16:35:08 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 16:35:08 +0200 (CEST) Subject: [pypy-svn] r25623 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060409143508.AA002101D7@code0.codespeak.net> Author: mwh Date: Sun Apr 9 16:35:07 2006 New Revision: 25623 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py Log: some changes to placate llinterp change the state header to be a gc struct too fix bug in slp_main_loop the test now works! Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sun Apr 9 16:35:07 2006 @@ -1,11 +1,10 @@ from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython import rarithmetic -STATE_HEADER = lltype.Struct('state_header', - ('f_back', lltype.Ptr(lltype.ForwardReference())), +STATE_HEADER = lltype.GcStruct('state_header', + ('f_back', lltype.Ptr(lltype.GcForwardReference())), ('restartstate', lltype.Signed), - ('function', llmemory.Address), -) + ('function', llmemory.Address),) STATE_HEADER.f_back.TO.become(STATE_HEADER) null_state = lltype.nullptr(STATE_HEADER) @@ -27,7 +26,7 @@ global_state = StacklessData() void_void_func = lltype.Ptr(lltype.FuncType([], lltype.Void)) -long_void_func = lltype.Ptr(lltype.FuncType([], lltype.Signed)) +long_void_func = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)) longlong_void_func = lltype.Ptr(lltype.FuncType([], lltype.SignedLongLong)) float_void_func = lltype.Ptr(lltype.FuncType([], lltype.Float)) pointer_void_func = lltype.Ptr(lltype.FuncType([], llmemory.Address) ) @@ -38,7 +37,7 @@ fn2() elif signature == 'long': fn3 = llmemory.cast_adr_to_ptr(fn, long_void_func) - global_state.retval_long = fn3() + global_state.retval_long = fn3(0) elif signature == 'longlong': fn3 = llmemory.cast_adr_to_ptr(fn, longlong_void_func) global_state.retval_longlong = fn3() @@ -64,7 +63,8 @@ currentframe = global_state.top while currentframe: - global_state.top = nextframe = currentframe.f_back + global_state.top = currentframe + nextframe = currentframe.f_back fn, signature, global_state.restart_substate = decode_state(currentframe) try: call_function(fn, signature) @@ -75,7 +75,7 @@ else: global_state.exception = None - currentframe = nextframe + currentframe = nextframe if global_state.exception is not None: raise global_state.exception Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Sun Apr 9 16:35:07 2006 @@ -86,10 +86,11 @@ ## ll_list = r_list_of_strings.convert_const(['']) ## from pypy.rpython.llinterp import LLInterpreter ## interp = LLInterpreter(t.rtyper) -## interp.eval_graph(graphof(t, entry_point), [ll_list]) +## res = interp.eval_graph(graphof(t, entry_point), [ll_list]) +## print res ## return - cbuilder = CStandaloneBuilder(t, entry_point, gcpolicy=gc.BoehmGcPolicy) + cbuilder = CStandaloneBuilder(t, entry_point)#, gcpolicy=gc.BoehmGcPolicy) cbuilder.generate_source() cbuilder.compile() return cbuilder.cmdexec('') Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 16:35:07 2006 @@ -259,11 +259,13 @@ var_unwind_exception = copyvar(self.translator, var_unwind_exception) fields = [] + n = [] for i, v in enumerate(varstosave): assert v.concretetype is not lltype.Void fields.append(('field_%d'%(i,), v.concretetype)) + n.append(repr(v.concretetype)) - frame_type = lltype.GcStruct("S", + frame_type = lltype.GcStruct("S" + '-'.join(n), ('header', STATE_HEADER), *fields) From arigo at codespeak.net Sun Apr 9 16:39:00 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 16:39:00 +0200 (CEST) Subject: [pypy-svn] r25624 - pypy/dist/pypy/rpython Message-ID: <20060409143900.46A1A101F8@code0.codespeak.net> Author: arigo Date: Sun Apr 9 16:38:59 2006 New Revision: 25624 Modified: pypy/dist/pypy/rpython/normalizecalls.py Log: (arre, arigo, pedronis around) Rewrite assign_inheritance_ids() in a muuuuch saner way. Modified: pypy/dist/pypy/rpython/normalizecalls.py ============================================================================== --- pypy/dist/pypy/rpython/normalizecalls.py (original) +++ pypy/dist/pypy/rpython/normalizecalls.py Sun Apr 9 16:38:59 2006 @@ -274,118 +274,53 @@ # ____________________________________________________________ -class MinIdSymbolic(ComputedIntSymbolic): - def __init__(self, classdef, rootid): - self.classdef = classdef - if classdef is None: - self.parent = None - elif classdef.basedef is None: - self.parent = rootid +class Max(object): + def __cmp__(self, other): + if self is other: + return 0 else: - self.parent = classdef.basedef.minid - if self.parent: - self.parent.children.append(self) - self.children = [] - if rootid is None: - self.rootid = self - else: - self.rootid = rootid + return 1 - def compute_fn(self): - if self.classdef.minid is self: - compute_inheritance_ids(self.classdef.bookkeeper) - return self.classdef.minid - - def __eq__(self, other): - if isinstance(other, MinIdSymbolic): - return self is other - elif isinstance(other, MaxIdSymbolic): - return False - raise NotImplementedError - - def __ne__(self, other): - return not (self == other) - - def __le__(self, other): - if isinstance(other, MinIdSymbolic): - common_classdef = self.classdef.commonbase(other.classdef) - if common_classdef is None: - baseid = self.rootid - else: - baseid = common_classdef.minid - if baseid is self: - return True - if baseid is other: - return False - current_self = self - while current_self.parent is not baseid: - current_self = current_self.parent - current_other = other - while current_other.parent is not baseid: - current_other = current_other.parent - selfindex = baseid.children.index(current_self) - otherindex = baseid.children.index(current_other) - return selfindex <= otherindex - elif isinstance(other, MaxIdSymbolic): - rightmost = other.minid - while rightmost.children: - rightmost = rightmost.children[-1] - return self <= rightmost - raise NotImplementedError - - def compute_inheritance_ids(self, id_=0): - if self.classdef is not None: - self.classdef.minid = id_ - maxid = id_ - for child in self.children: - maxid = child.compute_inheritance_ids(maxid + 1) - if self.classdef is not None: - self.classdef.maxid = maxid - return maxid - -class MaxIdSymbolic(ComputedIntSymbolic): - def __init__(self, minid): - self.minid = minid +MAX = Max() # a maximum object + + +class TotalOrderSymbolic(ComputedIntSymbolic): + + def __init__(self, orderwitness, peers): + self.orderwitness = orderwitness + self.peers = peers + self.value = None + peers.append(self) + + def __cmp__(self, other): + if not isinstance(other, TotalOrderSymbolic): + return NotImplemented + else: + return cmp(self.orderwitness, other.orderwitness) def compute_fn(self): - if self.minid.classdef.minid is self.minid: - compute_inheritance_ids(self.minid.classdef.bookkeeper) - return self.minid.classdef.maxid - -def compute_inheritance_ids(bookkeeper): - bookkeeper.annotator.rootid.compute_inheritance_ids() -# def assign_id(classdef, nextid): -# assert isinstance(classdef.minid, MinIdSymbolic) -# assert isinstance(classdef.maxid, MaxIdSymbolic) -# classdef.minid = nextid -# nextid += 1 -# for subclass in classdef.subdefs: -# nextid = assign_id(subclass, nextid) -# classdef.maxid = nextid -# return classdef.maxid -# id_ = 0 -# for classdef in bookkeeper.classdefs: -# if classdef.basedef is None: -# id_ = assign_id(classdef, id_) - + if self.value is None: + self.peers.sort() + for i, peer in enumerate(self.peers): + assert peer.value is None + peer.value = i + assert self.value is not None + return self.value def assign_inheritance_ids(annotator): - if hasattr(annotator, 'rootid'): - rootid = annotator.rootid - else: - rootid = MinIdSymbolic(None, None) - annotator.rootid = rootid - def assign_id(classdef): - if not hasattr(classdef, 'minid'): - classdef.minid = MinIdSymbolic(classdef, rootid) - if not hasattr(classdef, 'maxid'): - classdef.maxid = MaxIdSymbolic(classdef.minid) - for subclass in classdef.subdefs: - assign_id(subclass) + # we sort the classes by lexicographic order of reversed(mro), + # which gives a nice depth-first order. + bk = annotator.bookkeeper + try: + lst = bk._inheritance_id_symbolics + except AttributeError: + lst = bk._inheritance_id_symbolics = [] for classdef in annotator.bookkeeper.classdefs: - - if classdef.basedef is None: - assign_id(classdef) + if not hasattr(classdef, 'minid'): + witness = list(classdef.getmro()) + witness.reverse() + classdef.minid = TotalOrderSymbolic(witness, lst) + classdef.maxid = TotalOrderSymbolic(witness + [MAX], lst) # ____________________________________________________________ From arigo at codespeak.net Sun Apr 9 16:49:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 16:49:18 +0200 (CEST) Subject: [pypy-svn] r25625 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060409144918.BF5EE10200@code0.codespeak.net> Author: arigo Date: Sun Apr 9 16:49:17 2006 New Revision: 25625 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: (arre, arigo) Mostly revert r25613: these conversions are horribly messy to implement. They would make the back-ends' life not much fun too. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Sun Apr 9 16:49:17 2006 @@ -612,24 +612,6 @@ raise TypeError, "can only cast pointers to other pointers" return ptr._cast_to(PTRTYPE) -def cast_array_pointer(PTRTYPE, ptr): - CURTYPE = typeOf(ptr) - if not isinstance(CURTYPE, Ptr) or not isinstance(PTRTYPE, Ptr): - raise TypeError, "can only cast pointers to other pointers" - if (not isinstance(PTRTYPE.TO, Array) or - not PTRTYPE.TO._hints.get('nolength', False)): - raise TypeError, "can only convert to nolength Arrays" - if isinstance(CURTYPE.TO, Array): - if CURTYPE.TO._hints.get('nolength', False): - raise TypeError, "cannot convert from a nolength Array" - elif isinstance(CURTYPE.TO, FixedSizeArray): - pass - else: - raise TypeError, "can only convert from an array" - if CURTYPE.TO.OF != PTRTYPE.TO.OF: - raise TypeError, "pointers to arrays of different item types" - return ptr._cast_array_to(PTRTYPE) - def _expose(val, solid=False): """XXX A nice docstring here""" T = typeOf(val) @@ -710,8 +692,7 @@ def _setobj(self, pointing_to, solid=False): if pointing_to is None: obj0 = None - elif (solid or isinstance(self._T, (GC_CONTAINER, FuncType)) - or isinstance(pointing_to, _nolengtharray)): + elif solid or isinstance(self._T, (GC_CONTAINER, FuncType)): obj0 = pointing_to else: self._set_weak(True) @@ -861,9 +842,6 @@ raise TypeError("widening %r inside %r instead of %r" % (CURTYPE, PARENTTYPE, PTRTYPE.TO)) return _ptr(PTRTYPE, struc, solid=self._solid) - def _cast_array_to(self, PTRTYPE): - return _ptr(PTRTYPE, _nolengtharray(PTRTYPE.TO, self._obj)) - def _cast_to_int(self): obj = self._obj while obj._parentstructure(): @@ -1032,32 +1010,6 @@ def setitem(self, index, value): self.items[index] = value -class _nolengtharray(object): - _kind = "array" - - __slots__ = ('_TYPE', 'wr_parentarray') - - def __init__(self, TYPE, parentarray): - self._TYPE = TYPE - self.wr_parentarray = pickleable_weakref(parentarray) - - def getparentarray(self): - parentarray = self.wr_parentarray() - if parentarray is None: - raise RuntimeError("accessing already garbage collected %r" - % (self._T,)) - return parentarray - - def getlength(self): - # for debugging only - return self.getparentarray().getlength() - - def getitem(self, index): - return self.getparentarray().getitem(index) - - def setitem(self, index, value): - return self.getparentarray().setitem(index, value) - assert not '__dict__' in dir(_array) assert not '__dict__' in dir(_struct) Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Sun Apr 9 16:49:17 2006 @@ -519,50 +519,6 @@ a = malloc(A, 10) py.test.raises(TypeError, len, a) -def test_cast_length_to_no_length(): - A = Array(Char, hints={'nolength': True}) - S = GcStruct("str", ("hash", Signed), - ("chars", Array(Char))) - s = malloc(S, 5) - s.chars[0] = 'h' - s.chars[1] = 'e' - s.chars[2] = 'l' - s.chars[3] = 'l' - s.chars[4] = 'o' - a = cast_array_pointer(Ptr(A), s.chars) - assert a[0] == 'h' - assert a[1] == 'e' - assert a[2] == 'l' - assert a[3] == 'l' - assert a[4] == 'o' - py.test.raises(TypeError, len, a) - a[1] = 'E' - assert s.chars[1] == 'E' - s.chars[2] = 'L' - assert a[2] == 'L' - -def test_cast_fixed_to_no_length(): - A = Array(Char, hints={'nolength': True}) - S = GcStruct("str", ("hash", Signed), - ("chars", FixedSizeArray(Char, 5))) - s = malloc(S) - s.chars[0] = 'h' - s.chars[1] = 'e' - s.chars[2] = 'l' - s.chars[3] = 'l' - s.chars[4] = 'o' - a = cast_array_pointer(Ptr(A), s.chars) - assert a[0] == 'h' - assert a[1] == 'e' - assert a[2] == 'l' - assert a[3] == 'l' - assert a[4] == 'o' - py.test.raises(TypeError, len, a) - a[1] = 'E' - assert s.chars[1] == 'E' - s.chars[2] = 'L' - assert a[2] == 'L' - def test_dissect_ll_instance(): assert list(dissect_ll_instance(1)) == [(Signed, 1)] GcS = GcStruct("S", ('x', Signed)) From mwh at codespeak.net Sun Apr 9 17:03:45 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 17:03:45 +0200 (CEST) Subject: [pypy-svn] r25626 - in pypy/dist/pypy: rpython rpython/lltypesystem translator/c Message-ID: <20060409150345.DC5DB100E4@code0.codespeak.net> Author: mwh Date: Sun Apr 9 17:03:44 2006 New Revision: 25626 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/translator/c/funcgen.py Log: introduce an unsafe_call operation which calls a callable without providing any arguments for use by stackless Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Sun Apr 9 17:03:44 2006 @@ -437,6 +437,18 @@ log.warn("op_indirect_call with graphs=None:", f) return self.op_direct_call(f, *args) + def op_unsafe_call(self, f): + assert isinstance(f, llmemory.fakeaddress) + assert f.offset is None + obj = self.llinterpreter.typer.type_system.deref(f.ob) + assert hasattr(obj, 'graph') # don't want to think about that + graph = obj.graph + args = [] + for arg in obj.graph.startblock.inputargs: + args.append(arg.concretetype._defl()) + frame = self.__class__(graph, args, self.llinterpreter, self) + return frame.eval() + def op_malloc(self, obj): if self.llinterpreter.gc is not None: args = self.llinterpreter.gc.get_arg_malloc(obj) Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Sun Apr 9 17:03:44 2006 @@ -79,6 +79,7 @@ 'direct_call': LLOp(canraise=(Exception,)), 'indirect_call': LLOp(canraise=(Exception,)), 'safe_call': LLOp(), + 'unsafe_call': LLOp(canraise=(Exception,)), # __________ numeric operations __________ Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Sun Apr 9 17:03:44 2006 @@ -376,6 +376,16 @@ # is of type Void, which is removed by OP_DIRECT_CALL OP_INDIRECT_CALL = OP_DIRECT_CALL + def OP_UNSAFE_CALL(self, op): + line = '((%s (*)())(%s))();'%(self.lltypename(op.result).replace('@', ''), self.expr(op.args[0])) + if self.lltypemap(op.result) is not Void: + r = self.expr(op.result) + line = '%s = %s' % (r, line) + check = self.check_directcall_result(op) + if check: + return line + '\n' + check + return line + def check_directcall_result(self, op): return None From mwh at codespeak.net Sun Apr 9 17:06:39 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 17:06:39 +0200 (CEST) Subject: [pypy-svn] r25627 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060409150639.624E1100FA@code0.codespeak.net> Author: mwh Date: Sun Apr 9 17:06:38 2006 New Revision: 25627 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_transform.py Log: clean up stackless.code.call_function by using unsafe_call Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Sun Apr 9 17:06:38 2006 @@ -1,4 +1,4 @@ -from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython.lltypesystem import lltype, llmemory, lloperation from pypy.rpython import rarithmetic STATE_HEADER = lltype.GcStruct('state_header', @@ -25,32 +25,24 @@ global_state = StacklessData() -void_void_func = lltype.Ptr(lltype.FuncType([], lltype.Void)) -long_void_func = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)) -longlong_void_func = lltype.Ptr(lltype.FuncType([], lltype.SignedLongLong)) -float_void_func = lltype.Ptr(lltype.FuncType([], lltype.Float)) -pointer_void_func = lltype.Ptr(lltype.FuncType([], llmemory.Address) ) - def call_function(fn, signature): if signature == 'void': - fn2 = llmemory.cast_adr_to_ptr(fn, void_void_func) - fn2() + lloperation.llop.unsafe_call(lltype.Void, fn) elif signature == 'long': - fn3 = llmemory.cast_adr_to_ptr(fn, long_void_func) - global_state.retval_long = fn3(0) + global_state.retval_long = lloperation.llop.unsafe_call( + lltype.Signed, fn) elif signature == 'longlong': - fn3 = llmemory.cast_adr_to_ptr(fn, longlong_void_func) - global_state.retval_longlong = fn3() + global_state.retval_longlong = lloperation.llop.unsafe_call( + lltype.SignedLongLong, fn) elif signature == 'float': - fn3 = llmemory.cast_adr_to_ptr(fn, float_void_func) - global_state.retval_double = fn3() + global_state.retval_double = lloperation.llop.unsafe_call( + lltype.Float, fn) elif signature == 'pointer': - fn5 = llmemory.cast_adr_to_ptr(fn, pointer_void_func) - global_state.retval_void_p = fn5() + global_state.retval_void_p = lloperation.llop.unsafe_call( + llmemory.Address, fn) null_address = llmemory.fakeaddress(None) - class UnwindException(Exception): def __init__(self): self.frame_top = null_state # points to frame that first caught Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Sun Apr 9 17:06:38 2006 @@ -87,8 +87,7 @@ ## from pypy.rpython.llinterp import LLInterpreter ## interp = LLInterpreter(t.rtyper) ## res = interp.eval_graph(graphof(t, entry_point), [ll_list]) -## print res -## return +## return str(res) cbuilder = CStandaloneBuilder(t, entry_point)#, gcpolicy=gc.BoehmGcPolicy) cbuilder.generate_source() From mwh at codespeak.net Sun Apr 9 17:25:52 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 17:25:52 +0200 (CEST) Subject: [pypy-svn] r25628 - pypy/dist/pypy/translator/stackless/test Message-ID: <20060409152552.9F92D101EB@code0.codespeak.net> Author: mwh Date: Sun Apr 9 17:25:51 2006 New Revision: 25628 Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py Log: rewrite stackless test to run the test on llinterp too Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Sun Apr 9 17:25:51 2006 @@ -33,6 +33,19 @@ from pypy.translator.stackless import code +def test_simple_transform_llinterp(): + from pypy.translator.stackless.code import UnwindException + def check(x): + if x: + raise UnwindException + def g(x): + check(x) + return x + 1 + def example(x): + return g(x) + 1 + res = llinterp_stackless_function(example, example, g) + assert res == 3 + def test_simple_transform(): from pypy.translator.stackless.code import UnwindException def check(x): @@ -45,18 +58,8 @@ return g(x) + 1 res = run_stackless_function(example, example, g) assert res.strip() == "3" - -def run_stackless_function(fn, *stacklessfuncs): - def entry_point(argv): - try: - r = fn(len(argv)) - except code.UnwindException, u: - code.global_state.top = u.frame_top - code.slp_main_loop() - r = code.global_state.retval_long - os.write(1, str(r)+'\n') - return 0 +def rtype_stackless_function(fn, *stacklessfuncs): s_list_of_strings = annmodel.SomeList(ListDef(None, annmodel.SomeString())) s_list_of_strings.listdef.resize() t = TranslationContext() @@ -71,7 +74,7 @@ unwind_def.generalize_attr('frame_top', annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) unwind_def.generalize_attr('frame_bottom', annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) - annotator.build_types(entry_point, [s_list_of_strings]) + annotator.build_types(fn, [s_list_of_strings]) t.buildrtyper().specialize() st = StacklessTransfomer(t) @@ -81,15 +84,42 @@ checkgraph(graph) if conftest.option.view: t.view() + return t + +def run_stackless_function(fn, *stacklessfuncs): + def entry_point(argv): + try: + r = fn(len(argv)) + except code.UnwindException, u: + code.global_state.top = u.frame_top + code.slp_main_loop() + r = code.global_state.retval_long + os.write(1, str(r)+'\n') + return 0 -## r_list_of_strings = t.rtyper.getrepr(s_list_of_strings) -## ll_list = r_list_of_strings.convert_const(['']) -## from pypy.rpython.llinterp import LLInterpreter -## interp = LLInterpreter(t.rtyper) -## res = interp.eval_graph(graphof(t, entry_point), [ll_list]) -## return str(res) + t = rtype_stackless_function(entry_point, *stacklessfuncs) cbuilder = CStandaloneBuilder(t, entry_point)#, gcpolicy=gc.BoehmGcPolicy) cbuilder.generate_source() cbuilder.compile() - return cbuilder.cmdexec('') + return cbuilder.cmdexec('').strip() + +def llinterp_stackless_function(fn, *stacklessfuncs): + def entry_point(argv): + try: + r = fn(len(argv)) + except code.UnwindException, u: + code.global_state.top = u.frame_top + code.slp_main_loop() + r = code.global_state.retval_long + return r + + t = rtype_stackless_function(entry_point, *stacklessfuncs) + + r_list_of_strings = t.rtyper.getrepr( + t.annotator.binding(graphof(t, entry_point).startblock.inputargs[0])) + ll_list = r_list_of_strings.convert_const(['']) + from pypy.rpython.llinterp import LLInterpreter + interp = LLInterpreter(t.rtyper) + res = interp.eval_graph(graphof(t, entry_point), [ll_list]) + return res From mwh at codespeak.net Sun Apr 9 17:34:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 9 Apr 2006 17:34:53 +0200 (CEST) Subject: [pypy-svn] r25629 - in pypy/dist/pypy/translator: . backendopt c llvm/backendopt stackless test Message-ID: <20060409153453.7DA76101DB@code0.codespeak.net> Author: mwh Date: Sun Apr 9 17:34:50 2006 New Revision: 25629 Modified: pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/c/exceptiontransform.py pypy/dist/pypy/translator/llvm/backendopt/exception.py pypy/dist/pypy/translator/stackless/transform.py pypy/dist/pypy/translator/test/test_unsimplify.py pypy/dist/pypy/translator/unsimplify.py Log: the only reason split_block had a graph argument was to call checkgraph() on it, which was a performance disaster and was commented out a while ago. so kill the graph arg. Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Apr 9 17:34:50 2006 @@ -351,7 +351,7 @@ def do_inline(self, block, index_operation): splitlink = split_block_with_keepalive( - self.translator, self.graph, block, index_operation) + self.translator, block, index_operation) afterblock = splitlink.target # these variables have to be passed along all the links in the inlined # graph because the original function needs them in the blocks after Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Sun Apr 9 17:34:50 2006 @@ -62,9 +62,9 @@ keepalive_ops.append(SpaceOperation('keepalive', [v], v_keepalive)) return keepalive_ops -def split_block_with_keepalive(translator, graph, block, index_operation, +def split_block_with_keepalive(translator, block, index_operation, keep_alive_op_args=True): - splitlink = split_block(translator, graph, block, index_operation) + splitlink = split_block(translator, block, index_operation) afterblock = splitlink.target conservative_keepalives = needs_conservative_livevar_calculation(block) if conservative_keepalives: Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Sun Apr 9 17:34:50 2006 @@ -153,7 +153,7 @@ continue splitlink = support.split_block_with_keepalive( - self.translator, graph, block, i+1, False) + self.translator, block, i+1, False) afterblock = splitlink.target if lastblock is block: lastblock = afterblock Modified: pypy/dist/pypy/translator/llvm/backendopt/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/backendopt/exception.py (original) +++ pypy/dist/pypy/translator/llvm/backendopt/exception.py Sun Apr 9 17:34:50 2006 @@ -32,7 +32,7 @@ continue n_calls_patched += 1 - split_block(translator, graph, block, i+1) + split_block(translator, block, i+1) res = Variable() res.concretetype = Bool Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Sun Apr 9 17:34:50 2006 @@ -226,8 +226,7 @@ while i < len(block.operations): op = block.operations[i] if op.opname in ('direct_call', 'indirect_call'): - link = support.split_block_with_keepalive(self.translator, - self.curr_graph, block, i+1) + link = support.split_block_with_keepalive(self.translator, block, i+1) var_unwind_exception = varoftype(evalue) args = [v for v in link.args Modified: pypy/dist/pypy/translator/test/test_unsimplify.py ============================================================================== --- pypy/dist/pypy/translator/test/test_unsimplify.py (original) +++ pypy/dist/pypy/translator/test/test_unsimplify.py Sun Apr 9 17:34:50 2006 @@ -1,6 +1,7 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.translator.unsimplify import split_block from pypy.rpython.llinterp import LLInterpreter +from pypy.objspace.flow.model import checkgraph def translate(func, argtypes): t = TranslationContext() @@ -15,7 +16,8 @@ w = x * y return z + w graph, t = translate(f, [int, int]) - split_block(t, graph, graph.startblock, i) + split_block(t, graph.startblock, i) + checkgraph(graph) interp = LLInterpreter(t.rtyper) result = interp.eval_graph(graph, [1, 2]) assert result == 5 @@ -28,7 +30,8 @@ else: return y + 2 graph, t = translate(f, [int, int]) - split_block(t, graph, graph.startblock, i) + split_block(t, graph.startblock, i) + checkgraph(graph) interp = LLInterpreter(t.rtyper) result = interp.eval_graph(graph, [-12, 2]) assert result == 4 @@ -53,7 +56,8 @@ return 1 return x graph, t = translate(catches, [int]) - split_block(t, graph, graph.startblock, i) + split_block(t, graph.startblock, i) + checkgraph(graph) interp = LLInterpreter(t.rtyper) result = interp.eval_graph(graph, [0]) assert result == 0 Modified: pypy/dist/pypy/translator/unsimplify.py ============================================================================== --- pypy/dist/pypy/translator/unsimplify.py (original) +++ pypy/dist/pypy/translator/unsimplify.py Sun Apr 9 17:34:50 2006 @@ -35,7 +35,7 @@ link.target = newblock return newblock -def split_block(translator, graph, block, index): +def split_block(translator, block, index): """return a link where prevblock is the block leading up but excluding the index'th operation and target is a new block with the neccessary variables passed on. NOTE: if you call this after rtyping, you WILL need to worry @@ -85,7 +85,6 @@ block.exits = [link] block.exitswitch = None block.exc_handler = False - #checkgraph(graph) return link def remove_direct_loops(translator, graph): From arigo at codespeak.net Sun Apr 9 17:58:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 17:58:55 +0200 (CEST) Subject: [pypy-svn] r25630 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060409155855.9225D101DE@code0.codespeak.net> Author: arigo Date: Sun Apr 9 17:58:53 2006 New Revision: 25630 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Log: (arre, arigo) More support for working with fakeaddresses: adr.signed[n] adr.char[n] adr.address[n] now work as expected. They also do some more type-checking. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Sun Apr 9 17:58:53 2006 @@ -36,13 +36,14 @@ return NotImplemented return ItemOffset(self.TYPE, self.repeat * other) - def get(self, ob): - return ob[self.repeat] + def ref(self, firstitemref): + assert isinstance(firstitemref, _arrayitemref) + array = firstitemref.array + assert lltype.typeOf(array).TO.OF == self.TYPE + index = firstitemref.index + self.repeat + return _arrayitemref(array, index) - def set(self, ob, value): - ob[self.repeat] = value - class FieldOffset(AddressOffset): def __init__(self, TYPE, fldname): @@ -52,11 +53,10 @@ def __repr__(self): return "" % (self.TYPE, self.fldname) - def get(self, ob): - return getattr(ob, self.fldname) - - def set(self, ob, value): - setattr(ob, self.fldname, value) + def ref(self, containerref): + struct = containerref.get() + assert lltype.typeOf(struct).TO == self.TYPE + return _structfieldref(struct, self.fldname) class CompositeOffset(AddressOffset): @@ -68,11 +68,8 @@ def __repr__(self): return '< %r + %r >'%(self.first, self.second) - def get(self, ob): - return self.second.get(self.first.get(ob)) - - def set(self, ob, value): - return self.second.set(self.first.get(ob), value) + def ref(self, containerref): + return self.second.ref(self.first.ref(containerref)) class ArrayItemsOffset(AddressOffset): @@ -83,11 +80,10 @@ def __repr__(self): return '< ArrayItemsOffset >' - def get(self, ob): - return ob - - def set(self, ob, value): - raise Exception("can't assign to an array's items") + def ref(self, arrayref): + array = arrayref.get() + assert lltype.typeOf(array).TO == self.TYPE + return _arrayitemref(array, index=0) class ArrayLengthOffset(AddressOffset): @@ -97,12 +93,55 @@ def __repr__(self): return '< ArrayLengthOffset >' - def get(self, ob): - return len(ob) + def ref(self, arrayref): + array = arrayref.get() + assert lltype.typeOf(array).TO == self.TYPE + return _arraylenref(array) + - def set(self, ob, value): +class _arrayitemref(object): + def __init__(self, array, index): + self.array = array + self.index = index + def get(self): + return self.array[self.index] + def set(self, value): + self.array[self.index] = value + def type(self): + return lltype.typeOf(self.array).TO.OF + +class _arraylenref(object): + def __init__(self, array): + self.array = array + def get(self): + return len(self.array) + def set(self, value): raise Exception("can't assign to an array's length") + def type(self): + return lltype.Signed +class _structfieldref(object): + def __init__(self, struct, fieldname): + self.struct = struct + self.fieldname = fieldname + def get(self): + return getattr(self.struct, self.fieldname) + def set(self, value): + setattr(self.struct, self.fieldname, value) + def type(self): + return getattr(lltype.typeOf(self.struct), self.fieldname) + +class _obref(object): + def __init__(self, ob): + self.ob = ob + def get(self): + return self.ob + def set(self, value): + raise Exception("can't assign to whole object") + def type(self): + return lltype.typeOf(self.ob) + +# ____________________________________________________________ def sizeof(TYPE, n=None): if n is None: @@ -148,17 +187,17 @@ return fakeaddress(self.ob, offset) return NotImplemented + def ref(self): + ref = _obref(self.ob) + if self.offset is not None: + ref = self.offset.ref(ref) + return ref + def get(self): - if self.offset is None: - return self.ob - else: - return self.offset.get(self.ob) + return self.ref().get() def set(self, value): - if self.offset is None: - raise Exception("can't assign to whole object") - else: - self.offset.set(self.ob, value) + self.ref().set(value) def _cast_to_ptr(self, EXPECTED_TYPE): return lltype.cast_pointer(EXPECTED_TYPE, self.get()) @@ -166,43 +205,53 @@ def _cast_to_int(self): return self.get()._cast_to_int() -# XXX the indexing in code like -# addr.signed[0] = v -# is just silly. remove it. +# ____________________________________________________________ + +NULL = fakeaddress(None) # XXX this should be the same as memory.lladdress.NULL +Address = lltype.Primitive("Address", NULL) + class _fakeaccessor(object): def __init__(self, addr): self.addr = addr def __getitem__(self, index): - assert index == 0 - return self.convert(self.addr.get()) + addr = self.addr + if index != 0: + addr += ItemOffset(addr.ref().type(), index) + return self.convert(addr.get()) def __setitem__(self, index, value): - assert index == 0 - self.addr.set(value) - - -class _signed_fakeaccessor(_fakeaccessor): - TYPE = lltype.Signed + addr = self.addr + if index != 0: + addr += ItemOffset(addr.ref().type(), index) + addr.set(value) def convert(self, value): assert lltype.typeOf(value) == self.TYPE return value + +class _signed_fakeaccessor(_fakeaccessor): + TYPE = lltype.Signed + +class _char_fakeaccessor(_fakeaccessor): + TYPE = lltype.Char + class _address_fakeaccessor(_fakeaccessor): - TYPE = None + TYPE = Address def convert(self, value): - # XXX is this the right check for "Ptr-ness" ? - assert isinstance(value, lltype._ptr) - return fakeaddress(value) + if isinstance(value, lltype._ptr): + return fakeaddress(value) + elif isinstance(value, Address): + return value + else: + raise TypeError(value) fakeaddress.signed = property(_signed_fakeaccessor) +fakeaddress.char = property(_char_fakeaccessor) fakeaddress.address = property(_address_fakeaccessor) - -Address = lltype.Primitive("Address", fakeaddress(None)) - fakeaddress._TYPE = Address # the obtained address will not keep the object alive. e.g. if the object is Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Sun Apr 9 17:58:53 2006 @@ -3,35 +3,39 @@ import py def test_simple(): - class C: - def __init__(self, x): - self.x = x - c = C(1) - a = fakeaddress(c) - assert a.get() is c - b = a + FieldOffset('dummy', 'x') - assert b.get() == 1 - b.set(2) - assert c.x == 2 + S = lltype.GcStruct("S", ("x", lltype.Signed), ("y", lltype.Signed)) + s = lltype.malloc(S) + s.x = 123 + s.y = 456 + a = fakeaddress(s) + assert a.get() == s + b = a + FieldOffset(S, 'x') + assert b.get() == 123 + b.set(234) + assert s.x == 234 def test_composite(): - class C: - def __init__(self, x): - self.x = x - c = C(C(3)) - a = fakeaddress(c) - assert a.get() is c - b = a + FieldOffset('dummy', 'x') + FieldOffset('dummy', 'x') - assert b.get() == 3 - b.set(2) - assert c.x.x == 2 + S1 = lltype.GcStruct("S1", ("x", lltype.Signed), ("y", lltype.Signed)) + S2 = lltype.GcStruct("S2", ("s", S1)) + s2 = lltype.malloc(S2) + s2.s.x = 123 + s2.s.y = 456 + a = fakeaddress(s2) + assert a.get() == s2 + b = a + FieldOffset(S2, 's') + FieldOffset(S1, 'x') + assert b.get() == 123 + b.set(234) + assert s2.s.x == 234 def test_array(): - x = [2, 3, 5, 7, 11] + A = lltype.GcArray(lltype.Signed) + x = lltype.malloc(A, 5) + x[3] = 123 a = fakeaddress(x) - # is there a way to ensure that we add the ArrayItemsOffset ? - b = a + ArrayItemsOffset('dummy') + ItemOffset('dummy')*3 - assert b.get() == x[3] + b = a + ArrayItemsOffset(A) + b += ItemOffset(lltype.Signed)*2 + b += ItemOffset(lltype.Signed) + assert b.get() == 123 b.set(14) assert x[3] == 14 @@ -70,7 +74,6 @@ def test_cast_adr_to_ptr(): from pypy.rpython.memory.test.test_llinterpsim import interpret - from pypy.rpython.lltypesystem import lltype S = lltype.GcStruct("S", ("x", lltype.Signed)) Sptr = lltype.Ptr(S) def f(): @@ -83,7 +86,6 @@ def test_cast_adr_to_int(): from pypy.rpython.memory.test.test_llinterpsim import interpret - from pypy.rpython.lltypesystem import lltype S = lltype.GcStruct("S", ("x", lltype.Signed)) Sptr = lltype.Ptr(S) def f(): @@ -95,3 +97,36 @@ assert f() res = interpret(f, []) assert res + +def test_fakeaccessor(): + S = lltype.GcStruct("S", ("x", lltype.Signed), ("y", lltype.Signed)) + s = lltype.malloc(S) + s.x = 123 + s.y = 456 + adr = cast_ptr_to_adr(s) + adr += FieldOffset(S, "y") + assert adr.signed[0] == 456 + adr.signed[0] = 789 + assert s.y == 789 + + A = lltype.GcArray(lltype.Signed) + a = lltype.malloc(A, 5) + a[3] = 123 + adr = cast_ptr_to_adr(a) + assert (adr + ArrayLengthOffset(A)).signed[0] == 5 + assert (adr + ArrayItemsOffset(A)).signed[3] == 123 + (adr + ArrayItemsOffset(A)).signed[3] = 456 + assert a[3] == 456 + adr1000 = (adr + ArrayItemsOffset(A) + ItemOffset(lltype.Signed, 1000)) + assert adr1000.signed[-997] == 456 + + A = lltype.GcArray(lltype.Char) + a = lltype.malloc(A, 5) + a[3] = '*' + adr = cast_ptr_to_adr(a) + assert (adr + ArrayLengthOffset(A)).signed[0] == 5 + assert (adr + ArrayItemsOffset(A)).char[3] == '*' + (adr + ArrayItemsOffset(A)).char[3] = '+' + assert a[3] == '+' + adr1000 = (adr + ArrayItemsOffset(A) + ItemOffset(lltype.Char, 1000)) + assert adr1000.char[-997] == '+' From arigo at codespeak.net Sun Apr 9 18:29:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 18:29:35 +0200 (CEST) Subject: [pypy-svn] r25631 - pypy/dist/pypy/translator/c/test Message-ID: <20060409162935.58F581014F@code0.codespeak.net> Author: arigo Date: Sun Apr 9 18:29:34 2006 New Revision: 25631 Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py Log: More tests. No code change (yet). This is known to break on gcc 4.0, though. Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Sun Apr 9 18:29:34 2006 @@ -79,3 +79,37 @@ assert s == "HELLO" fn = self.getcompiled(llf) fn() + + def test_call_with_fixedsizearray(self): + A = FixedSizeArray(Signed, 5) + S = GcStruct('s', ('a', Ptr(A))) + a = malloc(A, immortal=True) + a[1] = 123 + def g(x): + return x[1] + def llf(): + s = malloc(S) + s.a = a + return g(s.a) + fn = self.getcompiled(llf) + res = fn() + assert res == 123 + + def test_more_prebuilt_arrays(self): + A = FixedSizeArray(Signed, 5) + S = GcStruct('s', ('a1', Ptr(A)), ('a2', A)) + s = malloc(S) + s.a1 = malloc(A, immortal=True) + s.a1[2] = 50 + s.a2[2] = 60 + def llf(n=int): + if n == 1: + a = s.a1 + else: + a = s.a2 + return a[2] + fn = self.getcompiled(llf) + res = fn(1) + assert res == 50 + res = fn(2) + assert res == 60 From arigo at codespeak.net Sun Apr 9 18:34:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 18:34:49 +0200 (CEST) Subject: [pypy-svn] r25632 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060409163449.0E05A101DA@code0.codespeak.net> Author: arigo Date: Sun Apr 9 18:34:47 2006 New Revision: 25632 Modified: pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/c/test/test_lltyped.py Log: More difficult tests. Fixed one bug thus exposed. Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Sun Apr 9 18:34:47 2006 @@ -252,6 +252,9 @@ return self.itemtypename.replace('@', '(@)[%d]' % FIXEDARRAY.length) def access_expr(self, baseexpr, index): + if not isinstance(index, int): + assert index.startswith('item') + index = int(index[4:]) return '%s[%d]' % (baseexpr, index) def definition(self): Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Sun Apr 9 18:34:47 2006 @@ -81,12 +81,12 @@ fn() def test_call_with_fixedsizearray(self): - A = FixedSizeArray(Signed, 5) + A = FixedSizeArray(Struct('s1', ('x', Signed)), 5) S = GcStruct('s', ('a', Ptr(A))) a = malloc(A, immortal=True) - a[1] = 123 + a[1].x = 123 def g(x): - return x[1] + return x[1].x def llf(): s = malloc(S) s.a = a @@ -96,18 +96,18 @@ assert res == 123 def test_more_prebuilt_arrays(self): - A = FixedSizeArray(Signed, 5) + A = FixedSizeArray(Struct('s1', ('x', Signed)), 5) S = GcStruct('s', ('a1', Ptr(A)), ('a2', A)) s = malloc(S) s.a1 = malloc(A, immortal=True) - s.a1[2] = 50 - s.a2[2] = 60 + s.a1[2].x = 50 + s.a2[2].x = 60 def llf(n=int): if n == 1: a = s.a1 else: a = s.a2 - return a[2] + return a[2].x fn = self.getcompiled(llf) res = fn(1) assert res == 50 From arigo at codespeak.net Sun Apr 9 18:52:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 18:52:49 +0200 (CEST) Subject: [pypy-svn] r25633 - pypy/dist/pypy/translator/c/test Message-ID: <20060409165249.06808101E2@code0.codespeak.net> Author: arigo Date: Sun Apr 9 18:52:49 2006 New Revision: 25633 Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py Log: (arre, arigo) Yet another passing-for-now-if-gcc<4.0 test. Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Sun Apr 9 18:52:49 2006 @@ -113,3 +113,15 @@ assert res == 50 res = fn(2) assert res == 60 + + def test_fnptr_with_fixedsizearray(self): + A = ForwardReference() + F = FuncType([Ptr(A)], Signed) + A.become(FixedSizeArray(Struct('s1', ('f', Ptr(F)), ('n', Signed)), 5)) + a = malloc(A, immortal=True) + a[3].n = 42 + def llf(n=int): + return bool(a[n].f) + fn = self.getcompiled(llf) + res = fn(4) + assert res == 0 From pedronis at codespeak.net Sun Apr 9 19:04:15 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 9 Apr 2006 19:04:15 +0200 (CEST) Subject: [pypy-svn] r25634 - pypy/dist/pypy/translator/backendopt Message-ID: <20060409170415.D2672101D6@code0.codespeak.net> Author: pedronis Date: Sun Apr 9 19:04:15 2006 New Revision: 25634 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: re-testing orthogonally the changes to inlining. the change in 25542 is OK. No bad performance effect vs. no-change on 25531. Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Apr 9 19:04:15 2006 @@ -527,6 +527,7 @@ for graph1, graph2 in callgraph: callers.setdefault(graph2, {})[graph1] = True callees.setdefault(graph1, {})[graph2] = True + # the -len(callers) change is OK heap = [(0.0, -len(callers[graph]), graph) for graph in callers] valid_weight = {} couldnt_inline = {} From arigo at codespeak.net Sun Apr 9 19:16:09 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 19:16:09 +0200 (CEST) Subject: [pypy-svn] r25635 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060409171609.3DF2610111@code0.codespeak.net> Author: arigo Date: Sun Apr 9 19:16:07 2006 New Revision: 25635 Modified: pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/c/test/test_lltyped.py Log: (arre, arigo) Don't confuse C programmers and compilers so much about pointers to arrays. Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Sun Apr 9 19:16:07 2006 @@ -91,8 +91,13 @@ if isinstance(T, Primitive): return PrimitiveType[T] elif isinstance(T, Ptr): - typename = self.gettype(T.TO) # who_asks not propagated - return typename.replace('@', '*@') + if isinstance(T.TO, FixedSizeArray): + # /me blames C + node = self.gettypedefnode(T.TO) + return node.getptrtype() + else: + typename = self.gettype(T.TO) # who_asks not propagated + return typename.replace('@', '*@') elif isinstance(T, (Struct, Array)): node = self.gettypedefnode(T, varlength=varlength) if who_asks is not None: Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Sun Apr 9 19:16:07 2006 @@ -428,7 +428,11 @@ OP_BARE_SETFIELD = OP_SETFIELD def OP_GETSUBSTRUCT(self, op): - return self.OP_GETFIELD(op, ampersand='&') + RESULT = self.lltypemap(op.result).TO + if isinstance(RESULT, FixedSizeArray): + return self.OP_GETFIELD(op, ampersand='') + else: + return self.OP_GETFIELD(op, ampersand='&') def OP_GETARRAYSIZE(self, op): ARRAY = self.lltypemap(op.args[0]).TO @@ -442,9 +446,7 @@ def OP_GETARRAYITEM(self, op): ARRAY = self.lltypemap(op.args[0]).TO items = self.expr(op.args[0]) - if isinstance(ARRAY, FixedSizeArray): - items = '(*%s)' % (items,) - else: + if not isinstance(ARRAY, FixedSizeArray): items += '->items' return self.generic_get(op, '%s[%s]' % (items, self.expr(op.args[1]))) @@ -452,9 +454,7 @@ def OP_SETARRAYITEM(self, op): ARRAY = self.lltypemap(op.args[0]).TO items = self.expr(op.args[0]) - if isinstance(ARRAY, FixedSizeArray): - items = '(*%s)' % (items,) - else: + if not isinstance(ARRAY, FixedSizeArray): items += '->items' return self.generic_set(op, '%s[%s]' % (items, self.expr(op.args[1]))) @@ -462,9 +462,7 @@ def OP_GETARRAYSUBSTRUCT(self, op): ARRAY = self.lltypemap(op.args[0]).TO items = self.expr(op.args[0]) - if isinstance(ARRAY, FixedSizeArray): - items = '*%s' % (items,) - else: + if not isinstance(ARRAY, FixedSizeArray): items += '->items' return '%s = %s + %s;' % (self.expr(op.result), items, Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Sun Apr 9 19:16:07 2006 @@ -251,6 +251,9 @@ FIXEDARRAY = self.FIXEDARRAY return self.itemtypename.replace('@', '(@)[%d]' % FIXEDARRAY.length) + def getptrtype(self): + return self.itemtypename.replace('@', '*@') + def access_expr(self, baseexpr, index): if not isinstance(index, int): assert index.startswith('item') @@ -454,6 +457,10 @@ if USESLOTS: __slots__ = () + def __init__(self, db, T, obj): + ContainerNode.__init__(self, db, T, obj) + self.ptrname = self.name + def basename(self): return self.T._name Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Sun Apr 9 19:16:07 2006 @@ -121,7 +121,10 @@ a = malloc(A, immortal=True) a[3].n = 42 def llf(n=int): - return bool(a[n].f) + if a[n].f: + return a[n].f(a) + else: + return -1 fn = self.getcompiled(llf) res = fn(4) - assert res == 0 + assert res == -1 From arigo at codespeak.net Sun Apr 9 19:36:22 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 9 Apr 2006 19:36:22 +0200 (CEST) Subject: [pypy-svn] r25637 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060409173622.5DC5310153@code0.codespeak.net> Author: arigo Date: Sun Apr 9 19:36:21 2006 New Revision: 25637 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Log: (arre, arigo) Support __eq__ on fakeaddresses. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Sun Apr 9 19:36:21 2006 @@ -187,6 +187,21 @@ return fakeaddress(self.ob, offset) return NotImplemented + def __eq__(self, other): + if not isinstance(other, fakeaddress): + return False + if self.ob is None: + return other.ob is None + if other.ob is None: + return False + ref1 = self.ref() + ref2 = other.ref() + return (ref1.__class__ is ref2.__class__ and + ref1.__dict__ == ref2.__dict__) + + def __ne__(self, other): + return not (self == other) + def ref(self): ref = _obref(self.ob) if self.offset is not None: Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Sun Apr 9 19:36:21 2006 @@ -130,3 +130,33 @@ assert a[3] == '+' adr1000 = (adr + ArrayItemsOffset(A) + ItemOffset(lltype.Char, 1000)) assert adr1000.char[-997] == '+' + +def test_fakeadr_eq(): + S = lltype.GcStruct("S", ("x", lltype.Signed), ("y", lltype.Signed)) + s = lltype.malloc(S) + + assert cast_ptr_to_adr(s) == cast_ptr_to_adr(s) + + adr1 = cast_ptr_to_adr(s) + FieldOffset(S, "x") + adr2 = cast_ptr_to_adr(s) + FieldOffset(S, "y") + adr3 = cast_ptr_to_adr(s) + FieldOffset(S, "y") + assert adr1 != adr2 + assert adr2 == adr3 + + A = lltype.GcArray(lltype.Char) + a = lltype.malloc(A, 5) + adr1 = cast_ptr_to_adr(a) + ArrayLengthOffset(A) + adr2 = cast_ptr_to_adr(a) + ArrayLengthOffset(A) + assert adr1 == adr2 + + adr1 = cast_ptr_to_adr(a) + ArrayItemsOffset(A) + adr2 = cast_ptr_to_adr(a) + ArrayItemsOffset(A) + assert adr1 == adr2 + adr2 += ItemOffset(lltype.Char, 0) + assert adr1 == adr2 + + adr1 += ItemOffset(lltype.Char, 2) + adr2 += ItemOffset(lltype.Char, 3) + assert adr1 != adr2 + adr2 += ItemOffset(lltype.Char, -1) + assert adr1 == adr2 From antocuni at codespeak.net Mon Apr 10 01:14:25 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 10 Apr 2006 01:14:25 +0200 (CEST) Subject: [pypy-svn] r25641 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060409231425.0281210126@code0.codespeak.net> Author: antocuni Date: Mon Apr 10 01:14:05 2006 New Revision: 25641 Modified: pypy/dist/pypy/translator/cli/class_.py pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/ilgenerator.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/rte.py pypy/dist/pypy/translator/cli/src/pypy.cs pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py pypy/dist/pypy/translator/cli/test/test_oo.py Log: Added support for classes as first class objects. The 'runtimenew' operation is executed by calling an external function defined in src/pypy.cs; that file is automatically compiled into pypy.dll when changed thanks to rte.py. Some minor bugs concerning the 'meta' fields has been corrected. Modified: pypy/dist/pypy/translator/cli/class_.py ============================================================================== --- pypy/dist/pypy/translator/cli/class_.py (original) +++ pypy/dist/pypy/translator/cli/class_.py Mon Apr 10 01:14:05 2006 @@ -44,7 +44,9 @@ ilasm.begin_class(self.name, self.get_base_class()) for f_name, (f_type, f_default) in self.classdef._fields.iteritems(): - ilasm.field(f_name, self.cts.lltype_to_cts(f_type)) + cts_type = self.cts.lltype_to_cts(f_type) + if cts_type != 'void': + ilasm.field(f_name, cts_type) # TODO: should the .ctor set the default values? self._ctor() Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Mon Apr 10 01:14:05 2006 @@ -6,7 +6,7 @@ from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong -from pypy.rpython.ootypesystem.ootype import Instance, Class +from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod from pypy.translator.cli.option import getoption from pypy.tool.ansi_print import ansi_log @@ -23,7 +23,7 @@ UnsignedLongLong: 'unsigned int64', Bool: 'bool', Float: 'float64', - Class: 'class [mscorlib]System.Type', # TODO: check this + Class: 'class [mscorlib]System.Type', } _pyexception_to_cts = { @@ -51,6 +51,8 @@ name = t._name self.db.pending_class(t) return 'class %s' % name + elif isinstance(t, StaticMethod): + return 'void' # TODO: is it correct to ignore StaticMethod? return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Mon Apr 10 01:14:05 2006 @@ -45,8 +45,12 @@ def record_const(self, value): const = AbstractConst.make(self, value) - name = const.get_name(len(self.consts)) - self.consts[const] = name + try: + name = self.consts[const] + except KeyError: + name = const.get_name(len(self.consts)) + self.consts[const] = name + return '%s.%s::%s' % (CONST_NAMESPACE, CONST_CLASS, name) def gen_constants(self, ilasm): @@ -96,21 +100,33 @@ self.cts = CTS(db) self.obj = obj + def __hash__(self): + return hash(self.obj) + + def __eq__(self, other): + return self.obj == other.obj + def get_name(self, n): name = self.obj._TYPE._name.replace('.', '_') return '%s_%d' % (name, n) def get_type(self): return self.cts.lltype_to_cts(self.obj._TYPE) - #return 'class %s' % self.obj._TYPE._name def init(self, ilasm): classdef = self.obj._TYPE ilasm.new('instance void class %s::.ctor()' % classdef._name) while classdef is not None: - for name, (type_, value) in classdef._fields.iteritems(): + for name, (type_, default) in classdef._fields.iteritems(): if isinstance(type_, ootype.StaticMethod): continue elif type_ is ootype.Class: - continue + value = getattr(self.obj, name) + self.cts.lltype_to_cts(value._INSTANCE) # force scheduling class generation + classname = value._INSTANCE._name + ilasm.opcode('dup') + ilasm.opcode('ldtoken', classname) + ilasm.call('class [mscorlib]System.Type class [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle)') + ilasm.opcode('stfld class [mscorlib]System.Type %s::%s' % (classdef._name, name)) classdef = classdef._superclass + Modified: pypy/dist/pypy/translator/cli/ilgenerator.py ============================================================================== --- pypy/dist/pypy/translator/cli/ilgenerator.py (original) +++ pypy/dist/pypy/translator/cli/ilgenerator.py Mon Apr 10 01:14:05 2006 @@ -37,6 +37,7 @@ self.out = outfile self.code = CodeGenerator(self.out) self.code.writeline('.assembly extern mscorlib {}') + self.code.writeline('.assembly extern pypy {}') self.code.writeline('.assembly %s {}' % name) def close(self): Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Mon Apr 10 01:14:05 2006 @@ -11,10 +11,12 @@ def _abs(type_): return [PushAllArgs, 'call %s class [mscorlib]System.Math::Abs(%s)' % (type_, type_)] +_runtimenew = 'object [pypy]pypy.runtime.Utils::RuntimeNew([mscorlib]System.Type)' opcodes = { # __________ object oriented operations __________ 'new': [New], + 'runtimenew': [PushAllArgs, 'call ' + _runtimenew], 'oosetfield': [SetField], 'oogetfield': [GetField], 'oosend': [CallMethod], Modified: pypy/dist/pypy/translator/cli/rte.py ============================================================================== --- pypy/dist/pypy/translator/cli/rte.py (original) +++ pypy/dist/pypy/translator/cli/rte.py Mon Apr 10 01:14:05 2006 @@ -6,6 +6,7 @@ import os import os.path import subprocess +import py SRC = 'pypy.cs' DLL = 'pypy.dll' @@ -33,10 +34,18 @@ return dll def compile(source): - compiler = subprocess.Popen(['mcs', '/t:library', source], + mcs = _get_compiler() + compiler = subprocess.Popen([mcs, '/t:library', source], stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdout, stderr = compiler.communicate() retval = compiler.wait() assert retval == 0, 'Failed to compile %s: the compiler said:\n %s' % (DLL, stderr) +def _get_compiler(): + try: + py.path.local.sysfind('mcs') # TODO: support windows + return 'mcs' + except py.error.ENOENT: + py.test.skip("%s is not on your path." % helper) + Modified: pypy/dist/pypy/translator/cli/src/pypy.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypy.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypy.cs Mon Apr 10 01:14:05 2006 @@ -1,8 +1,12 @@ using System; -namespace PyPy +namespace pypy.runtime { - public class Foo + public class Utils { + public static object RuntimeNew(Type t) + { + return t.GetConstructor(new Type[0]).Invoke(new object[0]); + } } } Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Mon Apr 10 01:14:05 2006 @@ -39,8 +39,7 @@ def bar(x, y): b = Derived(x) - return b.compute() -# return foo(Base, x) + foo(Derived, y) + return foo(Base, x) + foo(Derived, y) f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Mon Apr 10 01:14:05 2006 @@ -1,6 +1,7 @@ import os import subprocess import platform +import shutil import py from pypy.tool.udir import udir @@ -11,6 +12,7 @@ from pypy.translator.cli.node import Node from pypy.translator.cli.cts import CTS from pypy.translator.cli.database import LowLevelDatabase +from pypy.translator.cli.rte import get_pypy_dll FLOAT_PRECISION = 8 @@ -103,11 +105,11 @@ t.viewcg() if getoption('wd'): - tmpdir = py.path.local('.') + self.tmpdir = py.path.local('.') else: - tmpdir = udir + self.tmpdir = udir - return GenCli(tmpdir, t, TestEntryPoint(self.graph)) + return GenCli(self.tmpdir, t, TestEntryPoint(self.graph)) def __check_helper(self, helper): try: @@ -127,6 +129,9 @@ if getoption('source'): return None + pypy_dll = get_pypy_dll() # get or recompile pypy.dll + shutil.copy(pypy_dll, self.tmpdir.strpath) + self.__check_helper("ilasm") ilasm = subprocess.Popen(["ilasm", tmpfile], stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdout, stderr = ilasm.communicate() Modified: pypy/dist/pypy/translator/cli/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cli/test/test_oo.py Mon Apr 10 01:14:05 2006 @@ -36,11 +36,19 @@ INCREMENT = 2 def __init__(self, x, y): - MyClass.__init__(self, x, y) + MyClass.__init__(self, x+12, y+34) def compute(self): return self.x - self.y +# helper functions +def call_method(obj): + return obj.compute() + +def init_and_compute(cls, x, y): + return cls(x, y).compute() + +# test functions def oo_compute(x, y): obj = MyClass(x, y) return obj.compute() @@ -53,13 +61,10 @@ obj = MyDerivedClass(x, y) return obj.compute_and_multiply(2) -def helper(obj): - return obj.compute() - def oo_liskov(x, y): base = MyClass(x, y) derived = MyDerivedClass(x, y) - return helper(base) + helper(derived) + return call_method(base) + call_method(derived) def oo_static_method(x, y): base = MyClass(x, y) @@ -72,6 +77,9 @@ derived = MyDerivedClass(x, y) return base.class_attribute() + derived.class_attribute() +def oo_runtimenew(x, y): + return init_and_compute(MyClass, x, y) + init_and_compute(MyDerivedClass, x, y) + ##def oo_class_meth(x, y): ## return MyClass.class_meth(x, y) + MyDerivedClass.class_meth(x, y) From cfbolz at codespeak.net Mon Apr 10 01:40:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 10 Apr 2006 01:40:49 +0200 (CEST) Subject: [pypy-svn] r25642 - pypy/dist/pypy/translator/backendopt Message-ID: <20060409234049.1BE711017D@code0.codespeak.net> Author: cfbolz Date: Mon Apr 10 01:40:46 2006 New Revision: 25642 Modified: pypy/dist/pypy/translator/backendopt/mallocprediction.py Log: make final result bold Modified: pypy/dist/pypy/translator/backendopt/mallocprediction.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/mallocprediction.py (original) +++ pypy/dist/pypy/translator/backendopt/mallocprediction.py Mon Apr 10 01:40:46 2006 @@ -153,7 +153,7 @@ if not newcount: break count += newcount - log.inlineandremove("removed %s mallocs in total" % count) + log.inlineandremove.event("removed %s mallocs in total" % count) for graph in translator.graphs: removenoops.remove_superfluous_keep_alive(graph) removenoops.remove_duplicate_casts(graph, translator) From cfbolz at codespeak.net Mon Apr 10 01:49:39 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 10 Apr 2006 01:49:39 +0200 (CEST) Subject: [pypy-svn] r25643 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060409234939.5EED21017C@code0.codespeak.net> Author: cfbolz Date: Mon Apr 10 01:49:36 2006 New Revision: 25643 Modified: pypy/dist/pypy/translator/backendopt/propagate.py pypy/dist/pypy/translator/backendopt/test/test_propagate.py Log: some more propagate things: * fix bug related to getsubstruct: don't constant-fold it if the result is a non-gc-pointer, since the parent might go away * new transformation which removes (in some trivial cases) getfield operations. this happens if a setfield is followed by a getfield to the same object and field or if two getfields are after each other. The decision whether the object changed in between is extremely conservative. * an (in progress) attempt to propagate constants even more, which depends on the DataFlowFamilyBuilder. The approach seems to work, the test doesn't. Somehow I suspect a bug in DataFlowFamilyBuilder. Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Mon Apr 10 01:49:36 2006 @@ -7,6 +7,8 @@ from pypy.translator.simplify import get_graph from pypy.translator.backendopt.removenoops import remove_same_as from pypy.translator.backendopt.inline import OP_WEIGHTS +from pypy.translator.backendopt.ssa import DataFlowFamilyBuilder +from pypy.translator.backendopt.support import log, var_needsgc def do_atmost(n, f, *args): i = 0 @@ -49,7 +51,7 @@ link.target = link.target.exits[val].target link.args = args if candidates: - print "rewiring links in graph", graph.name + #print "rewiring links in graph", graph.name simplify.join_blocks(graph) return True return False @@ -75,30 +77,35 @@ def propagate_consts(graph): """replace a variable of the inputargs of a block by a constant - if all blocks leading to it have the same constant in that position""" + if all blocks leading to it have the same constant in that position + or if all non-constants that lead to it are in the same variable family + as the constant that we go to.""" entrymap = mkentrymap(graph) candidates = [] changed = False + variable_families = DataFlowFamilyBuilder(graph).get_variable_families() for block, ingoing in entrymap.iteritems(): if block in [graph.returnblock, graph.exceptblock]: continue for i in range(len(ingoing[0].args) - 1, -1, -1): vals = {} + var = block.inputargs[i] + var_rep = variable_families.find_rep(var) withvar = True for link in ingoing: if isinstance(link.args[i], Variable): - break + if variable_families.find_rep(link.args[i]) != var_rep: + break else: vals[link.args[i]] = True else: - withvar = False + withvar = False if len(vals) != 1 or withvar: continue - print "propagating constants in graph", graph.name + #print "propagating constants in graph", graph.name const = vals.keys()[0] for link in ingoing: del link.args[i] - var = block.inputargs[i] del block.inputargs[i] op = SpaceOperation("same_as", [const], var) block.operations.insert(0, op) @@ -136,14 +143,17 @@ if sum([isinstance(arg, Variable) for arg in op.args]): continue if lloperation.LL_OPERATIONS[op.opname].canfold: - print "folding operation", op, "in graph", graph.name + if op.opname in ("getsubstruct", "getarraysubstruct"): + if not var_needsgc(op.result): + continue try: llframe.eval_operation(op) except: - print "did not work" + pass else: res = Constant(llframe.getval(op.result)) - print "result", res.value + log.constantfolding("in graph %s, %s = %s" % + (graph.name, op, res)) res.concretetype = op.result.concretetype block.operations[i].opname = "same_as" block.operations[i].args = [res] @@ -249,6 +259,87 @@ else: return False +def iter_op_pairs(graph, opname1, opname2, equality): + for block in graph.iterblocks(): + num_operations = len(block.operations) + for current1 in range(num_operations): + if block.operations[current1].opname != opname1: + continue + op1 = block.operations[current1] + for current2 in range(current1 + 1, num_operations): + if block.operations[current2].opname != opname2: + continue + op2 = block.operations[current2] + if equality(op1, op2): + yield block, current1, current2 + return + +def can_be_same(val1, val2): + if isinstance(val1, Constant) and isinstance(val2, Constant): + return val1.value == val2.value + return val1.concretetype == val2.concretetype + +def remove_getfield(graph, translator): + """ this removes a getfield after a setfield, if they work on the same + object and field and if there is no setfield in between which can access + the same field""" + def equality(op1, op2): + if isinstance(op1.args[0], Constant): + if isinstance(op2.args[0], Constant): + return (op1.args[0].value == op2.args[0].value and + op1.args[1].value == op2.args[1].value) + return False + return (op1.args[0] == op2.args[0] and + op1.args[1].value == op2.args[1].value) + def remove_if_possible(block, index1, index2): + op1 = block.operations[index1] + op2 = block.operations[index2] + #print "found" + #print op1 + #print op2 + fieldname = op1.args[1].value + var_or_const = op1.args[0] + if op1.opname == "setfield": + value = op1.args[2] + else: + value = op1.result + for i in range(index1 + 1, index2): + op = block.operations[i] + if op.opname == "setfield": + if op.args[1].value != fieldname: + continue + if can_be_same(op.args[0], op1.args[0]): + break + if op.opname == "direct_call": + break # giving up for now + if op.opname == "indirect_call": + break # giving up for now + else: + op2.opname = "same_as" + op2.args = [value] + return 1 + return 0 + count = 0 + for block, index1, index2 in iter_op_pairs( + graph, "setfield", "getfield", equality): + count += remove_if_possible(block, index1, index2) + for block, index1, index2 in iter_op_pairs( + graph, "getfield", "getfield", equality): + count += remove_if_possible(block, index1, index2) + if count: + remove_same_as(graph) + return count + +def remove_all_getfields(graph, t): + count = 0 + while 1: + newcount = remove_getfield(graph, t) + count += newcount + if not newcount: + break + if count: + log.removegetfield("removed %s getfields in %s" % (count, graph.name)) + return count def propagate_all(translator): for graph in translator.graphs: @@ -260,5 +351,6 @@ changed = do_atmost(100, constant_folding, graph, translator) or changed changed = partial_folding(graph, translator) or changed + changed = remove_all_getfields(graph, translator) or changed return changed do_atmost(10, prop) Modified: pypy/dist/pypy/translator/backendopt/test/test_propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_propagate.py Mon Apr 10 01:49:36 2006 @@ -81,6 +81,24 @@ assert len(graph.startblock.exits[0].args) == 1 check_graph(graph, [1], None, t) +def test_propagate_despite_vars(): + py.test.skip("immediate test") + patterns = [1, 1, 2, 3, 5, 7, 12] + class A(object): pass + a = A() + a.x = 10 + a.y = 20 + def f(x): + result = 0 + for i in range(a.x): + for j in range(a.y): + result += i * j + return result + graph, t = get_graph(f, [int]) + propagate_consts(graph) + if conftest.option.view: + t.view() + def test_constant_fold(): def f(x): return 1 @@ -113,6 +131,20 @@ assert len(graph.startblock.operations) == 1 check_graph(graph, [10], g(10), t) +def test_dont_fold_getfield(): + # must not constant fold this, because the container might be collected + string = "blablabla" + def f(x): + return string[abs(x)] + graph, t = get_graph(f, [int]) + res = constant_folding(graph, t) + assert not res + if conftest.option.view: + t.view() + print graph.startblock.operations[1] + check_graph(graph, [0], "b", t) + + def test_fold_const_blocks(): def s(x): res = 0 @@ -184,4 +216,52 @@ if conftest.option.view: t.view() - +def test_remove_getfield_after_setfield(): + class A(object): + def __init__(self, x=42): + self.x = x + class B(object): + pass + global_b = B() + global_b.a = None + def f(x): + a = A(x) + global_b.a = a + global_b.a.x += 1 + return global_b.a.x + graph, t = get_graph(f, [int], all_opts=False) + assert len(graph.startblock.operations) == 11 + count = remove_all_getfields(graph, t) + if conftest.option.view: + t.view() + assert len(graph.startblock.operations) == 8 + check_graph(graph, [42], 43, t) + +def test_remove_getfield_after_getfield(): + class A(object): + def __init__(self, x=42): + self.x = x + class B(object): + def __init__(self, a): + self.a = a + class C: + pass + global_c = C() + global_c.b1 = None + global_c.b2 = None + def f(x): + a = A(x) + b = B(a) + global_c.b1 = b + a1 = global_c.b1.a + global_c.b2 = B(a1) + return global_c.b1.a.x + graph, t = get_graph(f, [int]) + assert len(graph.startblock.operations) == 23 + count = remove_all_getfields(graph, t) + assert count + if conftest.option.view: + t.view() + assert len(graph.startblock.operations) == 20 + check_graph(graph, [42], 42, t) + From cfbolz at codespeak.net Mon Apr 10 01:55:29 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 10 Apr 2006 01:55:29 +0200 (CEST) Subject: [pypy-svn] r25644 - in pypy/dist/pypy/translator: . test Message-ID: <20060409235529.99FC01017E@code0.codespeak.net> Author: cfbolz Date: Mon Apr 10 01:55:27 2006 New Revision: 25644 Modified: pypy/dist/pypy/translator/simplify.py pypy/dist/pypy/translator/test/test_simplify.py Log: make has_no_side_effects more precise: follow all graphs instead of giving up when seeing an indirect_call Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Mon Apr 10 01:55:27 2006 @@ -383,9 +383,11 @@ if graph.startblock not in translator.rtyper.already_seen: return False if seen is None: - seen = [] + seen = {} elif graph in seen: return True + newseen = seen.copy() + newseen[graph] = True try: def visit(block): if not isinstance(block, Block): @@ -395,12 +397,15 @@ g = get_graph(op.args[0], translator) if g is None: raise HasSideEffects - if not has_no_side_effects(translator, g, seen + [graph]): + if not has_no_side_effects(translator, g, newseen): raise HasSideEffects elif op.opname == "indirect_call": - # XXX can be improved: look at all graphs, - # if they are there - raise HasSideEffects + graphs = op.args[-1].value + if graphs is None: + raise HasSideEffects + for g in graphs: + if not has_no_side_effects(translator, g, newseen): + raise HasSideEffects elif lloperation.LL_OPERATIONS[op.opname].sideeffects: raise HasSideEffects traverse(visit, graph) Modified: pypy/dist/pypy/translator/test/test_simplify.py ============================================================================== --- pypy/dist/pypy/translator/test/test_simplify.py (original) +++ pypy/dist/pypy/translator/test/test_simplify.py Mon Apr 10 01:55:27 2006 @@ -37,7 +37,24 @@ a = rec(x) return x + 12 graph, _ = translate(f, [int]) - assert len(graph.startblock.operations) + assert len(graph.startblock.operations) == 1 + +def test_remove_call_with_indirect_call(): + def f1(x): + return x + 1 + def f2(x): + return x + 2 + def g(x): + if x == 32: + f = f1 + else: + f = f2 + return f(x) + def h(x): + a = g(x) + return x + 42 + graph, t = translate(h, [int]) + assert len(graph.startblock.operations) == 1 def test_dont_remove_if_exception_guarded(): def f(x): From arigo at codespeak.net Mon Apr 10 11:09:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 11:09:58 +0200 (CEST) Subject: [pypy-svn] r25651 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060410090958.8170A10187@code0.codespeak.net> Author: arigo Date: Mon Apr 10 11:09:57 2006 New Revision: 25651 Added: pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Modified: pypy/dist/pypy/rpython/rctypes/implementation.py Log: (arre yesterday, arigo) The first rctypes string tests are passing! Yay! Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Mon Apr 10 11:09:57 2006 @@ -25,6 +25,7 @@ import pypy.rpython.rctypes.rprimitive import pypy.rpython.rctypes.rpointer import pypy.rpython.rctypes.rfunc +import pypy.rpython.rctypes.rchar_p # ctypes_annotation_list contains various attributes that # are used by the pypy annotation. Added: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Mon Apr 10 11:09:57 2006 @@ -0,0 +1,112 @@ +from pypy.rpython import extregistry +from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython.rstr import string_repr +from pypy.rpython.rctypes.rmodel import CTypesValueRepr +from pypy.annotation import model as annmodel + +from ctypes import c_char_p + + +class CCharPRepr(CTypesValueRepr): + + def get_content_keepalives(self): + "Return an extra keepalive field used for the RPython string." + return [('keepalive_str', string_repr.lowleveltype)] + + def getstring(self, llops, v_box): + v_c_data = self.get_c_data(llops, v_box) + return llops.gendirectcall(ll_getstring, v_box, v_c_data) + + def setstring(self, llops, v_box, v_str): + v_c_data = self.get_c_data(llops, v_box) + llops.gendirectcall(ll_setstring, v_box, v_c_data, v_str) + + def rtype_getattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + v_char_p = hop.inputarg(self, 0) + return self.getstring(hop.llops, v_char_p) + + def rtype_setattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + v_char_p, v_attr, v_value = hop.inputargs(self, lltype.Void, + string_repr) + self.setstring(hop.llops, v_char_p, v_value) + + +CCHARP = llmemory.Address # char * +FIRSTITEMOFS = llmemory.ArrayItemsOffset(string_repr.lowleveltype.TO.chars) + +def ll_strlen(p): + i = 0 + while ord(p.char[i]) != 0: + i += 1 + return i + +def ll_strnlen(p, maxlen): + i = 0 + while i < maxlen and ord(p.char[i]) != 0: + i += 1 + return i + +def ll_str2charp(s): + return llmemory.cast_ptr_to_adr(s.chars) + FIRSTITEMOFS + +def ll_getstring(box, c_data): + p = c_data.value + if p: + if (box.keepalive_str and ll_str2charp(box.keepalive_str) == p): + maxlen = len(box.keepalive_str.chars) + length = ll_strnlen(p, maxlen) + if length == maxlen: + # no embedded zero in the string + return box.keepalive_str + else: + length = ll_strlen(p) + newstr = lltype.malloc(string_repr.lowleveltype.TO, length) + for i in range(length): + newstr.chars[i] = p.char[i] + return newstr + else: + return lltype.nullptr(string_repr.lowleveltype.TO) + +def ll_setstring(box, c_data, string): + if string: + c_data.value = ll_str2charp(string) + else: + c_data.value = llmemory.NULL + box.keepalive_str = string + + +def c_char_p_compute_result_annotation(s_arg=None): + return annmodel.SomeCTypesObject(c_char_p, + annmodel.SomeCTypesObject.OWNSMEMORY) + +def c_char_p_specialize_call(hop): + r_char_p = hop.r_result + v_result = r_char_p.allocate_instance(hop.llops) + if len(hop.args_s): + v_value, = hop.inputargs(string_repr) + r_char_p.setstring(hop.llops, v_result, v_value) + return v_result + +extregistry.register_value(c_char_p, + compute_result_annotation=c_char_p_compute_result_annotation, + specialize_call=c_char_p_specialize_call + ) + +def c_char_p_get_repr(rtyper, s_char_p): + return CCharPRepr(rtyper, s_char_p, CCHARP) + +entry = extregistry.register_type(c_char_p, + compute_annotation = annmodel.SomeCTypesObject(c_char_p, + annmodel.SomeCTypesObject.OWNSMEMORY), + get_repr = c_char_p_get_repr, + ) +def c_char_p_get_field_annotation(s_char_p, fieldname): + assert fieldname == 'value' + return annmodel.SomeString(can_be_None=True) +entry.get_field_annotation = c_char_p_get_field_annotation Added: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Mon Apr 10 11:09:57 2006 @@ -0,0 +1,46 @@ +""" +Test the rctypes implementation. +""" + +import py.test +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy.translator.c.test.test_genc import compile +from pypy.annotation.model import SomeCTypesObject, SomeObject +from pypy import conftest +import sys +from pypy.rpython.test.test_llinterp import interpret + +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + +from ctypes import c_char_p + +class Test_annotation: + def test_annotate_c_char_p(self): + def func(): + p = c_char_p("hello") + return p.value + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + + assert s.knowntype == str + + if conftest.option.view: + t.view() + +class Test_specialization: + def test_specialize_c_char_p(self): + def func(): + p = c_char_p("hello") + return p.value + + res = interpret(func, []) + assert ''.join(res.chars) == "hello" + +class Test_compilation: + pass From arigo at codespeak.net Mon Apr 10 11:37:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 11:37:46 +0200 (CEST) Subject: [pypy-svn] r25654 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060410093746.27DDF10167@code0.codespeak.net> Author: arigo Date: Mon Apr 10 11:37:44 2006 New Revision: 25654 Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: Support and test for prebuilt c_char_p and pointer instances. Renamed c_data_ref to c_data for more uniformity in the ll helpers. Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Mon Apr 10 11:37:44 2006 @@ -14,12 +14,16 @@ return [('keepalive_str', string_repr.lowleveltype)] def getstring(self, llops, v_box): - v_c_data = self.get_c_data(llops, v_box) - return llops.gendirectcall(ll_getstring, v_box, v_c_data) + return llops.gendirectcall(ll_getstring, v_box) def setstring(self, llops, v_box, v_str): - v_c_data = self.get_c_data(llops, v_box) - llops.gendirectcall(ll_setstring, v_box, v_c_data, v_str) + llops.gendirectcall(ll_setstring, v_box, v_str) + + def initialize_const(self, p, string): + if isinstance(string, c_char_p): + string = string.value + llstring = string_repr.convert_const(string) + ll_setstring(p, llstring) def rtype_getattr(self, hop): s_attr = hop.args_s[1] @@ -55,8 +59,8 @@ def ll_str2charp(s): return llmemory.cast_ptr_to_adr(s.chars) + FIRSTITEMOFS -def ll_getstring(box, c_data): - p = c_data.value +def ll_getstring(box): + p = box.c_data.value if p: if (box.keepalive_str and ll_str2charp(box.keepalive_str) == p): maxlen = len(box.keepalive_str.chars) @@ -73,11 +77,11 @@ else: return lltype.nullptr(string_repr.lowleveltype.TO) -def ll_setstring(box, c_data, string): +def ll_setstring(box, string): if string: - c_data.value = ll_str2charp(string) + box.c_data.value = ll_str2charp(string) else: - c_data.value = llmemory.NULL + box.c_data.value = llmemory.NULL box.keepalive_str = string @@ -98,13 +102,16 @@ specialize_call=c_char_p_specialize_call ) +def c_char_compute_annotation(the_type, instance): + return annmodel.SomeCTypesObject(c_char_p, + annmodel.SomeCTypesObject.OWNSMEMORY) + def c_char_p_get_repr(rtyper, s_char_p): return CCharPRepr(rtyper, s_char_p, CCHARP) entry = extregistry.register_type(c_char_p, - compute_annotation = annmodel.SomeCTypesObject(c_char_p, - annmodel.SomeCTypesObject.OWNSMEMORY), - get_repr = c_char_p_get_repr, + compute_annotation = c_char_compute_annotation, + get_repr = c_char_p_get_repr, ) def c_char_p_get_field_annotation(s_char_p, fieldname): assert fieldname == 'value' Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Mon Apr 10 11:37:44 2006 @@ -63,7 +63,7 @@ self.r_memoryowner = rtyper.getrepr(s_memoryowner) self.lowleveltype = lltype.Ptr( lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data_ref", lltype.Ptr(self.c_data_type) ), + ( "c_data", lltype.Ptr(self.c_data_type) ), ( "c_data_owner_keepalive", self.r_memoryowner.lowleveltype ), *content_keepalives ) @@ -80,7 +80,7 @@ return llops.genop('getsubstruct', inputargs, lltype.Ptr(self.c_data_type) ) else: - inputargs = [v_box, inputconst(lltype.Void, "c_data_ref")] + inputargs = [v_box, inputconst(lltype.Void, "c_data")] return llops.genop('getfield', inputargs, lltype.Ptr(self.c_data_type) ) @@ -99,12 +99,12 @@ def allocate_instance_ref(self, llops, v_c_data, v_c_data_owner=None): """Only if self.ownsmemory is false. This allocates a new instance - and initialize its c_data_ref field.""" + and initialize its c_data pointer.""" if self.ownsmemory: raise TyperError("allocate_instance_ref: %r owns its memory" % ( self,)) v_box = self.allocate_instance(llops) - inputargs = [v_box, inputconst(lltype.Void, "c_data_ref"), v_c_data] + inputargs = [v_box, inputconst(lltype.Void, "c_data"), v_c_data] llops.genop('setfield', inputargs) if v_c_data_owner is not None: assert (v_c_data_owner.concretetype == @@ -172,3 +172,34 @@ """Writes to the 'value' field of the raw data.""" v_c_data = self.get_c_data(llops, v_box) self.setvalue_inside_c_data(llops, v_c_data, v_value) + + def convert_const(self, value): + if isinstance(value, self.ctype): + key = "by_id", id(value) + keepalive = value + else: + if self.ownsmemory: + raise TyperError("convert_const(%r) but repr owns memory" % ( + value,)) + key = "by_value", value + keepalive = None + try: + return self.const_cache[key][0] + except KeyError: + p = lltype.malloc(self.r_memoryowner.lowleveltype.TO) + self.initialize_const(p, value) + if self.ownsmemory: + result = p + else: + # we must return a non-memory-owning box that keeps the + # memory-owning box alive + result = lltype.malloc(self.lowleveltype.TO) + result.c_data = p.c_data # initialize c_data pointer + result.c_data_owner_keepalive = p + self.const_cache[key] = result, keepalive + return result + + def initialize_const(self, p, value): + if isinstance(value, self.ctype): + value = value.value + p.c_data.value = value Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 10 11:37:44 2006 @@ -30,6 +30,13 @@ v_owner] llops.genop('setfield', inputargs) + def initialize_const(self, p, ptr): + llcontents = self.r_contents.convert_const(ptr.contents) + p.c_data.value = llcontents.c_data + # the following line is probably pointless, as 'llcontents' will be + # an immortal global constant just like 'p', but better safe than sorry + p.keepalive_contents = llcontents.c_data_owner_keepalive + def rtype_getattr(self, hop): s_attr = hop.args_s[1] assert s_attr.is_constant() Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Mon Apr 10 11:37:44 2006 @@ -34,34 +34,6 @@ """ return self.getvalue_from_c_data(llops, v_c_data) - def convert_const(self, ctype_value): - if isinstance(ctype_value, self.ctype): - key = "by_id", id(ctype_value) - value = ctype_value.value - keepalive = ctype_value - else: - if self.ownsmemory: - raise TyperError("convert_const(%r) but repr owns memory" % ( - ctype_value,)) - key = "by_value", ctype_value - value = ctype_value - keepalive = None - try: - return self.const_cache[key][0] - except KeyError: - p = lltype.malloc(self.r_memoryowner.lowleveltype.TO) - p.c_data.value = value - if self.ownsmemory: - result = p - else: - # we must return a non-memory-owning box that keeps the - # memory-owning box alive - result = lltype.malloc(self.lowleveltype.TO) - result.c_data_ref = p.c_data - result.c_data_owner_keepalive = p - self.const_cache[key] = result, keepalive - return result - def rtype_getattr(self, hop): s_attr = hop.args_s[1] assert s_attr.is_constant() Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Mon Apr 10 11:37:44 2006 @@ -27,9 +27,19 @@ t = TranslationContext() a = t.buildannotator() s = a.build_types(func, []) - assert s.knowntype == str + if conftest.option.view: + t.view() + + def test_annotate_prebuilt(self): + p = c_char_p("hello") + def func(): + return p.value + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + assert s.knowntype == str if conftest.option.view: t.view() @@ -42,5 +52,29 @@ res = interpret(func, []) assert ''.join(res.chars) == "hello" + def test_specialize_prebuilt(self): + p = c_char_p("hello") + def func(): + return p.value + + res = interpret(func, []) + assert ''.join(res.chars) == "hello" + class Test_compilation: - pass + def test_compile_c_char_p(self): + def func(): + p = c_char_p("hello") + return p.value + + fn = compile(func, []) + res = fn() + assert res == "hello" + + def test_compile_prebuilt(self): + p = c_char_p("hello") + def func(): + return p.value + + fn = compile(func, []) + res = fn() + assert res == "hello" Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Mon Apr 10 11:37:44 2006 @@ -83,6 +83,21 @@ if conftest.option.view: t.view() + def test_annotate_prebuilt(self): + c = c_int(10) + p = pointer(c) + def access_prebuilt(): + p.contents.value += 1 + return p[0] + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(access_prebuilt, []) + assert s.knowntype == int + + if conftest.option.view: + t.view() + class Test_specialization: def test_specialize_c_int_ptr(self): @@ -172,3 +187,13 @@ assert access_array() == 5 * 7 * 11 res = interpret(access_array, []) assert res == 5 * 7 * 11 + + def test_specialize_prebuilt(self): + c = c_int(10) + p = pointer(c) + def access_prebuilt(): + p.contents.value += 1 + return p[0] + + res = interpret(access_prebuilt, []) + assert res == 11 From arigo at codespeak.net Mon Apr 10 11:58:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 11:58:49 +0200 (CEST) Subject: [pypy-svn] r25656 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060410095849.C437E10122@code0.codespeak.net> Author: arigo Date: Mon Apr 10 11:58:48 2006 New Revision: 25656 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: Adding tests (some in-progress) about calling atoi(). Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Mon Apr 10 11:58:48 2006 @@ -1,21 +1,36 @@ +import py from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rctypes.test.test_rctypes import mylib from pypy.rpython.test.test_llinterp import interpret from pypy.translator.c.test.test_genc import compile from pypy import conftest +from pypy.rpython.rstr import string_repr -from ctypes import c_long +from ctypes import c_int, c_long, c_char_p labs = mylib.labs labs.restype = c_long labs.argtypes = [c_long] - def ll_labs(n): return abs(n) - labs.llinterp_friendly_version = ll_labs +atoi = mylib.atoi +atoi.restype = c_int +atoi.argtypes = [c_char_p] +def ll_atoi(s): + "Very approximative ll implementation of atoi(), for testing" + i = result = 0 + while i < len(s.chars): + if '0' <= s.chars[i] <= '9': + result = result * 10 + ord(s.chars[i]) - ord('0') + else: + break + i += 1 + return result +atoi.llinterp_friendly_version = ll_atoi + def test_labs(n=6): assert labs(n) == abs(n) @@ -23,6 +38,12 @@ assert labs(-42) == 42 return labs(n) +def test_ll_atoi(): + assert ll_atoi(string_repr.convert_const("")) == 0 + assert ll_atoi(string_repr.convert_const("42z7")) == 42 + assert ll_atoi(string_repr.convert_const("blah")) == 0 + assert ll_atoi(string_repr.convert_const("18238")) == 18238 + class Test_annotation: def test_annotate_labs(self): a = RPythonAnnotator() @@ -31,11 +52,29 @@ if conftest.option.view: a.translator.view() + def test_annotate_atoi(self): + def fn(s): + return atoi(s) + a = RPythonAnnotator() + s = a.build_types(fn, [str]) + assert s.knowntype == int + if conftest.option.view: + a.translator.view() + class Test_specialization: def test_specialize_labs(self): res = interpret(test_labs, [-11]) assert res == 11 + def test_specialize_atoi(self): + py.test.skip("in-progress") + choices = ["", "42z7", "blah", "18238"] + def fn(n): + return atoi(choices[n]) + + res = [interpret(fn, [i]) for i in range(4)] + assert res == [0, 42, 0, 18238] + class Test_compile: def test_compile_labs(self): fn = compile(test_labs, [int]) From mwh at codespeak.net Mon Apr 10 12:16:50 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 10 Apr 2006 12:16:50 +0200 (CEST) Subject: [pypy-svn] r25657 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060410101650.E39BD10180@code0.codespeak.net> Author: mwh Date: Mon Apr 10 12:16:49 2006 New Revision: 25657 Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py Log: (arigo, mwh) support stackless transforming direct_calls that are protected by except blocks. still very very broken in the 'resume with exception' case, but what else is new? Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Mon Apr 10 12:16:49 2006 @@ -34,10 +34,9 @@ from pypy.translator.stackless import code def test_simple_transform_llinterp(): - from pypy.translator.stackless.code import UnwindException def check(x): if x: - raise UnwindException + raise code.UnwindException def g(x): check(x) return x + 1 @@ -47,10 +46,9 @@ assert res == 3 def test_simple_transform(): - from pypy.translator.stackless.code import UnwindException def check(x): if x: - raise UnwindException # XXX or so + raise code.UnwindException # XXX or so def g(x): check(x) return x + 1 @@ -59,6 +57,26 @@ res = run_stackless_function(example, example, g) assert res.strip() == "3" +def test_protected_call(): + def check(x): + if x: + raise code.UnwindException + def g(x): + check(x) + return x + 1 + def example(x): + try: + y = g(x) + except Exception: + y = -1 + return y + 1 + res = llinterp_stackless_function(example, example, g) + assert res == 3 + res = run_stackless_function(example, example, g) + assert res == "3" + + + def rtype_stackless_function(fn, *stacklessfuncs): s_list_of_strings = annmodel.SomeList(ListDef(None, annmodel.SomeString())) s_list_of_strings.listdef.resize() Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 10 12:16:49 2006 @@ -226,11 +226,15 @@ while i < len(block.operations): op = block.operations[i] if op.opname in ('direct_call', 'indirect_call'): - link = support.split_block_with_keepalive(self.translator, block, i+1) + if i == len(block.operations) - 1 and block.exitswitch == model.c_last_exception: + link = block.exits[0] + else: + link = support.split_block_with_keepalive(self.translator, block, i+1) + block.exitswitch = model.c_last_exception var_unwind_exception = varoftype(evalue) args = [v for v in link.args - if v is not op.result and v.concretetype is not lltype.Void] + if v is not op.result and v.concretetype is not lltype.Void] save_block, frame_state_type = self.generate_save_block( args, var_unwind_exception) @@ -238,10 +242,11 @@ newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) - block.exitswitch = model.c_last_exception newlink.last_exception = model.Constant(code.UnwindException, etype) - newlink.last_exc_value = var_unwind_exception - block.recloseblock(link, newlink) # exits.append(newlink) + newlink.last_exc_value = var_unwind_exception + newexits = list(block.exits) + newexits.insert(1, newlink) + block.recloseblock(*newexits) self.translator.rtyper._convert_link(block, newlink) block = link.target From auc at codespeak.net Mon Apr 10 12:23:45 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 10 Apr 2006 12:23:45 +0200 (CEST) Subject: [pypy-svn] r25658 - in pypy/dist/pypy/objspace: . constraint constraint/test Message-ID: <20060410102345.263D71016D@code0.codespeak.net> Author: auc Date: Mon Apr 10 12:23:42 2006 New Revision: 25658 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py pypy/dist/pypy/objspace/constraint/test/test_fd.py pypy/dist/pypy/objspace/logic.py Log: tries to avoid too much un/wrapping Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Mon Apr 10 12:23:42 2006 @@ -15,10 +15,12 @@ class W_Variable(Wrappable): def __init__(self, obj_space, w_name): self._space = obj_space - self.name = w_name + self.w_name = w_name -W_Variable.typedef = typedef.TypeDef("W_Variable") + def name_w(self): + return self._space.str_w(self.w_name) +W_Variable.typedef = typedef.TypeDef("W_Variable") #-- Constraints ------------------------- @@ -35,9 +37,6 @@ def w_estimate_cost(self, w_cs): pass - def w_copy_to(self, w_computation_space): - pass - def w_revise(self, w_cs): pass @@ -46,9 +45,7 @@ affected_variables = interp2app(W_Constraint.w_affected_variables), is_variable_relevant = interp2app(W_Constraint.w_is_variable_relevant), estimate_cost = interp2app(W_Constraint.w_estimate_cost), - copy_to = interp2app(W_Constraint.w_copy_to), - revise = interp2app(W_Constraint.w_revise) - ) + revise = interp2app(W_Constraint.w_revise)) #-- Computation space ------------------- @@ -69,7 +66,7 @@ def w_dom(self, w_variable): assert isinstance(w_variable, W_Variable) - return self.domains.content[w_variable.name] + return self.domains.content[w_variable.w_name] def w_tell(self, w_constraint): assert isinstance(w_constraint, W_Constraint) Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Mon Apr 10 12:23:42 2006 @@ -32,18 +32,19 @@ """variables is a list of variables which appear in the formula""" W_Constraint.__init__(self, object_space) assert isinstance(w_variables, W_ListObject) - assert self._space.is_true(self._space.ge(self._space.len(w_variables), self._space.newint(1))) + assert self._space.is_true(self._space.ge(self._space.len(w_variables), + self._space.newint(1))) self._names_to_vars = {} for var in w_variables.wrappeditems: - self._names_to_vars[var.name] = var - self._variables = w_variables + self._names_to_vars[var.name_w()] = var + self._variables = w_variables.wrappeditems #unwrap once ... def w_affected_variables(self): """ Return a list of all variables affected by this constraint """ - return self._variables + return self._space.newlist(self._variables) - def w_is_variable_relevant(self, w_variable): - return variable in self._variables + def w_knows_var(self, w_variable): + return self._space.newbool(variable in self._variables) def w_estimate_cost(self, w_cs): """Return an estimate of the cost of the narrowing of the constraint""" @@ -58,23 +59,9 @@ W_AbstractConstraint.typedef = typedef.TypeDef("W_AbstractConstraint", affected_variables = interp2app(W_AbstractConstraint.w_affected_variables), - is_variable_relevant = interp2app(W_AbstractConstraint.w_is_variable_relevant), + knows_var = interp2app(W_AbstractConstraint.w_knows_var), estimate_cost = interp2app(W_AbstractConstraint.w_estimate_cost), - copy_to = interp2app(W_AbstractConstraint.w_copy_to), - revise = interp2app(W_AbstractConstraint.w_revise) -) - - - -def make_lambda_head(space, w_vars): - var_ids = ','.join([space.str_w(var.name) for var in w_vars.wrappeditems]) - return 'lambda ' + var_ids + ':' - -def expand_expr_template(space, w_expr, w_vars): - return space.str_w(w_expr) - for w_var in w_vars.wrappeditems: - expr.replace(w_var.name, w_var.name + '.val') - return expr + revise = interp2app(W_AbstractConstraint.w_revise)) class W_AllDistinct(W_AbstractConstraint): @@ -82,13 +69,9 @@ def __init__(self, object_space, w_variables): W_AbstractConstraint.__init__(self, object_space, w_variables) - assert len(w_variables.wrappeditems)>1 # worst case complexity self.__cost = len(w_variables.wrappeditems) * (len(w_variables.wrappeditems) - 1) / 2 - def w__repr__(self): - return self._space.newstring('' % str(self._variables)) - def w_estimate_cost(self, w_cs): assert isinstance(w_cs, W_ComputationSpace) return self._space.newint(self.__cost) @@ -104,7 +87,7 @@ assert isinstance(w_cs, W_ComputationSpace) variables = [(self._space.int_w(w_cs.w_dom(variable).w_size()), variable, w_cs.w_dom(variable)) - for variable in self._variables.wrappeditems] + for variable in self._variables] variables.sort() # if a domain has a size of 1, @@ -145,53 +128,58 @@ W_AllDistinct.typedef = typedef.TypeDef( "W_AllDistinct", W_AbstractConstraint.typedef, estimate_cost = interp2app(W_AllDistinct.w_estimate_cost), - revise = interp2app(W_AllDistinct.w_revise), - __repr__ = interp2app(W_AllDistinct.w__repr__) - ) + revise = interp2app(W_AllDistinct.w_revise)) # function bolted into the space to serve as constructor -def make_alldistinct(objectspace, w_variables): - return objectspace.wrap(W_AllDistinct(objectspace, w_variables)) +def make_alldistinct(object_space, w_variables): + return object_space.wrap(W_AllDistinct(object_space, w_variables)) app_make_alldistinct = gateway.interp2app(make_alldistinct) + +def make_filter(object_space, w_variables, w_formula): + """NOT RPYTHON""" + var_ids = ','.join([object_space.str_w(var.w_name) + for var in w_variables.wrappeditems]) + func_head = 'lambda ' + var_ids + ':' + func_obj = eval(func_head + object_space.str_w(w_formula), {}, {}) + return func_obj +app_make_filter = gateway.interp2app(make_filter) + class W_Expression(W_AbstractConstraint): """A constraint represented as a python expression.""" - _FILTER_CACHE = {} - def __init__(self, o_space, w_variables, w_formula): + def __init__(self, object_space, w_variables, w_formula): """variables is a list of variables which appear in the formula formula is a python expression that will be evaluated as a boolean""" - self.formula = w_formula - W_AbstractConstraint.__init__(self, o_space, w_variables) - self.filter_func = eval(make_lambda_head(self._space, w_variables) \ - + self._space.str_w(w_formula), {}, {}) + W_AbstractConstraint.__init__(self, object_space, w_variables) + self.formula = self._space.str_w(w_formula) + self.filter_func = make_filter(self._space, w_variables, w_formula) def test_solution(self, sol ): """test a solution against this constraint accept a mapping of variable names to value""" args = [] for var in self._variables: - args.append( sol[var.name] ) + args.append( sol[var.w_name] ) return self.filterFunc( *args ) def _init_result_cache(self): """key = (variable,value), value = [has_success,has_failure]""" result_cache = {} - for var_name in self._variables.wrappeditems: - result_cache[self._space.str_w(var_name.name)] = {} + for var in self._variables: + result_cache[var.name_w()] = {} return result_cache - def _assign_values(self, w_cs): variables = [] kwargs = {} - for variable in self._variables.wrappeditems: + for variable in self._variables: domain = w_cs.w_dom(variable) values = domain.w_get_values() variables.append((self._space.int_w(domain.w_size()), [variable, values, 0, self._space.len(values)])) - kwargs[self._space.str_w(variable.name)] = values.wrappeditems[0] + kwargs[variable.name_w()] = values.wrappeditems[0] # sort variables to instanciate those with fewer possible values first variables.sort() @@ -224,6 +212,8 @@ break else: continue + print kwargs.items() + print ffunc, kwargs.items() if ffunc(**kwargs): for var, val in kwargs.items(): result_cache[var][val] = 1 @@ -264,9 +254,6 @@ assert len(variables) == 2 Expression.__init__(self, variables, formula, type) - def copy_to(self, space): - raise NotImplementedError - def w_revise(self, domains): """specialized narrowing algorithm for binary expressions Runs much faster than the generic version""" @@ -321,89 +308,5 @@ return W_BinaryExpression(o_space, w_variables, w_formula) else: return W_Expression(o_space, w_variables, w_formula) - - app_make_expression = gateway.interp2app(make_expression) -# have a look at this later ... (really needed ?) -class W_BasicConstraint(W_Constraint): - """A BasicConstraint, which is never queued by the Repository - A BasicConstraint affects only one variable, and will be entailed - on the first call to narrow()""" - - def __init__(self, object_space, variable, reference, operator): - """variables is a list of variables on which - the constraint is applied""" - W_Constraint.__init__(self, object_space) - self._variable = variable - self._reference = reference - self._operator = operator - - def __repr__(self): - return '<%s %s %s>'% (self.__class__, self._variable, self._reference) - - def w_is_variable_relevant(self, w_variable): - return variable == self._variable - - def w_estimate_cost(self): - return self._space.newint(0) # get in the first place in the queue - - def w_affected_variables(self): - return [self._variable] - - def getVariable(self): - return self._variable - - def w_revise(self, w_domains): - domain = domains[self._variable] - operator = self._operator - ref = self._reference - try: - for val in domain.get_values() : - if not operator(val, ref) : - domain.remove_value(val) - except ConsistencyFailure: - raise ConsistencyFailure('inconsistency while applying %s' % \ - repr(self)) - return 1 - - def __eq__(self, other): - raise NotImplementedError - -W_BasicConstraint.typedef = typedef.TypeDef( - "W_BasicConstraint", - affected_variables = interp2app(W_BasicConstraint.w_affected_variables), - is_variable_relevant = interp2app(W_BasicConstraint.w_is_variable_relevant), - estimate_cost = interp2app(W_BasicConstraint.w_estimate_cost), - copy_to = interp2app(W_BasicConstraint.w_copy_to), - revise = interp2app(W_BasicConstraint.w_revise) - ) -class W_Equals(W_BasicConstraint): - """A basic constraint variable == constant value""" - def __init__(self, variable, reference): - W_BasicConstraint.__init__(self, variable, reference, operator.eq) - -class W_NotEquals(W_BasicConstraint): - """A basic constraint variable != constant value""" - def __init__(self, variable, reference): - W_BasicConstraint.__init__(self, variable, reference, operator.ne) - -class W_LesserThan(W_BasicConstraint): - """A basic constraint variable < constant value""" - def __init__(self, variable, reference): - W_BasicConstraint.__init__(self, variable, reference, operator.lt) - -class W_LesserOrEqual(W_BasicConstraint): - """A basic constraint variable <= constant value""" - def __init__(self, variable, reference): - W_BasicConstraint.__init__(self, variable, reference, operator.le) - -class W_GreaterThan(W_BasicConstraint): - """A basic constraint variable > constant value""" - def __init__(self, variable, reference): - W_BasicConstraint.__init__(self, variable, reference, operator.gt) - -class W_GreaterOrEqual(W_BasicConstraint): - """A basic constraint variable >= constant value""" - def __init__(self, variable, reference): - W_BasicConstraint.__init__(self, variable, reference, operator.ge) Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Mon Apr 10 12:23:42 2006 @@ -22,20 +22,20 @@ def __init__(self, space): self._space = space - self.__changed = self._space.newbool(False) + self.__changed = False def w_reset_flags(self): - self.__changed = self._space.newbool(False) + self.__changed = False def w_has_changed(self): - return self.__changed + return self._space.newbool(self.__changed) def w_size(self): pass def _value_removed(self): """The implementation of remove_value should call this method""" - self.__changed = self._space.newbool(True) + self.__changed = True if self._space.eq_w(self.w_size(), self._space.newint(0)): raise OperationError(self._space.w_RuntimeError, self._space.wrap('ConsistencyFailure')) @@ -54,16 +54,20 @@ This class uses a dictionnary to make sure that there are no duplicate values""" W_AbstractDomain.__init__(self, space) - self._values = space.newdict([]) + self._values = {} self.set_values(w_values) def set_values(self, w_values): + """Objects in the value set can't be unwrapped unless we + specialize on specific types - this might need speccialization + of revise & friends + """ for w_v in w_values.wrappeditems: - self._values.content[w_v] = self._space.w_True + self._values[w_v] = self._space.w_True def w_remove_value(self, w_value): """Remove value of domain and check for consistency""" - del self._values.content[w_value] + del self._values[w_value] self._value_removed() def w_remove_values(self, w_values): @@ -71,19 +75,19 @@ if self._space.is_true(self._space.gt(self._space.len(w_values), self._space.newint(0))) : for w_val in w_values.wrappeditems : - del self._values.content [w_val] + del self._values[w_val] self._value_removed() __delitem__ = w_remove_value def w_size(self): """computes the size of a finite domain""" - return self._space.newint(len(self._values.content)) + return self._space.newint(len(self._values)) __len__ = w_size def w_get_values(self): """return all the values in the domain in an indexable sequence""" - return self._space.newlist([x for x in self._values.content]) + return self._space.newlist([x for x in self._values]) def __iter__(self): return iter(self._values) @@ -98,7 +102,7 @@ def __eq__(self, w_other): if not isinstance(w_other, W_FiniteDomain): return self._space.newbool(False) - return self._space.eq(self._values, w_other._values) + return self._space.newbool(self._values == w_other._values) def __ne__(self, w_other): @@ -118,8 +122,8 @@ def intersection__FiniteDomain_FiniteDomain(space, w_fd1, w_fd2): - w_v1 = w_fd1._values.content - w_res = [w_v for w_v in w_fd2.w_get_values().wrappeditems + w_v1 = w_fd1._values + w_res = [w_v for w_v in w_fd2._values if w_v in w_v1] return make_fd(space, space.newlist(w_res)) Modified: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Mon Apr 10 12:23:42 2006 @@ -71,8 +71,7 @@ cstr = make_expression([v1], '2*v1==2') assert str(cstr).startswith(' Author: mwh Date: Mon Apr 10 12:44:41 2006 New Revision: 25659 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py Log: (arigo, mwh) support resuming with an exception by using a helper to read the return value or raise as appropriate. you can't catch the excepton thus raised, however... Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Mon Apr 10 12:44:41 2006 @@ -93,3 +93,8 @@ # global_state.top = None # XXX and then insert the rtyped graph of this into functions +def fetch_retval_long(): + if global_state.exception: + raise global_state.exception + else: + return global_state.retval_long Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Mon Apr 10 12:44:41 2006 @@ -5,6 +5,7 @@ from pypy.translator.c import gc from pypy.rpython.memory.gctransform import varoftype from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython import llinterp from pypy.translator.translator import TranslationContext, graphof from pypy.objspace.flow.model import checkgraph from pypy.annotation import model as annmodel @@ -74,6 +75,49 @@ assert res == 3 res = run_stackless_function(example, example, g) assert res == "3" + +def test_resume_with_exception(): + def check(x): + if x: + raise code.UnwindException + def g(x): + check(x) + if x: + raise KeyError + else: + return x + 1 + def h(x): + return g(x) + def example(x): + y = h(x) + return y + 1 + info = py.test.raises( + llinterp.LLException, + "llinterp_stackless_function(example, example, g, h)") + assert llinterp.type_name(info.value.args[0]) == 'KeyError' + +def test_resume_with_exception_handling(): + py.test.skip("in progress") + def check(x): + if x: + raise code.UnwindException + def g(x): + check(x) + if x: + raise KeyError + else: + return x + 1 + def h(x): + return g(x) + def example(x): + try: + y = h(x) + except KeyError: + y = -1 + return y + 1 + res = llinterp_stackless_function(example, example, g, h) + assert res == 0 + @@ -137,7 +181,6 @@ r_list_of_strings = t.rtyper.getrepr( t.annotator.binding(graphof(t, entry_point).startblock.inputargs[0])) ll_list = r_list_of_strings.convert_const(['']) - from pypy.rpython.llinterp import LLInterpreter - interp = LLInterpreter(t.rtyper) + interp = llinterp.LLInterpreter(t.rtyper) res = interp.eval_graph(graphof(t, entry_point), [ll_list]) return res Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 10 12:44:41 2006 @@ -108,6 +108,14 @@ graph=resume_state_graph), lltype.Ptr(RESUME_STATE_TYPE)) + FETCH_RETVAL_LONG_TYPE = lltype.FuncType([], lltype.Signed) + fetch_retval_long_graph = mixlevelannotator.getgraph( + code.fetch_retval_long, [], annmodel.SomeInteger()) + self.fetch_retval_long_ptr = model.Constant(lltype.functionptr( + FETCH_RETVAL_LONG_TYPE, "fetch_retval_long", + graph=fetch_retval_long_graph), + lltype.Ptr(FETCH_RETVAL_LONG_TYPE)) + mixlevelannotator.finish() s_global_state = bk.immutablevalue(code.global_state) @@ -196,7 +204,7 @@ for arg in resume_point.link_to_resumption.args: newarg = copyvar(self.translator, arg) if arg is resume_point.var_result: - ops.extend(self.ops_read_global_state_field(newarg, "retval_long")) + ops.append(model.SpaceOperation("direct_call", [self.fetch_retval_long_ptr], newarg)) else: # frame_state_type._names[0] is 'header' fname = model.Constant(frame_state_type._names[i+1], lltype.Void) From mwh at codespeak.net Mon Apr 10 12:55:10 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 10 Apr 2006 12:55:10 +0200 (CEST) Subject: [pypy-svn] r25660 - pypy/dist/pypy/translator/stackless Message-ID: <20060410105510.84499100EB@code0.codespeak.net> Author: mwh Date: Mon Apr 10 12:55:09 2006 New Revision: 25660 Modified: pypy/dist/pypy/translator/stackless/transform.py Log: (arigo, mwh) some notes on how to make test_resume_with_exception_handling pass. Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 10 12:55:09 2006 @@ -241,6 +241,19 @@ block.exitswitch = model.c_last_exception var_unwind_exception = varoftype(evalue) + # for the case where we are resuming to an except: + # block we need to store here a list of links that + # might be resumed to, and in insert_resume_handling + # we need to basically copy each link onto the + # resuming block. + # + # it probably also makes sense to compute the list of + # args to save once, here, and save that too. + # + # finally, it is important that the fetch_retval + # function be called right at the end of the resuming + # block, and that it is called even if the return + # value is not again used. args = [v for v in link.args if v is not op.result and v.concretetype is not lltype.Void] save_block, frame_state_type = self.generate_save_block( From stephan at codespeak.net Mon Apr 10 13:29:57 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Mon, 10 Apr 2006 13:29:57 +0200 (CEST) Subject: [pypy-svn] r25661 - pypy/dist/pypy/objspace/std Message-ID: <20060410112957.20AED100CE@code0.codespeak.net> Author: stephan Date: Mon Apr 10 13:29:55 2006 New Revision: 25661 Modified: pypy/dist/pypy/objspace/std/setobject.py pypy/dist/pypy/objspace/std/settype.py Log: somewhat cleaner set/frozenset implementation. set 'WITHSET = True' in model.py Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Mon Apr 10 13:29:55 2006 @@ -3,16 +3,18 @@ from pypy.objspace.std.model import WITHSET from pypy.objspace.std.stdtypedef import StdObjSpaceMultiMethod from pypy.rpython.objectmodel import r_dict +from pypy.rpython.rarithmetic import intmask from pypy.interpreter import gateway class W_BaseSetObject(W_Object): - def __init__(w_self, space, wrappeditems): + def __init__(w_self, space, wrappeditems=None): W_Object.__init__(w_self, space) w_self.setdata = setdata = r_dict(space.eq_w, space.hash_w) - iterable_w = space.unpackiterable(wrappeditems) - for w_item in iterable_w: - w_self.setdata[w_item] = None + if wrappeditems is not None: + iterable_w = space.unpackiterable(wrappeditems) + for w_item in iterable_w: + w_self.setdata[w_item] = None def __repr__(w_self): """representation for debugging purposes""" @@ -25,6 +27,10 @@ class W_FrozensetObject(W_BaseSetObject): from pypy.objspace.std.settype import frozenset_typedef as typedef + def __init__(w_self, space, wrappeditems): + W_BaseSetObject.__init__(w_self, space, wrappeditems) + w_self.hash = -1 + registerimplementation(W_SetObject) registerimplementation(W_FrozensetObject) @@ -73,6 +79,14 @@ # some helper functions +def _iter_to_dict(space, w_iterable): + data = r_dict(space.eq_w, space.hash_w) + iterable_w = space.unpackiterable(w_iterable) + for w_item in iterable_w: + data[w_item] = None + + return data + def _dict_to_set(space, rpdict): return space.newset(W_SetIterObject(space, rpdict)) @@ -82,6 +96,13 @@ # helper functions for set operation on dicts +def _is_setlike(space, w_obj): + if space.is_true(space.isinstance(w_obj, space.w_set)) or \ + space.is_true(space.isinstance(w_obj, space.w_frozenset)): + return True + else: + return False + def _union_dict(space, ldict, rdict, isupdate): if isupdate: ld = ldict @@ -145,16 +166,14 @@ #end helper functions -def set_update__Set_Set(space, w_left, w_other): +def set_update__Set_ANY(space, w_left, w_other): """Update a set with the union of itself and another.""" - ld, rd = w_left.setdata, w_other.setdata + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _union_dict(space, ld, rd, True) return space.w_None -set_update__Set_Frozenset = set_update__Set_Set - def inplace_or__Set_Set(space, w_left, w_other): - set_update__Set_Set(space, w_left, w_other) + set_update__Set_ANY(space, w_left, w_other) return w_left inplace_or__Set_Frozenset = inplace_or__Set_Set @@ -177,39 +196,37 @@ w_left.setdata.clear() return space.w_None -def set_difference__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_difference__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _difference_dict(space, ld, rd, False) return _dict_to_set(space, new_ld) -set_difference__Set_Frozenset = set_difference__Set_Set -sub__Set_Set = set_difference__Set_Set -sub__Set_Frozenset = set_difference__Set_Set +sub__Set_Set = set_difference__Set_ANY +sub__Set_Frozenset = set_difference__Set_ANY -def frozenset_difference__Frozenset_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def frozenset_difference__Frozenset_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _difference_dict(space, ld, rd, False) return _dict_to_frozenset(space, new_ld) -frozenset_difference__Frozenset_Frozenset = frozenset_difference__Frozenset_Set -sub__Frozenset_Set = frozenset_difference__Frozenset_Set -sub__Frozenset_Frozenset = frozenset_difference__Frozenset_Set +sub__Frozenset_Set = frozenset_difference__Frozenset_ANY +sub__Frozenset_Frozenset = frozenset_difference__Frozenset_ANY -def set_difference_update__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_difference_update__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _difference_dict(space, ld, rd, True) return space.w_None -set_difference_update__Set_Frozenset = set_difference_update__Set_Set - def inplace_sub__Set_Set(space, w_left, w_other): - set_difference_update__Set_Set(space, w_left, w_other) + set_difference_update__Set_ANY(space, w_left, w_other) return w_left inplace_sub__Set_Frozenset = inplace_sub__Set_Set -def eq__Set_Set(space, w_left, w_other): +def eq__Set_ANY(space, w_left, w_other): + if not _is_setlike(space, w_other): + return space.w_False if space.is_w(w_left, w_other): return space.w_True @@ -221,12 +238,18 @@ return space.w_False return space.w_True -eq__Set_Frozenset = eq__Set_Set -eq__Frozenset_Frozenset = eq__Set_Set -eq__Frozenset_Set = eq__Set_Set +eq__Frozenset_ANY = eq__Set_ANY def contains__Set_ANY(space, w_left, w_other): - return space.newbool(w_other in w_left.setdata) + try: + r = w_other in w_left.setdata + return space.newbool(r) + except Exception, exp: + if _is_setlike(space, w_other): + w_f = space.newfrozenset(w_other) + return space.newbool(w_f) + else: + return space.w_False contains__Frozenset_ANY = contains__Set_ANY @@ -283,10 +306,26 @@ raise OperationError(space.w_KeyError, space.call_method(w_item,'__repr__')) -def hash__Set(space, w_left): +def hash__Set(space, w_set): raise OperationError(space.w_TypeError, space.wrap('set objects are unhashable')) +def hash__Frozenset(space, w_set): + if w_set.hash != -1: + return space.wrap(w_set.hash) + hash = 1927868237 + hash *= (len(w_set.setdata) + 1) + for w_item in w_set.setdata.iterkeys(): + h = space.int_w(space.hash(w_item)) + hash ^= (h ^ (h << 16) ^ 89869747) * 3644798167 + hash = hash * 69069 + 907133923 + if hash == -1: + hash = 590923713 + hash = intmask(hash) + w_set.hash = hash + + return space.wrap(hash) + def set_pop__Set(space, w_left): if len(w_left.setdata) == 0: raise OperationError(space.w_KeyError, @@ -297,88 +336,75 @@ return w_value -def set_intersection__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_intersection__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _intersection_dict(space, ld, rd, False) return _dict_to_set(space, new_ld) -set_intersection__Set_Frozenset = set_intersection__Set_Set -and__Set_Set = set_intersection__Set_Set -and__Set_Frozenset = set_intersection__Set_Set +and__Set_Set = set_intersection__Set_ANY +and__Set_Frozenset = set_intersection__Set_ANY -def frozenset_intersection__Frozenset_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def frozenset_intersection__Frozenset_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _intersection_dict(space, ld, rd, False) return _dict_to_frozenset(space, new_ld) -frozenset_intersection__Frozenset_Frozenset = \ - frozenset_intersection__Frozenset_Set -and__Frozenset_Set = frozenset_intersection__Frozenset_Set -and__Frozenset_Frozenset = frozenset_intersection__Frozenset_Set +and__Frozenset_Set = frozenset_intersection__Frozenset_ANY +and__Frozenset_Frozenset = frozenset_intersection__Frozenset_ANY -def set_intersection_update__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_intersection_update__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _intersection_dict(space, ld, rd, True) return space.w_None -set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set - def inplace_and__Set_Set(space, w_left, w_other): - set_intersection_update__Set_Set(space, w_left, w_other) + set_intersection_update__Set_ANY(space, w_left, w_other) return w_left inplace_and__Set_Frozenset = inplace_and__Set_Set -def set_symmetric_difference__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_symmetric_difference__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) return _dict_to_set(space, new_ld) -set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set -xor__Set_Set = set_symmetric_difference__Set_Set -xor__Set_Frozenset = set_symmetric_difference__Set_Set +xor__Set_Set = set_symmetric_difference__Set_ANY +xor__Set_Frozenset = set_symmetric_difference__Set_ANY -def frozenset_symmetric_difference__Frozenset_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def frozenset_symmetric_difference__Frozenset_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) return _dict_to_frozenset(space, new_ld) -frozenset_symmetric_difference__Frozenset_Frozenset = \ - frozenset_symmetric_difference__Frozenset_Set -xor__Frozenset_Set = frozenset_symmetric_difference__Frozenset_Set -xor__Frozenset_Frozenset = frozenset_symmetric_difference__Frozenset_Set +xor__Frozenset_Set = frozenset_symmetric_difference__Frozenset_ANY +xor__Frozenset_Frozenset = frozenset_symmetric_difference__Frozenset_ANY -def set_symmetric_difference_update__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_symmetric_difference_update__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _symmetric_difference_dict(space, ld, rd, True) return space.w_None -set_symmetric_difference_update__Set_Frozenset = \ - set_symmetric_difference_update__Set_Set - def inplace_xor__Set_Set(space, w_left, w_other): - set_symmetric_difference_update__Set_Set(space, w_left, w_other) + set_symmetric_difference_update__Set_ANY(space, w_left, w_other) return w_left inplace_xor__Set_Frozenset = inplace_xor__Set_Set -def set_union__Set_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def set_union__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _union_dict(space, ld, rd, False) return _dict_to_set(space, new_ld) -set_union__Set_Frozenset = set_union__Set_Set -or__Set_Set = set_union__Set_Set -or__Set_Frozenset = set_union__Set_Set +or__Set_Set = set_union__Set_ANY +or__Set_Frozenset = set_union__Set_ANY -def frozenset_union__Frozenset_Set(space, w_left, w_other): - ld, rd = w_left.setdata, w_other.setdata +def frozenset_union__Frozenset_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _union_dict(space, ld, rd, False) return _dict_to_frozenset(space, new_ld) -frozenset_union__Frozenset_Frozenset = frozenset_union__Frozenset_Set -or__Frozenset_Set = frozenset_union__Frozenset_Set -or__Frozenset_Frozenset = frozenset_union__Frozenset_Set +or__Frozenset_Set = frozenset_union__Frozenset_ANY +or__Frozenset_Frozenset = frozenset_union__Frozenset_ANY def len__Set(space, w_left): return space.newint(len(w_left.setdata)) @@ -398,8 +424,14 @@ cmp__Frozenset_Frozenset = cmp__Set_Set cmp__Frozenset_Set = cmp__Set_Set +def init__Set(space, w_set, __args__): + w_iterable, = __args__.parse('set', + (['some_iterable'], None, None), + [W_SetObject(space,None)]) + W_SetObject.__init__(w_set, space, w_iterable) + app = gateway.applevel(""" - def ne__Set_Set(s, o): + def ne__Set_ANY(s, o): return not s == o def gt__Set_Set(s, o): @@ -416,10 +448,8 @@ """, filename=__file__) -ne__Set_Set = app.interphook("ne__Set_Set") -ne__Set_Frozenset = ne__Set_Set -ne__Frozenset_Set = ne__Set_Set -ne__Frozenset_Frozenset = ne__Set_Set +ne__Set_ANY = app.interphook("ne__Set_ANY") +ne__Frozenset_ANY = ne__Set_ANY gt__Set_Set = app.interphook("gt__Set_Set") gt__Set_Frozenset = gt__Set_Set Modified: pypy/dist/pypy/objspace/std/settype.py ============================================================================== --- pypy/dist/pypy/objspace/std/settype.py (original) +++ pypy/dist/pypy/objspace/std/settype.py Mon Apr 10 13:29:55 2006 @@ -37,9 +37,6 @@ from pypy.objspace.std.setobject import W_SetObject if w_iterable is None: w_iterable = space.newtuple([]) - elif (space.is_w(w_settype, space.w_set) and - space.is_w(space.type(w_iterable), space.w_set)): - return w_iterable w_obj = space.allocate_instance(W_SetObject, w_settype) W_SetObject.__init__(w_obj, space, w_iterable) @@ -49,9 +46,6 @@ from pypy.objspace.std.setobject import W_FrozensetObject if w_iterable is None: w_iterable = space.newtuple([]) - elif (space.is_w(w_frozensettype, space.w_frozenset) and - space.is_w(space.type(w_iterable), space.w_frozenset)): - return w_iterable w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype) W_FrozensetObject.__init__(w_obj, space, w_iterable) From auc at codespeak.net Mon Apr 10 15:52:51 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 10 Apr 2006 15:52:51 +0200 (CEST) Subject: [pypy-svn] r25662 - in pypy/dist/pypy/objspace: . constraint constraint/test Message-ID: <20060410135251.B81381011F@code0.codespeak.net> Author: auc Date: Mon Apr 10 15:52:49 2006 New Revision: 25662 Modified: pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py pypy/dist/pypy/objspace/logic.py Log: uglification of expression/revise & friends Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Mon Apr 10 15:52:49 2006 @@ -8,8 +8,6 @@ from pypy.objspace.constraint.computationspace import W_ComputationSpace from pypy.objspace.constraint.computationspace import W_Constraint - -#from variable import NoDom import operator #-- Exceptions --------------------------------------- @@ -165,21 +163,21 @@ def _init_result_cache(self): """key = (variable,value), value = [has_success,has_failure]""" - result_cache = {} + result_cache = self._space.newdict({}) for var in self._variables: - result_cache[var.name_w()] = {} + result_cache.content[var.w_name] = self._space.newdict({}) return result_cache def _assign_values(self, w_cs): variables = [] - kwargs = {} + kwargs = self._space.newdict({}) for variable in self._variables: domain = w_cs.w_dom(variable) values = domain.w_get_values() variables.append((self._space.int_w(domain.w_size()), - [variable, values, 0, + [variable, values, self._space.newint(0), self._space.len(values)])) - kwargs[variable.name_w()] = values.wrappeditems[0] + kwargs.content[variable.w_name] = values.wrappeditems[0] # sort variables to instanciate those with fewer possible values first variables.sort() @@ -188,13 +186,13 @@ yield kwargs # try to instanciate the next variable for size, curr in variables: - if (curr[2] + 1) < curr[-1]: - curr[2] += 1 - kwargs[curr[0].name] = curr[1][curr[2]] + if self._space.int_w(curr[2]) + 1 < self._space.int_w(curr[-1]): + curr[2] = self._space.add(curr[2], self._space.newint(1)) + kwargs.content[curr[0].w_name] = curr[1].wrappeditems[self._space.int_w(curr[2])] break else: - curr[2] = 0 - kwargs[curr[0].name] = curr[1][0] + curr[2] = self._space.newint(0) + kwargs.content[curr[0].w_name] = curr[1].wrappeditems[0] else: # it's over go_on = 0 @@ -202,28 +200,30 @@ def w_revise(self, w_cs): """generic propagation algorithm for n-ary expressions""" assert isinstance(w_cs, W_ComputationSpace) - maybe_entailed = 1 + maybe_entailed = self._space.newint(1) ffunc = self.filter_func result_cache = self._init_result_cache() for kwargs in self._assign_values(w_cs): if maybe_entailed: - for var, val in kwargs.iteritems(): - if val not in result_cache[var]: + for varname, val in kwargs.content.iteritems(): + if val not in result_cache.content[varname].content: break else: continue - print kwargs.items() - print ffunc, kwargs.items() - if ffunc(**kwargs): - for var, val in kwargs.items(): - result_cache[var][val] = 1 + if self._space.is_true(self._space.call(self._space.wrap(ffunc), + self._space.newlist([]), kwargs)): + for var, val in kwargs.content.items(): + result_cache.content[var].content[val] = self._space.w_True else: - maybe_entailed = 0 + maybe_entailed = self._space.newint(0) try: - for var, keep in result_cache.iteritems(): - domain = w_cs.w_dom(self._names_to_vars[var]) - domain.remove_values([val for val in domain if val not in keep]) + for varname, keep in result_cache.content.items(): + print keep + domain = w_cs.w_dom(self._names_to_vars[self._space.str_w(varname)]) + domain.w_remove_values(self._space.newlist([val + for val in domain._values + if val not in keep.content])) except ConsistencyFailure: raise ConsistencyFailure('Inconsistency while applying %s' % \ @@ -231,7 +231,7 @@ except KeyError: # There are no more value in result_cache pass - + return maybe_entailed Modified: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Mon Apr 10 15:52:49 2006 @@ -75,6 +75,6 @@ csp = newspace() v1 = csp.var('v1', FiniteDomain([1, 2])) cstr = make_expression([v1], '2*v1==2') - assert cstr.revise(csp) == 1 + assert cstr.revise(csp) == 0 assert csp.dom(v1).get_values() == [1] Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Mon Apr 10 15:52:49 2006 @@ -807,10 +807,6 @@ space.ExceptionTable.append('UnificationError') space.ExceptionTable.sort() # hmmm - # capture the original is_ op (?) - is_nb_ = space.is_ - space.is_nb_ = is_nb_ - space.setitem(space.builtin.w_dict, space.wrap('newvar'), space.wrap(app_newvar)) space.setitem(space.builtin.w_dict, space.wrap('is_free'), @@ -862,6 +858,10 @@ space.wrap(app_wait)) space.setitem(space.builtin.w_dict, space.wrap('wait_needed'), space.wrap(app_wait_needed)) + + # capture a bunch of non-blocking ops + space.is_nb_ = space.is_ + patch_space_in_place(space, 'logic', proxymaker) return space From arigo at codespeak.net Mon Apr 10 15:59:14 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 15:59:14 +0200 (CEST) Subject: [pypy-svn] r25663 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060410135914.832A410128@code0.codespeak.net> Author: arigo Date: Mon Apr 10 15:59:13 2006 New Revision: 25663 Added: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Log: A test for regular ctypes. The purpose of this test file is to check how ctypes really work, down to what aliases what and what exact types operations return. Added: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 10 15:59:13 2006 @@ -0,0 +1,58 @@ +""" +The purpose of this test file is to check how ctypes really work, +down to what aliases what and what exact types operations return. +""" + +from ctypes import * + +def test_primitive_pointer(): + x = c_int(5) + assert x.value == 5 + x.value = 6 + assert x.value == 6 + + p = pointer(x) + assert isinstance(p.contents, c_int) + p.contents.value += 1 + assert x.value == 7 + + y = c_int(12) + p.contents = y + p.contents.value += 2 + assert y.value == 14 + assert x.value == 7 + + pp = pointer(p) + pp.contents.contents = x + p.contents.value += 2 + assert x.value == 9 + + assert isinstance(p[0], int) + p[0] += 1 + assert x.value == 10 + z = c_int(86) + p[0] = z + assert x.value == 86 + z.value = 84 + assert x.value == 86 + + assert isinstance(pp[0], POINTER(c_int)) + assert pp[0].contents.value == x.value == 86 + pp[0].contents = z + assert p.contents.value == z.value == 84 + + q = pointer(y) + pp[0] = q + assert pp.contents.contents.value == y.value == 14 + + +def test_char_p(): + x = c_char_p("hello\x00world") + assert x.value == "hello" + x.value = "world" + assert x.value == "world" + + p = pointer(x) + assert p[0] == x.value == "world" + p[0] = "other" + assert x.value == p.contents.value == p[0] == "other" From arigo at codespeak.net Mon Apr 10 16:18:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 16:18:16 +0200 (CEST) Subject: [pypy-svn] r25664 - pypy/dist/pypy/tool Message-ID: <20060410141816.DC08D1014B@code0.codespeak.net> Author: arigo Date: Mon Apr 10 16:18:15 2006 New Revision: 25664 Modified: pypy/dist/pypy/tool/pydis.py Log: Fix imports. Modified: pypy/dist/pypy/tool/pydis.py ============================================================================== --- pypy/dist/pypy/tool/pydis.py (original) +++ pypy/dist/pypy/tool/pydis.py Mon Apr 10 16:18:15 2006 @@ -8,8 +8,8 @@ import autopath import sys -from pypy.tool.opcode import * -from pypy.tool.opcode import __all__ as _opcodes_all +from pypy.tool.stdlib_opcode import * +from pypy.tool.stdlib_opcode import __all__ as _opcodes_all __all__ = ["dis","pydisassemble","distb","disco"] + _opcodes_all del _opcodes_all From arigo at codespeak.net Mon Apr 10 16:50:31 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 16:50:31 +0200 (CEST) Subject: [pypy-svn] r25665 - pypy/dist/pypy/objspace/std Message-ID: <20060410145031.B43B9100B6@code0.codespeak.net> Author: arigo Date: Mon Apr 10 16:50:30 2006 New Revision: 25665 Modified: pypy/dist/pypy/objspace/std/stdtypedef.py Log: Safety-check: registering several multimethods of the same name on the same type is not a good idea. Modified: pypy/dist/pypy/objspace/std/stdtypedef.py ============================================================================== --- pypy/dist/pypy/objspace/std/stdtypedef.py (original) +++ pypy/dist/pypy/objspace/std/stdtypedef.py Mon Apr 10 16:50:30 2006 @@ -93,8 +93,13 @@ def hack_out_multimethods(ns): "NOT_RPYTHON: initialization-time only." result = [] + seen = {} for value in ns.itervalues(): if isinstance(value, StdObjSpaceMultiMethod): + if value.name in seen: + raise Exception("duplicate multimethod name %r" % + (value.name,)) + seen[value.name] = True result.append(value) return result From arigo at codespeak.net Mon Apr 10 17:03:03 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 17:03:03 +0200 (CEST) Subject: [pypy-svn] r25666 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060410150303.D058310116@code0.codespeak.net> Author: arigo Date: Mon Apr 10 17:03:02 2006 New Revision: 25666 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: Setting 'contents' and __setitem__ on ctypes pointers. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Mon Apr 10 17:03:02 2006 @@ -1,11 +1,10 @@ from ctypes import ARRAY, c_int from pypy.annotation.model import SomeCTypesObject, SomeBuiltin from pypy.rpython import extregistry -from pypy.rpython.rmodel import Repr, inputconst +from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype -from pypy.rpython.rmodel import IntegerRepr -from pypy.rpython.rctypes.rmodel import CTypesRefRepr +from pypy.rpython.rctypes.rmodel import CTypesRefRepr, reccopy ArrayType = type(ARRAY(c_int, 10)) @@ -46,25 +45,6 @@ return r_array.r_item.return_c_data(hop.llops, v_c_data) -def reccopy(llops, v_source, v_dest): - # helper (XXX move somewhere else) to copy recursively a structure - # or array onto another. - T = v_source.concretetype.TO - assert T == v_dest.concretetype.TO - assert isinstance(T, lltype.Struct) - for name in T._names: - FIELDTYPE = getattr(T, name) - cname = inputconst(lltype.Void, name) - if isinstance(FIELDTYPE, lltype.ContainerType): - RESTYPE = lltype.Ptr(FIELDTYPE) - v_subsrc = llops.genop('getsubstruct', [v_source, cname], RESTYPE) - v_subdst = llops.genop('getsubstruct', [v_dest, cname], RESTYPE) - reccopy(llops, v_subsrc, v_subdst) - else: - v_value = llops.genop('getfield', [v_source, cname], FIELDTYPE) - llops.genop('setfield', [v_dest, cname, v_value]) - - def arraytype_specialize_call(hop): r_array = hop.r_result return hop.genop("malloc", [ Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Mon Apr 10 17:03:02 2006 @@ -203,3 +203,22 @@ if isinstance(value, self.ctype): value = value.value p.c_data.value = value + +# ____________________________________________________________ + +def reccopy(llops, v_source, v_dest): + # helper to copy recursively a structure or array onto another. + T = v_source.concretetype.TO + assert T == v_dest.concretetype.TO + assert isinstance(T, lltype.Struct) + for name in T._names: + FIELDTYPE = getattr(T, name) + cname = inputconst(lltype.Void, name) + if isinstance(FIELDTYPE, lltype.ContainerType): + RESTYPE = lltype.Ptr(FIELDTYPE) + v_subsrc = llops.genop('getsubstruct', [v_source, cname], RESTYPE) + v_subdst = llops.genop('getsubstruct', [v_dest, cname], RESTYPE) + reccopy(llops, v_subsrc, v_subdst) + else: + v_value = llops.genop('getfield', [v_source, cname], FIELDTYPE) + llops.genop('setfield', [v_dest, cname, v_value]) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 10 17:03:02 2006 @@ -3,7 +3,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel from pypy.annotation.pairtype import pairtype -from pypy.rpython.rctypes.rmodel import CTypesValueRepr +from pypy.rpython.rctypes.rmodel import CTypesValueRepr, reccopy from ctypes import POINTER, pointer, c_int @@ -37,6 +37,12 @@ # an immortal global constant just like 'p', but better safe than sorry p.keepalive_contents = llcontents.c_data_owner_keepalive + def setcontents(self, llops, v_ptr, v_contentsbox): + v_c_data = self.r_contents.get_c_data(llops, v_contentsbox) + v_owner = self.r_contents.get_c_data_owner(llops, v_contentsbox) + self.setvalue(llops, v_ptr, v_c_data) + self.setkeepalive(llops, v_ptr, v_owner) + def rtype_getattr(self, hop): s_attr = hop.args_s[1] assert s_attr.is_constant() @@ -45,6 +51,14 @@ v_c_ptr = self.getvalue(hop.llops, v_ptr) return self.r_contents.allocate_instance_ref(hop.llops, v_c_ptr) + def rtype_setattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'contents' + v_ptr, v_attr, v_newcontents = hop.inputargs(self, lltype.Void, + self.r_contents) + self.setcontents(hop.llops, v_ptr, v_newcontents) + class __extend__(pairtype(PointerRepr, IntegerRepr)): @@ -58,14 +72,19 @@ raise NotImplementedError("XXX: pointer[non-zero-index]") def rtype_setitem((r_ptr, _), hop): + # p[0] = x is not the same as p.contents.value = x + # it makes a copy of the data in 'x' just like rarray.rtype_setitem() self = r_ptr - v_ptr, v_index, v_newvalue = hop.inputargs(self, lltype.Signed, XXX) - + v_ptr, v_index, v_contentsbox = hop.inputargs(self, lltype.Signed, + self.r_contents) + v_new_c_data = self.r_contents.get_c_data(hop.llops, v_contentsbox) + v_target = self.getvalue(hop.llops, v_ptr) if hop.args_s[1].is_constant() and hop.args_s[1].const == 0: - v_c_ptr = self.getvalue(hop. llops, v_ptr) - XXX + pass else: raise NotImplementedError("XXX: pointer[non-zero-index] = value") + # copy the whole structure's content over + reccopy(hop.llops, v_new_c_data, v_target) #def registerPointerType(ptrtype): # """Adds a new pointer type to the extregistry. @@ -155,10 +174,7 @@ v_result = r_ptr.allocate_instance(hop.llops) if len(hop.args_s): v_contentsbox, = hop.inputargs(r_ptr.r_contents) - v_c_data = r_ptr.r_contents.get_c_data(hop.llops, v_contentsbox) - v_owner = r_ptr.r_contents.get_c_data_owner(hop.llops, v_contentsbox) - r_ptr.setvalue(hop.llops, v_result, v_c_data) - r_ptr.setkeepalive(hop.llops, v_result, v_owner) + r_ptr.setcontents(hop.llops, v_result, v_contentsbox) return v_result def pointerinstance_compute_annotation(type, instance): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Mon Apr 10 17:03:02 2006 @@ -98,6 +98,23 @@ if conftest.option.view: t.view() + def test_annotate_ass_contents(self): + def fn(): + x = c_int(5) + y = c_int(6) + p = pointer(x) + p.contents = y + y.value = 12 + return p.contents.value + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(fn, []) + assert s.knowntype == int + + if conftest.option.view: + t.view() + class Test_specialization: def test_specialize_c_int_ptr(self): @@ -174,13 +191,15 @@ """ Make sure that pointers work the same way as arrays. """ - py.test.skip("in-progress") def access_array(): - my_pointer = pointer(c_int(11)) + my_int = c_int(11) + my_pointer = pointer(my_int) x = my_pointer[0] my_pointer[0] = c_int(7) + assert my_int.value == 7 y = my_pointer[0] my_pointer[0] = 5 + assert my_int.value == 5 z = my_pointer.contents.value return x * y * z @@ -197,3 +216,15 @@ res = interpret(access_prebuilt, []) assert res == 11 + + def test_specialize_ass_contents(self): + def fn(): + x = c_int(5) + y = c_int(6) + p = pointer(x) + p.contents = y + y.value = 12 + return p.contents.value + + res = interpret(fn, []) + assert res == 12 From stephan at codespeak.net Mon Apr 10 17:26:18 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Mon, 10 Apr 2006 17:26:18 +0200 (CEST) Subject: [pypy-svn] r25667 - pypy/dist/pypy/objspace/std Message-ID: <20060410152618.22C7310150@code0.codespeak.net> Author: stephan Date: Mon Apr 10 17:26:16 2006 New Revision: 25667 Added: pypy/dist/pypy/objspace/std/frozensettype.py Modified: pypy/dist/pypy/objspace/std/model.py pypy/dist/pypy/objspace/std/setobject.py pypy/dist/pypy/objspace/std/settype.py Log: created extra 'frozensettype.py' file to avoid multimethod registering problems. The 'setobject.py' could stay more or less the same, but it had to be taken care that 'register_all' gets only the appropriate selection of methods for each type. Added: pypy/dist/pypy/objspace/std/frozensettype.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/std/frozensettype.py Mon Apr 10 17:26:16 2006 @@ -0,0 +1,35 @@ +from pypy.interpreter.error import OperationError +from pypy.objspace.std.objspace import register_all +from pypy.objspace.std.stdtypedef import StdTypeDef, newmethod +from pypy.objspace.std.stdtypedef import StdObjSpaceMultiMethod as SMM +from pypy.interpreter.gateway import NoneNotWrapped +from pypy.interpreter import gateway +from pypy.objspace.std.model import WITHSET + +frozenset_copy = SMM('copy', 1) +frozenset_difference = SMM('difference', 2) +frozenset_intersection = SMM('intersection', 2) +frozenset_issubset = SMM('issubset', 2) +frozenset_issuperset = SMM('issuperset', 2) +frozenset_symmetric_difference = SMM('symmetric_difference', 2) +frozenset_union = SMM('union', 2) + +register_all(vars(), globals()) + +def descr__frozenset__new__(space, w_frozensettype, w_iterable=NoneNotWrapped): + from pypy.objspace.std.setobject import W_FrozensetObject + if w_iterable is None: + w_iterable = space.newtuple([]) + w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype) + W_FrozensetObject.__init__(w_obj, space, w_iterable) + + return w_obj + +frozenset_typedef = StdTypeDef("frozenset", + __doc__ = """frozenset(iterable) --> frozenset object + +Build an immutable unordered collection.""", + __new__ = newmethod(descr__frozenset__new__), + ) + +frozenset_typedef.registermethods(globals()) Modified: pypy/dist/pypy/objspace/std/model.py ============================================================================== --- pypy/dist/pypy/objspace/std/model.py (original) +++ pypy/dist/pypy/objspace/std/model.py Mon Apr 10 17:26:16 2006 @@ -23,7 +23,7 @@ from pypy.objspace.std.complextype import complex_typedef if WITHSET: from pypy.objspace.std.settype import set_typedef - from pypy.objspace.std.settype import frozenset_typedef + from pypy.objspace.std.frozensettype import frozenset_typedef from pypy.objspace.std.tupletype import tuple_typedef from pypy.objspace.std.listtype import list_typedef from pypy.objspace.std.dicttype import dict_typedef Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Mon Apr 10 17:26:16 2006 @@ -25,7 +25,7 @@ from pypy.objspace.std.settype import set_typedef as typedef class W_FrozensetObject(W_BaseSetObject): - from pypy.objspace.std.settype import frozenset_typedef as typedef + from pypy.objspace.std.frozensettype import frozenset_typedef as typedef def __init__(w_self, space, wrappeditems): W_BaseSetObject.__init__(w_self, space, wrappeditems) @@ -189,7 +189,7 @@ def set_copy__Set(space, w_left): return space.newset(w_left) -def frozenset_copy_Frozenset(space, w_left): +def frozenset_copy__Frozenset(space, w_left): return space.newfrozenset(w_left) def set_clear__Set(space, w_left): @@ -464,5 +464,18 @@ repr__Set = app.interphook('repr__Set') repr__Frozenset = app.interphook('repr__Frozenset') +from pypy.objspace.std import frozensettype from pypy.objspace.std import settype -register_all(vars(), settype) + +# make sure that the 'register_all' function gets only the appropriate +# methods + +sdg = [(n, m) for n, m in vars().items() if n.find('__Frozenset') == -1] +fdg = [(n, m) for n, m in vars().items() if n.find('__Set') == -1] + +register_all(dict(sdg), settype) +register_all(dict(fdg), frozensettype) + +# this doesn't work: +#register_all(vars(), frozensettype) +#register_all(vars(), settype) Modified: pypy/dist/pypy/objspace/std/settype.py ============================================================================== --- pypy/dist/pypy/objspace/std/settype.py (original) +++ pypy/dist/pypy/objspace/std/settype.py Mon Apr 10 17:26:16 2006 @@ -23,14 +23,6 @@ set_union = SMM('union', 2) set_update = SMM('update', 2) -frozenset_copy = SMM('copy', 1) -frozenset_difference = SMM('difference', 2) -frozenset_intersection = SMM('intersection', 2) -frozenset_issubset = SMM('issubset', 2) -frozenset_issuperset = SMM('issuperset', 2) -frozenset_symmetric_difference = SMM('symmetric_difference', 2) -frozenset_union = SMM('union', 2) - register_all(vars(), globals()) def descr__set__new__(space, w_settype, w_iterable=NoneNotWrapped): @@ -42,15 +34,6 @@ return w_obj -def descr__frozenset__new__(space, w_frozensettype, w_iterable=NoneNotWrapped): - from pypy.objspace.std.setobject import W_FrozensetObject - if w_iterable is None: - w_iterable = space.newtuple([]) - w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype) - W_FrozensetObject.__init__(w_obj, space, w_iterable) - - return w_obj - set_typedef = StdTypeDef("set", __doc__ = """set(iterable) --> set object @@ -60,13 +43,4 @@ set_typedef.registermethods(globals()) -frozenset_typedef = StdTypeDef("frozenset", - __doc__ = """frozenset(iterable) --> frozenset object - -Build an immutable unordered collection.""", - __new__ = newmethod(descr__frozenset__new__), - ) - -frozenset_typedef.registermethods(globals()) - setiter_typedef = StdTypeDef("setiterator") From arigo at codespeak.net Mon Apr 10 17:40:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 17:40:47 +0200 (CEST) Subject: [pypy-svn] r25668 - pypy/dist/pypy/objspace/std Message-ID: <20060410154047.5C44110160@code0.codespeak.net> Author: arigo Date: Mon Apr 10 17:40:45 2006 New Revision: 25668 Modified: pypy/dist/pypy/objspace/std/register_all.py pypy/dist/pypy/objspace/std/setobject.py Log: Allow several alternate namespaces in calls to register_all(). This works in the case of setobject.py because in this case multimethods are looked up by namespace key and not by multimethod name (so no conflict between set_xxx and frozenset_xxx). Modified: pypy/dist/pypy/objspace/std/register_all.py ============================================================================== --- pypy/dist/pypy/objspace/std/register_all.py (original) +++ pypy/dist/pypy/objspace/std/register_all.py Mon Apr 10 17:40:45 2006 @@ -4,7 +4,7 @@ 'or': 'or_', } -def register_all(module_dict, alt_ns=None): +def register_all(module_dict, *alt_ns): """register implementations for multimethods. By default a (name, object) pair of the given module dictionary @@ -14,9 +14,7 @@ """ from pypy.objspace.std.objspace import StdObjSpace from pypy.objspace.std.model import W_ANY, W_Object - namespaces = [StdObjSpace.MM, StdObjSpace] - if alt_ns: - namespaces.insert(0, alt_ns) + namespaces = list(alt_ns) + [StdObjSpace.MM, StdObjSpace] for name, obj in module_dict.items(): if name.startswith('app_'): Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Mon Apr 10 17:40:45 2006 @@ -467,15 +467,4 @@ from pypy.objspace.std import frozensettype from pypy.objspace.std import settype -# make sure that the 'register_all' function gets only the appropriate -# methods - -sdg = [(n, m) for n, m in vars().items() if n.find('__Frozenset') == -1] -fdg = [(n, m) for n, m in vars().items() if n.find('__Set') == -1] - -register_all(dict(sdg), settype) -register_all(dict(fdg), frozensettype) - -# this doesn't work: -#register_all(vars(), frozensettype) -#register_all(vars(), settype) +register_all(vars(), settype, frozensettype) From arigo at codespeak.net Mon Apr 10 17:43:29 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 17:43:29 +0200 (CEST) Subject: [pypy-svn] r25669 - in pypy/dist/pypy/rpython: lltypesystem rctypes rctypes/test Message-ID: <20060410154329.2732610162@code0.codespeak.net> Author: arigo Date: Mon Apr 10 17:43:27 2006 New Revision: 25669 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: Hack in lltype to allow __getitem__ just past the end of rpystrings to return the hidden final \x00 character. This allows some good progress on string support in rctypes. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Mon Apr 10 17:43:27 2006 @@ -763,6 +763,11 @@ def __getitem__(self, i): # ! can only return basic or ptr ! if isinstance(self._T, (Array, FixedSizeArray)): if not (0 <= i < self._obj.getlength()): + if (self._T._hints.get('isrpystring', False) and + i == self._obj.getlength()): + # special hack for the null terminator + assert self._T.OF == Char + return '\x00' raise IndexError("array index out of bounds") o = self._obj.getitem(i) return _expose(o, self._solid) @@ -792,6 +797,12 @@ return self._obj.getlength() raise TypeError("%r instance is not an array" % (self._T,)) + def __iter__(self): + # this is a work-around for the 'isrpystring' hack in __getitem__, + # which otherwise causes list(p) to include the extra \x00 character. + for i in range(len(self)): + yield self[i] + def __repr__(self): return '<%s>' % (self,) Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Mon Apr 10 17:43:27 2006 @@ -1,8 +1,9 @@ from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.rpython.rstr import string_repr +from pypy.rpython.rstr import StringRepr, string_repr from pypy.rpython.rctypes.rmodel import CTypesValueRepr from pypy.annotation import model as annmodel +from pypy.annotation.pairtype import pairtype from ctypes import c_char_p @@ -41,6 +42,16 @@ self.setstring(hop.llops, v_char_p, v_value) +class __extend__(pairtype(StringRepr, CCharPRepr)): + def convert_from_to((r_from, r_to), v, llops): + # r_from could be char_repr: first convert it to string_repr + v = llops.convertvar(v, r_from, string_repr) + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + r_temp.setstring(llops, v_owned_box, v) + return llops.convertvar(v_owned_box, r_temp, r_to) + + CCHARP = llmemory.Address # char * FIRSTITEMOFS = llmemory.ArrayItemsOffset(string_repr.lowleveltype.TO.chars) @@ -62,7 +73,7 @@ def ll_getstring(box): p = box.c_data.value if p: - if (box.keepalive_str and ll_str2charp(box.keepalive_str) == p): + if box.keepalive_str and ll_str2charp(box.keepalive_str) == p: maxlen = len(box.keepalive_str.chars) length = ll_strnlen(p, maxlen) if length == maxlen: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Mon Apr 10 17:43:27 2006 @@ -16,7 +16,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_char_p +from ctypes import c_char_p, pointer class Test_annotation: def test_annotate_c_char_p(self): @@ -60,6 +60,28 @@ res = interpret(func, []) assert ''.join(res.chars) == "hello" + def test_zero_terminates(self): + def func(): + p = c_char_p('world') + p.value = "hello\x00world" + return p.value + + res = interpret(func, []) + assert ''.join(res.chars) == "hello" + + def test_pointer_access(self): + def func(): + q = c_char_p('ka') + p = c_char_p('world') + pp = pointer(p) + pp[0] = q.value + return p.value + + assert func() == 'ka' + res = interpret(func, []) + assert ''.join(res.chars) == 'ka' + + class Test_compilation: def test_compile_c_char_p(self): def func(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Mon Apr 10 17:43:27 2006 @@ -5,6 +5,7 @@ from pypy.translator.c.test.test_genc import compile from pypy import conftest from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem import lltype, llmemory from ctypes import c_int, c_long, c_char_p @@ -19,14 +20,11 @@ atoi = mylib.atoi atoi.restype = c_int atoi.argtypes = [c_char_p] -def ll_atoi(s): +def ll_atoi(s_addr): "Very approximative ll implementation of atoi(), for testing" i = result = 0 - while i < len(s.chars): - if '0' <= s.chars[i] <= '9': - result = result * 10 + ord(s.chars[i]) - ord('0') - else: - break + while '0' <= s_addr.char[i] <= '9': + result = result * 10 + ord(s_addr.char[i]) - ord('0') i += 1 return result atoi.llinterp_friendly_version = ll_atoi @@ -39,10 +37,16 @@ return labs(n) def test_ll_atoi(): - assert ll_atoi(string_repr.convert_const("")) == 0 - assert ll_atoi(string_repr.convert_const("42z7")) == 42 - assert ll_atoi(string_repr.convert_const("blah")) == 0 - assert ll_atoi(string_repr.convert_const("18238")) == 18238 + keepalive = [] + def str2addr(string): + llstring = string_repr.convert_const(string) + keepalive.append(llstring) + return (llmemory.cast_ptr_to_adr(llstring.chars) + + llmemory.ArrayItemsOffset(lltype.typeOf(llstring.chars).TO)) + assert ll_atoi(str2addr("")) == 0 + assert ll_atoi(str2addr("42z7")) == 42 + assert ll_atoi(str2addr("blah")) == 0 + assert ll_atoi(str2addr("18238")) == 18238 class Test_annotation: def test_annotate_labs(self): @@ -67,7 +71,6 @@ assert res == 11 def test_specialize_atoi(self): - py.test.skip("in-progress") choices = ["", "42z7", "blah", "18238"] def fn(n): return atoi(choices[n]) From arigo at codespeak.net Mon Apr 10 18:21:42 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 18:21:42 +0200 (CEST) Subject: [pypy-svn] r25670 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060410162142.C3A0E100A7@code0.codespeak.net> Author: arigo Date: Mon Apr 10 18:21:36 2006 New Revision: 25670 Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Log: Some more conversions for ctypes primitives. Some head-scratching about keepalives in ctypes (not even rctypes!). Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Mon Apr 10 18:21:36 2006 @@ -50,11 +50,19 @@ self.setvalue(hop.llops, v_primitive, v_value) -class __extend__(pairtype(IntegerRepr, PrimitiveRepr)): +class __extend__(pairtype(IntegerRepr, PrimitiveRepr), + pairtype(FloatRepr, PrimitiveRepr), + pairtype(CharRepr, PrimitiveRepr)): def convert_from_to((r_from, r_to), v, llops): + # first convert 'v' to the precise expected low-level type + r_input = r_to.rtyper.primitive_to_repr[r_to.ll_type] + v = llops.convertvar(v, r_from, r_input) + # allocate a memory-owning box to hold a copy of the ll value 'v' r_temp = r_to.r_memoryowner v_owned_box = r_temp.allocate_instance(llops) r_temp.setvalue(llops, v_owned_box, v) + # return this box possibly converted to the expected output repr, + # which might be a memory-aliasing box return llops.convertvar(v_owned_box, r_temp, r_to) Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 10 18:21:36 2006 @@ -3,6 +3,7 @@ down to what aliases what and what exact types operations return. """ +import py from ctypes import * def test_primitive_pointer(): @@ -11,39 +12,46 @@ x.value = 6 assert x.value == 6 - p = pointer(x) + p = pointer(x) # p ---> x = 6 assert isinstance(p.contents, c_int) p.contents.value += 1 - assert x.value == 7 + assert x.value == 7 # p ---> x = 7 y = c_int(12) - p.contents = y - p.contents.value += 2 + p.contents = y # p ---> y = 12 + p.contents.value += 2 # p ---> y = 14 assert y.value == 14 assert x.value == 7 - pp = pointer(p) - pp.contents.contents = x - p.contents.value += 2 + pp = pointer(p) # pp ---> p ---> y = 14 + pp.contents.contents = x # pp ---> p ---> x = 7 + p.contents.value += 2 # pp ---> p ---> x = 9 assert x.value == 9 assert isinstance(p[0], int) - p[0] += 1 + p[0] += 1 # pp ---> p ---> x = 10 assert x.value == 10 z = c_int(86) - p[0] = z + p[0] = z # pp ---> p ---> x = 86 (not z!) assert x.value == 86 z.value = 84 assert x.value == 86 assert isinstance(pp[0], POINTER(c_int)) assert pp[0].contents.value == x.value == 86 - pp[0].contents = z + pp[0].contents = z # pp ---> p ---> z = 84 assert p.contents.value == z.value == 84 + py.test.skip("the rest fails sometimes -- did I get my keepalive" + "expectations wrong?") + q = pointer(y) - pp[0] = q - assert pp.contents.contents.value == y.value == 14 + pp[0] = q # pp ---> p ---> y = 14 + assert y.value == 14 # (^^^ not q! ) + assert p.contents.value == 14 + assert pp.contents.contents.value == 14 + q.contents = x + assert pp.contents.contents.value == 14 def test_char_p(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Mon Apr 10 18:21:36 2006 @@ -10,6 +10,7 @@ from pypy import conftest import sys from pypy.rpython.test.test_llinterp import interpret +from pypy.rpython.rarithmetic import r_longlong, r_ulonglong try: import ctypes @@ -18,7 +19,7 @@ from ctypes import c_char, c_byte, c_ubyte, c_short, c_ushort, c_int, c_uint from ctypes import c_long, c_ulong, c_longlong, c_ulonglong, c_float -from ctypes import c_double, c_char_p +from ctypes import c_double, c_char_p, pointer class Test_annotation: def test_simple(self): @@ -198,6 +199,58 @@ # the precision to compare. assert ("%.2f" % res) == ("%.2f" % 4.3) + def test_value_for_various_types(self): + def func(): + x = c_ushort(5) + x.value += 1 + assert x.value == 6 + x = c_char('A') + x.value = chr(ord(x.value) + 1) + assert x.value == 'B' + x = c_longlong(5) + x.value += 1 + assert x.value == r_longlong(6) + x = c_ulonglong(5) + x.value += 1 + assert x.value == r_ulonglong(6) + x = c_float(2.5) + x.value += 0.25 + assert x.value == 2.75 + x.value -= 1 + assert x.value == 1.75 + x = c_double(2.5) + x.value += 0.25 + assert x.value == 2.75 + x.value -= 1 + assert x.value == 1.75 + interpret(func, []) + + def test_convert_from_llvalue(self): + def func(): + x = c_ushort(5) + pointer(x)[0] += 1 + assert x.value == 6 + x = c_char('A') + pointer(x)[0] = chr(ord(pointer(x)[0]) + 1) + assert x.value == 'B' + x = c_longlong(5) + pointer(x)[0] += 1 + assert x.value == r_longlong(6) + x = c_ulonglong(5) + pointer(x)[0] += 1 + assert x.value == r_ulonglong(6) + x = c_float(2.5) + pointer(x)[0] += 0.25 + assert x.value == 2.75 + pointer(x)[0] -= 1 + assert x.value == 1.75 + x = c_double(2.5) + pointer(x)[0] += 0.25 + assert x.value == 2.75 + pointer(x)[0] -= 1 + assert x.value == 1.75 + interpret(func, []) + class Test_compilation: def test_compile_c_int(self): def create_c_int(): From auc at codespeak.net Mon Apr 10 18:27:40 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 10 Apr 2006 18:27:40 +0200 (CEST) Subject: [pypy-svn] r25671 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060410162740.B439A100ED@code0.codespeak.net> Author: auc Date: Mon Apr 10 18:27:37 2006 New Revision: 25671 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py pypy/dist/pypy/objspace/constraint/test/test_fd.py Log: constraint propagation, less wrapping Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Mon Apr 10 18:27:37 2006 @@ -52,31 +52,83 @@ class W_ComputationSpace(Wrappable): def __init__(self, obj_space): self._space = obj_space - self.domains = self._space.newdict({}) + # there, var -> dom + self.var_doms = self._space.newdict({}) + # constraint set self.constraints = self._space.newdict({}) + # var -> constraints + self.var_const_map = {} + # freshly added constraints (tell -> propagate) + self.to_check = {} def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) - if w_name in self.domains.content: + if w_name in self.var_doms.content: raise OperationError(self._space.w_RuntimeError, self._space.wrap("Name already used")) - self.domains.content[w_name] = w_domain - return W_Variable(self._space, w_name) + var = W_Variable(self._space, w_name) + self.var_doms.content[var] = w_domain + return var def w_dom(self, w_variable): assert isinstance(w_variable, W_Variable) - return self.domains.content[w_variable.w_name] + return self.var_doms.content[w_variable] def w_tell(self, w_constraint): assert isinstance(w_constraint, W_Constraint) self.constraints.content[w_constraint] = self._space.w_True + for var in w_constraint.affected_variables(): + self.var_const_map.setdefault(var, []) + self.var_const_map[var].append(w_constraint) + self.to_check[w_constraint] = True + + def dependant_constraints(self, var): + return self.var_const_map[var] + + def w_propagate(self): + return self.propagate() + + def propagate(self): + const_q = [(const.estimate_cost_w(self), const) + for const in self.to_check] + self.to_check = {} + assert const_q != [] + const_q.sort() + const_q.reverse() # for pop() friendlyness + affected_constraints = {} + while True: + if not const_q: + const_q = [(const.estimate_cost_w(self), const) + for const in affected_constraints] + if not const_q: + break + const_q.sort() + affected_constraints.clear() + cost, const = const_q.pop() + entailed = const.revise(self) + for var in const.affected_variables(): + dom = self.w_dom(var) + if not dom.has_changed(): + continue + for dependant_const in self.dependant_constraints(var): + if dependant_const is not const: + affected_constraints[dependant_const] = True + dom.w_reset_flags() + if entailed: + # we should also remove the constraint from + # the set of satifiable constraints of the space + if const in affected_constraints: + affected_constraints.remove(const) + + W_ComputationSpace.typedef = typedef.TypeDef( "W_ComputationSpace", var = interp2app(W_ComputationSpace.w_var), dom = interp2app(W_ComputationSpace.w_dom), - tell = interp2app(W_ComputationSpace.w_tell)) + tell = interp2app(W_ComputationSpace.w_tell), + propagate = interp2app(W_ComputationSpace.w_propagate)) def newspace(space): Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Mon Apr 10 18:27:37 2006 @@ -41,12 +41,23 @@ """ Return a list of all variables affected by this constraint """ return self._space.newlist(self._variables) + def affected_variables(self): + return self._variables + def w_knows_var(self, w_variable): return self._space.newbool(variable in self._variables) def w_estimate_cost(self, w_cs): """Return an estimate of the cost of the narrowing of the constraint""" assert isinstance(w_cs, W_ComputationSpace) + return self._space.newint(self.estimate_cost_w(w_cs)) + + def w_revise(self, w_cs): + assert isinstance(w_cs, W_ComputationSpace) + return self._space.newbool(self.revise(w_cs)) + + def estimate_cost_w(self, w_cs): + assert isinstance(w_cs, W_ComputationSpace) return reduce(operator.mul, [w_cs.w_dom(var).size() for var in self._variables]) @@ -70,9 +81,9 @@ # worst case complexity self.__cost = len(w_variables.wrappeditems) * (len(w_variables.wrappeditems) - 1) / 2 - def w_estimate_cost(self, w_cs): + def estimate_cost_w(self, w_cs): assert isinstance(w_cs, W_ComputationSpace) - return self._space.newint(self.__cost) + return self.__cost def test_solution(self, sol): """test a solution against this constraint @@ -81,7 +92,7 @@ value_set = set(values) return len(value_set) == len(sol) - def w_revise(self, w_cs): + def revise(self, w_cs): assert isinstance(w_cs, W_ComputationSpace) variables = [(self._space.int_w(w_cs.w_dom(variable).w_size()), variable, w_cs.w_dom(variable)) @@ -111,17 +122,16 @@ print "AllDistinct failed" raise OperationError(self._space.w_RuntimeError, self._space.wrap("Consistency Failure")) -# raise ConsistencyFailure() # the constraint is entailed if all domains have a size of 1 for variable in variables: if self._space.is_true(self._space.ne(variable[2].w_size(), self._space.newint(1))): - return self._space.newint(0) + return False # Question : did we *really* completely check # our own alldistinctness predicate ? - return self._space.newint(1) + return True W_AllDistinct.typedef = typedef.TypeDef( "W_AllDistinct", W_AbstractConstraint.typedef, @@ -197,10 +207,10 @@ # it's over go_on = 0 - def w_revise(self, w_cs): + def revise(self, w_cs): """generic propagation algorithm for n-ary expressions""" assert isinstance(w_cs, W_ComputationSpace) - maybe_entailed = self._space.newint(1) + maybe_entailed = True ffunc = self.filter_func result_cache = self._init_result_cache() for kwargs in self._assign_values(w_cs): @@ -215,7 +225,7 @@ for var, val in kwargs.content.items(): result_cache.content[var].content[val] = self._space.w_True else: - maybe_entailed = self._space.newint(0) + maybe_entailed = False try: for varname, keep in result_cache.content.items(): @@ -233,7 +243,6 @@ pass return maybe_entailed - def __repr__(self): return '<%s>' % self.formula @@ -304,6 +313,7 @@ """create a new constraint of type Expression or BinaryExpression The chosen class depends on the number of variables in the constraint""" # encode unicode + return W_Expression(o_space, w_variables, w_formula) if o_space.eq_w(o_space.len(w_variables), o_space.newint(2)): return W_BinaryExpression(o_space, w_variables, w_formula) else: Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Mon Apr 10 18:27:37 2006 @@ -30,7 +30,13 @@ def w_has_changed(self): return self._space.newbool(self.__changed) + def has_changed(self): + return self.__changed + def w_size(self): + return self._space.newint(self.size()) + + def size(self): pass def _value_removed(self): @@ -77,12 +83,10 @@ for w_val in w_values.wrappeditems : del self._values[w_val] self._value_removed() - __delitem__ = w_remove_value - def w_size(self): + def size(self): """computes the size of a finite domain""" - return self._space.newint(len(self._values)) - __len__ = w_size + return len(self._values) def w_get_values(self): """return all the values in the domain @@ -108,8 +112,10 @@ def __ne__(self, w_other): return not self == w_other + # function bolted into the space to serve as constructor def make_fd(space, w_values): + assert isinstance(w_values, W_ListObject) return space.wrap(W_FiniteDomain(space, w_values)) app_make_fd = gateway.interp2app(make_fd) Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Mon Apr 10 18:27:37 2006 @@ -14,16 +14,14 @@ cspace = newspace() v = cspace.var("foo", FiniteDomain([1,2,3])) assert str(v).startswith(' Author: antocuni Date: Mon Apr 10 18:44:17 2006 New Revision: 25672 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: Added ootype.List._GENERIC_METHODS attribute. It contains the same information as the former _METHODS, with the difference that the ITEMTYPE is explicitly marked by a placeholder. This way backends can know which arguments are parametized on ITEMTYPE and which have fixed type. The _METHODS attribute is automatically computed from _GENERIC_METHODS. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 10 18:44:17 2006 @@ -182,21 +182,36 @@ class List(OOType): + ITEMTYPE_T = object() # placeholder, see below def __init__(self, ITEMTYPE): self._ITEMTYPE = ITEMTYPE self._null = _null_list(self) - # This defines the abstract list interface that backends will have - # to map to their native list implementations. - self._METHODS = frozendict({ + # This defines the abstract list interface that backends will + # have to map to their native list implementations. + # 'ITEMTYPE_T' is used as a placeholder for indicating + # arguments that should have ITEMTYPE type. + + generic_types = {self.ITEMTYPE_T: ITEMTYPE} + self._GENERIC_METHODS = frozendict({ # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) "length": Meth([], Signed), - "append": Meth([ITEMTYPE], Void), - "getitem": Meth([Signed], ITEMTYPE), - "setitem": Meth([Signed, ITEMTYPE], Void), + "append": Meth([self.ITEMTYPE_T], Void), + "getitem": Meth([Signed], self.ITEMTYPE_T), + "setitem": Meth([Signed, self.ITEMTYPE_T], Void), }) + self._setup_methods(generic_types) + + def _setup_methods(self, generic_types): + methods = {} + for name, meth in self._GENERIC_METHODS.iteritems(): + args = [generic_types.get(arg, arg) for arg in meth.ARGS] + result = generic_types.get(meth.RESULT, meth.RESULT) + methods[name] = Meth(args, result) + self._METHODS = frozendict(methods) + # NB: We are expecting Lists of the same ITEMTYPE to compare/hash # equal. We don't redefine __eq__/__hash__ since the implementations # from LowLevelType work fine, especially in the face of recursive @@ -612,3 +627,6 @@ ROOT = Instance('Root', None, _is_root=True) + +print List(Signed)._METHODS + From antocuni at codespeak.net Mon Apr 10 18:50:18 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 10 Apr 2006 18:50:18 +0200 (CEST) Subject: [pypy-svn] r25673 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060410165018.980F71015E@code0.codespeak.net> Author: antocuni Date: Mon Apr 10 18:50:12 2006 New Revision: 25673 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: Forgotten to remove a 'print'... Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 10 18:50:12 2006 @@ -627,6 +627,3 @@ ROOT = Instance('Root', None, _is_root=True) - -print List(Signed)._METHODS - From arigo at codespeak.net Mon Apr 10 19:28:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 19:28:52 +0200 (CEST) Subject: [pypy-svn] r25674 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060410172852.4A8B5100E8@code0.codespeak.net> Author: arigo Date: Mon Apr 10 19:28:51 2006 New Revision: 25674 Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Log: The result of some ctypes source code reading... Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 10 19:28:51 2006 @@ -1,4 +1,5 @@ from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst +from pypy.rpython.error import TyperError from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel @@ -82,7 +83,8 @@ if hop.args_s[1].is_constant() and hop.args_s[1].const == 0: pass else: - raise NotImplementedError("XXX: pointer[non-zero-index] = value") + # not supported by ctypes either + raise TyperError("assignment to pointer[x] with x != 0") # copy the whole structure's content over reccopy(hop.llops, v_new_c_data, v_target) Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 10 19:28:51 2006 @@ -42,16 +42,15 @@ pp[0].contents = z # pp ---> p ---> z = 84 assert p.contents.value == z.value == 84 - py.test.skip("the rest fails sometimes -- did I get my keepalive" - "expectations wrong?") - - q = pointer(y) - pp[0] = q # pp ---> p ---> y = 14 - assert y.value == 14 # (^^^ not q! ) - assert p.contents.value == 14 - assert pp.contents.contents.value == 14 - q.contents = x - assert pp.contents.contents.value == 14 +## *** the rest is commented out because it should work but occasionally +## *** trigger a ctypes bug (SourceForge bug #1467852). *** +## q = pointer(y) +## pp[0] = q # pp ---> p ---> y = 14 +## assert y.value == 14 # (^^^ not q! ) +## assert p.contents.value == 14 +## assert pp.contents.contents.value == 14 +## q.contents = x +## assert pp.contents.contents.value == 14 def test_char_p(): From arigo at codespeak.net Mon Apr 10 20:05:56 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 10 Apr 2006 20:05:56 +0200 (CEST) Subject: [pypy-svn] r25675 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060410180556.5497A100CF@code0.codespeak.net> Author: arigo Date: Mon Apr 10 20:05:55 2006 New Revision: 25675 Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: Clean-ups of old commented-out code, and commenting out some tests. Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 10 20:05:55 2006 @@ -88,81 +88,7 @@ # copy the whole structure's content over reccopy(hop.llops, v_new_c_data, v_target) -#def registerPointerType(ptrtype): -# """Adds a new pointer type to the extregistry. -# -# Since pointers can be created to primitive ctypes objects, arrays, -# structs and structs are not predefined each new pointer type is -# registered in the extregistry as it is identified. -# -# The new pointers that are created have a "contents" attribute -# which, when retrieved, in effect dereferences the pointer and -# returns the referenced value. -# """ -# def compute_result_annotation(s_self, s_arg): -# return annmodel.SomeCTypesObject(ptrtype, -# annmodel.SomeCTypesObject.OWNSMEMORY) -# -# def specialize_call(hop): -# raise RuntimeError('foo') -# -# contentsType = annmodel.SomeCTypesObject(ptrtype._type_, -# annmodel.SomeCTypesObject.MEMORYALIAS) -# -# def get_repr(rtyper, s_pointer): -# return PointerRepr(rtyper, s_pointer, contentsType) -# -# type_entry = extregistry.register_type(ptrtype, -# specialize_call=specialize_call, -# get_repr=get_repr) -# type_entry.get_field_annotation = {'contents': contentsType}.__getitem__ -# -# return extregistry.register_value(ptrtype, -# compute_result_annotation=compute_result_annotation, -# specialize_call=specialize_call) -# -#def pointer_compute_annotation(metatype, the_type): -# """compute the annotation of POINTER() calls to create a ctypes -# pointer for the given type -# """ -# -# def pointer_compute_result_annotation(s_arg): -# """Called to compute the result annotation of -# POINTER(). This happens to return a new -# class which itself is treated as SomeBuiltin because when -# called it creates a new pointer. -# -# NOTE: To handle a myriad of possible pointer types, each -# ctypes type that is passed to POINTER() calls is itself -# registered if it isn't already. -# """ -# ptrtype = POINTER(s_arg.const) -# -# if not extregistry.is_registered_type(ptrtype): -# entry = registerPointerType(ptrtype) -# else: -# entry = extregistry.lookup(ptrtype) -# -# s_self = annmodel.SomeCTypesObject(ptrtype, -# annmodel.SomeCTypesObject.OWNSMEMORY) -# -# -# return annmodel.SomeBuiltin(entry.compute_result_annotation, -# s_self=s_self, -# methodname=ptrtype.__name__) -# -# # annotation of POINTER (not the call) is SomeBuitin which provides -# # a way of computing the result annotation of POINTER() -# return annmodel.SomeBuiltin(pointer_compute_result_annotation, -# methodname=the_type.__name__) -# -#def pointer_specialize_call(hop): -# raise RuntimeError("foo") -# -# handles POINTER() calls -#value_entry = extregistry.register_value(POINTER, -# compute_annotation=pointer_compute_annotation, -# specialize_call=pointer_specialize_call) +# ____________________________________________________________ def pointertype_compute_annotation(metatype, type): def compute_result_annotation(*arg_s): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Mon Apr 10 20:05:55 2006 @@ -19,8 +19,6 @@ c_int_10 = ARRAY(c_int,10) -#py.test.skip("Reworking primitive types") - class Test_annotation: def test_annotate_array(self): def create_array(): @@ -50,21 +48,21 @@ if conftest.option.view: t.view() - def x_test_annotate_array_slice_access(self): - def slice_access(): - my_array = c_int_10() - #f#my_array[0:7] = c_int(1) * 7 - my_array[0:5] = range(5) - - return my_array[0:5] - - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(slice_access, []) - #d#t.view() - #d#print "v90:", s, type(s) - assert s.knowntype == list - s.listdef.listitem.s_value.knowntype == int +## def test_annotate_array_slice_access(self): +## def slice_access(): +## my_array = c_int_10() +## #f#my_array[0:7] = c_int(1) * 7 +## my_array[0:5] = range(5) + +## return my_array[0:5] + +## t = TranslationContext() +## a = t.buildannotator() +## s = a.build_types(slice_access, []) +## #d#t.view() +## #d#print "v90:", s, type(s) +## assert s.knowntype == list +## s.listdef.listitem.s_value.knowntype == int def test_annotate_array_access_variable(self): def access_with_variable(): @@ -82,25 +80,25 @@ assert s.knowntype == int #t#t.view() - def test_annotate_array_access_index_error_on_positive_index(self): - def access_with_invalid_positive_index(): - my_array = c_int_10() - return my_array[10] +## def test_annotate_array_access_index_error_on_positive_index(self): +## def access_with_invalid_positive_index(): +## my_array = c_int_10() +## return my_array[10] - t = TranslationContext() - a = t.buildannotator() +## t = TranslationContext() +## a = t.buildannotator() - py.test.raises(IndexError, "s = a.build_types(access_with_invalid_positive_index,[])") +## py.test.raises(IndexError, "s = a.build_types(access_with_invalid_positive_index,[])") - def test_annotate_array_access_index_error_on_negative_index(self): - def access_with_invalid_negative_index(): - my_array = c_int_10() - return my_array[-11] +## def test_annotate_array_access_index_error_on_negative_index(self): +## def access_with_invalid_negative_index(): +## my_array = c_int_10() +## return my_array[-11] - t = TranslationContext() - a = t.buildannotator() +## t = TranslationContext() +## a = t.buildannotator() - py.test.raises(IndexError, "s = a.build_types(access_with_invalid_negative_index,[])") +## py.test.raises(IndexError, "s = a.build_types(access_with_invalid_negative_index,[])") class Test_specialization: def test_specialize_array(self): From cfbolz at codespeak.net Mon Apr 10 20:46:57 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 10 Apr 2006 20:46:57 +0200 (CEST) Subject: [pypy-svn] r25676 - in pypy/dist/pypy/translator: . backendopt backendopt/test Message-ID: <20060410184657.733C810101@code0.codespeak.net> Author: cfbolz Date: Mon Apr 10 20:46:55 2006 New Revision: 25676 Modified: pypy/dist/pypy/translator/backendopt/propagate.py pypy/dist/pypy/translator/backendopt/test/test_propagate.py pypy/dist/pypy/translator/simplify.py Log: fix constant folding of calls Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Mon Apr 10 20:46:55 2006 @@ -133,6 +133,9 @@ raise TooManyOperations return super(CountingLLFrame, self).eval_operation(operation) +def op_can_be_folded(op): + return not lloperation.LL_OPERATIONS[op.opname].canfold + def constant_folding(graph, translator): """do constant folding if the arguments of an operations are constants""" lli = LLInterpreter(translator.rtyper) @@ -158,23 +161,24 @@ block.operations[i].opname = "same_as" block.operations[i].args = [res] changed = True - # disabling the code for now, since it is not correct - elif 0: #op.opname == "direct_call": + elif op.opname == "direct_call": called_graph = get_graph(op.args[0], translator) if (called_graph is not None and - simplify.has_no_side_effects(translator, called_graph) and + simplify.has_no_side_effects( + translator, called_graph, + is_operation_false=op_can_be_folded) and (block.exitswitch != c_last_exception or i != len(block.operations) - 1)): args = [arg.value for arg in op.args[1:]] countingframe = CountingLLFrame(called_graph, args, lli) - print "folding call", op, "in graph", graph.name + #print "folding call", op, "in graph", graph.name try: res = countingframe.eval() except: - print "did not work" + #print "did not work" pass else: - print "result", res + #print "result", res res = Constant(res) res.concretetype = op.result.concretetype block.operations[i].opname = "same_as" @@ -341,16 +345,19 @@ log.removegetfield("removed %s getfields in %s" % (count, graph.name)) return count +def propagate_all_per_graph(graph, translator): + def prop(): + changed = False + changed = rewire_links(graph) or changed + changed = propagate_consts(graph) or changed + changed = coalesce_links(graph) or changed + changed = do_atmost(100, constant_folding, graph, + translator) or changed + changed = partial_folding(graph, translator) or changed + changed = remove_all_getfields(graph, translator) or changed + return changed + do_atmost(10, prop) + def propagate_all(translator): for graph in translator.graphs: - def prop(): - changed = False - changed = rewire_links(graph) or changed - changed = propagate_consts(graph) or changed - changed = coalesce_links(graph) or changed - changed = do_atmost(100, constant_folding, graph, - translator) or changed - changed = partial_folding(graph, translator) or changed - changed = remove_all_getfields(graph, translator) or changed - return changed - do_atmost(10, prop) + propagate_all_per_graph(graph, translator) Modified: pypy/dist/pypy/translator/backendopt/test/test_propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_propagate.py Mon Apr 10 20:46:55 2006 @@ -82,22 +82,33 @@ check_graph(graph, [1], None, t) def test_propagate_despite_vars(): - py.test.skip("immediate test") patterns = [1, 1, 2, 3, 5, 7, 12] class A(object): pass - a = A() - a.x = 10 - a.y = 20 - def f(x): + global_a = A() + global_a.x = 10 + global_a.y = 20 + def f(x): + a = A() + a.x = 10 + a.y = 20 result = 0 + i = 0 for i in range(a.x): - for j in range(a.y): + j = 0 + for i in range(global_a.y): result += i * j + j += 1 return result graph, t = get_graph(f, [int]) - propagate_consts(graph) if conftest.option.view: t.view() + while propagate_consts(graph): + pass + assert len(graph.startblock.exits[0].args) == 2 + innerloopblock = graph.startblock.exits[0].target.exits[0].target.exits[0].target + assert len(innerloopblock.inputargs) == 4 + assert len(innerloopblock.exits[0].args) == 4 + check_graph(graph, [0], f(0), t) def test_constant_fold(): def f(x): @@ -112,8 +123,6 @@ check_graph(graph, [1], g(1), t) def test_constant_fold_call(): - # fix the logic and try again :-) - py.test.skip("constant folding calls is disabled, for sanity reasons") def s(x): res = 0 i = 1 @@ -131,6 +140,21 @@ assert len(graph.startblock.operations) == 1 check_graph(graph, [10], g(10), t) +def test_dont_constant_fold_call(): + class A(object): + pass + global_a = A() + global_a.x = 1 + def g(): + return global_a.x + def f(x): + global_a.x = x + return g() + graph, t = get_graph(f, [int], inline_threshold=0) + while constant_folding(graph, t): + pass + check_graph(graph, [10], 10, t) + def test_dont_fold_getfield(): # must not constant fold this, because the container might be collected string = "blablabla" Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Mon Apr 10 20:46:55 2006 @@ -9,6 +9,7 @@ from pypy.objspace.flow.model import Variable, Constant, Block, Link from pypy.objspace.flow.model import c_last_exception from pypy.objspace.flow.model import checkgraph, traverse, mkentrymap +from pypy.rpython.lltypesystem import lloperation def get_graph(arg, translator): from pypy.translator.translator import graphof @@ -373,8 +374,11 @@ class HasSideEffects(Exception): pass -def has_no_side_effects(translator, graph, seen=None): - from pypy.rpython.lltypesystem import lloperation +def op_has_side_effects(op): + return lloperation.LL_OPERATIONS[op.opname].sideeffects + +def has_no_side_effects(translator, graph, seen=None, + is_operation_false=op_has_side_effects): #is the graph specialized? if no we can't say anything #don't cache the result though if translator.rtyper is None: @@ -406,7 +410,7 @@ for g in graphs: if not has_no_side_effects(translator, g, newseen): raise HasSideEffects - elif lloperation.LL_OPERATIONS[op.opname].sideeffects: + elif is_operation_false(op): raise HasSideEffects traverse(visit, graph) except HasSideEffects: From antocuni at codespeak.net Mon Apr 10 23:18:45 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 10 Apr 2006 23:18:45 +0200 (CEST) Subject: [pypy-svn] r25677 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060410211845.44AC31015C@code0.codespeak.net> Author: antocuni Date: Mon Apr 10 23:18:14 2006 New Revision: 25677 Added: pypy/dist/pypy/translator/cli/sdk.py (contents, props changed) pypy/dist/pypy/translator/cli/src/pypylib.cs - copied, changed from r25666, pypy/dist/pypy/translator/cli/src/pypy.cs pypy/dist/pypy/translator/cli/test/test_list.py (contents, props changed) Removed: pypy/dist/pypy/translator/cli/src/pypy.cs Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/ilgenerator.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/rte.py pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py Log: Added support for rpython lists. Platform-specific issues are now grouped togheter in sdk.py. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Mon Apr 10 23:18:14 2006 @@ -6,7 +6,7 @@ from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong -from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod +from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod, List from pypy.translator.cli.option import getoption from pypy.tool.ansi_print import ansi_log @@ -22,8 +22,11 @@ SignedLongLong: 'int64', UnsignedLongLong: 'unsigned int64', Bool: 'bool', - Float: 'float64', + Float: 'float64', Class: 'class [mscorlib]System.Type', + + # TODO: it seems a hack + List.ITEMTYPE_T: '!0', } _pyexception_to_cts = { @@ -53,6 +56,9 @@ return 'class %s' % name elif isinstance(t, StaticMethod): return 'void' # TODO: is it correct to ignore StaticMethod? + elif isinstance(t, List): + item_type = self.lltype_to_cts(t._ITEMTYPE) + return 'class [pypylib]pypy.runtime.List`1<%s>' % item_type return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) @@ -62,6 +68,9 @@ def llconst_to_cts(self, const): return self.lltype_to_cts(const.concretetype), const.value + def ctor_name(self, t): + return 'instance void %s::.ctor()' % self.lltype_to_cts(t) + def graph_to_signature(self, graph, is_method = False, func_name = None): ret_type, ret_var = self.llvar_to_cts(graph.getreturnvar()) func_name = func_name or graph.name @@ -75,6 +84,24 @@ return '%s %s(%s)' % (ret_type, func_name, arg_list) + def method_signature(self, obj, name): + # TODO: use callvirt only when strictly necessary + if isinstance(obj, Instance): + owner, meth = obj._lookup(name) + class_name = obj._name + full_name = 'class %s::%s' % (class_name, name) + return self.graph_to_signature(meth.graph, True, full_name), True + + elif isinstance(obj, List): + meth = obj._GENERIC_METHODS[name] + class_name = self.lltype_to_cts(obj) + ret_type = self.lltype_to_cts(meth.RESULT) + arg_types = [self.lltype_to_cts(arg) for arg in meth.ARGS] + arg_list = ', '.join(arg_types) + return '%s %s::%s(%s)' % (ret_type, class_name, name, arg_list), False + else: + assert False + def split_class_name(self, class_name): parts = class_name.rsplit('.', 1) if len(parts) == 2: Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Mon Apr 10 23:18:14 2006 @@ -238,18 +238,11 @@ field_type = self.cts.lltype_to_cts(type_) return '%s %s::%s' % (field_type, class_name, field) - def ctor_name(self, ooinstance): - return 'instance void class %s::.ctor()' % self.class_name(ooinstance) - # following methods belongs to the Generator interface def function_signature(self, graph): return self.cts.graph_to_signature(graph, False) - def method_signature(self, graph, class_name, name): - full_name = '%s::%s' % (class_name, name) - return self.cts.graph_to_signature(graph, True, full_name) - def class_name(self, ooinstance): return ooinstance._name @@ -261,7 +254,7 @@ self.ilasm.call(func_name) def new(self, obj): - self.ilasm.new(self.ctor_name(obj)) + self.ilasm.new(self.cts.ctor_name(obj)) def set_field(self, obj, name): self.ilasm.opcode('stfld ' + self.field_name(obj, name)) @@ -270,11 +263,9 @@ self.ilasm.opcode('ldfld ' + self.field_name(obj, name)) def call_method(self, obj, name): - owner, meth = obj._lookup(name) - signature = self.method_signature(meth.graph, self.class_name(obj), name) - # TODO: there are cases when we don't need callvirt but a - # plain call is sufficient - self.ilasm.call_method(signature) + # TODO: use callvirt only when strictly necessary + signature, virtual = self.cts.method_signature(obj, name) + self.ilasm.call_method(signature, virtual) def load(self, v): if isinstance(v, flowmodel.Variable): Modified: pypy/dist/pypy/translator/cli/ilgenerator.py ============================================================================== --- pypy/dist/pypy/translator/cli/ilgenerator.py (original) +++ pypy/dist/pypy/translator/cli/ilgenerator.py Mon Apr 10 23:18:14 2006 @@ -125,8 +125,11 @@ def call(self, func): self.code.writeline('call ' + func) - def call_method(self, meth): - self.code.writeline('callvirt instance ' + meth) + def call_method(self, meth, virtual): + if virtual: + self.code.writeline('callvirt instance ' + meth) + else: + self.code.writeline('call instance ' + meth) def new(self, class_): self.code.writeline('newobj ' + class_) Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Mon Apr 10 23:18:14 2006 @@ -54,22 +54,6 @@ cls = getattr(graph.func, 'class_', None) self._render_function(generator, graph, op.args) -## if cls is None: -## self._render_function(generator, graph, op.args) -## else: -## self._render_unbound_meth(generator, cls, graph, op.args) - - def _render_unbound_meth(self, generator, cls, graph, args): - 0/0 - this = args[1] - # TODO: make sure that 'this' is compatible with 'cls' - for func_arg in args[1:]: - generator.load(func_arg) - - # TODO: it doesn't work if cls is in another namespace - cls_name, meth_name = graph.name.rsplit('.', 1) - meth_sig = generator.method_signature(graph, cls_name, meth_name) - generator.call(graph, meth_sig) # TODO def _render_function(self, generator, graph, args): func_sig = generator.function_signature(graph) Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Mon Apr 10 23:18:14 2006 @@ -11,7 +11,7 @@ def _abs(type_): return [PushAllArgs, 'call %s class [mscorlib]System.Math::Abs(%s)' % (type_, type_)] -_runtimenew = 'object [pypy]pypy.runtime.Utils::RuntimeNew([mscorlib]System.Type)' +_runtimenew = 'object [pypylib]pypy.runtime.Utils::RuntimeNew([mscorlib]System.Type)' opcodes = { # __________ object oriented operations __________ Modified: pypy/dist/pypy/translator/cli/rte.py ============================================================================== --- pypy/dist/pypy/translator/cli/rte.py (original) +++ pypy/dist/pypy/translator/cli/rte.py Mon Apr 10 23:18:14 2006 @@ -6,10 +6,11 @@ import os import os.path import subprocess -import py -SRC = 'pypy.cs' -DLL = 'pypy.dll' +from pypy.translator.cli.sdk import SDK + +SRC = 'pypylib.cs' +DLL = 'pypylib.dll' def _filename(name): rel_path = os.path.join(os.path.dirname(__file__), 'src/' + name) @@ -29,23 +30,19 @@ pass if recompile: - compile(source) + compile(source, dll) return dll -def compile(source): - mcs = _get_compiler() - compiler = subprocess.Popen([mcs, '/t:library', source], +def compile(source, dll): + csc = SDK.csc() + compiler = subprocess.Popen([csc, '/t:library', '/out:%s' % dll, source], stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdout, stderr = compiler.communicate() retval = compiler.wait() assert retval == 0, 'Failed to compile %s: the compiler said:\n %s' % (DLL, stderr) -def _get_compiler(): - try: - py.path.local.sysfind('mcs') # TODO: support windows - return 'mcs' - except py.error.ENOENT: - py.test.skip("%s is not on your path." % helper) +if __name__ == '__main__': + get_pypy_dll() Added: pypy/dist/pypy/translator/cli/sdk.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/sdk.py Mon Apr 10 23:18:14 2006 @@ -0,0 +1,40 @@ +import platform +import py + +class AbstractSDK(object): + @classmethod + def _check_helper(cls, helper): + try: + py.path.local.sysfind(helper) + return helper + except py.error.ENOENT: + py.test.skip("%s is not on your path." % helper) + + @classmethod + def runtime(cls): + for item in cls.RUNTIME: + cls._check_helper(item) + return cls.RUNTIME + + @classmethod + def ilasm(cls): + return cls._check_helper(cls.ILASM) + + @classmethod + def csc(cls): + return cls._check_helper(cls.CSC) + +class MicrosoftSDK(AbstractSDK): + RUNTIME = [] + ILASM = 'ilasm' + CSC = 'csc' + +class MonoSDK(AbstractSDK): + RUNTIME = ['mono'] + ILASM = 'ilasm2' + CSC = 'gmcs' + +if platform.system() == 'Windows': + SDK = MicrosoftSDK +else: + SDK = MonoSDK Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Mon Apr 10 23:18:14 2006 @@ -23,23 +23,13 @@ else: print 'OK' -class Base: - def __init__(self, x): - self.x = x - - def compute(self): - return self.x - -class Derived(Base): - def __init__(self, x): - Base.__init__(self, x+42) - -def foo(cls, x): - return cls(x).compute() +def foo(mylist, i): + return mylist[i] def bar(x, y): - b = Derived(x) - return foo(Base, x) + foo(Derived, y) + mylist = [1,2,3,x,y] + mylist[0] = 432 + return foo(mylist, (x+y)%5) f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Mon Apr 10 23:18:14 2006 @@ -1,6 +1,5 @@ import os import subprocess -import platform import shutil import py @@ -12,6 +11,7 @@ from pypy.translator.cli.node import Node from pypy.translator.cli.cts import CTS from pypy.translator.cli.database import LowLevelDatabase +from pypy.translator.cli.sdk import SDK from pypy.translator.cli.rte import get_pypy_dll FLOAT_PRECISION = 8 @@ -111,19 +111,6 @@ return GenCli(self.tmpdir, t, TestEntryPoint(self.graph)) - def __check_helper(self, helper): - try: - py.path.local.sysfind(helper) - except py.error.ENOENT: - py.test.skip("%s is not on your path." % helper) - - def __get_runtime(self): - if platform.system() == 'Windows': - return [] - else: - self.__check_helper('mono') - return ['mono'] - def _build_exe(self): tmpfile = self._gen.generate_source() if getoption('source'): @@ -132,10 +119,10 @@ pypy_dll = get_pypy_dll() # get or recompile pypy.dll shutil.copy(pypy_dll, self.tmpdir.strpath) - self.__check_helper("ilasm") - ilasm = subprocess.Popen(["ilasm", tmpfile], stdout=subprocess.PIPE, stderr=subprocess.PIPE) - stdout, stderr = ilasm.communicate() - retval = ilasm.wait() + ilasm = SDK.ilasm() + proc = subprocess.Popen([ilasm, tmpfile], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = proc.communicate() + retval = proc.wait() assert retval == 0, 'ilasm failed to assemble %s (%s):\n%s' % (self.graph.name, tmpfile, stdout) return tmpfile.replace('.il', '.exe') @@ -143,8 +130,7 @@ if self._exe is None: py.test.skip("Compilation disabled") - runtime = self.__get_runtime() - arglist = runtime + [self._exe] + map(str, args) + arglist = SDK.runtime() + [self._exe] + map(str, args) mono = subprocess.Popen(arglist, stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdout, stderr = mono.communicate() retval = mono.wait() Added: pypy/dist/pypy/translator/cli/test/test_list.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/test/test_list.py Mon Apr 10 23:18:14 2006 @@ -0,0 +1,30 @@ +from pypy.translator.cli.test.runtest import check + +def test_list(): + for name, func in globals().iteritems(): + if not name.startswith('list_'): + continue + + yield check, func, [int, int], (42, 13) + + +def create(x, y): + return [x, y, x+y, x*y] + +def sum_(lst): + total = 0 + i = 0 + while i < len(lst): + total += lst[i] + i += 1 + + return total + +def list_sum(x, y): + return sum_(create(x, y)) + +def list_setitem(x, y): + lst = create(x, y) + lst[0] = 0 + lst[1] = 0 + return sum_(lst) From mwh at codespeak.net Mon Apr 10 23:35:45 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 10 Apr 2006 23:35:45 +0200 (CEST) Subject: [pypy-svn] r25678 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060410213545.E821A1015F@code0.codespeak.net> Author: mwh Date: Mon Apr 10 23:35:44 2006 New Revision: 25678 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py Log: what seems like some good progress with the stackless transformation. support for catching a resumed exception. this required surprisingly few special cases. the test armin and i wrote earlier today now passes. support for multiple return types. only float is tested, though, and the code is pretty horrible in places. lots and lots of smaller cleanups. probably the next step is much more extensive testing. Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Mon Apr 10 23:35:44 2006 @@ -2,56 +2,65 @@ from pypy.rpython import rarithmetic STATE_HEADER = lltype.GcStruct('state_header', - ('f_back', lltype.Ptr(lltype.GcForwardReference())), - ('restartstate', lltype.Signed), - ('function', llmemory.Address),) + ('f_back', lltype.Ptr(lltype.GcForwardReference())), + ('restartstate', lltype.Signed), + ('function', llmemory.Address), + ('retval_type', lltype.Signed)) STATE_HEADER.f_back.TO.become(STATE_HEADER) null_state = lltype.nullptr(STATE_HEADER) def decode_state(currentframe): - return currentframe.function, "long", currentframe.restartstate + return (currentframe.function, + currentframe.retval_type, + currentframe.restartstate) class StacklessData: def __init__(self): self.top = null_state - self.bottom = null_state self.restart_substate = 0 self.retval_long = 0 self.retval_longlong = rarithmetic.r_longlong(0) - self.retval_double = 0.0 - self.retval_void_p = llmemory.fakeaddress(None) + self.retval_float = 0.0 + self.retval_void_p = llmemory.NULL self.exception = None global_state = StacklessData() -def call_function(fn, signature): - if signature == 'void': +RETVAL_VOID, RETVAL_LONG, RETVAL_LONGLONG, RETVAL_FLOAT, RETVAL_VOID_P = \ + range(5) + +def call_function(fn, retval_code): + if retval_code == RETVAL_VOID: lloperation.llop.unsafe_call(lltype.Void, fn) - elif signature == 'long': + elif retval_code == RETVAL_LONG: global_state.retval_long = lloperation.llop.unsafe_call( lltype.Signed, fn) - elif signature == 'longlong': + elif retval_code == RETVAL_LONGLONG: global_state.retval_longlong = lloperation.llop.unsafe_call( lltype.SignedLongLong, fn) - elif signature == 'float': - global_state.retval_double = lloperation.llop.unsafe_call( + elif retval_code == RETVAL_FLOAT: + global_state.retval_float = lloperation.llop.unsafe_call( lltype.Float, fn) - elif signature == 'pointer': + elif retval_code == RETVAL_VOID_P: global_state.retval_void_p = lloperation.llop.unsafe_call( llmemory.Address, fn) -null_address = llmemory.fakeaddress(None) - class UnwindException(Exception): def __init__(self): - self.frame_top = null_state # points to frame that first caught - # the UnwindException - self.frame_bottom = null_state - # walking frame_top.f_back.f_back... goes to frame_bottom - # + # frame_top points to frame that first caught the + # UnwindException, whilst frame_bottom points to the frame + # that most recently caught the UnwindException. frame_bottom + # is only needed to efficiently tack frames on to the end of + # the stack. walking frame_top.f_back.f_back... goes to + # frame_bottom + self.frame_top = null_state + self.frame_bottom = null_state def slp_main_loop(): + """ + slp_main_loop() keeps resuming... + """ currentframe = global_state.top while currentframe: @@ -60,7 +69,7 @@ fn, signature, global_state.restart_substate = decode_state(currentframe) try: call_function(fn, signature) - except UnwindException, u: #XXX annotation support needed + except UnwindException, u: #XXX annotation support needed nextframe = u.frame_top except Exception, e: global_state.exception = e @@ -81,6 +90,8 @@ u.frame_bottom = frame_state def resume_state(): + """Return and zero the 'restart_substate', the index of the resume + point to jump to or zero for the not resuming case.""" x = global_state.restart_substate global_state.restart_substate = 0 return x @@ -93,8 +104,31 @@ # global_state.top = None # XXX and then insert the rtyped graph of this into functions +def fetch_retval_void(): + if global_state.exception: + raise global_state.exception + def fetch_retval_long(): if global_state.exception: raise global_state.exception else: return global_state.retval_long + +def fetch_retval_longlong(): + if global_state.exception: + raise global_state.exception + else: + return global_state.retval_longlong + +def fetch_retval_float(): + if global_state.exception: + raise global_state.exception + else: + return global_state.retval_float + +def fetch_retval_void_p(): + if global_state.exception: + raise global_state.exception + else: + return global_state.retval_void_p + Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Mon Apr 10 23:35:44 2006 @@ -46,6 +46,18 @@ res = llinterp_stackless_function(example, example, g) assert res == 3 +def test_simple_transform_llinterp_float(): + def check(x): + if x: + raise code.UnwindException + def g(x): + check(x) + return x + 0.125 + def example(x): + return int((g(x) + 1)*1000.0) + res = llinterp_stackless_function(example, example, g) + assert res == 2125 + def test_simple_transform(): def check(x): if x: @@ -97,7 +109,6 @@ assert llinterp.type_name(info.value.args[0]) == 'KeyError' def test_resume_with_exception_handling(): - py.test.skip("in progress") def check(x): if x: raise code.UnwindException @@ -117,9 +128,6 @@ return y + 1 res = llinterp_stackless_function(example, example, g, h) assert res == 0 - - - def rtype_stackless_function(fn, *stacklessfuncs): s_list_of_strings = annmodel.SomeList(ListDef(None, annmodel.SomeString())) @@ -133,10 +141,14 @@ # helpers which can cause slp_main_loop to get re-annotated after # it is rtyped. which is bad. unwind_def = bk.getuniqueclassdef(code.UnwindException) - unwind_def.generalize_attr('frame_top', annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) - unwind_def.generalize_attr('frame_bottom', annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) + unwind_def.generalize_attr('frame_top', + annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) + unwind_def.generalize_attr('frame_bottom', + annmodel.SomePtr(lltype.Ptr(code.STATE_HEADER))) - annotator.build_types(fn, [s_list_of_strings]) + s_returnvar = annotator.build_types(fn, [s_list_of_strings]) + if not isinstance(s_returnvar, annmodel.SomeInteger): + raise Exception, "this probably isn't going to work" t.buildrtyper().specialize() st = StacklessTransfomer(t) @@ -161,7 +173,7 @@ t = rtype_stackless_function(entry_point, *stacklessfuncs) - cbuilder = CStandaloneBuilder(t, entry_point)#, gcpolicy=gc.BoehmGcPolicy) + cbuilder = CStandaloneBuilder(t, entry_point) cbuilder.generate_source() cbuilder.compile() return cbuilder.cmdexec('').strip() @@ -173,7 +185,7 @@ except code.UnwindException, u: code.global_state.top = u.frame_top code.slp_main_loop() - r = code.global_state.retval_long + return code.global_state.retval_long return r t = rtype_stackless_function(entry_point, *stacklessfuncs) Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 10 23:35:44 2006 @@ -36,38 +36,41 @@ else: raise Exception("don't know about %r" % (T,)) -## def func(x): -## return g() + x + 1 - -## STATE_func_0 = lltype.Struct('STATE_func_0', -## ('header', STATE_HEADER), -## ('saved_long_0', Signed)) - -## def func(x): -## if global_state.top: -## if global_state.restart_substate == 0: -## frame = cast_pointer(lltype.Ptr(STATE_func_0), global_state.top) -## x = frame.saved_long_0 -## retval = global_state.long_retval -## else: -## abort() -## else: -## try: -## retval = g(x) -## except UnwindException, u: -## state = lltype.raw_malloc(STATE_func_0) -## state.saved_long_0 = x -## state.header.f_back = u.frame -## state.header.state = XXX -## u.frame = state.header -## raise - -## return retval + x + 1 +# a simple example of what the stackless transform does +# +# def func(x): +# return g() + x + 1 +# +# STATE_func_0 = lltype.Struct('STATE_func_0', +# ('header', code.STATE_HEADER), +# ('saved_long_0', Signed)) +# +# def func(x): +# if global_state.restart_substate == 0: +# try: +# retval = g(x) +# except code.UnwindException, u: +# state = lltype.malloc(STATE_func_0) +# state.header.restartstate = 1 +# state.header.function = llmemory.cast_ptr_to_adr(func) +# state.header.retval_type = code.RETVAL_LONG +# state.saved_long_0 = x +# code.add_frame_state(u, state.header) +# raise +# elif global_state.restart_substate == 1: +# state = lltype.cast_pointer(lltype.Ptr(STATE_func_0), +# global_state.top) +# x = state.saved_long_0 +# retval = global_state.long_retval +# else: +# abort() +# return retval + x + 1 class ResumePoint: - def __init__(self, var_result, link_to_resumption, frame_state_type): + def __init__(self, var_result, args, links_to_resumption, frame_state_type): self.var_result = var_result - self.link_to_resumption = link_to_resumption + self.args = args + self.links_to_resumption = links_to_resumption self.frame_state_type = frame_state_type class StacklessTransfomer(object): @@ -86,8 +89,9 @@ mixlevelannotator = MixLevelHelperAnnotator(translator.rtyper) l2a = annmodel.lltype_to_annotation - annotations = [annmodel.SomeInstance(bk.getuniqueclassdef(code.UnwindException)), - annmodel.SomePtr(lltype.Ptr(STATE_HEADER))] + annotations = [ + annmodel.SomeInstance(bk.getuniqueclassdef(code.UnwindException)), + annmodel.SomePtr(lltype.Ptr(STATE_HEADER))] add_frame_state_graph = mixlevelannotator.getgraph( code.add_frame_state, @@ -108,6 +112,14 @@ graph=resume_state_graph), lltype.Ptr(RESUME_STATE_TYPE)) + FETCH_RETVAL_VOID_TYPE = lltype.FuncType([], lltype.Void) + fetch_retval_void_graph = mixlevelannotator.getgraph( + code.fetch_retval_void, [], annmodel.s_None) + self.fetch_retval_void_ptr = model.Constant(lltype.functionptr( + FETCH_RETVAL_VOID_TYPE, "fetch_retval_void", + graph=fetch_retval_void_graph), + lltype.Ptr(FETCH_RETVAL_VOID_TYPE)) + FETCH_RETVAL_LONG_TYPE = lltype.FuncType([], lltype.Signed) fetch_retval_long_graph = mixlevelannotator.getgraph( code.fetch_retval_long, [], annmodel.SomeInteger()) @@ -116,13 +128,38 @@ graph=fetch_retval_long_graph), lltype.Ptr(FETCH_RETVAL_LONG_TYPE)) + FETCH_RETVAL_LONGLONG_TYPE = lltype.FuncType([], lltype.Signed) + fetch_retval_longlong_graph = mixlevelannotator.getgraph( # WAA! + code.fetch_retval_longlong, [], annmodel.SomeInteger(size=2)) + self.fetch_retval_longlong_ptr = model.Constant(lltype.functionptr( + FETCH_RETVAL_LONGLONG_TYPE, "fetch_retval_longlong", + graph=fetch_retval_longlong_graph), + lltype.Ptr(FETCH_RETVAL_LONGLONG_TYPE)) + + FETCH_RETVAL_FLOAT_TYPE = lltype.FuncType([], lltype.Float) + fetch_retval_float_graph = mixlevelannotator.getgraph( + code.fetch_retval_float, [], annmodel.SomeFloat()) + self.fetch_retval_float_ptr = model.Constant(lltype.functionptr( + FETCH_RETVAL_FLOAT_TYPE, "fetch_retval_float", + graph=fetch_retval_float_graph), + lltype.Ptr(FETCH_RETVAL_FLOAT_TYPE)) + + FETCH_RETVAL_VOID_P_TYPE = lltype.FuncType([], llmemory.Address) + fetch_retval_void_p_graph = mixlevelannotator.getgraph( + code.fetch_retval_void_p, [], annmodel.SomeAddress()) + self.fetch_retval_void_p_ptr = model.Constant(lltype.functionptr( + FETCH_RETVAL_VOID_P_TYPE, "fetch_retval_void_p", + graph=fetch_retval_void_p_graph), + lltype.Ptr(FETCH_RETVAL_VOID_P_TYPE)) + mixlevelannotator.finish() s_global_state = bk.immutablevalue(code.global_state) r_global_state = translator.rtyper.getrepr(s_global_state) - self.ll_global_state = model.Constant(r_global_state.convert_const(code.global_state), - r_global_state.lowleveltype) - + self.ll_global_state = model.Constant( + r_global_state.convert_const(code.global_state), + r_global_state.lowleveltype) + def frame_type_for_vars(self, vars): types = [storage_type(v.concretetype) for v in vars] @@ -138,7 +175,8 @@ fields = [] for i, k in enumerate(key): for j in range(k): - fields.append(('state_%s_%d'%(STORAGE_FIELDS[i], j), STORAGE_TYPES[i])) + fields.append( + ('state_%s_%d'%(STORAGE_FIELDS[i], j), STORAGE_TYPES[i])) T = lltype.Struct("state_%d_%d_%d_%d"%tuple(key), ('header', STATE_HEADER), *fields) @@ -171,7 +209,8 @@ ops.append(model.SpaceOperation( "getfield", - [self.ll_global_state, model.Constant(llfieldname, lltype.Void)], + [self.ll_global_state, + model.Constant(llfieldname, lltype.Void)], tmpvar)) if tmpvar is not targetvar: ops.append(model.SpaceOperation( @@ -181,11 +220,14 @@ def insert_resume_handling(self, graph): old_start_block = graph.startblock - newinputargs = [copyvar(self.translator, v) for v in old_start_block.inputargs] + newinputargs = [copyvar(self.translator, v) + for v in old_start_block.inputargs] new_start_block = model.Block(newinputargs) var_resume_state = varoftype(lltype.Signed) new_start_block.operations.append( - model.SpaceOperation("direct_call", [self.resume_state_ptr], var_resume_state)) + model.SpaceOperation("direct_call", + [self.resume_state_ptr], + var_resume_state)) not_resuming_link = model.Link(newinputargs, old_start_block, 0) not_resuming_link.llexitcase = 0 resuming_links = [] @@ -197,25 +239,62 @@ frame_top = varoftype(lltype.Ptr(frame_state_type)) ops.extend(self.ops_read_global_state_field(frame_top, "top")) ops.append(model.SpaceOperation( - "setfield", [self.ll_global_state, model.Constant("inst_top", lltype.Void), - model.Constant(null_state, lltype.typeOf(null_state))], - varoftype(lltype.Void))) - i = 0 - for arg in resume_point.link_to_resumption.args: - newarg = copyvar(self.translator, arg) - if arg is resume_point.var_result: - ops.append(model.SpaceOperation("direct_call", [self.fetch_retval_long_ptr], newarg)) - else: - # frame_state_type._names[0] is 'header' - fname = model.Constant(frame_state_type._names[i+1], lltype.Void) - ops.append(model.SpaceOperation( - 'getfield', [frame_top, fname], newarg)) - i += 1 - newargs.append(newarg) + "setfield", + [self.ll_global_state, + model.Constant("inst_top", lltype.Void), + model.Constant(null_state, lltype.typeOf(null_state))], + varoftype(lltype.Void))) + varmap = {} + for i, arg in enumerate(resume_point.args): + newarg = varmap[arg] = copyvar(self.translator, arg) + assert arg is not resume_point.var_result + fname = model.Constant(frame_state_type._names[i+1], lltype.Void) + ops.append(model.SpaceOperation( + 'getfield', [frame_top, fname], newarg)) + + r = storage_type(resume_point.var_result.concretetype) + if r is not None: + rettype = STORAGE_TYPES[r] + else: + rettype = lltype.Void + if rettype == lltype.Signed: + getretval = self.fetch_retval_long_ptr + if rettype == lltype.SignedLongLong: + getretval = self.fetch_retval_longlong_ptr + elif rettype == lltype.Void: + getretval = self.fetch_retval_void_ptr + elif rettype == lltype.Float: + getretval = self.fetch_retval_float_ptr + elif rettype == llmemory.Address: + getretval = self.fetch_retval_void_p_ptr + varmap[resume_point.var_result] = retval = ( + copyvar(self.translator, resume_point.var_result)) + ops.append(model.SpaceOperation("direct_call", [getretval], retval)) + newblock.operations.extend(ops) - newblock.closeblock(model.Link(newargs, resume_point.link_to_resumption.target)) + + def rename(arg): + if isinstance(arg, model.Variable): + if arg in varmap: + return varmap[arg] + else: + assert arg in [l.last_exception, l.last_exc_value] + r = copyvar(self.translator, arg) + varmap[arg] = r + return r + else: + return arg + + newblock.closeblock(*[l.copy(rename) + for l in resume_point.links_to_resumption]) + # this check is a bit implicit! + if len(resume_point.links_to_resumption) > 1: + newblock.exitswitch = model.c_last_exception + else: + newblock.exitswitch = None - resuming_links.append(model.Link([], newblock, resume_point_index+1)) + resuming_links.append( + model.Link([], newblock, resume_point_index+1)) resuming_links[-1].llexitcase = resume_point_index+1 new_start_block.exitswitch = var_resume_state new_start_block.closeblock(not_resuming_link, *resuming_links) @@ -234,11 +313,14 @@ while i < len(block.operations): op = block.operations[i] if op.opname in ('direct_call', 'indirect_call'): - if i == len(block.operations) - 1 and block.exitswitch == model.c_last_exception: + if i == len(block.operations) - 1 \ + and block.exitswitch == model.c_last_exception: link = block.exits[0] else: - link = support.split_block_with_keepalive(self.translator, block, i+1) + link = support.split_block_with_keepalive( + self.translator, block, i+1) block.exitswitch = model.c_last_exception + link.llexitcase = None var_unwind_exception = varoftype(evalue) # for the case where we are resuming to an except: @@ -254,16 +336,25 @@ # function be called right at the end of the resuming # block, and that it is called even if the return # value is not again used. - args = [v for v in link.args - if v is not op.result and v.concretetype is not lltype.Void] + args = [] + for l in block.exits: + for arg in link.args: + if isinstance(arg, model.Variable) \ + and arg.concretetype is not lltype.Void \ + and arg is not op.result \ + and arg not in args: + args.append(arg) + save_block, frame_state_type = self.generate_save_block( args, var_unwind_exception) - self.resume_points.append(ResumePoint(op.result, link, frame_state_type)) + self.resume_points.append( + ResumePoint(op.result, args, block.exits, frame_state_type)) newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) - newlink.last_exception = model.Constant(code.UnwindException, etype) + newlink.last_exception = model.Constant(code.UnwindException, + etype) newlink.last_exc_value = var_unwind_exception newexits = list(block.exits) newexits.insert(1, newlink) @@ -299,20 +390,23 @@ saveops = save_state_block.operations frame_state_var = varoftype(lltype.Ptr(frame_type)) - saveops.append(model.SpaceOperation('malloc', - [model.Constant(frame_type, lltype.Void)], - frame_state_var)) + saveops.append(model.SpaceOperation( + 'malloc', + [model.Constant(frame_type, lltype.Void)], + frame_state_var)) saveops.extend(self.generate_saveops(frame_state_var, inputargs)) var_exc = varoftype(self.unwind_exception_type) - saveops.append(model.SpaceOperation("cast_pointer", [var_unwind_exception], - var_exc)) + saveops.append(model.SpaceOperation( + "cast_pointer", + [var_unwind_exception], + var_exc)) var_header = varoftype(lltype.Ptr(STATE_HEADER)) - # XXX should this be getsubstruct? - saveops.append(model.SpaceOperation("getsubstruct", [frame_state_var, model.Constant("header", lltype.Void)], var_header)) + saveops.append(model.SpaceOperation( + "cast_pointer", [frame_state_var], var_header)) saveops.append(model.SpaceOperation( "direct_call", @@ -320,39 +414,51 @@ varoftype(lltype.Void))) saveops.append(model.SpaceOperation( - "setfield", [var_header, model.Constant("restartstate", lltype.Void), - model.Constant(len(self.resume_points)+1, lltype.Signed)], - varoftype(lltype.Void))) - # XXX add returntypes + "setfield", + [var_header, model.Constant("restartstate", lltype.Void), + model.Constant(len(self.resume_points)+1, lltype.Signed)], + varoftype(lltype.Void))) funcptr = rtyper.type_system.getcallable(self.curr_graph) saveops.append(model.SpaceOperation( - "setfield", [var_header, model.Constant("function", lltype.Void), - model.Constant(llmemory.fakeaddress(funcptr), llmemory.Address)], - varoftype(lltype.Void))) + "setfield", + [var_header, model.Constant("function", lltype.Void), + model.Constant(llmemory.fakeaddress(funcptr), llmemory.Address)], + varoftype(lltype.Void))) + rettype = lltype.typeOf(funcptr).TO.RESULT + retval_type = {None: code.RETVAL_VOID, + 0: code.RETVAL_VOID_P, + 1: code.RETVAL_LONG, + 2: code.RETVAL_FLOAT, + 3: code.RETVAL_LONGLONG}[storage_type(rettype)] + + saveops.append(model.SpaceOperation( + "setfield", [var_header, model.Constant("retval_type", lltype.Void), + model.Constant(retval_type, lltype.Signed)], + varoftype(lltype.Void))) type_repr = rclass.get_type_repr(rtyper) - c_unwindexception = model.Constant(type_repr.convert_const(code.UnwindException), etype) + c_unwindexception = model.Constant( + type_repr.convert_const(code.UnwindException), etype) if not hasattr(self.curr_graph.exceptblock.inputargs[0], 'concretetype'): self.curr_graph.exceptblock.inputargs[0].concretetype = etype if not hasattr(self.curr_graph.exceptblock.inputargs[1], 'concretetype'): self.curr_graph.exceptblock.inputargs[1].concretetype = evalue - save_state_block.closeblock(model.Link([c_unwindexception, var_unwind_exception], - self.curr_graph.exceptblock)) - self.translator.rtyper._convert_link(save_state_block, save_state_block.exits[0]) + save_state_block.closeblock(model.Link( + [c_unwindexception, var_unwind_exception], + self.curr_graph.exceptblock)) + self.translator.rtyper._convert_link( + save_state_block, save_state_block.exits[0]) return save_state_block, frame_type - def generate_saveops(self, frame_state_var, varstosave): frame_type = frame_state_var.concretetype.TO ops = [] for i, var in enumerate(varstosave): t = storage_type(var.concretetype) fname = model.Constant(frame_type._names[i+1], lltype.Void) - ops.append(model.SpaceOperation('setfield', - [frame_state_var, fname, var], - varoftype(lltype.Void))) - + ops.append(model.SpaceOperation( + 'setfield', + [frame_state_var, fname, var], + varoftype(lltype.Void))) return ops - - From tismer at codespeak.net Tue Apr 11 01:36:24 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 11 Apr 2006 01:36:24 +0200 (CEST) Subject: [pypy-svn] r25679 - pypy/dist/pypy/objspace/flow Message-ID: <20060410233624.AD4341015A@code0.codespeak.net> Author: tismer Date: Tue Apr 11 01:36:20 2006 New Revision: 25679 Modified: pypy/dist/pypy/objspace/flow/flowcontext.py Log: added a facility to have a callback on every link created. purpose: late detecion of constants Modified: pypy/dist/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/dist/pypy/objspace/flow/flowcontext.py (original) +++ pypy/dist/pypy/objspace/flow/flowcontext.py Tue Apr 11 01:36:20 2006 @@ -133,7 +133,7 @@ vars2.append(Variable()) egg = EggBlock(vars2, block, case) ec.pendingblocks.append(egg) - link = Link(vars, egg, case) + link = ec.make_link(vars, egg, case) if attach: link.extravars(**attach) egg.extravars(**attach) # xxx @@ -211,6 +211,7 @@ initialblock = SpamBlock(FrameState(frame).copy()) self.pendingblocks = [initialblock] self.graph = FunctionGraph(name or code.co_name, initialblock) + self.make_link = Link # overridable for transition tracking def create_frame(self): # create an empty frame suitable for the code object @@ -225,6 +226,13 @@ def guessbool(self, w_condition, **kwds): return self.recorder.guessbool(self, w_condition, **kwds) + def start_monitoring(self, monitorfunc): + def make_link(*args): + link = Link(*args); + monitorfunc(link) + return link + self.make_link = make_link + def guessexception(self, *classes): def replace_exc_values(case): if case is not Exception: @@ -274,12 +282,12 @@ msg = "implicit %s shouldn't occur" % exc_cls.__name__ w_type = Constant(AssertionError) w_value = Constant(AssertionError(msg)) - link = Link([w_type, w_value], self.graph.exceptblock) + link = self.make_link([w_type, w_value], self.graph.exceptblock) self.recorder.crnt_block.closeblock(link) except OperationError, e: #print "OE", e.w_type, e.w_value - link = Link([e.w_type, e.w_value], self.graph.exceptblock) + link = self.make_link([e.w_type, e.w_value], self.graph.exceptblock) self.recorder.crnt_block.closeblock(link) except StopFlowing: @@ -290,7 +298,7 @@ else: assert w_result is not None - link = Link([w_result], self.graph.returnblock) + link = self.make_link([w_result], self.graph.returnblock) self.recorder.crnt_block.closeblock(link) del self.recorder @@ -346,7 +354,8 @@ newblock = SpamBlock(newstate) # unconditionally link the current block to the newblock outputargs = currentstate.getoutputargs(newstate) - currentblock.closeblock(Link(outputargs, newblock)) + link = self.make_link(outputargs, newblock) + currentblock.closeblock(link) # phew if not finished: if block is not None: @@ -356,7 +365,7 @@ block.operations = () block.exitswitch = None outputargs = block.framestate.getoutputargs(newstate) - block.recloseblock(Link(outputargs, newblock)) + block.recloseblock(self.make_link(outputargs, newblock)) candidates.remove(block) candidates.insert(0, newblock) self.pendingblocks.append(newblock) From tismer at codespeak.net Tue Apr 11 01:49:51 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 11 Apr 2006 01:49:51 +0200 (CEST) Subject: [pypy-svn] r25680 - pypy/dist/pypy/translator Message-ID: <20060410234951.5197A10161@code0.codespeak.net> Author: tismer Date: Tue Apr 11 01:49:49 2006 New Revision: 25680 Modified: pypy/dist/pypy/translator/simplify.py Log: allow removal of unused imports. Reason: constant imports detected during flowing. XXX maybe it is better to not do this statically, but carry a list of things to be cleaned up which became useless after flowing? Maybe in the space instance or graph? Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Tue Apr 11 01:49:49 2006 @@ -449,6 +449,7 @@ CanRemoveBuiltins = { isinstance: True, hasattr: True, + __import__: True, # imports for side-effects must store the result } def transform_dead_op_vars_in_blocks(blocks, translator=None): From tismer at codespeak.net Tue Apr 11 01:54:01 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 11 Apr 2006 01:54:01 +0200 (CEST) Subject: [pypy-svn] r25681 - pypy/dist/pypy/objspace/flow Message-ID: <20060410235401.4B6A210165@code0.codespeak.net> Author: tismer Date: Tue Apr 11 01:53:58 2006 New Revision: 25681 Modified: pypy/dist/pypy/objspace/flow/objspace.py Log: a more or less small addition to flow space: Automatic decision if an import is immediate or not. The implementation gave me quite a good time until I found out how to do this. But now we should be able to drop 'do_imports_immediately' by using 'deferred constantness' (The concept is documented inside :-) Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Tue Apr 11 01:53:58 2006 @@ -63,9 +63,8 @@ #self.make_sys() # objects which should keep their SomeObjectness self.not_really_const = NOT_REALLY_CONST - # variables which might in turn turn into constants. - # purpose: allow for importing into globals. - self.maybe_const = {} # variable -> constant + # tracking variables which might in turn turn into constants. + self.const_tracker = None def enter_cache_building_mode(self): # when populating the caches, the flow space switches to @@ -168,7 +167,8 @@ to_check = obj if hasattr(to_check, 'im_self'): to_check = to_check.im_self - if (not isinstance(to_check, (type, types.ClassType)) and # classes/types are assumed immutable + if (not isinstance(to_check, (type, types.ClassType, types.ModuleType)) and + # classes/types/modules are assumed immutable hasattr(to_check, '__class__') and to_check.__class__.__module__ != '__builtin__'): frozen = hasattr(to_check, '_freeze_') and to_check._freeze_() if not frozen: @@ -594,9 +594,53 @@ for line in ObjSpace.MethodTable: make_op(*line) -# override getattr for not really const objects +""" +Strategy for a new import logic +------------------------------- + +It is an old problem to decide whether to use do_imports_immediately. +In general, it would be nicer not to use this flag for RPython, in order +to make it easy to support imports at run-time for extensions. + +On the other hand, there are situations where this is absolutely needed: +Some of the ll helper functions need to import something late, to +avoid circular imports. Not doing the import immediately would cause +a crash, because the imported object would become SomeObject. + +We would like to have control over imports even on a per-import policy. + +As a general solution, I came up with the following trick, or maybe it's +not a trick but a good concept: + +By declaring the imported subject as a global, you trigger the immediate +import. This is consistent with the RPython concept that globals +should never change, just with the addition that objects may be added. +In addition, we consider global modules to be immutable, making attribute +access a constant operation. + +As a generalisation, we can enforce that getattr/setattr on any +object that is unwrappable for computation is evaluated +immediately. This gives us early detection of programming errors. +XXX this step isn't done, yet, need to discuss this. + +Implementation +-------------- + +It is not completely trivial, since we have to intercept the process +of flowing, to keep trak of which variable might become a constant. +Finally I ended up with a rather simple solution: +Flowcontext monitors every link creation, by no longer using +Link() directly, but redirecting this to a function make_link, +which can be patched to record the creation of links. + +The actual tracking and constant resolving is implemented in the +ConstTracker class below. + +""" def override(): + from __builtin__ import getattr as _getattr # uhmm + def getattr(self, w_obj, w_name): # handling special things like sys # (maybe this will vanish with a unique import logic) @@ -604,23 +648,90 @@ const_w = self.not_really_const[w_obj] if w_name not in const_w: return self.do_operation_with_implicit_exceptions('getattr', w_obj, w_name) - # tracking variables which might be constants - return self.regular_getattr(w_obj, w_name) + w_res = self.regular_getattr(w_obj, w_name) + # tracking variables which might be(come) constants + if self.const_tracker: + self.track_possible_constant(w_res, _getattr, w_obj, w_name) + return w_res FlowObjSpace.regular_getattr = FlowObjSpace.getattr FlowObjSpace.getattr = getattr - # protect us from globals access + # protect us from globals access but support constant import into globals def setitem(self, w_obj, w_key, w_val): ec = self.getexecutioncontext() if not (ec and w_obj is ec.w_globals): return self.regular_setitem(w_obj, w_key, w_val) - raise SyntaxError, "attempt to write global attribute %r in %r" % (w_key, ec.graph.func) + globals = self.unwrap(w_obj) + try: + key = self.unwrap_for_computation(self.resolve_constant(w_key)) + val = self.unwrap_for_computation(self.resolve_constant(w_val)) + if key not in globals or val == globals[key]: + globals[key] = val + return self.w_None + except UnwrapException: + pass + raise SyntaxError, "attempt to modify global attribute %r in %r" % (w_key, ec.graph.func) FlowObjSpace.regular_setitem = FlowObjSpace.setitem FlowObjSpace.setitem = setitem + def track_possible_constant(self, w_ret, func, *args_w): + if not self.const_tracker: + self.const_tracker = ConstTracker(self) + tracker = self.const_tracker + tracker.track_call(w_ret, func, *args_w) + self.getexecutioncontext().start_monitoring(tracker.monitor_transition) + + FlowObjSpace.track_possible_constant = track_possible_constant + + def resolve_constant(self, w_obj): + if self.const_tracker: + w_obj = self.const_tracker.resolve_const(w_obj) + return w_obj + + FlowObjSpace.resolve_constant = resolve_constant + override() + +class ConstTracker(object): + def __init__(self, space): + assert isinstance(space, FlowObjSpace) + self.space = space + self.known_consts = {} + self.tracked_vars = {} + self.mapping = {} + + def track_call(self, w_res, callable, *args_w): + """ defer evaluation of this expression until a const is needed + """ + self.mapping[w_res] = w_res + self.tracked_vars[w_res] = callable, args_w + + def monitor_transition(self, link): + for vin, vout in zip(link.args, link.target.inputargs): + # we record all true transitions, but no cycles. + if vin in self.mapping and vout not in self.mapping: + # the mapping leads directly to the origin. + self.mapping[vout] = self.mapping[vin] + + def resolve_const(self, w_obj): + """ compute a latent constant expression """ + if isinstance(w_obj, Constant): + return w_obj + w = self.mapping.get(w_obj, w_obj) + if w in self.known_consts: + return self.known_consts[w] + if w not in self.tracked_vars: + raise SyntaxError, 'RPython: cannot compute a constant for %s in %s' % ( + w_obj, self.space.getexecutioncontext().graph.func) + callable, args_w = self.tracked_vars.pop(w) + args_w = [self.resolve_const(w_x) for w_x in args_w] + args = [self.space.unwrap_for_computation(w_x) for w_x in args_w] + w_ret = self.space.wrap(callable(*args)) + self.known_consts[w] = w_ret + return w_ret + # ______________________________________________________________________ # End of objspace.py From tismer at codespeak.net Tue Apr 11 01:55:13 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 11 Apr 2006 01:55:13 +0200 (CEST) Subject: [pypy-svn] r25682 - pypy/dist/pypy/objspace/flow Message-ID: <20060410235513.36EF410168@code0.codespeak.net> Author: tismer Date: Tue Apr 11 01:55:11 2006 New Revision: 25682 Modified: pypy/dist/pypy/objspace/flow/specialcase.py Log: rewrote the import special case logic. It uses the new delayed constantness feature of flow space. Modified: pypy/dist/pypy/objspace/flow/specialcase.py ============================================================================== --- pypy/dist/pypy/objspace/flow/specialcase.py (original) +++ pypy/dist/pypy/objspace/flow/specialcase.py Tue Apr 11 01:55:11 2006 @@ -6,19 +6,21 @@ def sc_import(space, fn, args): w_name, w_glob, w_loc, w_frm = args.fixedunpack(4) - try: - name, glob, loc, frm = (space.unwrap(w_name), space.unwrap(w_glob), - space.unwrap(w_loc), space.unwrap(w_frm)) - except UnwrapException: + if not isinstance(w_loc, Constant): # import * in a function gives us the locals as Variable - # we forbid it as a SyntaxError + # we always forbid it as a SyntaxError raise SyntaxError, "RPython: import * is not allowed in functions" if space.do_imports_immediately: + name, glob, loc, frm = (space.unwrap(w_name), space.unwrap(w_glob), + space.unwrap(w_loc), space.unwrap(w_frm)) return space.wrap(__import__(name, glob, loc, frm)) # redirect it, but avoid exposing the globals w_glob = Constant({}) - return space.do_operation('simple_call', Constant(__import__), - w_name, w_glob, w_loc, w_frm) + w_ret = space.do_operation('simple_call', Constant(__import__), + w_name, w_glob, w_loc, w_frm) + # let the space decide later if this should be a constant import + space.track_possible_constant(w_ret, __import__, w_name, w_glob, w_loc, w_frm) + return w_ret def sc_operator(space, fn, args): args_w, kwds_w = args.unpack() From tismer at codespeak.net Tue Apr 11 05:47:28 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 11 Apr 2006 05:47:28 +0200 (CEST) Subject: [pypy-svn] r25683 - in pypy/dist/pypy/objspace/flow: . test Message-ID: <20060411034728.8348310166@code0.codespeak.net> Author: tismer Date: Tue Apr 11 05:47:24 2006 New Revision: 25683 Modified: pypy/dist/pypy/objspace/flow/objspace.py pypy/dist/pypy/objspace/flow/test/test_objspace.py Log: testing lazy constant. const propagation for modules not yet implemented (do we want this?) Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Tue Apr 11 05:47:24 2006 @@ -193,6 +193,8 @@ self.executioncontext = ec from pypy.objspace.flow import specialcase specialcase.setup(self) + self.const_tracker = None + # maybe it would be cleaner to have the tracker in the EC itself? def exception_match(self, w_exc_type, w_check_class): self.executioncontext.recorder.crnt_block.exc_handler = True Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_objspace.py Tue Apr 11 05:47:24 2006 @@ -655,7 +655,31 @@ def test_import_something(self): def f(): from some.unknown.module import stuff - self.codetest(f) + g = self.codetest(f) + + def test_lazy_constant_simple(self): + def f(): + global xxx_global + from pypy.objspace.flow.test.test_objspace import \ + user_defined_function as xxx_global, DATA + return xxx_global + g = self.codetest(f) + consts = [self.space.unwrap(c) for c in self.space.const_tracker.known_consts.values()] + assert user_defined_function in consts + assert DATA not in consts + global xxx_global; xxx_global = 42; del xxx_global + + def not_yet_test_lazy_constant_harder(self): + def f(): + global xxx_global + import pypy as xxx_global + ret = xxx_global.objspace.flow.test.test_objspace.user_defined_function + return ret + g = self.codetest(f) + consts = [self.space.unwrap(c) for c in self.space.const_tracker.known_consts.values()] + assert user_defined_function in consts + assert DATA not in consts + global xxx_global; xxx_global = 42; del xxx_global DATA = {'x': 5, 'y': 6} From tismer at codespeak.net Tue Apr 11 07:41:13 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 11 Apr 2006 07:41:13 +0200 (CEST) Subject: [pypy-svn] r25684 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060411054113.1C10210174@code0.codespeak.net> Author: tismer Date: Tue Apr 11 07:41:11 2006 New Revision: 25684 Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py Log: o wonder, even one more ll operation, needed for armin's var-arg case of wrapper.py Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Tue Apr 11 07:41:11 2006 @@ -315,6 +315,7 @@ 'check_no_more_arg': LLOp(canraise=(Exception,)), 'decode_arg': LLOp(canraise=(Exception,)), 'decode_arg_def': LLOp(canraise=(Exception,)), + 'getslice': LLOp(canraise=(Exception,)), } # __________ operations on PyObjects __________ From arigo at codespeak.net Tue Apr 11 10:16:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 10:16:59 +0200 (CEST) Subject: [pypy-svn] r25688 - pypy/dist/pypy/lib/test2 Message-ID: <20060411081659.916F010171@code0.codespeak.net> Author: arigo Date: Tue Apr 11 10:16:58 2006 New Revision: 25688 Modified: pypy/dist/pypy/lib/test2/test_marshal_extra.py Log: Python 2.5 compatibility. Modified: pypy/dist/pypy/lib/test2/test_marshal_extra.py ============================================================================== --- pypy/dist/pypy/lib/test2/test_marshal_extra.py (original) +++ pypy/dist/pypy/lib/test2/test_marshal_extra.py Tue Apr 11 10:16:58 2006 @@ -47,13 +47,19 @@ except NameError: pass # Python < 2.4 +if getattr(cpy_marshal, 'version', 0) > 1: + cpy_dump_version = (1,) +else: + cpy_dump_version = () + def test_cases(): for case in TESTCASES: yield dumps_and_reload, case yield loads_from_cpython, case yield dumps_to_cpython, case - yield dumps_subclass, case + if case is not StopIteration: + yield dumps_subclass, case yield load_from_cpython, case yield dump_to_cpython, case @@ -66,7 +72,7 @@ def loads_from_cpython(case): print 'load_from_cpython', `case` try: - s = cpy_marshal.dumps(case) + s = cpy_marshal.dumps(case, *cpy_dump_version) except ValueError: py.test.skip("this version of CPython doesn't support this object") obj = marshal.loads(s) @@ -75,7 +81,7 @@ def dumps_to_cpython(case): print 'dump_to_cpython', `case` try: - cpy_marshal.dumps(case) + cpy_marshal.dumps(case, *cpy_dump_version) except ValueError: py.test.skip("this version of CPython doesn't support this object") s = marshal.dumps(case) @@ -99,7 +105,7 @@ f1 = open(p, "w") try: try: - s = cpy_marshal.dump(case, f1) + s = cpy_marshal.dump(case, f1, *cpy_dump_version) finally: f1.close() except ValueError: @@ -115,7 +121,7 @@ def dump_to_cpython(case): try: - cpy_marshal.dumps(case) + cpy_marshal.dumps(case, *cpy_dump_version) except ValueError: py.test.skip("this version of CPython doesn't support this object") From arigo at codespeak.net Tue Apr 11 10:42:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 10:42:47 +0200 (CEST) Subject: [pypy-svn] r25689 - in pypy/dist/pypy/translator: . test Message-ID: <20060411084247.B895B1016A@code0.codespeak.net> Author: arigo Date: Tue Apr 11 10:42:46 2006 New Revision: 25689 Modified: pypy/dist/pypy/translator/geninterplevel.py pypy/dist/pypy/translator/test/snippet.py pypy/dist/pypy/translator/test/test_geninterp.py Log: Fix for negative longs in geninterp. Modified: pypy/dist/pypy/translator/geninterplevel.py ============================================================================== --- pypy/dist/pypy/translator/geninterplevel.py (original) +++ pypy/dist/pypy/translator/geninterplevel.py Tue Apr 11 10:42:46 2006 @@ -127,9 +127,9 @@ 'def %s(value):\n' ' dic = space.newdict([])\n' ' space.exec_("", dic, dic) # init __builtins__\n' - ' return space.eval("long(%%r, 16)" %% value, dic, dic)' % unique) + ' return space.eval(value, dic, dic)' % unique) self.initcode.append1('%s = %s(%r)' % ( - name, unique, hex(value)[2:-1] ) ) + name, unique, repr(value) ) ) def bltinmod_helper(self, mod): name = self.uniquename("mod_%s" % mod.__name__) Modified: pypy/dist/pypy/translator/test/snippet.py ============================================================================== --- pypy/dist/pypy/translator/test/snippet.py (original) +++ pypy/dist/pypy/translator/test/snippet.py Tue Apr 11 10:42:46 2006 @@ -623,6 +623,9 @@ def t_issubclass(x, y): return issubclass(type(x), (int, long)) and issubclass(type(y), int) +def t_neg_long(): + return -132L + # --------------------(Currently) Non runnable Functions --------------------- Modified: pypy/dist/pypy/translator/test/test_geninterp.py ============================================================================== --- pypy/dist/pypy/translator/test/test_geninterp.py (original) +++ pypy/dist/pypy/translator/test/test_geninterp.py Tue Apr 11 10:42:46 2006 @@ -270,3 +270,8 @@ fn = self.build_interpfunc(snippet.t_issubclass) result = fn(1, 2) assert result == True + + def test_negative_long(self): + fn = self.build_interpfunc(snippet.t_neg_long) + result = fn() + assert result == -132L and type(result) is long From arigo at codespeak.net Tue Apr 11 11:21:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 11:21:35 +0200 (CEST) Subject: [pypy-svn] r25690 - pypy/dist/pypy/annotation Message-ID: <20060411092135.9AA201010D@code0.codespeak.net> Author: arigo Date: Tue Apr 11 11:21:34 2006 New Revision: 25690 Modified: pypy/dist/pypy/annotation/binaryop.py Log: Remove useless unions(). Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Tue Apr 11 11:21:34 2006 @@ -723,14 +723,6 @@ def sub((s_addr, s_int)): return SomeAddress(is_null=False) -class __extend__(pairtype(SomeAddress, SomeImpossibleValue)): - def union((s_addr, s_imp)): - return s_addr - -class __extend__(pairtype(SomeImpossibleValue, SomeAddress)): - def union((s_imp, s_addr)): - return s_addr - class __extend__(pairtype(SomeAddress, SomeObject)): def union((s_addr, s_obj)): raise UnionError, "union of address and anything else makes no sense" From arigo at codespeak.net Tue Apr 11 11:23:39 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 11:23:39 +0200 (CEST) Subject: [pypy-svn] r25691 - pypy/dist/pypy/rpython Message-ID: <20060411092339.1C21B10179@code0.codespeak.net> Author: arigo Date: Tue Apr 11 11:23:38 2006 New Revision: 25691 Modified: pypy/dist/pypy/rpython/llinterp.py Log: new operation 'getslice' was not added to llinterp.py. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Tue Apr 11 11:23:38 2006 @@ -376,12 +376,15 @@ def op_decode_arg(self, fname, i, name, vargs, vkwds): raise NotImplementedError("decode_arg") - def op_decode_arg_def(fname, i, name, vargs, vkwds, default): + def op_decode_arg_def(self, fname, i, name, vargs, vkwds, default): raise NotImplementedError("decode_arg_def") - def op_check_no_more_arg(fname, n, vargs): + def op_check_no_more_arg(self, fname, n, vargs): raise NotImplementedError("check_no_more_arg") + def op_getslice(self, vargs, start, stop_should_be_None): + raise NotImplementedError("getslice") # only for argument parsing + def op_setfield(self, obj, fieldname, fieldvalue): # obj should be pointer FIELDTYPE = getattr(lltype.typeOf(obj).TO, fieldname) From arigo at codespeak.net Tue Apr 11 11:29:51 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 11:29:51 +0200 (CEST) Subject: [pypy-svn] r25692 - pypy/dist/pypy/annotation Message-ID: <20060411092951.E89C01017A@code0.codespeak.net> Author: arigo Date: Tue Apr 11 11:29:51 2006 New Revision: 25692 Added: pypy/dist/pypy/annotation/binaryop.py - copied unchanged from r25689, pypy/dist/pypy/annotation/binaryop.py Log: Revert r25690. From stephan at codespeak.net Tue Apr 11 12:07:02 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 11 Apr 2006 12:07:02 +0200 (CEST) Subject: [pypy-svn] r25693 - pypy/dist/pypy/objspace/std Message-ID: <20060411100702.AA92310178@code0.codespeak.net> Author: stephan Date: Tue Apr 11 12:07:01 2006 New Revision: 25693 Modified: pypy/dist/pypy/objspace/std/frozensettype.py pypy/dist/pypy/objspace/std/setobject.py Log: all tests from lib-python/modified-2.4.1/test/test_set.py are passing now, but tests for deepcopy and pickle. (Haven't checked compilability yet) Modified: pypy/dist/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/dist/pypy/objspace/std/frozensettype.py (original) +++ pypy/dist/pypy/objspace/std/frozensettype.py Tue Apr 11 12:07:01 2006 @@ -18,8 +18,11 @@ def descr__frozenset__new__(space, w_frozensettype, w_iterable=NoneNotWrapped): from pypy.objspace.std.setobject import W_FrozensetObject + from pypy.objspace.std.setobject import _is_frozenset_exact if w_iterable is None: w_iterable = space.newtuple([]) + if _is_frozenset_exact(space, w_iterable): + return w_iterable w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype) W_FrozensetObject.__init__(w_obj, space, w_iterable) Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Tue Apr 11 12:07:01 2006 @@ -11,16 +11,16 @@ def __init__(w_self, space, wrappeditems=None): W_Object.__init__(w_self, space) w_self.setdata = setdata = r_dict(space.eq_w, space.hash_w) - if wrappeditems is not None: - iterable_w = space.unpackiterable(wrappeditems) - for w_item in iterable_w: - w_self.setdata[w_item] = None + _initialize_set(space, w_self, wrappeditems) def __repr__(w_self): """representation for debugging purposes""" reprlist = [repr(w_item) for w_item in w_self.setdata.keys()] return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist)) + def _newobj(w_self, space): + return space.call(space.type(w_self),None) + class W_SetObject(W_BaseSetObject): from pypy.objspace.std.settype import set_typedef as typedef @@ -87,12 +87,18 @@ return data -def _dict_to_set(space, rpdict): - return space.newset(W_SetIterObject(space, rpdict)) - -def _dict_to_frozenset(space, rpdict): - #return space.newfrozenset(space.newtuple(rpdict.keys())) - return space.newfrozenset(W_SetIterObject(space, rpdict)) +def _initialize_set(space, w_obj, wrappeditems=None): + w_obj.setdata.clear() + if wrappeditems is not None: + w_iterator = space.iter(wrappeditems) + while True: + try: + w_item = space.next(w_iterator) + except OperationError, e: + if not e.match(space, space.w_StopIteration): + raise + break + w_obj.setdata[w_item] = None # helper functions for set operation on dicts @@ -103,6 +109,12 @@ else: return False +def _is_frozenset_exact(space, w_obj): + if space.eq_w(space.type(w_obj), space.w_frozenset): + return True + else: + return False + def _union_dict(space, ldict, rdict, isupdate): if isupdate: ld = ldict @@ -187,10 +199,15 @@ return space.w_None def set_copy__Set(space, w_left): - return space.newset(w_left) + w_obj = w_left._newobj(space) + w_obj.setdata.update(w_left.setdata) + return w_obj def frozenset_copy__Frozenset(space, w_left): - return space.newfrozenset(w_left) + if _is_frozenset_exact(space, w_left): + return w_left + else: + return set_copy__Set(space,w_left) def set_clear__Set(space, w_left): w_left.setdata.clear() @@ -199,18 +216,15 @@ def set_difference__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _difference_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) + w_obj = w_left._newobj(space) + w_obj.setdata.update(new_ld) + return w_obj sub__Set_Set = set_difference__Set_ANY sub__Set_Frozenset = set_difference__Set_ANY - -def frozenset_difference__Frozenset_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _difference_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -sub__Frozenset_Set = frozenset_difference__Frozenset_ANY -sub__Frozenset_Frozenset = frozenset_difference__Frozenset_ANY +frozenset_difference__Frozenset_ANY = set_difference__Set_ANY +sub__Frozenset_Set = set_difference__Set_ANY +sub__Frozenset_Frozenset = set_difference__Set_ANY def set_difference_update__Set_ANY(space, w_left, w_other): @@ -247,17 +261,18 @@ except Exception, exp: if _is_setlike(space, w_other): w_f = space.newfrozenset(w_other) - return space.newbool(w_f) + r = w_f in w_left.setdata + return space.newbool(r) else: - return space.w_False + raise exp contains__Frozenset_ANY = contains__Set_ANY -def set_issubset__Set_Set(space, w_left, w_other): +def set_issubset__Set_ANY(space, w_left, w_other): if space.is_w(w_left, w_other): return space.w_True - ld, rd = w_left.setdata, w_other.setdata + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) if len(ld) > len(rd): return space.w_False @@ -266,19 +281,17 @@ return space.w_False return space.w_True -set_issubset__Set_Frozenset = set_issubset__Set_Set -frozenset_issubset__Frozenset_Set = set_issubset__Set_Set -frozenset_issubset__Frozenset_Frozenset = set_issubset__Set_Set - -le__Set_Set = set_issubset__Set_Set -le__Set_Frozenset = set_issubset__Set_Set -le__Frozenset_Frozenset = set_issubset__Set_Set +frozenset_issubset__Frozenset_ANY = set_issubset__Set_ANY -def set_issuperset__Set_Set(space, w_left, w_other): +le__Set_Set = set_issubset__Set_ANY +le__Set_Frozenset = set_issubset__Set_ANY +le__Frozenset_Frozenset = set_issubset__Set_ANY + +def set_issuperset__Set_ANY(space, w_left, w_other): if space.is_w(w_left, w_other): return space.w_True - ld, rd = w_left.setdata, w_other.setdata + ld, rd = w_left.setdata, _iter_to_dict(space, w_other) if len(ld) < len(rd): return space.w_False @@ -287,17 +300,23 @@ return space.w_False return space.w_True -set_issuperset__Set_Frozenset = set_issuperset__Set_Set -frozenset_issuperset__Frozenset_Set = set_issuperset__Set_Set -frozenset_issuperset__Frozenset_Frozenset = set_issuperset__Set_Set - -ge__Set_Set = set_issuperset__Set_Set -ge__Set_Frozenset = set_issuperset__Set_Set -ge__Frozenset_Frozenset = set_issuperset__Set_Set +frozenset_issuperset__Frozenset_ANY = set_issuperset__Set_ANY + +ge__Set_Set = set_issuperset__Set_ANY +ge__Set_Frozenset = set_issuperset__Set_ANY +ge__Frozenset_Frozenset = set_issuperset__Set_ANY def set_discard__Set_ANY(space, w_left, w_item): - if w_item in w_left.setdata: - del w_left.setdata[w_item] + try: + if w_item in w_left.setdata: + del w_left.setdata[w_item] + except Exception, exp: + if _is_setlike(space, w_item): + w_f = space.newfrozenset(w_item) + if w_f in w_left.setdata: + del w_left.setdata[w_f] + else: + raise exp def set_remove__Set_ANY(space, w_left, w_item): try: @@ -305,6 +324,16 @@ except KeyError: raise OperationError(space.w_KeyError, space.call_method(w_item,'__repr__')) + except Exception, exp: + if _is_setlike(space, w_item): + w_f = space.newfrozenset(w_item) + try: + del w_left.setdata[w_f] + except KeyError: + raise OperationError(space.w_KeyError, + space.call_method(w_item,'__repr__')) + else: + raise exp def hash__Set(space, w_set): raise OperationError(space.w_TypeError, @@ -339,18 +368,15 @@ def set_intersection__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _intersection_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) + w_obj = w_left._newobj(space) + w_obj.setdata.update(new_ld) + return w_obj and__Set_Set = set_intersection__Set_ANY and__Set_Frozenset = set_intersection__Set_ANY - -def frozenset_intersection__Frozenset_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _intersection_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -and__Frozenset_Set = frozenset_intersection__Frozenset_ANY -and__Frozenset_Frozenset = frozenset_intersection__Frozenset_ANY +frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY +and__Frozenset_Set = set_intersection__Set_ANY +and__Frozenset_Frozenset = set_intersection__Set_ANY def set_intersection_update__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, _iter_to_dict(space, w_other) @@ -366,18 +392,17 @@ def set_symmetric_difference__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) + w_obj = w_left._newobj(space) + w_obj.setdata.update(new_ld) + return w_obj xor__Set_Set = set_symmetric_difference__Set_ANY xor__Set_Frozenset = set_symmetric_difference__Set_ANY -def frozenset_symmetric_difference__Frozenset_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -xor__Frozenset_Set = frozenset_symmetric_difference__Frozenset_ANY -xor__Frozenset_Frozenset = frozenset_symmetric_difference__Frozenset_ANY +frozenset_symmetric_difference__Frozenset_ANY = \ + set_symmetric_difference__Set_ANY +xor__Frozenset_Set = set_symmetric_difference__Set_ANY +xor__Frozenset_Frozenset = set_symmetric_difference__Set_ANY def set_symmetric_difference_update__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, _iter_to_dict(space, w_other) @@ -393,18 +418,15 @@ def set_union__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, _iter_to_dict(space, w_other) new_ld, rd = _union_dict(space, ld, rd, False) - return _dict_to_set(space, new_ld) + w_obj = w_left._newobj(space) + w_obj.setdata.update(new_ld) + return w_obj or__Set_Set = set_union__Set_ANY or__Set_Frozenset = set_union__Set_ANY - -def frozenset_union__Frozenset_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _union_dict(space, ld, rd, False) - return _dict_to_frozenset(space, new_ld) - -or__Frozenset_Set = frozenset_union__Frozenset_ANY -or__Frozenset_Frozenset = frozenset_union__Frozenset_ANY +frozenset_union__Frozenset_ANY = set_union__Set_ANY +or__Frozenset_Set = set_union__Set_ANY +or__Frozenset_Frozenset = set_union__Set_ANY def len__Set(space, w_left): return space.newint(len(w_left.setdata)) @@ -428,7 +450,7 @@ w_iterable, = __args__.parse('set', (['some_iterable'], None, None), [W_SetObject(space,None)]) - W_SetObject.__init__(w_set, space, w_iterable) + _initialize_set(space, w_set, w_iterable) app = gateway.applevel(""" def ne__Set_ANY(s, o): @@ -441,10 +463,7 @@ return s != o and s.issubset(o) def repr__Set(s): - return 'set(%s)' % [x for x in s] - - def repr__Frozenset(s): - return 'frozenset(%s)' % [x for x in s] + return '%s(%s)' % (s.__class__.__name__, [x for x in s]) """, filename=__file__) @@ -462,7 +481,7 @@ lt__Frozenset_Frozenset = lt__Set_Set repr__Set = app.interphook('repr__Set') -repr__Frozenset = app.interphook('repr__Frozenset') +repr__Frozenset = app.interphook('repr__Set') from pypy.objspace.std import frozensettype from pypy.objspace.std import settype From auc at codespeak.net Tue Apr 11 12:20:20 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 11 Apr 2006 12:20:20 +0200 (CEST) Subject: [pypy-svn] r25694 - in pypy/dist/pypy/objspace: . constraint constraint/test Message-ID: <20060411102020.2140910189@code0.codespeak.net> Author: auc Date: Tue Apr 11 12:20:17 2006 New Revision: 25694 Added: pypy/dist/pypy/objspace/constraint/test/test_distributor.py Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/distributor.py pypy/dist/pypy/objspace/logic.py Log: instantiate distributors Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Tue Apr 11 12:20:17 2006 @@ -47,6 +47,18 @@ estimate_cost = interp2app(W_Constraint.w_estimate_cost), revise = interp2app(W_Constraint.w_revise)) + +#-- Distributors (standards) ------------ + +class W_Distributor(Wrappable): + + def __init__(self, object_space, fanout): + self._space = object_space + self.fanout = fanout + +W_Distributor.typedef = typedef.TypeDef("W_Distributor") + + #-- Computation space ------------------- class W_ComputationSpace(Wrappable): Modified: pypy/dist/pypy/objspace/constraint/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/distributor.py Tue Apr 11 12:20:17 2006 @@ -1,5 +1,12 @@ import math +from pypy.interpreter.error import OperationError +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter import baseobjspace, typedef, gateway +from pypy.interpreter.gateway import interp2app + +from pypy.objspace.constraint.computationspace import W_Distributor + def arrange_domains(cs, variables): """build a data structure from var to dom that satisfies distribute & friends""" @@ -8,17 +15,17 @@ new_doms[var] = cs.dom(var).copy() return new_doms -class AbstractDistributor(object): + +class W_AbstractDistributor(W_Distributor): """_distribute is left unimplemented.""" - def __init__(self, c_space, nb_subspaces=2): - self.nb_subspaces = nb_subspaces - self.cs = c_space - self.verbose = 0 - - def set_space(self, space): - self.cs = space - + def w_fanout(self): + return self._space.newint(self.fanout) + + def fanout(self): + """return number of possible splits""" + return self.fanout + def find_smallest_domain(self): """returns the variable having the smallest domain. (or one of such varibles if there is a tie) @@ -33,13 +40,11 @@ return best - def nb_subdomains(self): - """return number of possible splits""" - return self.nb_subspaces - - + def w_distribute(self, w_cs, w_choice): + assert isintance(w_choice, W_IntegerObject) + self.distribute(w_cs, self._space.int_w(w_choice)) - def distribute(self, choice): + def distribute(self, w_cs, choice_w): variable = self._find_distribution_variable() self._do_distribute(self.cs.dom(variable), choice) for const in self.cs.dependant_constraints(variable): @@ -52,17 +57,20 @@ """remove values from domain depending on choice""" raise NotImplementedError +W_Distributor.typedef = typedef.TypeDef("W_AbstractDistributor", + W_Distributor.typedef, + fanout = interp2app(W_AbstractDistributor.w_fanout), + distribute = interp2app(W_AbstractDistributor.w_distribute)) - - -class NaiveDistributor(AbstractDistributor): +class W_NaiveDistributor(W_AbstractDistributor): """distributes domains by splitting the smallest domain in 2 new domains The first new domain has a size of one, and the second has all the other values""" - def __init__(self, c_space): - AbstractDistributor.__init__(self, c_space, 2) + def __init__(self, object_space, fanout_w): + # default fanout is 2, see make_naive_distributor + W_Distributor.__init__(self, object_space, fanout_w) def _do_distribute(self, domain, choice): values = domain.get_values() @@ -70,17 +78,26 @@ domain.remove_values(values[1:]) else: domain.remove_value(values[0]) - -class SplitDistributor(AbstractDistributor): +W_NaiveDistributor.typedef = typedef.TypeDef( + "W_NaiveDistributor", + W_AbstractDistributor.typedef) + +def make_naive_distributor(object_space, fanout=2): + return object_space.wrap(W_NaiveDistributor(object_space, fanout)) +app_make_naive_distributor = interp2app(make_naive_distributor, + unwrap_spec = [baseobjspace.ObjSpace, int]) + + +class W_SplitDistributor(W_AbstractDistributor): """distributes domains by splitting the smallest domain in nb_subspaces equal parts or as equal as possible. If nb_subspaces is 0, then the smallest domain is split in domains of size 1""" - def __init__(self, c_space, nb_subspaces=3): - AbstractDistributor.__init__(self, c_space, nb_subspaces) - + def __init__(self, object_space, fanout_w): + # default fanout is 3, see make_split_distributor + W_Distributor.__init__(self, object_space, fanout_w) def nb_subdomains(self): to_split = self.find_smallest_domain() @@ -100,11 +117,21 @@ domain.remove_values(values[:start]) domain.remove_values(values[end:]) +def make_split_distributor(object_space, fanout=3): + return object_space.wrap(W_SplitDistributor(object_space, fanout)) +app_make_split_distributor = interp2app(make_split_distributor, + unwrap_spec = [baseobjspace.ObjSpace, int]) + -class DichotomyDistributor(SplitDistributor): +class W_DichotomyDistributor(W_SplitDistributor): """distributes domains by splitting the smallest domain in two equal parts or as equal as possible""" - def __init__(self, c_space): - SplitDistributor.__init__(self, c_space, 2) + def __init__(self, object_space, w_fanout): + W_SplitDistributor.__init__(self, object_space, w_fanout) + +def make_dichotomy_distributor(object_space): + return make_split_distributor(object_space, 2) +app_make_dichotomy_distributor = interp2app(make_dichotomy_distributor) + -DefaultDistributor = NaiveDistributor +#DefaultDistributor = W_NaiveDistributor Added: pypy/dist/pypy/objspace/constraint/test/test_distributor.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/test/test_distributor.py Tue Apr 11 12:20:17 2006 @@ -0,0 +1,14 @@ +from pypy.conftest import gettestobjspace + + +class AppTest_Distributor(object): + + def setup_class(cls): + cls.space = gettestobjspace('logic') + + def test_instantiate(self): + d1 = NaiveDistributor() + d2 = SplitDistributor(4) + d3 = DichotomyDistributor() + + Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Tue Apr 11 12:20:17 2006 @@ -754,21 +754,25 @@ return proxy -#------ domains ----------------- +#------ domains ------------------ from pypy.objspace.constraint import domain all_mms.update(domain.all_mms) W_FiniteDomain = domain.W_FiniteDomain -#-------- computationspace -------------- +#-------- computationspace -------- from pypy.objspace.constraint import computationspace all_mms.update(computationspace.all_mms) W_ComputationSpace = computationspace.W_ComputationSpace -# ---- constraints +# ---- constraints ---------------- from pypy.objspace.constraint import constraint +#----- distributors --------------- +from pypy.objspace.constraint import distributor + + #-- THE SPACE --------------------------------------- #class UnificationError(w_RuntimeError): @@ -836,6 +840,14 @@ space.wrap(constraint.app_make_filter)) space.setitem(space.builtin.w_dict, space.wrap('AllDistinct'), space.wrap(constraint.app_make_alldistinct)) + #-- distributor -- + space.setitem(space.builtin.w_dict, space.wrap('NaiveDistributor'), + space.wrap(distributor.app_make_naive_distributor)) + space.setitem(space.builtin.w_dict, space.wrap('SplitDistributor'), + space.wrap(distributor.app_make_split_distributor)) + space.setitem(space.builtin.w_dict, space.wrap('DichotomyDistributor'), + space.wrap(distributor.app_make_dichotomy_distributor)) + if USE_COROUTINES: import os From arigo at codespeak.net Tue Apr 11 12:39:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 12:39:52 +0200 (CEST) Subject: [pypy-svn] r25695 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060411103952.78A9610186@code0.codespeak.net> Author: arigo Date: Tue Apr 11 12:39:50 2006 New Revision: 25695 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: rctypes "progress": * union for SomeCTypesObject * prebuilt arrays Blocker: the skipped test in test_rfunc.py. The problem is that (c_char*10) is implemented as an array of structs containing each a 'value' field of type char. This makes conversion to a regular array of chars quite difficult. I am not sure that there is a better solution than rewriting rctypes pointers to use addresses systematically. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Tue Apr 11 12:39:50 2006 @@ -724,10 +724,14 @@ return SomeAddress(is_null=False) class __extend__(pairtype(SomeAddress, SomeImpossibleValue)): + # need to override this specifically to hide the 'raise UnionError' + # of pairtype(SomeAddress, SomeObject). def union((s_addr, s_imp)): return s_addr class __extend__(pairtype(SomeImpossibleValue, SomeAddress)): + # need to override this specifically to hide the 'raise UnionError' + # of pairtype(SomeObject, SomeAddress). def union((s_imp, s_addr)): return s_addr @@ -781,3 +785,19 @@ listdef = ListDef(None, SomeCTypesObject(s_cto.knowntype._type_)) return SomeList(listdef) +class __extend__(pairtype(SomeCTypesObject, SomeCTypesObject)): + def union((s_cto1, s_cto2)): + if s_cto1.knowntype == s_cto2.knowntype: + states = {} + for s in [s_cto1, s_cto2]: + if s.memorystate != SomeCTypesObject.NOMEMORY: + states[s.memorystate] = True + if len(states) == 0: + state = SomeCTypesObject.NOMEMORY + elif len(states) == 1: + [state] = states.keys() + else: + state = SomeCTypesObject.MIXEDMEMORYOWNERSHIP + return SomeCTypesObject(s_cto1.knowntype, state) + else: + return SomeObject() Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Tue Apr 11 12:39:50 2006 @@ -434,7 +434,7 @@ MEMORYALIAS = "MEMORYALIAS" MIXEDMEMORYOWNERSHIP = "MIXEDMEMORYOWNERSHIP" - def __init__(self, knowntype, memorystate=None): + def __init__(self, knowntype, memorystate): if memorystate is None: memorystate = knowntype.default_memorystate self.knowntype = knowntype Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Tue Apr 11 12:39:50 2006 @@ -4,7 +4,7 @@ from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype -from pypy.rpython.rctypes.rmodel import CTypesRefRepr, reccopy +from pypy.rpython.rctypes.rmodel import CTypesRefRepr, genreccopy, reccopy ArrayType = type(ARRAY(c_int, 10)) @@ -25,6 +25,11 @@ super(ArrayRepr, self).__init__(rtyper, s_array, c_data_type) + def initialize_const(self, p, value): + for i in range(self.length): + llitem = self.r_item.convert_const(value[i]) + reccopy(llitem.c_data, p.c_data[i]) + def get_c_data_of_item(self, llops, v_array, v_index): v_c_array = self.get_c_data(llops, v_array) return llops.genop('getarraysubstruct', [v_c_array, v_index], @@ -37,7 +42,7 @@ v_item_c_data = r_array.r_item.get_c_data(hop.llops, v_item) v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) # copy the whole structure's content over - reccopy(hop.llops, v_item_c_data, v_c_data) + genreccopy(hop.llops, v_item_c_data, v_c_data) def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) @@ -61,7 +66,12 @@ compute_annotation=arraytype_compute_annotation, specialize_call=arraytype_specialize_call) +def array_instance_compute_annotation(type, instance): + return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) + def arraytype_get_repr(rtyper, s_array): return ArrayRepr(rtyper, s_array) -extregistry.register_metatype(ArrayType, get_repr=arraytype_get_repr) +extregistry.register_metatype(ArrayType, + compute_annotation=array_instance_compute_annotation, + get_repr=arraytype_get_repr) Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Tue Apr 11 12:39:50 2006 @@ -1,11 +1,13 @@ from pypy.rpython import extregistry +from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.rstr import StringRepr, string_repr from pypy.rpython.rctypes.rmodel import CTypesValueRepr +from pypy.rpython.rctypes.rarray import ArrayRepr from pypy.annotation import model as annmodel from pypy.annotation.pairtype import pairtype -from ctypes import c_char_p +from ctypes import c_char, c_char_p class CCharPRepr(CTypesValueRepr): @@ -51,6 +53,25 @@ r_temp.setstring(llops, v_owned_box, v) return llops.convertvar(v_owned_box, r_temp, r_to) +##class __extend__(pairtype(ArrayRepr, CCharPRepr)): +## def convert_from_to((r_from, r_to), v, llops): +## if r_from.r_item.ctype != c_char: +## return NotImplemented +## # warning: no keepalives, only for short-lived conversions like +## # in argument passing +## r_temp = r_to.r_memoryowner +## v_owned_box = r_temp.allocate_instance(llops) +## v_char_array_ptr = r_from.get_c_data(llops, v) +## v_char_array_adr = llops.genop('cast_ptr_to_adr', [v_char_array_ptr], +## resulttype = llmemory.Address) +## cofs = inputconst(lltype.Signed, +## llmemory.itemoffsetof(r_from.ll_type) + +## llmemory.offsetof(r_from.ll_type.OF, 'value')) +## v_first_char_adr = llops.genop('adr_add', [v_char_array_adr, cofs], +## resulttype = llmemory.Address) +## r_temp.setvalue(llops, v_owned_box, v_first_char_adr) +## return llops.convertvar(v_owned_box, r_temp, r_to) + CCHARP = llmemory.Address # char * FIRSTITEMOFS = llmemory.ArrayItemsOffset(string_repr.lowleveltype.TO.chars) Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 11 12:39:50 2006 @@ -74,6 +74,32 @@ "Return extra keepalive fields used for the content of this object." return [] + def convert_const(self, value): + if isinstance(value, self.ctype): + key = "by_id", id(value) + keepalive = value + else: + if self.ownsmemory: + raise TyperError("convert_const(%r) but repr owns memory" % ( + value,)) + key = "by_value", value + keepalive = None + try: + return self.const_cache[key][0] + except KeyError: + p = lltype.malloc(self.r_memoryowner.lowleveltype.TO) + self.initialize_const(p, value) + if self.ownsmemory: + result = p + else: + # we must return a non-memory-owning box that keeps the + # memory-owning box alive + result = lltype.malloc(self.lowleveltype.TO) + result.c_data = p.c_data # initialize c_data pointer + result.c_data_owner_keepalive = p + self.const_cache[key] = result, keepalive + return result + def get_c_data(self, llops, v_box): if self.ownsmemory: inputargs = [v_box, inputconst(lltype.Void, "c_data")] @@ -173,32 +199,6 @@ v_c_data = self.get_c_data(llops, v_box) self.setvalue_inside_c_data(llops, v_c_data, v_value) - def convert_const(self, value): - if isinstance(value, self.ctype): - key = "by_id", id(value) - keepalive = value - else: - if self.ownsmemory: - raise TyperError("convert_const(%r) but repr owns memory" % ( - value,)) - key = "by_value", value - keepalive = None - try: - return self.const_cache[key][0] - except KeyError: - p = lltype.malloc(self.r_memoryowner.lowleveltype.TO) - self.initialize_const(p, value) - if self.ownsmemory: - result = p - else: - # we must return a non-memory-owning box that keeps the - # memory-owning box alive - result = lltype.malloc(self.lowleveltype.TO) - result.c_data = p.c_data # initialize c_data pointer - result.c_data_owner_keepalive = p - self.const_cache[key] = result, keepalive - return result - def initialize_const(self, p, value): if isinstance(value, self.ctype): value = value.value @@ -206,11 +206,35 @@ # ____________________________________________________________ -def reccopy(llops, v_source, v_dest): - # helper to copy recursively a structure or array onto another. +def reccopy(source, dest): + # copy recursively a structure or array onto another. + T = lltype.typeOf(source).TO + assert T == lltype.typeOf(dest).TO + assert isinstance(T, lltype.Struct) # XXX implement arrays + for name in T._names: + FIELDTYPE = getattr(T, name) + if isinstance(FIELDTYPE, lltype.ContainerType): + subsrc = getattr(source, name) + subdst = getattr(dest, name) + reccopy(subsrc, subdst) + else: + llvalue = getattr(source, name) + setattr(dest, name, llvalue) + +def reccopy_arrayitem(source, destarray, destindex): + ITEMTYPE = lltype.typeOf(destarray).TO.OF + if isinstance(ITEMTYPE, lltype.Primitive): + destarray[destindex] = source + else: + reccopy(source, destarray[destindex]) + +def genreccopy(llops, v_source, v_dest): + # helper to generate the llops that copy recursively a structure + # or array onto another. 'v_source' and 'v_dest' can also be pairs + # (v, i) to mean the ith item of the array that v points to. T = v_source.concretetype.TO assert T == v_dest.concretetype.TO - assert isinstance(T, lltype.Struct) + assert isinstance(T, lltype.Struct) # XXX implement arrays for name in T._names: FIELDTYPE = getattr(T, name) cname = inputconst(lltype.Void, name) @@ -218,7 +242,16 @@ RESTYPE = lltype.Ptr(FIELDTYPE) v_subsrc = llops.genop('getsubstruct', [v_source, cname], RESTYPE) v_subdst = llops.genop('getsubstruct', [v_dest, cname], RESTYPE) - reccopy(llops, v_subsrc, v_subdst) + genreccopy(llops, v_subsrc, v_subdst) else: v_value = llops.genop('getfield', [v_source, cname], FIELDTYPE) llops.genop('setfield', [v_dest, cname, v_value]) + +def genreccopy_arrayitem(llops, v_source, v_destarray, v_destindex): + ITEMTYPE = v_destarray.concretetype.TO.OF + if isinstance(ITEMTYPE, lltype.Primitive): + llops.genop('setarrayitem', [v_destarray, v_destindex, v_source]) + else: + v_dest = llops.genop('getarraysubstruct', [v_destarray, v_destindex], + resulttype = lltype.Ptr(ITEMTYPE)) + genreccopy(llops, v_source, v_dest) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 11 12:39:50 2006 @@ -4,7 +4,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation import model as annmodel from pypy.annotation.pairtype import pairtype -from pypy.rpython.rctypes.rmodel import CTypesValueRepr, reccopy +from pypy.rpython.rctypes.rmodel import CTypesValueRepr, genreccopy from ctypes import POINTER, pointer, c_int @@ -86,7 +86,7 @@ # not supported by ctypes either raise TyperError("assignment to pointer[x] with x != 0") # copy the whole structure's content over - reccopy(hop.llops, v_new_c_data, v_target) + genreccopy(hop.llops, v_new_c_data, v_target) # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Tue Apr 11 12:39:50 2006 @@ -15,7 +15,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int, ARRAY, POINTER, pointer +from ctypes import c_int, c_short, ARRAY, POINTER, pointer c_int_10 = ARRAY(c_int,10) @@ -100,6 +100,23 @@ ## py.test.raises(IndexError, "s = a.build_types(access_with_invalid_negative_index,[])") + def test_annotate_prebuilt(self): + my_array_2 = (c_short*10)(0, 1, 4, 9, 16, 25, 36, 49, 64, 81) + my_array_3 = (c_short*10)(0, 1, 8, 27, 64, 125, 216, 343, 512, 729) + def func(i, n): + if i == 2: + array = my_array_2 + else: + array = my_array_3 + return array[n] + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, [int, int]) + if conftest.option.view: + a.translator.view() + assert s.knowntype == int + class Test_specialization: def test_specialize_array(self): def create_array(): @@ -124,6 +141,21 @@ res = interpret(access_array, [44]) assert res == 1 + def test_specialize_prebuilt(self): + my_array_2 = (c_short*10)(0, 1, 4, 9, 16, 25, 36, 49, 64, 81) + my_array_3 = (c_short*10)(0, 1, 8, 27, 64, 125, 216, 343, 512, 729) + def func(i, n): + if i == 2: + array = my_array_2 + else: + array = my_array_3 + return array[n] + + res = interpret(func, [2, 6]) + assert res == 36 + res = interpret(func, [3, 7]) + assert res == 343 + class Test_compilation: def test_compile_array_access(self): def access_array(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Tue Apr 11 12:39:50 2006 @@ -7,7 +7,7 @@ from pypy.rpython.rstr import string_repr from pypy.rpython.lltypesystem import lltype, llmemory -from ctypes import c_int, c_long, c_char_p +from ctypes import c_int, c_long, c_char_p, c_char labs = mylib.labs @@ -36,6 +36,17 @@ assert labs(-42) == 42 return labs(n) +def test_atoi(): + assert atoi("") == 0 + assert atoi("42z7") == 42 + assert atoi("blah") == 0 + assert atoi("18238") == 18238 + A = c_char * 10 + assert atoi(A('\x00')) == 0 + assert atoi(A('4', '2', 'z', '7', '\x00')) == 42 + assert atoi(A('b', 'l', 'a', 'h', '\x00')) == 0 + assert atoi(A('1', '8', '2', '3', '8', '\x00')) == 18238 + def test_ll_atoi(): keepalive = [] def str2addr(string): @@ -78,6 +89,20 @@ res = [interpret(fn, [i]) for i in range(4)] assert res == [0, 42, 0, 18238] + def test_specialize_atoi_char_array(self): + py.test.skip("in-progress") + A = c_char * 10 + choices = [A('\x00'), + A('4', '2', 'z', '7', '\x00'), + A('b', 'l', 'a', 'h', '\x00'), + A('1', '8', '2', '3', '8', '\x00')] + def fn(n): + return atoi(choices[n]) + + assert fn(3) == 18238 + res = [interpret(fn, [i]) for i in range(4)] + assert res == [0, 42, 0, 18238] + class Test_compile: def test_compile_labs(self): fn = compile(test_labs, [int]) From auc at codespeak.net Tue Apr 11 13:03:00 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 11 Apr 2006 13:03:00 +0200 (CEST) Subject: [pypy-svn] r25696 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060411110300.0688F1018D@code0.codespeak.net> Author: auc Date: Tue Apr 11 13:02:58 2006 New Revision: 25696 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/distributor.py pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py pypy/dist/pypy/objspace/constraint/test/test_distributor.py Log: distribute with naive distributor Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Tue Apr 11 13:02:58 2006 @@ -52,9 +52,9 @@ class W_Distributor(Wrappable): - def __init__(self, object_space, fanout): + def __init__(self, object_space, fanout_w): self._space = object_space - self.fanout = fanout + self.fanout = fanout_w W_Distributor.typedef = typedef.TypeDef("W_Distributor") @@ -64,39 +64,45 @@ class W_ComputationSpace(Wrappable): def __init__(self, obj_space): self._space = obj_space - # there, var -> dom - self.var_doms = self._space.newdict({}) + # var -> dom + self.var_dom = self._space.newdict({}) # constraint set self.constraints = self._space.newdict({}) # var -> constraints - self.var_const_map = {} + self.var_const = {} # freshly added constraints (tell -> propagate) self.to_check = {} def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) - if w_name in self.var_doms.content: + if w_name in self.var_dom.content: raise OperationError(self._space.w_RuntimeError, self._space.wrap("Name already used")) var = W_Variable(self._space, w_name) - self.var_doms.content[var] = w_domain + self.var_dom.content[var] = w_domain return var def w_dom(self, w_variable): assert isinstance(w_variable, W_Variable) - return self.var_doms.content[w_variable] + return self.var_dom.content[w_variable] def w_tell(self, w_constraint): assert isinstance(w_constraint, W_Constraint) self.constraints.content[w_constraint] = self._space.w_True for var in w_constraint.affected_variables(): - self.var_const_map.setdefault(var, []) - self.var_const_map[var].append(w_constraint) + self.var_const.setdefault(var, []) + self.var_const[var].append(w_constraint) self.to_check[w_constraint] = True + def w_dependant_constraints(self, w_var): + return self._space.newlist(self.dependant_constraints(w_var)) + def dependant_constraints(self, var): - return self.var_const_map[var] + try: + return self.var_const[var] + except KeyError: + return [] def w_propagate(self): return self.propagate() @@ -140,7 +146,8 @@ var = interp2app(W_ComputationSpace.w_var), dom = interp2app(W_ComputationSpace.w_dom), tell = interp2app(W_ComputationSpace.w_tell), - propagate = interp2app(W_ComputationSpace.w_propagate)) + propagate = interp2app(W_ComputationSpace.w_propagate), + dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints)) def newspace(space): Modified: pypy/dist/pypy/objspace/constraint/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/distributor.py Tue Apr 11 13:02:58 2006 @@ -5,6 +5,8 @@ from pypy.interpreter import baseobjspace, typedef, gateway from pypy.interpreter.gateway import interp2app +from pypy.objspace.std.intobject import W_IntObject + from pypy.objspace.constraint.computationspace import W_Distributor def arrange_domains(cs, variables): @@ -26,32 +28,33 @@ """return number of possible splits""" return self.fanout - def find_smallest_domain(self): + def _find_smallest_domain(self, w_cs): """returns the variable having the smallest domain. (or one of such varibles if there is a tie) """ - vars_ = [var for var in self.cs.get_variables_with_a_domain() - if self.cs.dom(var).size() > 1] + vars_ = [var for var, dom in w_cs.var_dom.content.items() + if dom.size() > 1] best = vars_[0] for var in vars_: - if self.cs.dom(var).size() < self.cs.dom(best).size(): + if w_cs.var_dom.content[var].size() < w_cs.var_dom.content[best].size(): best = var return best def w_distribute(self, w_cs, w_choice): - assert isintance(w_choice, W_IntegerObject) + assert isinstance(w_choice, W_IntObject) self.distribute(w_cs, self._space.int_w(w_choice)) def distribute(self, w_cs, choice_w): - variable = self._find_distribution_variable() - self._do_distribute(self.cs.dom(variable), choice) - for const in self.cs.dependant_constraints(variable): - self.cs.event_set.add(const) + variable = self.find_distribution_variable(w_cs) + self._do_distribute(w_cs.var_dom.content[variable], + choice_w) + for const in w_cs.dependant_constraints(variable): + w_cs.to_check[const] = True - def _find_distribution_variable(self): - return self.find_smallest_domain() + def find_distribution_variable(self, w_cs): + return self._find_smallest_domain(w_cs) def _do_distribute(self, domain, choice): """remove values from domain depending on choice""" @@ -75,9 +78,9 @@ def _do_distribute(self, domain, choice): values = domain.get_values() if choice == 0: - domain.remove_values(values[1:]) + domain.w_remove_values(values[1:]) else: - domain.remove_value(values[0]) + domain.w_remove_value(values[0]) W_NaiveDistributor.typedef = typedef.TypeDef( "W_NaiveDistributor", @@ -133,5 +136,3 @@ return make_split_distributor(object_space, 2) app_make_dichotomy_distributor = interp2app(make_dichotomy_distributor) - -#DefaultDistributor = W_NaiveDistributor Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Tue Apr 11 13:02:58 2006 @@ -91,7 +91,10 @@ def w_get_values(self): """return all the values in the domain in an indexable sequence""" - return self._space.newlist([x for x in self._values]) + return self._space.newlist(self.get_values()) + + def get_values(self): + return [x for x in self._values] def __iter__(self): return iter(self._values) Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Tue Apr 11 13:02:58 2006 @@ -29,6 +29,8 @@ v2 = csp.var('v2', FiniteDomain([1, 2])) cstr = AllDistinct([v1, v2]) csp.tell(cstr) + for v in (v1, v2): + assert cstr in csp.dependant_constraints(v) def test_propagate(self): csp = newspace() Modified: pypy/dist/pypy/objspace/constraint/test/test_distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_distributor.py Tue Apr 11 13:02:58 2006 @@ -10,5 +10,15 @@ d1 = NaiveDistributor() d2 = SplitDistributor(4) d3 = DichotomyDistributor() - - + assert d1.fanout() == 2 + assert d2.fanout() == 4 + assert d3.fanout() == 2 + + def test_naive_distribute(self): + spc = newspace() + x = spc.var('x', FiniteDomain([1])) + y = spc.var('y', FiniteDomain([1, 2])) + z = spc.var('z', FiniteDomain([1, 2, 3])) + d = NaiveDistributor() + d.distribute(spc, 2) + assert spc.dom(y) == FiniteDomain([2]) From antocuni at codespeak.net Tue Apr 11 14:51:01 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 11 Apr 2006 14:51:01 +0200 (CEST) Subject: [pypy-svn] r25697 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060411125101.14B281017B@code0.codespeak.net> Author: antocuni Date: Tue Apr 11 14:50:52 2006 New Revision: 25697 Modified: pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/ilgenerator.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/test/compile.py Log: Corrected some minor bugs that prevented test_oo to pass under Windows. Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Tue Apr 11 14:50:52 2006 @@ -79,9 +79,13 @@ class AbstractConst(object): def make(db, const): if isinstance(const, ootype._view): + static_type = const._TYPE const = const._inst + else: + static_type = None + if isinstance(const, ootype._instance): - return InstanceConst(db, const) + return InstanceConst(db, const, static_type) else: assert False, 'Unknown constant: %s' % const make = staticmethod(make) @@ -96,9 +100,14 @@ pass class InstanceConst(AbstractConst): - def __init__(self, db, obj): + def __init__(self, db, obj, static_type): self.cts = CTS(db) self.obj = obj + if static_type is None: + self.static_type = obj._TYPE + else: + self.static_type = static_type + self.cts.lltype_to_cts(obj._TYPE) # force scheduling of obj's class def __hash__(self): return hash(self.obj) @@ -111,7 +120,7 @@ return '%s_%d' % (name, n) def get_type(self): - return self.cts.lltype_to_cts(self.obj._TYPE) + return self.cts.lltype_to_cts(self.static_type) def init(self, ilasm): classdef = self.obj._TYPE Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Tue Apr 11 14:50:52 2006 @@ -253,6 +253,15 @@ self.db.pending_function(graph) self.ilasm.call(func_name) + def call_signature(self, signature): + self.ilasm.call(signature) + + def cast_to(self, lltype): + cts_type = self.cts.lltype_to_cts(lltype) + if cts_type.startswith('class '): + cts_type = cts_type[len('class '):] + self.ilasm.opcode('castclass', cts_type) + def new(self, obj): self.ilasm.new(self.cts.ctor_name(obj)) Modified: pypy/dist/pypy/translator/cli/ilgenerator.py ============================================================================== --- pypy/dist/pypy/translator/cli/ilgenerator.py (original) +++ pypy/dist/pypy/translator/cli/ilgenerator.py Tue Apr 11 14:50:52 2006 @@ -37,7 +37,7 @@ self.out = outfile self.code = CodeGenerator(self.out) self.code.writeline('.assembly extern mscorlib {}') - self.code.writeline('.assembly extern pypy {}') + self.code.writeline('.assembly extern pypylib {}') self.code.writeline('.assembly %s {}' % name) def close(self): Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Tue Apr 11 14:50:52 2006 @@ -92,6 +92,12 @@ generator.call_method(this.concretetype, method.value) +class _RuntimeNew(MicroInstruction): + def render(self, generator, op): + generator.load(op.args[0]) + generator.call_signature('object [pypylib]pypy.runtime.Utils::RuntimeNew(class [mscorlib]System.Type)') + generator.cast_to(op.result.concretetype) + PushAllArgs = _PushAllArgs() StoreResult = _StoreResult() @@ -100,3 +106,4 @@ SetField = _SetField() GetField = _GetField() CallMethod = _CallMethod() +RuntimeNew = _RuntimeNew() Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Tue Apr 11 14:50:52 2006 @@ -1,5 +1,5 @@ -from pypy.translator.cli.metavm import PushArg, PushAllArgs,\ - StoreResult, Call, InstructionList, New, SetField, GetField, CallMethod +from pypy.translator.cli.metavm import PushArg, PushAllArgs, StoreResult, Call,\ + InstructionList, New, SetField, GetField, CallMethod, RuntimeNew # some useful instruction patterns Not = ['ldc.i4.0', 'ceq'] @@ -11,12 +11,11 @@ def _abs(type_): return [PushAllArgs, 'call %s class [mscorlib]System.Math::Abs(%s)' % (type_, type_)] -_runtimenew = 'object [pypylib]pypy.runtime.Utils::RuntimeNew([mscorlib]System.Type)' opcodes = { # __________ object oriented operations __________ 'new': [New], - 'runtimenew': [PushAllArgs, 'call ' + _runtimenew], + 'runtimenew': [RuntimeNew], 'oosetfield': [SetField], 'oogetfield': [GetField], 'oosend': [CallMethod], Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Tue Apr 11 14:50:52 2006 @@ -23,13 +23,49 @@ else: print 'OK' -def foo(mylist, i): - return mylist[i] + +class MyClass: + INCREMENT = 1 + + def __init__(self, x, y): + self.x = x + self.y = y + + def compute(self): + return self.x + self.y + + def compute_and_multiply(self, factor): + return self.compute() * factor + + def static_meth(x, y): + return x*y + static_meth = staticmethod(static_meth) + +## def class_meth(cls, x, y): +## return x*y + cls.INCREMENT +## class_meth = classmethod(class_meth) + + def class_attribute(self): + return self.x + self.INCREMENT + +class MyDerivedClass(MyClass): + INCREMENT = 2 + + def __init__(self, x, y): + MyClass.__init__(self, x+12, y+34) + + def compute(self): + return self.x - self.y + + + +def init_and_compute(cls, x, y): + return cls(x, y).compute() + def bar(x, y): - mylist = [1,2,3,x,y] - mylist[0] = 432 - return foo(mylist, (x+y)%5) + return init_and_compute(MyClass, x, y) + init_and_compute(MyDerivedClass, x, y) + f = compile_function(bar, [int, int]) From auc at codespeak.net Tue Apr 11 15:00:35 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 11 Apr 2006 15:00:35 +0200 (CEST) Subject: [pypy-svn] r25698 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060411130035.284F41017F@code0.codespeak.net> Author: auc Date: Tue Apr 11 15:00:33 2006 New Revision: 25698 Modified: pypy/dist/pypy/objspace/constraint/distributor.py pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_distributor.py Log: distribute with split distributor Modified: pypy/dist/pypy/objspace/constraint/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/distributor.py Tue Apr 11 15:00:33 2006 @@ -34,22 +34,20 @@ """ vars_ = [var for var, dom in w_cs.var_dom.content.items() if dom.size() > 1] - best = vars_[0] for var in vars_: if w_cs.var_dom.content[var].size() < w_cs.var_dom.content[best].size(): best = var - return best def w_distribute(self, w_cs, w_choice): assert isinstance(w_choice, W_IntObject) - self.distribute(w_cs, self._space.int_w(w_choice)) + self.distribute(w_cs, self._space.int_w(w_choice) -1) def distribute(self, w_cs, choice_w): variable = self.find_distribution_variable(w_cs) - self._do_distribute(w_cs.var_dom.content[variable], - choice_w) + domain = w_cs.w_dom(variable) + self._do_distribute(w_cs, domain, choice_w) for const in w_cs.dependant_constraints(variable): w_cs.to_check[const] = True @@ -75,7 +73,7 @@ # default fanout is 2, see make_naive_distributor W_Distributor.__init__(self, object_space, fanout_w) - def _do_distribute(self, domain, choice): + def _do_distribute(self, w_cs, domain, choice): values = domain.get_values() if choice == 0: domain.w_remove_values(values[1:]) @@ -102,19 +100,19 @@ # default fanout is 3, see make_split_distributor W_Distributor.__init__(self, object_space, fanout_w) - def nb_subdomains(self): - to_split = self.find_smallest_domain() - if self.nb_subspaces: - return min(self.nb_subspaces, - self.cs.dom(to_split).size()) + def _subdomains(self, w_cs): + """returns the min number of partitions + for a domain to be distributed""" + to_split = self._find_smallest_domain(w_cs) + if self.fanout: + return min(self.fanout, + w_cs.w_dom(to_split).size()) else: - return self.cs.dom(to_split).size() + return w_cs.w_dom(to_split).size() - - def _do_distribute(self, domain, choice): - nb_subspaces = self.nb_subdomains() + def _do_distribute(self, w_cs, domain, choice): values = domain.get_values() - nb_elts = max(1, len(values)*1./nb_subspaces) + nb_elts = max(1, len(values)*1./self._subdomains(w_cs)) start, end = (int(math.floor(choice * nb_elts)), int(math.floor((choice + 1) * nb_elts))) domain.remove_values(values[:start]) Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Tue Apr 11 15:00:33 2006 @@ -78,9 +78,11 @@ def w_remove_values(self, w_values): """Remove values of domain and check for consistency""" - if self._space.is_true(self._space.gt(self._space.len(w_values), - self._space.newint(0))) : - for w_val in w_values.wrappeditems : + self.remove_values(w_values.wrappeditems) + + def remove_values(self, values): + if len(values) > 0: + for w_val in values: del self._values[w_val] self._value_removed() Modified: pypy/dist/pypy/objspace/constraint/test/test_distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_distributor.py Tue Apr 11 15:00:33 2006 @@ -22,3 +22,17 @@ d = NaiveDistributor() d.distribute(spc, 2) assert spc.dom(y) == FiniteDomain([2]) + + def test_split_distributor(self): + spc = newspace() + x = spc.var('x', FiniteDomain([1])) + y = spc.var('y', FiniteDomain([1, 2])) + z = spc.var('z', FiniteDomain([1, 2, 3])) + w = spc.var('w', FiniteDomain([1, 2, 3, 4, 5, 6])) + d = SplitDistributor(3) + d.distribute(spc, 2) + assert spc.dom(y).size() == 1 + d.distribute(spc, 1) + assert spc.dom(z).size() == 1 + d.distribute(spc, 3) + assert spc.dom(w).size() == 2 From auc at codespeak.net Tue Apr 11 15:50:02 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 11 Apr 2006 15:50:02 +0200 (CEST) Subject: [pypy-svn] r25699 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060411135002.9DDFB1018E@code0.codespeak.net> Author: auc Date: Tue Apr 11 15:49:59 2006 New Revision: 25699 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/distributor.py pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: cspace cloning Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Tue Apr 11 15:49:59 2006 @@ -65,31 +65,43 @@ def __init__(self, obj_space): self._space = obj_space # var -> dom - self.var_dom = self._space.newdict({}) + self.var_dom = {} # constraint set - self.constraints = self._space.newdict({}) + self.constraints = {} # var -> constraints self.var_const = {} # freshly added constraints (tell -> propagate) self.to_check = {} + def w_clone(self): + new = newspace(self._space) + for var, dom in self.var_dom.items(): + new.var_dom[var] = dom.w_copy() + # !! be sure to not put state in constraints + new.constraints = self.constraints + for const in self.to_check: + new.to_check[const] = True + for var, const in self.var_const.items(): + new.var_const[var] = const + return new + def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) - if w_name in self.var_dom.content: + if w_name in self.var_dom: raise OperationError(self._space.w_RuntimeError, self._space.wrap("Name already used")) var = W_Variable(self._space, w_name) - self.var_dom.content[var] = w_domain + self.var_dom[var] = w_domain return var def w_dom(self, w_variable): assert isinstance(w_variable, W_Variable) - return self.var_dom.content[w_variable] + return self.var_dom[w_variable] def w_tell(self, w_constraint): assert isinstance(w_constraint, W_Constraint) - self.constraints.content[w_constraint] = self._space.w_True + self.constraints[w_constraint] = self._space.w_True for var in w_constraint.affected_variables(): self.var_const.setdefault(var, []) self.var_const[var].append(w_constraint) @@ -104,8 +116,8 @@ except KeyError: return [] - def w_propagate(self): - return self.propagate() + def w_ask(self): + self.propagate() def propagate(self): const_q = [(const.estimate_cost_w(self), const) @@ -138,18 +150,18 @@ # the set of satifiable constraints of the space if const in affected_constraints: affected_constraints.remove(const) - - + W_ComputationSpace.typedef = typedef.TypeDef( "W_ComputationSpace", var = interp2app(W_ComputationSpace.w_var), dom = interp2app(W_ComputationSpace.w_dom), tell = interp2app(W_ComputationSpace.w_tell), - propagate = interp2app(W_ComputationSpace.w_propagate), + ask = interp2app(W_ComputationSpace.w_ask), + clone = interp2app(W_ComputationSpace.w_clone), dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints)) -def newspace(space): - return W_ComputationSpace(space) +def newspace(object_space): + return W_ComputationSpace(object_space) app_newspace = gateway.interp2app(newspace) Modified: pypy/dist/pypy/objspace/constraint/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/distributor.py Tue Apr 11 15:49:59 2006 @@ -32,11 +32,11 @@ """returns the variable having the smallest domain. (or one of such varibles if there is a tie) """ - vars_ = [var for var, dom in w_cs.var_dom.content.items() + vars_ = [var for var, dom in w_cs.var_dom.items() if dom.size() > 1] best = vars_[0] for var in vars_: - if w_cs.var_dom.content[var].size() < w_cs.var_dom.content[best].size(): + if w_cs.var_dom[var].size() < w_cs.var_dom[best].size(): best = var return best Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Tue Apr 11 15:49:59 2006 @@ -143,7 +143,8 @@ W_FiniteDomain, W_FiniteDomain) all_mms['intersection'] = intersection_mm -W_FiniteDomain.typedef = typedef.TypeDef("W_FiniteDomain", +W_FiniteDomain.typedef = typedef.TypeDef( + "W_FiniteDomain", W_AbstractDomain.typedef, remove_value = interp2app(W_FiniteDomain.w_remove_value), remove_values = interp2app(W_FiniteDomain.w_remove_values), Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Tue Apr 11 15:49:59 2006 @@ -32,7 +32,7 @@ for v in (v1, v2): assert cstr in csp.dependant_constraints(v) - def test_propagate(self): + def test_ask(self): csp = newspace() x = csp.var('x', FiniteDomain([1])) y = csp.var('y', FiniteDomain([1, 2])) @@ -40,8 +40,25 @@ csp.tell(make_expression([x, y], 'x Author: stephan Date: Tue Apr 11 16:54:48 2006 New Revision: 25701 Modified: pypy/dist/pypy/objspace/std/frozensettype.py pypy/dist/pypy/objspace/std/setobject.py pypy/dist/pypy/objspace/std/settype.py Log: all tests in lib-python/modified-2.4.1/test/test_set.py are now passing, but the pickling for subtypes of set and frozenset (Actually, when running the commands in CLI, there is not problem). But, translation doesn't work yet. Still investigation (and fighting with the debugging process). As always, pypy.objspace.std.model.WITHSET must be set to True to enable native set/frozenset implementation. Modified: pypy/dist/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/dist/pypy/objspace/std/frozensettype.py (original) +++ pypy/dist/pypy/objspace/std/frozensettype.py Tue Apr 11 16:54:48 2006 @@ -13,6 +13,7 @@ frozenset_issuperset = SMM('issuperset', 2) frozenset_symmetric_difference = SMM('symmetric_difference', 2) frozenset_union = SMM('union', 2) +frozenset_reduce = SMM('__reduce__',1) register_all(vars(), globals()) Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Tue Apr 11 16:54:48 2006 @@ -12,6 +12,7 @@ W_Object.__init__(w_self, space) w_self.setdata = setdata = r_dict(space.eq_w, space.hash_w) _initialize_set(space, w_self, wrappeditems) + w_self.in_init = True def __repr__(w_self): """representation for debugging purposes""" @@ -346,7 +347,7 @@ hash *= (len(w_set.setdata) + 1) for w_item in w_set.setdata.iterkeys(): h = space.int_w(space.hash(w_item)) - hash ^= (h ^ (h << 16) ^ 89869747) * 3644798167 + hash ^= ((h ^ (h << 16) ^ 89869747) * 3644798167) hash = hash * 69069 + 907133923 if hash == -1: hash = 590923713 @@ -449,8 +450,11 @@ def init__Set(space, w_set, __args__): w_iterable, = __args__.parse('set', (['some_iterable'], None, None), - [W_SetObject(space,None)]) - _initialize_set(space, w_set, w_iterable) + #[W_SetObject(space,None)]) + [space.newtuple([])]) + if not w_set.in_init: + _initialize_set(space, w_set, w_iterable) + w_set.in_init = False app = gateway.applevel(""" def ne__Set_ANY(s, o): @@ -465,6 +469,9 @@ def repr__Set(s): return '%s(%s)' % (s.__class__.__name__, [x for x in s]) + def reduce__Set(s): + return (s.__class__,(tuple(s),),None) + """, filename=__file__) ne__Set_ANY = app.interphook("ne__Set_ANY") @@ -483,6 +490,9 @@ repr__Set = app.interphook('repr__Set') repr__Frozenset = app.interphook('repr__Set') +set_reduce__Set = app.interphook('reduce__Set') +frozenset_reduce__Frozenset = app.interphook('reduce__Set') + from pypy.objspace.std import frozensettype from pypy.objspace.std import settype Modified: pypy/dist/pypy/objspace/std/settype.py ============================================================================== --- pypy/dist/pypy/objspace/std/settype.py (original) +++ pypy/dist/pypy/objspace/std/settype.py Tue Apr 11 16:54:48 2006 @@ -22,6 +22,7 @@ set_symmetric_difference_update = SMM('symmetric_difference_update', 2) set_union = SMM('union', 2) set_update = SMM('update', 2) +set_reduce = SMM('__reduce__',1) register_all(vars(), globals()) From auc at codespeak.net Tue Apr 11 17:09:43 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 11 Apr 2006 17:09:43 +0200 (CEST) Subject: [pypy-svn] r25702 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060411150943.3415110198@code0.codespeak.net> Author: auc Date: Tue Apr 11 17:09:41 2006 New Revision: 25702 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: commit on cspace Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Tue Apr 11 17:09:41 2006 @@ -60,10 +60,12 @@ #-- Computation space ------------------- +from pypy.objspace.constraint.distributor import make_dichotomy_distributor class W_ComputationSpace(Wrappable): def __init__(self, obj_space): self._space = obj_space + self.distributor = make_dichotomy_distributor(self._space) # var -> dom self.var_dom = {} # constraint set @@ -73,8 +75,14 @@ # freshly added constraints (tell -> propagate) self.to_check = {} + #-- public interface --------------- + + def w_ask(self): + self.propagate() + def w_clone(self): new = newspace(self._space) + new.distributor = self.distributor for var, dom in self.var_dom.items(): new.var_dom[var] = dom.w_copy() # !! be sure to not put state in constraints @@ -85,6 +93,12 @@ new.var_const[var] = const return new + def w_commit(self, w_choice): + self.distributor.w_distribute(self, w_choice) + + def w_set_distributor(self, w_distributor): + self.distributor = w_distributor + def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) @@ -110,15 +124,15 @@ def w_dependant_constraints(self, w_var): return self._space.newlist(self.dependant_constraints(w_var)) + + #-- everything else --------------- + def dependant_constraints(self, var): try: return self.var_const[var] except KeyError: return [] - def w_ask(self): - self.propagate() - def propagate(self): const_q = [(const.estimate_cost_w(self), const) for const in self.to_check] @@ -159,6 +173,7 @@ tell = interp2app(W_ComputationSpace.w_tell), ask = interp2app(W_ComputationSpace.w_ask), clone = interp2app(W_ComputationSpace.w_clone), + commit = interp2app(W_ComputationSpace.w_commit), dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints)) Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Tue Apr 11 17:09:41 2006 @@ -32,7 +32,7 @@ for v in (v1, v2): assert cstr in csp.dependant_constraints(v) - def test_ask(self): + def test_propagate(self): csp = newspace() x = csp.var('x', FiniteDomain([1])) y = csp.var('y', FiniteDomain([1, 2])) @@ -40,7 +40,7 @@ csp.tell(make_expression([x, y], 'x Author: auc Date: Tue Apr 11 18:35:15 2006 New Revision: 25704 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: ask Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Tue Apr 11 18:35:15 2006 @@ -74,11 +74,26 @@ self.var_const = {} # freshly added constraints (tell -> propagate) self.to_check = {} + self.status = None #-- public interface --------------- def w_ask(self): - self.propagate() + if self.status is not None: return self.status + try: + if len(self.to_check) > 0: + self.propagate() + except: # FIXME: ConcistencyFailure + self.status = self._space.newint(0) + return self.status + try: + self.distributor.find_distribution_variable(self) + except: # FIXME: indexError ? + self.status = self._space.newint(1) + return self.status + self.status = self._space.newint(self.distributor.fanout) + print "duh ?" + return self.status def w_clone(self): new = newspace(self._space) Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Tue Apr 11 18:35:15 2006 @@ -229,7 +229,6 @@ try: for varname, keep in result_cache.content.items(): - print keep domain = w_cs.w_dom(self._names_to_vars[self._space.str_w(varname)]) domain.w_remove_values(self._space.newlist([val for val in domain._values Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Tue Apr 11 18:35:15 2006 @@ -76,4 +76,32 @@ csp.commit(2) assert csp.dom(z).size() == 2 - + def test_ask_success(self): + csp = newspace() + x = csp.var('x', FiniteDomain([1])) + y = csp.var('y', FiniteDomain([1, 2])) + z = csp.var('z', FiniteDomain([1, 2, 3])) + csp.tell(make_expression([x, y], 'x 1 From arigo at codespeak.net Tue Apr 11 18:52:11 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 11 Apr 2006 18:52:11 +0200 (CEST) Subject: [pypy-svn] r25705 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060411165211.75BC51018F@code0.codespeak.net> Author: arigo Date: Tue Apr 11 18:52:10 2006 New Revision: 25705 Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Log: Change the c_data layout for primitives: instead of a struct with a single field called 'value', it is now a FixedSizeArray of a single item. The goal is then to consider pointers to a FixedSizeArray of a single primitive item as pretty much the same as the (non-lltype) notion of pointer to primitive. Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Tue Apr 11 18:52:10 2006 @@ -92,7 +92,7 @@ return llmemory.cast_ptr_to_adr(s.chars) + FIRSTITEMOFS def ll_getstring(box): - p = box.c_data.value + p = box.c_data[0] if p: if box.keepalive_str and ll_str2charp(box.keepalive_str) == p: maxlen = len(box.keepalive_str.chars) @@ -111,9 +111,9 @@ def ll_setstring(box, string): if string: - box.c_data.value = ll_str2charp(string) + box.c_data[0] = ll_str2charp(string) else: - box.c_data.value = llmemory.NULL + box.c_data[0] = llmemory.NULL box.keepalive_str = string Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 11 18:52:10 2006 @@ -18,7 +18,7 @@ # * 'c_data_type' is a low-level container type that also represents # the raw C data; the difference is that we can take # an lltype pointer to it. For primitives or pointers - # this is a Struct with a single field 'value' of + # this is a FixedSizeArray with a single item of # type 'll_type'. Otherwise, c_data_type == ll_type. # # * 'lowleveltype' is the Repr's choosen low-level type for the RPython @@ -177,17 +177,14 @@ semantics, like primitives and pointers.""" def get_c_data_type(self, ll_type): - return lltype.Struct('C_Data_%s' % (self.ctype.__name__,), - ('value', ll_type) ) + return lltype.FixedSizeArray(ll_type, 1) def getvalue_from_c_data(self, llops, v_c_data): - cname = inputconst(lltype.Void, 'value') - return llops.genop('getfield', [v_c_data, cname], + return llops.genop('getarrayitem', [v_c_data, C_ZERO], resulttype=self.ll_type) def setvalue_inside_c_data(self, llops, v_c_data, v_value): - cname = inputconst(lltype.Void, 'value') - llops.genop('setfield', [v_c_data, cname, v_value]) + llops.genop('setarrayitem', [v_c_data, C_ZERO, v_value]) def getvalue(self, llops, v_box): """Reads from the 'value' field of the raw data.""" @@ -202,24 +199,39 @@ def initialize_const(self, p, value): if isinstance(value, self.ctype): value = value.value - p.c_data.value = value + p.c_data[0] = value # ____________________________________________________________ +C_ZERO = inputconst(lltype.Signed, 0) + def reccopy(source, dest): # copy recursively a structure or array onto another. T = lltype.typeOf(source).TO assert T == lltype.typeOf(dest).TO - assert isinstance(T, lltype.Struct) # XXX implement arrays - for name in T._names: - FIELDTYPE = getattr(T, name) - if isinstance(FIELDTYPE, lltype.ContainerType): - subsrc = getattr(source, name) - subdst = getattr(dest, name) - reccopy(subsrc, subdst) - else: - llvalue = getattr(source, name) - setattr(dest, name, llvalue) + if isinstance(T, (lltype.Array, lltype.FixedSizeArray)): + assert len(source) == len(dest) + ITEMTYPE = T.OF + for i in range(len(source)): + if isinstance(ITEMTYPE, lltype.ContainerType): + subsrc = source[i] + subdst = dest[i] + reccopy(subsrc, subdst) + else: + llvalue = source[i] + dest[i] = llvalue + elif isinstance(T, lltype.Struct): + for name in T._names: + FIELDTYPE = getattr(T, name) + if isinstance(FIELDTYPE, lltype.ContainerType): + subsrc = getattr(source, name) + subdst = getattr(dest, name) + reccopy(subsrc, subdst) + else: + llvalue = getattr(source, name) + setattr(dest, name, llvalue) + else: + raise TypeError(T) def reccopy_arrayitem(source, destarray, destindex): ITEMTYPE = lltype.typeOf(destarray).TO.OF @@ -234,18 +246,45 @@ # (v, i) to mean the ith item of the array that v points to. T = v_source.concretetype.TO assert T == v_dest.concretetype.TO - assert isinstance(T, lltype.Struct) # XXX implement arrays - for name in T._names: - FIELDTYPE = getattr(T, name) - cname = inputconst(lltype.Void, name) - if isinstance(FIELDTYPE, lltype.ContainerType): - RESTYPE = lltype.Ptr(FIELDTYPE) - v_subsrc = llops.genop('getsubstruct', [v_source, cname], RESTYPE) - v_subdst = llops.genop('getsubstruct', [v_dest, cname], RESTYPE) - genreccopy(llops, v_subsrc, v_subdst) - else: - v_value = llops.genop('getfield', [v_source, cname], FIELDTYPE) - llops.genop('setfield', [v_dest, cname, v_value]) + + if isinstance(T, lltype.FixedSizeArray): + # XXX don't do that if the length is large + ITEMTYPE = T.OF + for i in range(T.length): + c_i = inputconst(lltype.Signed, i) + if isinstance(ITEMTYPE, lltype.ContainerType): + RESTYPE = lltype.Ptr(ITEMTYPE) + v_subsrc = llops.genop('getarraysubstruct', [v_source, c_i], + resulttype = RESTYPE) + v_subdst = llops.genop('getarraysubstruct', [v_dest, c_i], + resulttype = RESTYPE) + genreccopy(llops, v_subsrc, v_subdst) + else: + v_value = llops.genop('getarrayitem', [v_source, c_i], + resulttype = ITEMTYPE) + llops.genop('setarrayitem', [v_dest, c_i, v_value]) + + elif isinstance(T, lltype.Array): + raise NotImplementedError("XXX genreccopy() for arrays") + + elif isinstance(T, lltype.Struct): + for name in T._names: + FIELDTYPE = getattr(T, name) + cname = inputconst(lltype.Void, name) + if isinstance(FIELDTYPE, lltype.ContainerType): + RESTYPE = lltype.Ptr(FIELDTYPE) + v_subsrc = llops.genop('getsubstruct', [v_source, cname], + resulttype = RESTYPE) + v_subdst = llops.genop('getsubstruct', [v_dest, cname], + resulttype = RESTYPE) + genreccopy(llops, v_subsrc, v_subdst) + else: + v_value = llops.genop('getfield', [v_source, cname], + resulttype = FIELDTYPE) + llops.genop('setfield', [v_dest, cname, v_value]) + + else: + raise TypeError(T) def genreccopy_arrayitem(llops, v_source, v_destarray, v_destindex): ITEMTYPE = v_destarray.concretetype.TO.OF Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 11 18:52:10 2006 @@ -33,7 +33,7 @@ def initialize_const(self, p, ptr): llcontents = self.r_contents.convert_const(ptr.contents) - p.c_data.value = llcontents.c_data + p.c_data[0] = llcontents.c_data # the following line is probably pointless, as 'llcontents' will be # an immortal global constant just like 'p', but better safe than sorry p.keepalive_contents = llcontents.c_data_owner_keepalive Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Tue Apr 11 18:52:10 2006 @@ -124,8 +124,8 @@ res = interpret(create_array, []) c_data = res.c_data - assert c_data[0].value == 0 - assert c_data[9].value == 0 + assert c_data[0][0] == 0 + assert c_data[9][0] == 0 py.test.raises(IndexError, "c_data[10]") assert len(c_data) == 10 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Tue Apr 11 18:52:10 2006 @@ -124,14 +124,14 @@ return c_int(42) res = interpret(create_c_int, []) c_data = res.c_data - assert c_data.value == 42 + assert c_data[0] == 42 def test_specialize_c_int_default_value(self): def create_c_int(): return c_int() res = interpret(create_c_int, []) c_data = res.c_data - assert c_data.value == 0 + assert c_data[0] == 0 def test_specialize_c_int_access_value(self): def create_c_int(): @@ -161,14 +161,14 @@ return c_float(4.2) res = interpret(create_c_float, []) c_data = res.c_data - assert c_data.value == 4.2 + assert c_data[0] == 4.2 def test_specialize_c_float_default_value(self): def create_c_float(): return c_float() res = interpret(create_c_float, []) c_data = res.c_data - assert c_data.value == 0.0 + assert c_data[0] == 0.0 def test_specialize_c_float_access_value(self): def create_c_float(): From tismer at codespeak.net Wed Apr 12 08:01:51 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 12 Apr 2006 08:01:51 +0200 (CEST) Subject: [pypy-svn] r25710 - in pypy/dist/pypy: objspace/flow objspace/flow/test translator Message-ID: <20060412060151.46D4110155@code0.codespeak.net> Author: tismer Date: Wed Apr 12 08:01:45 2006 New Revision: 25710 Modified: pypy/dist/pypy/objspace/flow/flowcontext.py pypy/dist/pypy/objspace/flow/objspace.py pypy/dist/pypy/objspace/flow/specialcase.py pypy/dist/pypy/objspace/flow/test/test_objspace.py pypy/dist/pypy/translator/simplify.py Log: removing the concept of delayed constants. :-X Modified: pypy/dist/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/dist/pypy/objspace/flow/flowcontext.py (original) +++ pypy/dist/pypy/objspace/flow/flowcontext.py Wed Apr 12 08:01:45 2006 @@ -211,7 +211,8 @@ initialblock = SpamBlock(FrameState(frame).copy()) self.pendingblocks = [initialblock] self.graph = FunctionGraph(name or code.co_name, initialblock) - self.make_link = Link # overridable for transition tracking + + make_link = Link # overridable for transition tracking def create_frame(self): # create an empty frame suitable for the code object @@ -226,13 +227,6 @@ def guessbool(self, w_condition, **kwds): return self.recorder.guessbool(self, w_condition, **kwds) - def start_monitoring(self, monitorfunc): - def make_link(*args): - link = Link(*args); - monitorfunc(link) - return link - self.make_link = make_link - def guessexception(self, *classes): def replace_exc_values(case): if case is not Exception: Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Wed Apr 12 08:01:45 2006 @@ -645,95 +645,27 @@ def getattr(self, w_obj, w_name): # handling special things like sys - # (maybe this will vanish with a unique import logic) + # unfortunately this will never vanish with a unique import logic :-( if w_obj in self.not_really_const: const_w = self.not_really_const[w_obj] if w_name not in const_w: return self.do_operation_with_implicit_exceptions('getattr', w_obj, w_name) - w_res = self.regular_getattr(w_obj, w_name) - # tracking variables which might be(come) constants - if self.const_tracker: - self.track_possible_constant(w_res, _getattr, w_obj, w_name) - return w_res + return self.regular_getattr(w_obj, w_name) FlowObjSpace.regular_getattr = FlowObjSpace.getattr FlowObjSpace.getattr = getattr - # protect us from globals access but support constant import into globals + # protect us from globals write access def setitem(self, w_obj, w_key, w_val): ec = self.getexecutioncontext() if not (ec and w_obj is ec.w_globals): return self.regular_setitem(w_obj, w_key, w_val) - globals = self.unwrap(w_obj) - try: - key = self.unwrap_for_computation(self.resolve_constant(w_key)) - val = self.unwrap_for_computation(self.resolve_constant(w_val)) - if key not in globals or val == globals[key]: - globals[key] = val - return self.w_None - except UnwrapException: - pass raise SyntaxError, "attempt to modify global attribute %r in %r" % (w_key, ec.graph.func) FlowObjSpace.regular_setitem = FlowObjSpace.setitem FlowObjSpace.setitem = setitem - def track_possible_constant(self, w_ret, func, *args_w): - if not self.const_tracker: - self.const_tracker = ConstTracker(self) - tracker = self.const_tracker - tracker.track_call(w_ret, func, *args_w) - self.getexecutioncontext().start_monitoring(tracker.monitor_transition) - - FlowObjSpace.track_possible_constant = track_possible_constant - - def resolve_constant(self, w_obj): - if self.const_tracker: - w_obj = self.const_tracker.resolve_const(w_obj) - return w_obj - - FlowObjSpace.resolve_constant = resolve_constant - override() - -class ConstTracker(object): - def __init__(self, space): - assert isinstance(space, FlowObjSpace) - self.space = space - self.known_consts = {} - self.tracked_vars = {} - self.mapping = {} - - def track_call(self, w_res, callable, *args_w): - """ defer evaluation of this expression until a const is needed - """ - self.mapping[w_res] = w_res - self.tracked_vars[w_res] = callable, args_w - - def monitor_transition(self, link): - for vin, vout in zip(link.args, link.target.inputargs): - # we record all true transitions, but no cycles. - if vin in self.mapping and vout not in self.mapping: - # the mapping leads directly to the origin. - self.mapping[vout] = self.mapping[vin] - - def resolve_const(self, w_obj): - """ compute a latent constant expression """ - if isinstance(w_obj, Constant): - return w_obj - w = self.mapping.get(w_obj, w_obj) - if w in self.known_consts: - return self.known_consts[w] - if w not in self.tracked_vars: - raise SyntaxError, 'RPython: cannot compute a constant for %s in %s' % ( - w_obj, self.space.getexecutioncontext().graph.func) - callable, args_w = self.tracked_vars.pop(w) - args_w = [self.resolve_const(w_x) for w_x in args_w] - args = [self.space.unwrap_for_computation(w_x) for w_x in args_w] - w_ret = self.space.wrap(callable(*args)) - self.known_consts[w] = w_ret - return w_ret - # ______________________________________________________________________ # End of objspace.py Modified: pypy/dist/pypy/objspace/flow/specialcase.py ============================================================================== --- pypy/dist/pypy/objspace/flow/specialcase.py (original) +++ pypy/dist/pypy/objspace/flow/specialcase.py Wed Apr 12 08:01:45 2006 @@ -16,11 +16,8 @@ return space.wrap(__import__(name, glob, loc, frm)) # redirect it, but avoid exposing the globals w_glob = Constant({}) - w_ret = space.do_operation('simple_call', Constant(__import__), + return space.do_operation('simple_call', Constant(__import__), w_name, w_glob, w_loc, w_frm) - # let the space decide later if this should be a constant import - space.track_possible_constant(w_ret, __import__, w_name, w_glob, w_loc, w_frm) - return w_ret def sc_operator(space, fn, args): args_w, kwds_w = args.unpack() Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_objspace.py Wed Apr 12 08:01:45 2006 @@ -657,29 +657,6 @@ from some.unknown.module import stuff g = self.codetest(f) - def test_lazy_constant_simple(self): - def f(): - global xxx_global - from pypy.objspace.flow.test.test_objspace import \ - user_defined_function as xxx_global, DATA - return xxx_global - g = self.codetest(f) - consts = [self.space.unwrap(c) for c in self.space.const_tracker.known_consts.values()] - assert user_defined_function in consts - assert DATA not in consts - global xxx_global; xxx_global = 42; del xxx_global - - def not_yet_test_lazy_constant_harder(self): - def f(): - global xxx_global - import pypy as xxx_global - ret = xxx_global.objspace.flow.test.test_objspace.user_defined_function - return ret - g = self.codetest(f) - consts = [self.space.unwrap(c) for c in self.space.const_tracker.known_consts.values()] - assert user_defined_function in consts - assert DATA not in consts - global xxx_global; xxx_global = 42; del xxx_global DATA = {'x': 5, 'y': 6} Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Wed Apr 12 08:01:45 2006 @@ -449,7 +449,6 @@ CanRemoveBuiltins = { isinstance: True, hasattr: True, - __import__: True, # imports for side-effects must store the result } def transform_dead_op_vars_in_blocks(blocks, translator=None): From arigo at codespeak.net Wed Apr 12 10:28:09 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 10:28:09 +0200 (CEST) Subject: [pypy-svn] r25712 - pypy/dist/pypy/objspace/flow Message-ID: <20060412082809.4948D100D4@code0.codespeak.net> Author: arigo Date: Wed Apr 12 10:28:07 2006 New Revision: 25712 Modified: pypy/dist/pypy/objspace/flow/objspace.py Log: Leftovers. Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Wed Apr 12 10:28:07 2006 @@ -63,8 +63,6 @@ #self.make_sys() # objects which should keep their SomeObjectness self.not_really_const = NOT_REALLY_CONST - # tracking variables which might in turn turn into constants. - self.const_tracker = None def enter_cache_building_mode(self): # when populating the caches, the flow space switches to @@ -193,8 +191,6 @@ self.executioncontext = ec from pypy.objspace.flow import specialcase specialcase.setup(self) - self.const_tracker = None - # maybe it would be cleaner to have the tracker in the EC itself? def exception_match(self, w_exc_type, w_check_class): self.executioncontext.recorder.crnt_block.exc_handler = True From hpk at codespeak.net Wed Apr 12 14:50:59 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 12 Apr 2006 14:50:59 +0200 (CEST) Subject: [pypy-svn] r25722 - pypy/dist/pypy/translator/microbench Message-ID: <20060412125059.8893210146@code0.codespeak.net> Author: hpk Date: Wed Apr 12 14:50:59 2006 New Revision: 25722 Added: pypy/dist/pypy/translator/microbench/test_dict.py Log: add a few dictionary (r_dict) related performance tests comparing a default pypy-c and cpython-2.4.1 gives on my machine: 3.30x slower on test_dict.test_dict_setitem() 3.50x slower on test_dict.test_dict_raw_range() 4.43x slower on test_dict.test_dict_class_dict_getmethod() 5.18x slower on test_dict.test_dict_getitem() 7.71x slower on test_dict.test_dict_instance_setnewattr_instance_dict() 8.00x slower on test_dict.test_dict_instance_getattr_instance_dict() 8.14x slower on test_dict.test_dict_instance_setattr_instance_dict() Added: pypy/dist/pypy/translator/microbench/test_dict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/microbench/test_dict.py Wed Apr 12 14:50:59 2006 @@ -0,0 +1,66 @@ + +iterations = 100000 +def test_dict_setitem(): + d = {} + for x in range(iterations): + d[x] = None + d[x] = None + d[x] = None + d[x] = None + +d = {} +for x in range(iterations): + d[x] = x + +def test_dict_getitem(): + for x in range(iterations): + y = d[x] + y = d[x] + y = d[x] + y = d[x] + +def test_dict_raw_range(): + for x in range(iterations): + pass + +class A: + def __init__(self): + self.a = 3 + self.b = 4 + def f(self): + pass + def g(self): + pass + + +def test_dict_class_dict_getmethod(): + a = A() + for x in range(iterations): + a.f + a.f + a.f + a.f + +def test_dict_instance_getattr_instance_dict(): + a = A() + for x in range(iterations): + a.a + a.b + a.a + a.b + +def test_dict_instance_setattr_instance_dict(): + a = A() + for x in range(iterations): + a.a = 3 + a.b = 4 + a.a = 3 + a.b = 4 + +def test_dict_instance_setnewattr_instance_dict(): + a = A() + for x in range(iterations): + a.c = 3 + a.d = 4 + a.e = 5 + a.f = 6 From auc at codespeak.net Wed Apr 12 16:11:13 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 12 Apr 2006 16:11:13 +0200 (CEST) Subject: [pypy-svn] r25723 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060412141113.54E971014A@code0.codespeak.net> Author: auc Date: Wed Apr 12 16:11:11 2006 New Revision: 25723 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: fixed varname uniqueness Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Wed Apr 12 16:11:11 2006 @@ -62,12 +62,14 @@ #-- Computation space ------------------- from pypy.objspace.constraint.distributor import make_dichotomy_distributor + class W_ComputationSpace(Wrappable): def __init__(self, obj_space): self._space = obj_space self.distributor = make_dichotomy_distributor(self._space) - # var -> dom + # var -> dom, name->var self.var_dom = {} + self.name_var = {} # constraint set self.constraints = {} # var -> constraints @@ -75,6 +77,7 @@ # freshly added constraints (tell -> propagate) self.to_check = {} self.status = None + self.sol_set = obj_space.newlist([]) #-- public interface --------------- @@ -117,11 +120,12 @@ def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) - if w_name in self.var_dom: + if w_name in self.name_var: raise OperationError(self._space.w_RuntimeError, self._space.wrap("Name already used")) var = W_Variable(self._space, w_name) self.var_dom[var] = w_domain + self.name_var[w_name] = var return var def w_dom(self, w_variable): @@ -139,6 +143,9 @@ def w_dependant_constraints(self, w_var): return self._space.newlist(self.dependant_constraints(w_var)) + def w_define_problem(self, w_problem): + print "there" + self.sol_set = self._space.call(w_problem, self) #-- everything else --------------- @@ -189,7 +196,8 @@ ask = interp2app(W_ComputationSpace.w_ask), clone = interp2app(W_ComputationSpace.w_clone), commit = interp2app(W_ComputationSpace.w_commit), - dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints)) + dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints), + define_problem = interp2app(W_ComputationSpace.w_define_problem)) def newspace(object_space): Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Wed Apr 12 16:11:11 2006 @@ -14,8 +14,8 @@ cspace = newspace() v = cspace.var("foo", FiniteDomain([1,2,3])) assert str(v).startswith(' Author: auc Date: Wed Apr 12 16:39:36 2006 New Revision: 25724 Added: pypy/dist/pypy/objspace/constraint/applevel/ pypy/dist/pypy/objspace/constraint/applevel/problems.py pypy/dist/pypy/objspace/constraint/applevel/solver.py pypy/dist/pypy/objspace/constraint/test/test_solver.py Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: solver instantiation Added: pypy/dist/pypy/objspace/constraint/applevel/problems.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/applevel/problems.py Wed Apr 12 16:39:36 2006 @@ -0,0 +1,108 @@ + +def dummy_problem(computation_space): + ret = computation_space.var('__dummy__') + computation_space.set_dom(ret, c.FiniteDomain([])) + return (ret,) + +def send_more_money(computation_space): + cs = computation_space + + variables = (s, e, n, d, m, o, r, y) = cs.make_vars('s', 'e', 'n', 'd', 'm', 'o', 'r', 'y') + + digits = range(10) + for var in variables: + cs.set_dom(var, c.FiniteDomain(digits)) + + # use fd.AllDistinct + for v1 in variables: + for v2 in variables: + if v1 != v2: + cs.add_constraint([v1, v2], + '%s != %s' % (v1.name, v2.name)) + + # use fd.NotEquals + cs.add_constraint([s], 's != 0') + cs.add_constraint([m], 'm != 0') + cs.add_constraint([s, e, n, d, m, o, r, y], + '1000*s+100*e+10*n+d+1000*m+100*o+10*r+e == 10000*m+1000*o+100*n+10*e+y') + cs.set_distributor(di.DichotomyDistributor(cs)) + print cs.constraints + return (s, e, n, d, m, o, r, y) + +def conference_scheduling(computation_space): + cs = computation_space + + dom_values = [(room,slot) + for room in ('room A','room B','room C') + for slot in ('day 1 AM','day 1 PM','day 2 AM', + 'day 2 PM')] + + variables = [cs.var(v, FiniteDomain(dom_values)) + for v in ('c01','c02','c03','c04','c05', + 'c06','c07','c08','c09','c10')] + + for conf in ('c03','c04','c05','c06'): + v = cs.find_var(conf) + cs.tell(make_expression([v], "%s[0] == 'room C'" % conf)) + + for conf in ('c01','c05','c10'): + v = cs.find_var(conf) + cs.tell(make_expression([v], "%s[1].startswith('day 1')" % conf)) + + for conf in ('c02','c03','c04','c09'): + v = cs.find_var(conf) + cs.tell(make_expression([v], "%s[1].startswith('day 2')" % conf)) + + groups = (('c01','c02','c03','c10'), + ('c02','c06','c08','c09'), + ('c03','c05','c06','c07'), + ('c01','c03','c07','c08')) + + for group in groups: + cs.tell(AllDistinct([cs.find_var(v) for v in group])) +## cs.add_expression(AllDistinct(cs, tuple([cs.find_var(v) +## for v in group]))) + +## for g in groups: +## for conf1 in g: +## for conf2 in g: +## v1, v2 = cs.find_vars(conf1, conf2) +## if conf2 > conf1: +## cs.add_constraint([v1,v2], '%s[1] != %s[1]'% (v1.name,v2.name)) + + for conf1 in variables: + for conf2 in variables: + if conf2 > conf1: + cs.tell(make_expression([conf1,conf2], + '%s != %s'%(conf1.name(),conf2.name()))) + return variables + +def sudoku(computation_space): + cs = computation_space + import constraint as c + + variables = [cs.var('v%i%i'%(x,y)) for x in range(1,10) for y in range(1,10)] + + # Make the variables + for v in variables: + cs.set_dom(v, c.FiniteDomain(range(1,10))) + # Add constraints for rows (sum should be 45) + for i in range(1,10): + row = [ v for v in variables if v.name[1] == str(i)] + cs.add_constraint(row, 'sum([%s]) == 45' % ', '.join([v.name for v in row])) + # Add constraints for columns (sum should be 45) + for i in range(1,10): + row = [ v for v in variables if v.name[2] == str(i)] + cs.add_constraint(row, 'sum([%s]) == 45' % ', '.join([v.name for v in row])) + # Add constraints for subsquares (sum should be 45) + offsets = [(r,c) for r in [-1,0,1] for c in [-1,0,1]] + subsquares = [(r,c) for r in [2,5,8] for c in [2,5,8]] + for rc in subsquares: + sub = [cs.find_var('v%d%d'% (rc[0] + off[0],rc[1] + off[1])) for off in offsets] + cs.add_constraint(sub, 'sum([%s]) == 45' % ', '.join([v.name for v in sub])) + for v in sub: + for m in sub[sub.index(v)+1:]: + cs.add_constraint([v,m], '%s != %s' % (v.name, m.name)) + #print cs.constraints + return tuple(variables) + Added: pypy/dist/pypy/objspace/constraint/applevel/solver.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/applevel/solver.py Wed Apr 12 16:39:36 2006 @@ -0,0 +1,41 @@ + +class StrategyDistributionMismatch(Exception): + pass + + +from collections import deque + +class Depth: pass +class Breadth: pass + +#-- pythonic lazy solve_all (takes space) + +def lazily_iter_solve_all(space, direction=Depth): + + sp_stack = deque([]) + sp_stack.append(space) + + if direction == Depth: + def collect(space): + sp_stack.append(space) + else: + def collect(space): + sp_stack.appendleft(space) + + while len(sp_stack): + space = sp_stack.pop() + print ' '*len(sp_stack), "ask ..." + status = space.ask() + if status == 1: + print ' '*len(sp_stack), "solution !" + yield space.merge() + elif status > 1: + print ' '*len(sp_stack), "branches ..." + sp1 = space.clone() + sp1.commit(1) + collect(sp1) + sp2 = space.clone() + sp2.commit(2) + collect(sp2) + +solve = lazily_iter_solve_all Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Wed Apr 12 16:39:36 2006 @@ -19,8 +19,13 @@ def name_w(self): return self._space.str_w(self.w_name) + + def w_name(self): + return self.w_name -W_Variable.typedef = typedef.TypeDef("W_Variable") +W_Variable.typedef = typedef.TypeDef( + "W_Variable", + name = interp2app(W_Variable.w_name)) #-- Constraints ------------------------- @@ -120,14 +125,18 @@ def w_var(self, w_name, w_domain): assert isinstance(w_name, W_StringObject) assert isinstance(w_domain, W_AbstractDomain) - if w_name in self.name_var: + name = self._space.str_w(w_name) + if name in self.name_var: raise OperationError(self._space.w_RuntimeError, self._space.wrap("Name already used")) var = W_Variable(self._space, w_name) self.var_dom[var] = w_domain - self.name_var[w_name] = var + self.name_var[name] = var return var + def w_find_var(self, w_name): + return self.name_var[self._space.str_w(w_name)] + def w_dom(self, w_variable): assert isinstance(w_variable, W_Variable) return self.var_dom[w_variable] @@ -147,6 +156,9 @@ print "there" self.sol_set = self._space.call(w_problem, self) + def w_set_root(self, w_solution_list): + self.sol_set = w_solution_list + #-- everything else --------------- def dependant_constraints(self, var): @@ -191,13 +203,15 @@ W_ComputationSpace.typedef = typedef.TypeDef( "W_ComputationSpace", var = interp2app(W_ComputationSpace.w_var), + find_var = interp2app(W_ComputationSpace.w_find_var), dom = interp2app(W_ComputationSpace.w_dom), tell = interp2app(W_ComputationSpace.w_tell), ask = interp2app(W_ComputationSpace.w_ask), clone = interp2app(W_ComputationSpace.w_clone), commit = interp2app(W_ComputationSpace.w_commit), dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints), - define_problem = interp2app(W_ComputationSpace.w_define_problem)) + define_problem = interp2app(W_ComputationSpace.w_define_problem), + set_root = interp2app(W_ComputationSpace.w_set_root)) def newspace(object_space): Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Wed Apr 12 16:39:36 2006 @@ -16,7 +16,6 @@ assert str(v).startswith('" From antocuni at codespeak.net Wed Apr 12 18:29:44 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 12 Apr 2006 18:29:44 +0200 (CEST) Subject: [pypy-svn] r25725 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060412162944.F37071008E@code0.codespeak.net> Author: antocuni Date: Wed Apr 12 18:29:32 2006 New Revision: 25725 Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/opcodes.py pypy/dist/pypy/translator/cli/test/compile.py Log: Refactored exception handling: the first try was to translate Python exception to CLI exceptions, but this would have caused some trobules because the two inheritance hierarchies don't overlap. Now Python exceptions has its own CLI hierarchy, generated in exactly the same way that is for other classes. This means that we can no longer rely on CLI '.ovf' opcodes that raises OverflowException on error, because Python's OverflowError doesn't map to CLI's OverfloweException. For now the tricky and inefficient solution is to explicitly catch OverflowException and throw a fresh OverflowError instead. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Wed Apr 12 18:29:32 2006 @@ -49,16 +49,21 @@ def __init__(self, db): self.db = db - def lltype_to_cts(self, t): + def __class(self, result, include_class): + if include_class: + return 'class ' + result + else: + return result + + def lltype_to_cts(self, t, include_class=True): if isinstance(t, Instance): - name = t._name self.db.pending_class(t) - return 'class %s' % name + return self.__class(t._name, include_class) elif isinstance(t, StaticMethod): return 'void' # TODO: is it correct to ignore StaticMethod? elif isinstance(t, List): item_type = self.lltype_to_cts(t._ITEMTYPE) - return 'class [pypylib]pypy.runtime.List`1<%s>' % item_type + return self.__class('[pypylib]pypy.runtime.List`1<%s>' % item_type, include_class) return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) @@ -75,7 +80,7 @@ ret_type, ret_var = self.llvar_to_cts(graph.getreturnvar()) func_name = func_name or graph.name - args = graph.getargs() + args = [arg for arg in graph.getargs() if arg.concretetype is not Void] if is_method: args = args[1:] @@ -108,6 +113,3 @@ return parts else: return None, parts[0] - - def pyexception_to_cts(self, exc): - return _get_from_dict(_pyexception_to_cts, exc, 'Unknown exception %s' % exc) Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Wed Apr 12 18:29:32 2006 @@ -19,6 +19,7 @@ self.functions = {} # graph --> function_name self.methods = {} # graph --> method_name self.consts = {} # value --> const_name + self.const_names = set() def pending_function(self, graph): self.pending_node(Function(self, graph)) @@ -48,8 +49,11 @@ try: name = self.consts[const] except KeyError: - name = const.get_name(len(self.consts)) + name = const.get_name() + if name in self.const_names: + name += '__%d' % len(self.consts) self.consts[const] = name + self.const_names.add(name) return '%s.%s::%s' % (CONST_NAMESPACE, CONST_CLASS, name) @@ -90,7 +94,7 @@ assert False, 'Unknown constant: %s' % const make = staticmethod(make) - def get_name(self, n): + def get_name(self): pass def get_type(self): @@ -115,9 +119,8 @@ def __eq__(self, other): return self.obj == other.obj - def get_name(self, n): - name = self.obj._TYPE._name.replace('.', '_') - return '%s_%d' % (name, n) + def get_name(self): + return self.obj._TYPE._name.replace('.', '_') def get_type(self): return self.cts.lltype_to_cts(self.static_type) Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Wed Apr 12 18:29:32 2006 @@ -56,7 +56,7 @@ if self.is_method: args = self.args[1:] # self is implicit - meth_type = 'virtual' + meth_type = 'virtual' # TODO: mark as virtual only when strictly necessary else: args = self.args meth_type = 'static' @@ -100,7 +100,12 @@ continue # see above assert issubclass(link.exitcase, Exception) - cts_exc = self.cts.pyexception_to_cts(link.exitcase) + #cts_exc = self.cts.pyexception_to_cts(link.exitcase) + #cts_exc = str(link.exitcase) # TODO: is it a bit hackish? + ll_meta_exc = link.llexitcase + self.db.record_const(ll_meta_exc) + ll_exc = ll_meta_exc._inst.class_._INSTANCE + cts_exc = self.cts.lltype_to_cts(ll_exc, False) self.ilasm.begin_catch(cts_exc) target = link.target @@ -198,7 +203,8 @@ self.locals = locals def _set_args(self): - self.args = map(self.cts.llvar_to_cts, self.graph.getargs()) + args = [arg for arg in self.graph.getargs() if arg.concretetype is not Void] + self.args = map(self.cts.llvar_to_cts, args) self.argset = set([argname for argtype, argname in self.args]) def _get_block_name(self, block): @@ -257,9 +263,7 @@ self.ilasm.call(signature) def cast_to(self, lltype): - cts_type = self.cts.lltype_to_cts(lltype) - if cts_type.startswith('class '): - cts_type = cts_type[len('class '):] + cts_type = self.cts.lltype_to_cts(lltype, False) self.ilasm.opcode('castclass', cts_type) def new(self, obj): Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Wed Apr 12 18:29:32 2006 @@ -28,6 +28,9 @@ def render(self, generator, op): pass + def __str__(self): + return self.__class__.__name__ + class PushArg(MicroInstruction): def __init__(self, n): Modified: pypy/dist/pypy/translator/cli/opcodes.py ============================================================================== --- pypy/dist/pypy/translator/cli/opcodes.py (original) +++ pypy/dist/pypy/translator/cli/opcodes.py Wed Apr 12 18:29:32 2006 @@ -11,6 +11,29 @@ def _abs(type_): return [PushAllArgs, 'call %s class [mscorlib]System.Math::Abs(%s)' % (type_, type_)] +def _check(op): + if isinstance(op, str): + op = [PushAllArgs, op, StoreResult] + + label = '__check_block_%d' % _check.count + _check.count += 1 + + return [ + '.try {' + ] + op + [ # do the real operations + 'leave %s }' % label, # continue normal execution + + # if overflow, raise a pypy's OverflowError + 'catch [mscorlib]System.OverflowException {', + 'newobj instance void class exceptions.OverflowError::.ctor()', + 'dup', + 'ldsfld class Object_meta pypy.runtime.Constants::exceptions_OverflowError_meta', + 'stfld class Object_meta Object::meta', + 'throw }', + + '%s: nop' % label # continue normal execution + ] +_check.count = 0 opcodes = { # __________ object oriented operations __________ @@ -21,6 +44,7 @@ 'oosend': [CallMethod], 'ooupcast': DoNothing, 'oodowncast': DoNothing, # TODO: is it really safe? + 'oois': 'ceq', 'same_as': DoNothing, # TODO: does same_as really do nothing else than renaming? @@ -43,9 +67,9 @@ 'int_is_true': DoNothing, 'int_neg': 'neg', - 'int_neg_ovf': ['ldc.i4.0', PushAllArgs, 'sub.ovf'], + 'int_neg_ovf': _check(['ldc.i4.0', PushAllArgs, 'sub.ovf', StoreResult]), 'int_abs': _abs('int32'), - 'int_abs_ovf': _abs('int32'), + 'int_abs_ovf': _check(_abs('int32')), 'int_invert': 'not', 'int_add': 'add', @@ -64,9 +88,9 @@ 'int_lshift': 'shl', 'int_rshift': 'shr', 'int_xor': 'xor', - 'int_add_ovf': 'add.ovf', - 'int_sub_ovf': 'sub.ovf', - 'int_mul_ovf': 'mul.ovf', + 'int_add_ovf': _check('add.ovf'), + 'int_sub_ovf': _check('sub.ovf'), + 'int_mul_ovf': _check('mul.ovf'), 'int_floordiv_ovf': 'div', # these can't overflow! 'int_mod_ovf': 'rem', 'int_lt_ovf': 'clt', @@ -79,8 +103,8 @@ 'int_or_ovf': 'or', # are they the same? - 'int_lshift_ovf': [PushArg(0), 'conv.i8', PushArg(1), 'shl', 'conv.ovf.i4'], - 'int_lshift_ovf_val': [PushArg(0), 'conv.i8', PushArg(1), 'shl', 'conv.ovf.i4'], + 'int_lshift_ovf': _check([PushArg(0),'conv.i8',PushArg(1), 'shl', 'conv.ovf.i4', StoreResult]), + 'int_lshift_ovf_val': _check([PushArg(0),'conv.i8',PushArg(1), 'shl', 'conv.ovf.i4', StoreResult]), 'int_rshift_ovf': 'shr', # these can't overflow! 'int_xor_ovf': 'xor', @@ -191,7 +215,10 @@ if type(value) is str: value = InstructionList([PushAllArgs, value, StoreResult]) elif value is not None: - value = InstructionList(value + [StoreResult]) - + if StoreResult not in value: + value.append(StoreResult) + value = InstructionList(value) + opcodes[key] = value + Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Wed Apr 12 18:29:32 2006 @@ -24,53 +24,16 @@ print 'OK' -class MyClass: - INCREMENT = 1 - - def __init__(self, x, y): - self.x = x - self.y = y - - def compute(self): - return self.x + self.y - - def compute_and_multiply(self, factor): - return self.compute() * factor - - def static_meth(x, y): - return x*y - static_meth = staticmethod(static_meth) - -## def class_meth(cls, x, y): -## return x*y + cls.INCREMENT -## class_meth = classmethod(class_meth) - - def class_attribute(self): - return self.x + self.INCREMENT - -class MyDerivedClass(MyClass): - INCREMENT = 2 - - def __init__(self, x, y): - MyClass.__init__(self, x+12, y+34) - - def compute(self): - return self.x - self.y - - - -def init_and_compute(cls, x, y): - return cls(x, y).compute() - - def bar(x, y): - return init_and_compute(MyClass, x, y) + init_and_compute(MyDerivedClass, x, y) - + try: + return ovfcheck(x+y) + except OverflowError: + return 0 f = compile_function(bar, [int, int]) try: - check(bar, f, 42, 13) + check(bar, f, sys.maxint, 13) except py.test.Item.Skipped: print 'Test skipped' From arigo at codespeak.net Wed Apr 12 19:26:08 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 19:26:08 +0200 (CEST) Subject: [pypy-svn] r25727 - in pypy/dist/pypy: annotation rpython/lltypesystem rpython/lltypesystem/test Message-ID: <20060412172608.9EA8F10075@code0.codespeak.net> Author: arigo Date: Wed Apr 12 19:26:06 2006 New Revision: 25727 Modified: pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: Add a new lltype function: cast_subarray_pointer(). Allows casting from a regular Array to a FixedSizeArray that represents a slice of the original Array. (Motivation: rctypes.) Implementation-wise, this requires a new class _subarray in lltype.py. There is no new ll operation to support this, as it is rare and we shouldn't burden the back-end about it. Instead, it can be done with a sequence of address manipulations. Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Wed Apr 12 19:26:06 2006 @@ -421,6 +421,14 @@ cast_p = lltype.cast_pointer(PtrT.const, s_p.ll_ptrtype._defl()) return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) +def cast_subarray_pointer(PtrT, s_p, s_offset): + assert isinstance(s_p, SomePtr), "casting of non-pointer: %r" % s_p + assert PtrT.is_constant() + cast_p = lltype.cast_subarray_pointer(PtrT.const, + s_p.ll_ptrtype._example(), + 0) + return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) + def cast_ptr_to_int(s_ptr): # xxx return SomeInteger() @@ -437,6 +445,7 @@ BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr BUILTIN_ANALYZERS[lltype.cast_pointer] = cast_pointer +BUILTIN_ANALYZERS[lltype.cast_subarray_pointer] = cast_subarray_pointer BUILTIN_ANALYZERS[lltype.cast_ptr_to_int] = cast_ptr_to_int BUILTIN_ANALYZERS[lltype.getRuntimeTypeInfo] = getRuntimeTypeInfo BUILTIN_ANALYZERS[lltype.runtime_type_info] = runtime_type_info Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Wed Apr 12 19:26:06 2006 @@ -215,7 +215,18 @@ self.ref().set(value) def _cast_to_ptr(self, EXPECTED_TYPE): - return lltype.cast_pointer(EXPECTED_TYPE, self.get()) + ref = self.ref() + if (isinstance(ref, _arrayitemref) and + isinstance(EXPECTED_TYPE.TO, lltype.FixedSizeArray) and + isinstance(lltype.typeOf(ref.array).TO, lltype.Array)): + # special case that requires cast_subarray_pointer + return lltype.cast_subarray_pointer(EXPECTED_TYPE, + ref.array, + ref.index) + else: + # regular case + assert isinstance(ref.type(), lltype.ContainerType) + return lltype.cast_pointer(EXPECTED_TYPE, ref.get()) def _cast_to_int(self): return self.get()._cast_to_int() Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Wed Apr 12 19:26:06 2006 @@ -6,6 +6,7 @@ from pypy.tool.picklesupport import getstate_with_slots, setstate_with_slots, pickleable_weakref from types import NoneType from sys import maxint +import weakref log = py.log.Producer('lltype') @@ -612,6 +613,29 @@ raise TypeError, "can only cast pointers to other pointers" return ptr._cast_to(PTRTYPE) +def cast_subarray_pointer(ARRAYPTRTYPE, arrayptr, baseoffset): + CURPTRTYPE = typeOf(arrayptr) + if not isinstance(CURPTRTYPE, Ptr) or not isinstance(ARRAYPTRTYPE, Ptr): + raise TypeError, "can only cast pointers to other pointers" + ARRAYTYPE = ARRAYPTRTYPE.TO + if (not isinstance(CURPTRTYPE.TO, Array) or + not isinstance(ARRAYTYPE, FixedSizeArray)): + raise TypeError, "for now, can only cast Array to FixedSizeArray" + if CURPTRTYPE.TO.OF != ARRAYTYPE.OF: + raise TypeError, "mismatching array item types" + if not arrayptr: + raise RuntimeError("cast_subarray_pointer: NULL argument") + try: + cache = _subarray._cache[arrayptr._obj] + except KeyError: + cache = _subarray._cache[arrayptr._obj] = {} + key = (ARRAYTYPE, baseoffset) + try: + subarray = cache[key] + except KeyError: + subarray = cache[key] = _subarray(ARRAYTYPE, arrayptr._obj, baseoffset) + return _ptr(ARRAYPTRTYPE, subarray) + def _expose(val, solid=False): """XXX A nice docstring here""" T = typeOf(val) @@ -762,9 +786,9 @@ def __getitem__(self, i): # ! can only return basic or ptr ! if isinstance(self._T, (Array, FixedSizeArray)): - if not (0 <= i < self._obj.getlength()): - if (self._T._hints.get('isrpystring', False) and - i == self._obj.getlength()): + start, stop = self._obj.getbounds() + if not (start <= i < stop): + if self._T._hints.get('isrpystring', False) and i == stop: # special hack for the null terminator assert self._T.OF == Char return '\x00' @@ -782,8 +806,9 @@ if T2 != T1: raise TypeError("%r items:\n" "expect %r\n" - " got %r" % (self._T, T1, T2)) - if not (0 <= i < self._obj.getlength()): + " got %r" % (self._T, T1, T2)) + start, stop = self._obj.getbounds() + if not (start <= i < stop): raise IndexError("array index out of bounds") self._obj.setitem(i, val) return @@ -971,6 +996,9 @@ assert isinstance(self._TYPE, FixedSizeArray) return self._TYPE.length + def getbounds(self): + return 0, self.getlength() + def getitem(self, index): # for FixedSizeArray kind of structs assert isinstance(self._TYPE, FixedSizeArray) return getattr(self, 'item%d' % index) @@ -1015,6 +1043,9 @@ def getlength(self): return len(self.items) + def getbounds(self): + return 0, self.getlength() + def getitem(self, index): return self.items[index] @@ -1025,6 +1056,32 @@ assert not '__dict__' in dir(_struct) +class _subarray(_parentable): # only for cast_subarray_pointer() + _kind = "subarray" + _cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays} + + def __init__(self, TYPE, arrayobj, baseoffset): + _parentable.__init__(self, TYPE) + self._setparentstructure(arrayobj, baseoffset) + + def getlength(self): + assert isinstance(self._TYPE, FixedSizeArray) + return self._TYPE.length + + def getbounds(self): + baseoffset = self._parent_index + start, stop = self._parentstructure().getbounds() + return start - baseoffset, stop - baseoffset + + def getitem(self, index): + baseoffset = self._parent_index + return self._parentstructure().getitem(baseoffset + index) + + def setitem(self, index, value): + baseoffset = self._parent_index + self._parentstructure().setitem(baseoffset + index, value) + + class _func(object): def __init__(self, TYPE, **attrs): self._TYPE = TYPE Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Wed Apr 12 19:26:06 2006 @@ -160,3 +160,15 @@ assert adr1 != adr2 adr2 += ItemOffset(lltype.Char, -1) assert adr1 == adr2 + +def test_cast_subarray_pointer(): + A = lltype.GcArray(lltype.Signed) + a = lltype.malloc(A, 5) + a[3] = 132 + + SUBARRAY = lltype.FixedSizeArray(lltype.Signed, 1) + adr = cast_ptr_to_adr(a) + itemoffsetof(A, 3) + subarray = cast_adr_to_ptr(adr, lltype.Ptr(SUBARRAY)) + assert subarray[0] == 132 + subarray[0] += 2 + assert a[3] == 134 Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Wed Apr 12 19:26:06 2006 @@ -577,3 +577,36 @@ assert s.a[3] == 17 assert len(s.a) == 5 py.test.raises(TypeError, "s.a = a") + +def test_cast_subarray_pointer(): + A = GcArray(Signed) + a = malloc(A, 5) + a[0] = 0 + a[1] = 10 + a[2] = 20 + a[3] = 30 + a[4] = 40 + BOX = Ptr(FixedSizeArray(Signed, 2)) + b01 = cast_subarray_pointer(BOX, a, 0) + b12 = cast_subarray_pointer(BOX, a, 1) + b23 = cast_subarray_pointer(BOX, a, 2) + b34 = cast_subarray_pointer(BOX, a, 3) + assert b01[0] == 0 + assert b01[1] == 10 + assert b12[0] == 10 + assert b12[1] == 20 + assert b23[0] == 20 + assert b23[1] == 30 + assert b34[0] == 30 + assert b34[1] == 40 + b23[0] = 23 + assert a[2] == 23 + b12[1] += 1 + assert a[2] == 24 + # out-of-bound access is allowed, if it's within the parent's bounds + assert len(b23) == 2 + assert b23[-1] == 10 + assert b12[3] == 40 + py.test.raises(IndexError, "b01[-1]") + py.test.raises(IndexError, "b34[2]") + py.test.raises(IndexError, "b12[4]") From arigo at codespeak.net Wed Apr 12 19:30:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 19:30:46 +0200 (CEST) Subject: [pypy-svn] r25728 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060412173046.0E1BD10076@code0.codespeak.net> Author: arigo Date: Wed Apr 12 19:30:45 2006 New Revision: 25728 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py Log: Move the hack about the null terminator of rpystrings into the class _array, so that _subarrays inherit it. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Wed Apr 12 19:30:45 2006 @@ -788,10 +788,6 @@ if isinstance(self._T, (Array, FixedSizeArray)): start, stop = self._obj.getbounds() if not (start <= i < stop): - if self._T._hints.get('isrpystring', False) and i == stop: - # special hack for the null terminator - assert self._T.OF == Char - return '\x00' raise IndexError("array index out of bounds") o = self._obj.getitem(i) return _expose(o, self._solid) @@ -1044,10 +1040,23 @@ return len(self.items) def getbounds(self): - return 0, self.getlength() + stop = len(self.items) + if self._TYPE._hints.get('isrpystring', False): + # special hack for the null terminator + assert self._TYPE.OF == Char + stop += 1 + return 0, stop def getitem(self, index): - return self.items[index] + try: + return self.items[index] + except IndexError: + if (self._TYPE._hints.get('isrpystring', False) and + index == len(self.items)): + # special hack for the null terminator + assert self._TYPE.OF == Char + return '\x00' + raise def setitem(self, index, value): self.items[index] = value From tismer at codespeak.net Wed Apr 12 19:39:11 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 12 Apr 2006 19:39:11 +0200 (CEST) Subject: [pypy-svn] r25729 - in pypy/dist/pypy: rpython rpython/lltypesystem translator/c translator/c/test translator/c/winproj/extension Message-ID: <20060412173911.AE929100B5@code0.codespeak.net> Author: tismer Date: Wed Apr 12 19:39:05 2006 New Revision: 25729 Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj pypy/dist/pypy/translator/c/wrapper.py Log: made object instantiation implicit, to be called during __init__. This is much more efficient than the former approach, although the wrapping is slightly more delicate. This also required to add an extra hook which I dislike, since the rtyper needs to know which function is actually being wrapped, in order to figure out whether we are wrapping __init__ or something else. Temporary (but tested) check-in before refactoring. Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Wed Apr 12 19:39:05 2006 @@ -594,6 +594,7 @@ # # _________________________ Conversions for CPython _________________________ +# part I: wrapping, destructor, preserving object identity def call_destructor(thing, repr): ll_call_destructor(thing, repr) @@ -602,13 +603,13 @@ return 42 # will be mapped def rtype_destruct_object(hop): - v_any, c_spec = hop.inputargs(*hop.args_r) + v_inst, c_spec = hop.inputargs(*hop.args_r) repr = c_spec.value if repr.has_wrapper: null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) # XXX this is a hack! We need an operation to remove a broken PyObject - repr.setfield(v_any, '_wrapper_', null, hop.llops, opname='bare_setfield') - hop.genop('gc_unprotect', [v_any]) + repr.setfield(v_inst, '_wrapper_', null, hop.llops, opname='bare_setfield') + hop.genop('gc_unprotect', [v_inst]) extregistry.register_value(ll_call_destructor, compute_result_annotation=lambda *args: None, @@ -623,16 +624,16 @@ def rtype_wrap_object_create(hop): gencapi = hop.llops.gencapicall pyptr = hop.r_result - v_any, c_spec = hop.inputargs(*hop.args_r) + v_inst, c_spec = hop.inputargs(*hop.args_r) repr = c_spec.value f = call_destructor - hop.genop('gc_protect', [v_any]) + hop.genop('gc_protect', [v_inst]) ARG = repr.lowleveltype reprPBC = hop.rtyper.annotator.bookkeeper.immutablevalue(repr) fp_dtor = hop.rtyper.annotate_helper_fn(f, [ARG, reprPBC]) FUNC = FuncType([ARG, Void], Void) c_dtor = hop.inputconst(Ptr(FUNC), fp_dtor) - res = gencapi('PyCObject_FromVoidPtr', [v_any, c_dtor], resulttype=hop.r_result) + res = gencapi('PyCObject_FromVoidPtr', [v_inst, c_dtor], resulttype=pyptr) if repr.has_wrapper: cobj = res c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) @@ -640,7 +641,7 @@ res = gencapi('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyptr) c_self = hop.inputconst(pyobj_repr, '__self__') v_result = hop.genop('setattr', [res, c_self, cobj], resulttype=pyptr) - repr.setfield(v_any, '_wrapper_', res, hop.llops) + repr.setfield(v_inst, '_wrapper_', res, hop.llops) hop.genop('gc_unprotect', [res]) # yes a weak ref return res @@ -655,10 +656,10 @@ return 42 def rtype_wrap_object_fetch(hop): - v_any, c_spec = hop.inputargs(*hop.args_r) + v_inst, c_spec = hop.inputargs(*hop.args_r) repr = c_spec.value if repr.has_wrapper: - return repr.getfield(v_any, '_wrapper_', hop.llops) + return repr.getfield(v_inst, '_wrapper_', hop.llops) else: null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) return null @@ -673,15 +674,6 @@ ret = create_pywrapper(obj, repr) return ret -class __extend__(pairtype(PyObjRepr, InstanceRepr)): - def convert_from_to((r_from, r_to), v, llops): - if r_to.has_wrapper: - c_self = inputconst(pyobj_repr, '__self__') - v = llops.genop('getattr', [v, c_self], resulttype=r_from) - v_adr = llops.gencapicall('PyCObject_AsVoidPtr', [v], resulttype=r_to) - llops.genop('gc_protect', [v_adr]) - return v_adr - class __extend__(pairtype(InstanceRepr, PyObjRepr)): def convert_from_to((r_from, r_to), v, llops): c_repr = inputconst(Void, r_from) @@ -690,6 +682,38 @@ else: return llops.gendirectcall(create_pywrapper, v, c_repr) +# part II: unwrapping, creating the instance + +class __extend__(pairtype(PyObjRepr, InstanceRepr)): + def convert_from_to((r_from, r_to), v, llops): + if r_to.has_wrapper: + init, context = llops.rtyper.get_wrapping_hint(r_to.classdef) + if context is init: + # saving an extra __new__ method, we create the instance on __init__ + v_inst = r_to.new_instance(llops) + gencapi = llops.gencapicall + rtyper = llops.rtyper + pyptr = r_from + f = call_destructor + ARG = r_to.lowleveltype + reprPBC = rtyper.annotator.bookkeeper.immutablevalue(r_to) + fp_dtor = rtyper.annotate_helper_fn(f, [ARG, reprPBC]) + FUNC = FuncType([ARG, Void], Void) + c_dtor = inputconst(Ptr(FUNC), fp_dtor) + v_cobj = gencapi('PyCObject_FromVoidPtr', [v_inst, c_dtor], resulttype=pyptr) + c_self = inputconst(pyobj_repr, '__self__') + llops.genop('setattr', [v, c_self, v_cobj], resulttype=pyptr) + r_to.setfield(v_inst, '_wrapper_', v, llops) + llops.genop('gc_protect', [v_inst]) + llops.genop('gc_unprotect', [v]) + return v_inst + + c_self = inputconst(pyobj_repr, '__self__') + v = llops.genop('getattr', [v, c_self], resulttype=r_from) + v_inst = llops.gencapicall('PyCObject_AsVoidPtr', [v], resulttype=r_to) + llops.genop('gc_protect', [v_inst]) + return v_inst + # ____________________________________________________________ def ll_both_none(ins1, ins2): Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Wed Apr 12 19:39:05 2006 @@ -57,6 +57,7 @@ self.instance_reprs = {} self.pbc_reprs = {} self.classdefs_with_wrapper = {} + self.wrapper_context = None # or add an extra arg to convertvar? self.concrete_calltables = {} self.class_pbc_attributes = {} self.oo_meth_impls = {} @@ -89,7 +90,13 @@ self.crash_on_first_typeerror = True def add_wrapper(self, clsdef): - self.classdefs_with_wrapper[clsdef] = clsdef + # record that this class has a wrapper, and what the __init__ is + init = getattr(clsdef.classdesc.pyobj.__init__, 'im_func', None) + self.classdefs_with_wrapper[clsdef] = init + + def set_wrapper_context(self, obj): + # not nice, but we sometimes need to know which function we are wrapping + self.wrapper_context = obj def add_pendingsetup(self, repr): assert isinstance(repr, Repr) @@ -546,6 +553,9 @@ def needs_wrapper(self, clsdef): return clsdef in self.classdefs_with_wrapper + def get_wrapping_hint(self, clsdef): + return self.classdefs_with_wrapper[clsdef], self.wrapper_context + def getcallable(self, graph): def getconcretetype(v): return self.bindingrepr(v).lowleveltype Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Wed Apr 12 19:39:05 2006 @@ -21,8 +21,7 @@ class LowLevelDatabase(object): - def __init__(self, translator=None, standalone=False, gcpolicy=None, thread_enabled=False, - instantiators={}): + def __init__(self, translator=None, standalone=False, gcpolicy=None, thread_enabled=False): self.translator = translator self.standalone = standalone self.structdefnodes = {} @@ -38,7 +37,7 @@ self.infs = [] self.namespace = CNameManager() if not standalone: - self.pyobjmaker = PyObjMaker(self.namespace, self.get, translator, instantiators) + self.pyobjmaker = PyObjMaker(self.namespace, self.get, translator) if gcpolicy is None: from pypy.translator.c import gc polname = conftest.option.gcpolicy Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Wed Apr 12 19:39:05 2006 @@ -33,11 +33,10 @@ self.libraries = libraries self.exports = {} - def build_database(self, exports=[], instantiators={}): + def build_database(self, exports=[]): translator = self.translator db = LowLevelDatabase(translator, standalone=self.standalone, - gcpolicy=self.gcpolicy, thread_enabled=self.thread_enabled, - instantiators=instantiators) + gcpolicy=self.gcpolicy, thread_enabled=self.thread_enabled) if self.stackless: from pypy.translator.c.stackless import StacklessData Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Wed Apr 12 19:39:05 2006 @@ -23,7 +23,7 @@ reconstruct them. """ - def __init__(self, namespace, getvalue, translator=None, instantiators={}): + def __init__(self, namespace, getvalue, translator=None): self.namespace = namespace self.getvalue = getvalue self.translator = translator @@ -38,7 +38,6 @@ self.wrappers = {} # {'pycfunctionvariable': ('name', 'wrapperfn')} self.import_hints = {} # I don't seem to need it any longer. # leaving the import support intact, doesn't hurt. - self.instantiators = instantiators def nameof(self, obj, debug=None): if debug: @@ -336,7 +335,8 @@ return name def nameof_classobj(self, cls): - if cls in self.instantiators: + clsdef = self.translator.annotator.bookkeeper.getuniqueclassdef(cls) + if self.translator.rtyper.needs_wrapper(clsdef): return self.wrap_exported_class(cls) if cls.__doc__ and cls.__doc__.lstrip().startswith('NOT_RPYTHON'): @@ -551,16 +551,9 @@ if baseargs: baseargs = '(%s)' % baseargs - # fishing for the instantiator - instantiator = self.nameof(self.instantiators[cls]) a = self.initcode.append a('class %s%s:' % (name, baseargs) ) a(' __metaclass__ = type') a(' __slots__ = ["__self__"] # for PyCObject') - a(' def __new__(cls, *args, **kwds):') - a(' return %s()' % instantiator ) - # XXX - # I would like to use instantiator directly, but don't know - # how to create a function that ignores all args self.later(initclassobj()) return name Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Wed Apr 12 19:39:05 2006 @@ -4,7 +4,6 @@ from pypy.rpython import extregistry from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem import lltype -from pypy.rpython.objectmodel import instantiate from pypy.rpython import robject, rclass import sys @@ -35,7 +34,6 @@ t.buildannotator() rtyper = t.buildrtyper() bk = rtyper.annotator.bookkeeper - instantiators = {} t.annotator.build_types(func, get_annotation(func)) if not exports: exports = [] @@ -43,14 +41,7 @@ exports = exports + [('__all__', all)] for obj in exports: if isinstance(obj, type): - cls = obj - def make(): - obj = instantiate(cls) - return obj - make.__name__ = cls.__name__ + '__new__' - t.annotator.build_types(make, []) - instantiators[cls] = make - clsdef = bk.getuniqueclassdef(cls) + clsdef = bk.getuniqueclassdef(obj) rtyper.add_wrapper(clsdef) elif callable(obj): t.annotator.build_types(obj, get_annotation(obj)) @@ -71,7 +62,7 @@ cbuilder = CExtModuleBuilder(t, func, gcpolicy=gcpolicy) # explicit build of database - db = cbuilder.build_database(exports=exports, instantiators=instantiators) + db = cbuilder.build_database(exports=exports) cbuilder.generate_source(db) cbuilder.compile() @@ -263,9 +254,9 @@ DemoClass.__init__(self, b, a) self.c = c -# def demo(self, *other): + def demo(self, *other): #if other: print other - # return float(DemoClass.demo(self)) + return float(DemoClass.demo(self)) def otherdemo(self): return 'this is the DemoSubclass', self.a, self.b @@ -319,38 +310,8 @@ # _______________________________________________ # creating our own setup function for the module -def do_the_import(): - from twisted.internet import reactor - return reactor - -def rtype_wraptest(hop): - #v_obj, = hop.inputargs((robject.pyobj_repr, )) - from pypy.objspace.flow.model import Constant - v_obj = Constant(hop.args_s[0].const) - v = hop.genop('simple_call', [v_obj], resulttype = robject.pyobj_repr) - return v - -def wraptest(obj): - return obj -extregistry.register_value(wraptest, - compute_result_annotation = annmodel.SomeObject(), - specialize_call = rtype_wraptest) - -# not sure what to do with the above. -# use genpickle facility to produce a plain function? -# create a space and run geninterp on it? -# tweak flow space to delay the imports for this func? - -### XXX write up the rules how to use this ### - -def t2(): - global sys - import sys - def setup_new_module(mod, modname): # note the name clash with py.test on setup_module - #t1() - #t2() return from types import module m = module(modname) @@ -392,6 +353,7 @@ res = obj.demo() assert res == DemoClass(2, 3).demo() + if __name__=='__main__': test_expose_classes() Modified: pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj ============================================================================== --- pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj (original) +++ pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Wed Apr 12 19:39:05 2006 @@ -208,7 +208,7 @@ + RelativePath="..\..\..\..\..\..\..\Documents and Settings\ctismer\Local Settings\Temp\usession-1965\testing_1\testing_1.c"> Modified: pypy/dist/pypy/translator/c/wrapper.py ============================================================================== --- pypy/dist/pypy/translator/c/wrapper.py (original) +++ pypy/dist/pypy/translator/c/wrapper.py Wed Apr 12 19:39:05 2006 @@ -98,9 +98,12 @@ assert rtyper is not None, ( "needs the rtyper to perform argument conversions") r_arg = rtyper.bindingrepr(inputargs[i]) + # give the rtyper a chance to know which function we are wrapping + rtyper.set_wrapper_context(func) varguments[i] = newops.convertvar(varguments[i], r_from = pyobj_repr, r_to = r_arg) + rtyper.set_wrapper_context(None) # "result = direct_call(func, argument_0, argument_1, ..)" vlist = [inputconst(typeOf(f), f)] + varguments From arigo at codespeak.net Wed Apr 12 19:48:31 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 19:48:31 +0200 (CEST) Subject: [pypy-svn] r25730 - pypy/dist/pypy/rpython Message-ID: <20060412174831.8D1C1100D3@code0.codespeak.net> Author: arigo Date: Wed Apr 12 19:48:30 2006 New Revision: 25730 Modified: pypy/dist/pypy/rpython/rbuiltin.py Log: Argh, forgot this file in the previous check-in. Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Wed Apr 12 19:48:30 2006 @@ -4,7 +4,7 @@ from pypy.rpython.lltypesystem import lltype, rclass, llmemory from pypy.rpython import rarithmetic, objectmodel, rstack, rint, raddress from pypy.rpython.error import TyperError -from pypy.rpython.rmodel import Repr, IntegerRepr +from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.rrange import rtype_builtin_range, rtype_builtin_xrange from pypy.rpython import rstr from pypy.rpython import rptr @@ -307,6 +307,34 @@ return hop.genop('cast_pointer', [v_input], # v_type implicit in r_result resulttype = hop.r_result.lowleveltype) +def rtype_cast_subarray_pointer(hop): + assert hop.args_s[0].is_constant() + assert isinstance(hop.args_r[1], rptr.PtrRepr) + v_type, v_input, v_baseoffset = hop.inputargs(lltype.Void, hop.args_r[1], + lltype.Signed) + hop.exception_cannot_occur() + # instead of adding a 'cast_subarray_pointer' ll operation, + # we do it with address manipulations + return gen_cast_array_pointer(hop.llops, hop.r_result.lowleveltype, + v_input, v_baseoffset) + +def gen_cast_array_pointer(llops, ARRAYPTRTYPE, v_array, v_baseoffset): + "Helper to generate the equivalent of a 'cast_subarray_pointer' operation." + from pypy.rpython.lltypesystem import llmemory + SRCTYPE = v_array.concretetype.TO + v_array_adr = llops.genop('cast_ptr_to_adr', [v_array], + resulttype = llmemory.Address) + c_ofs1 = inputconst(lltype.Signed, llmemory.ArrayItemsOffset(SRCTYPE)) + v_array_items_adr = llops.genop('adr_add', [v_array_adr, c_ofs1], + resulttype = llmemory.Address) + c_ofs2 = inputconst(lltype.Signed, llmemory.ItemOffset(SRCTYPE.OF)) + v_ofs3 = llops.genop('int_mul', [c_ofs2, v_baseoffset], + resulttype = lltype.Signed) + v_base_adr = llops.genop('adr_add', [v_array_items_adr, v_ofs3], + resulttype = llmemory.Address) + return llops.genop('cast_adr_to_ptr', [v_base_adr], + resulttype = ARRAYPTRTYPE) + def rtype_cast_primitive(hop): assert hop.args_s[0].is_constant() TGT = hop.args_s[0].const @@ -341,6 +369,7 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer +BUILTIN_TYPER[lltype.cast_subarray_pointer] = rtype_cast_subarray_pointer BUILTIN_TYPER[lltype.cast_ptr_to_int] = rtype_cast_ptr_to_int BUILTIN_TYPER[lltype.typeOf] = rtype_const_result BUILTIN_TYPER[lltype.nullptr] = rtype_const_result From arigo at codespeak.net Wed Apr 12 19:49:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 19:49:01 +0200 (CEST) Subject: [pypy-svn] r25731 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060412174901.DC42F10108@code0.codespeak.net> Author: arigo Date: Wed Apr 12 19:49:00 2006 New Revision: 25731 Modified: pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/c/test/test_lltyped.py Log: C back-end hacks for _subarray. Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Wed Apr 12 19:49:00 2006 @@ -3,7 +3,7 @@ Struct, Array, FixedSizeArray, FuncType, PyObjectType, typeOf, \ GcStruct, GcArray, GC_CONTAINER, ContainerType, \ parentlink, Ptr, PyObject, Void, OpaqueType, Float, \ - RuntimeTypeInfo, getRuntimeTypeInfo, Char + RuntimeTypeInfo, getRuntimeTypeInfo, Char, _subarray from pypy.translator.c.funcgen import FunctionCodeGenerator from pypy.translator.c.external import CExternalFunctionCodeGenerator from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring @@ -459,14 +459,15 @@ def __init__(self, db, T, obj): ContainerNode.__init__(self, db, T, obj) - self.ptrname = self.name + if not isinstance(obj, _subarray): # XXX hackish + self.ptrname = self.name def basename(self): return self.T._name def enum_dependencies(self): - for name in self.T._names: # _names == ['item0', 'item1', ...] - yield getattr(self.obj, name) + for i in range(self.obj.getlength()): + yield self.obj.getitem(i) def getlength(self): return 1 # not variable-sized! Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Wed Apr 12 19:49:00 2006 @@ -128,3 +128,35 @@ fn = self.getcompiled(llf) res = fn(4) assert res == -1 + + def test_cast_subarray_pointer(self): + A = GcArray(Signed) + a = malloc(A, 5) + a[0] = 0 + a[1] = 10 + a[2] = 20 + a[3] = 30 + a[4] = 40 + BOX = Ptr(FixedSizeArray(Signed, 2)) + b01 = cast_subarray_pointer(BOX, a, 0) + b12 = cast_subarray_pointer(BOX, a, 1) + b23 = cast_subarray_pointer(BOX, a, 2) + b34 = cast_subarray_pointer(BOX, a, 3) + def llf(n=int): + saved = a[n] + a[n] = 1000 + try: + return b01[0] + b12[0] + b23[1] + b34[1] + finally: + a[n] = saved + fn = self.getcompiled(llf) + res = fn(0) + assert res == 1000 + 10 + 30 + 40 + res = fn(1) + assert res == 0 + 1000 + 30 + 40 + res = fn(2) + assert res == 0 + 10 + 30 + 40 + res = fn(3) + assert res == 0 + 10 + 1000 + 40 + res = fn(4) + assert res == 0 + 10 + 30 + 1000 From arigo at codespeak.net Wed Apr 12 20:15:27 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 20:15:27 +0200 (CEST) Subject: [pypy-svn] r25732 - in pypy/dist/pypy: rpython/lltypesystem rpython/lltypesystem/test translator/c/test Message-ID: <20060412181527.6B82A10115@code0.codespeak.net> Author: arigo Date: Wed Apr 12 20:15:25 2006 New Revision: 25732 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py pypy/dist/pypy/translator/c/test/test_lltyped.py Log: Actually, cast_subarray_pointer() works for slicing from FixedSizeArray to FixedSizeArray too. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Wed Apr 12 20:15:25 2006 @@ -618,9 +618,9 @@ if not isinstance(CURPTRTYPE, Ptr) or not isinstance(ARRAYPTRTYPE, Ptr): raise TypeError, "can only cast pointers to other pointers" ARRAYTYPE = ARRAYPTRTYPE.TO - if (not isinstance(CURPTRTYPE.TO, Array) or + if (not isinstance(CURPTRTYPE.TO, (Array, FixedSizeArray)) or not isinstance(ARRAYTYPE, FixedSizeArray)): - raise TypeError, "for now, can only cast Array to FixedSizeArray" + raise TypeError, "can only cast from arrays to FixedSizeArray" if CURPTRTYPE.TO.OF != ARRAYTYPE.OF: raise TypeError, "mismatching array item types" if not arrayptr: Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Wed Apr 12 20:15:25 2006 @@ -579,34 +579,34 @@ py.test.raises(TypeError, "s.a = a") def test_cast_subarray_pointer(): - A = GcArray(Signed) - a = malloc(A, 5) - a[0] = 0 - a[1] = 10 - a[2] = 20 - a[3] = 30 - a[4] = 40 - BOX = Ptr(FixedSizeArray(Signed, 2)) - b01 = cast_subarray_pointer(BOX, a, 0) - b12 = cast_subarray_pointer(BOX, a, 1) - b23 = cast_subarray_pointer(BOX, a, 2) - b34 = cast_subarray_pointer(BOX, a, 3) - assert b01[0] == 0 - assert b01[1] == 10 - assert b12[0] == 10 - assert b12[1] == 20 - assert b23[0] == 20 - assert b23[1] == 30 - assert b34[0] == 30 - assert b34[1] == 40 - b23[0] = 23 - assert a[2] == 23 - b12[1] += 1 - assert a[2] == 24 - # out-of-bound access is allowed, if it's within the parent's bounds - assert len(b23) == 2 - assert b23[-1] == 10 - assert b12[3] == 40 - py.test.raises(IndexError, "b01[-1]") - py.test.raises(IndexError, "b34[2]") - py.test.raises(IndexError, "b12[4]") + for a in [malloc(GcArray(Signed), 5), + malloc(FixedSizeArray(Signed, 5), immortal=True)]: + a[0] = 0 + a[1] = 10 + a[2] = 20 + a[3] = 30 + a[4] = 40 + BOX = Ptr(FixedSizeArray(Signed, 2)) + b01 = cast_subarray_pointer(BOX, a, 0) + b12 = cast_subarray_pointer(BOX, a, 1) + b23 = cast_subarray_pointer(BOX, a, 2) + b34 = cast_subarray_pointer(BOX, a, 3) + assert b01[0] == 0 + assert b01[1] == 10 + assert b12[0] == 10 + assert b12[1] == 20 + assert b23[0] == 20 + assert b23[1] == 30 + assert b34[0] == 30 + assert b34[1] == 40 + b23[0] = 23 + assert a[2] == 23 + b12[1] += 1 + assert a[2] == 24 + # out-of-bound access is allowed, if it's within the parent's bounds + assert len(b23) == 2 + assert b23[-1] == 10 + assert b12[3] == 40 + py.test.raises(IndexError, "b01[-1]") + py.test.raises(IndexError, "b34[2]") + py.test.raises(IndexError, "b12[4]") Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Wed Apr 12 20:15:25 2006 @@ -130,33 +130,33 @@ assert res == -1 def test_cast_subarray_pointer(self): - A = GcArray(Signed) - a = malloc(A, 5) - a[0] = 0 - a[1] = 10 - a[2] = 20 - a[3] = 30 - a[4] = 40 - BOX = Ptr(FixedSizeArray(Signed, 2)) - b01 = cast_subarray_pointer(BOX, a, 0) - b12 = cast_subarray_pointer(BOX, a, 1) - b23 = cast_subarray_pointer(BOX, a, 2) - b34 = cast_subarray_pointer(BOX, a, 3) - def llf(n=int): - saved = a[n] - a[n] = 1000 - try: - return b01[0] + b12[0] + b23[1] + b34[1] - finally: - a[n] = saved - fn = self.getcompiled(llf) - res = fn(0) - assert res == 1000 + 10 + 30 + 40 - res = fn(1) - assert res == 0 + 1000 + 30 + 40 - res = fn(2) - assert res == 0 + 10 + 30 + 40 - res = fn(3) - assert res == 0 + 10 + 1000 + 40 - res = fn(4) - assert res == 0 + 10 + 30 + 1000 + for a in [malloc(GcArray(Signed), 5), + malloc(FixedSizeArray(Signed, 5), immortal=True)]: + a[0] = 0 + a[1] = 10 + a[2] = 20 + a[3] = 30 + a[4] = 40 + BOX = Ptr(FixedSizeArray(Signed, 2)) + b01 = cast_subarray_pointer(BOX, a, 0) + b12 = cast_subarray_pointer(BOX, a, 1) + b23 = cast_subarray_pointer(BOX, a, 2) + b34 = cast_subarray_pointer(BOX, a, 3) + def llf(n=int): + saved = a[n] + a[n] = 1000 + try: + return b01[0] + b12[0] + b23[1] + b34[1] + finally: + a[n] = saved + fn = self.getcompiled(llf) + res = fn(0) + assert res == 1000 + 10 + 30 + 40 + res = fn(1) + assert res == 0 + 1000 + 30 + 40 + res = fn(2) + assert res == 0 + 10 + 30 + 40 + res = fn(3) + assert res == 0 + 10 + 1000 + 40 + res = fn(4) + assert res == 0 + 10 + 30 + 1000 From tismer at codespeak.net Wed Apr 12 20:19:02 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 12 Apr 2006 20:19:02 +0200 (CEST) Subject: [pypy-svn] r25733 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060412181902.F26F510119@code0.codespeak.net> Author: tismer Date: Wed Apr 12 20:19:01 2006 New Revision: 25733 Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py Log: factorizing out common stuff, better naming Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Wed Apr 12 20:19:01 2006 @@ -621,29 +621,35 @@ def ll_create_pywrapper(thing, repr): return 42 +def into_cobject(v_inst, repr, llops): + llops.genop('gc_protect', [v_inst]) + ARG = repr.lowleveltype + reprPBC = llops.rtyper.annotator.bookkeeper.immutablevalue(repr) + fp_dtor = llops.rtyper.annotate_helper_fn(call_destructor, [ARG, reprPBC]) + FUNC = FuncType([ARG, Void], Void) + c_dtor = inputconst(Ptr(FUNC), fp_dtor) + return llops.gencapicall('PyCObject_FromVoidPtr', [v_inst, c_dtor], resulttype=pyobj_repr) + +def outof_cobject(v_obj, repr, llops): + v_inst = llops.gencapicall('PyCObject_AsVoidPtr', [v_obj], resulttype=repr) + llops.genop('gc_protect', [v_inst]) + return v_inst + def rtype_wrap_object_create(hop): gencapi = hop.llops.gencapicall - pyptr = hop.r_result v_inst, c_spec = hop.inputargs(*hop.args_r) repr = c_spec.value - f = call_destructor - hop.genop('gc_protect', [v_inst]) - ARG = repr.lowleveltype - reprPBC = hop.rtyper.annotator.bookkeeper.immutablevalue(repr) - fp_dtor = hop.rtyper.annotate_helper_fn(f, [ARG, reprPBC]) - FUNC = FuncType([ARG, Void], Void) - c_dtor = hop.inputconst(Ptr(FUNC), fp_dtor) - res = gencapi('PyCObject_FromVoidPtr', [v_inst, c_dtor], resulttype=pyptr) + v_res = into_cobject(v_inst, repr, hop.llops) if repr.has_wrapper: - cobj = res + v_cobj = v_res c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) c_0 = hop.inputconst(Signed, 0) - res = gencapi('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyptr) + v_res = gencapi('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyobj_repr) c_self = hop.inputconst(pyobj_repr, '__self__') - v_result = hop.genop('setattr', [res, c_self, cobj], resulttype=pyptr) - repr.setfield(v_inst, '_wrapper_', res, hop.llops) - hop.genop('gc_unprotect', [res]) # yes a weak ref - return res + hop.genop('setattr', [v_res, c_self, v_cobj], resulttype=pyobj_repr) + repr.setfield(v_inst, '_wrapper_', v_res, hop.llops) + hop.genop('gc_unprotect', [v_res]) # yes a weak ref + return v_res extregistry.register_value(ll_create_pywrapper, compute_result_annotation=annmodel.SomePtr(Ptr(PyObject)), @@ -693,26 +699,16 @@ v_inst = r_to.new_instance(llops) gencapi = llops.gencapicall rtyper = llops.rtyper - pyptr = r_from - f = call_destructor - ARG = r_to.lowleveltype - reprPBC = rtyper.annotator.bookkeeper.immutablevalue(r_to) - fp_dtor = rtyper.annotate_helper_fn(f, [ARG, reprPBC]) - FUNC = FuncType([ARG, Void], Void) - c_dtor = inputconst(Ptr(FUNC), fp_dtor) - v_cobj = gencapi('PyCObject_FromVoidPtr', [v_inst, c_dtor], resulttype=pyptr) + v_cobj = into_cobject(v_inst, r_to, llops) c_self = inputconst(pyobj_repr, '__self__') - llops.genop('setattr', [v, c_self, v_cobj], resulttype=pyptr) + llops.genop('setattr', [v, c_self, v_cobj], resulttype=pyobj_repr) r_to.setfield(v_inst, '_wrapper_', v, llops) - llops.genop('gc_protect', [v_inst]) llops.genop('gc_unprotect', [v]) return v_inst c_self = inputconst(pyobj_repr, '__self__') v = llops.genop('getattr', [v, c_self], resulttype=r_from) - v_inst = llops.gencapicall('PyCObject_AsVoidPtr', [v], resulttype=r_to) - llops.genop('gc_protect', [v_inst]) - return v_inst + return outof_cobject(v, r_to, llops) # ____________________________________________________________ From arigo at codespeak.net Wed Apr 12 20:21:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 20:21:10 +0200 (CEST) Subject: [pypy-svn] r25734 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060412182110.CB8211013E@code0.codespeak.net> Author: arigo Date: Wed Apr 12 20:21:09 2006 New Revision: 25734 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Log: Fix for addresses involved in FixedSizeArray-to-FixedSizeArray slicing. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Wed Apr 12 20:21:09 2006 @@ -218,7 +218,8 @@ ref = self.ref() if (isinstance(ref, _arrayitemref) and isinstance(EXPECTED_TYPE.TO, lltype.FixedSizeArray) and - isinstance(lltype.typeOf(ref.array).TO, lltype.Array)): + isinstance(lltype.typeOf(ref.array).TO, (lltype.FixedSizeArray, + lltype.Array))): # special case that requires cast_subarray_pointer return lltype.cast_subarray_pointer(EXPECTED_TYPE, ref.array, Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Wed Apr 12 20:21:09 2006 @@ -162,13 +162,14 @@ assert adr1 == adr2 def test_cast_subarray_pointer(): - A = lltype.GcArray(lltype.Signed) - a = lltype.malloc(A, 5) - a[3] = 132 - - SUBARRAY = lltype.FixedSizeArray(lltype.Signed, 1) - adr = cast_ptr_to_adr(a) + itemoffsetof(A, 3) - subarray = cast_adr_to_ptr(adr, lltype.Ptr(SUBARRAY)) - assert subarray[0] == 132 - subarray[0] += 2 - assert a[3] == 134 + for a in [lltype.malloc(lltype.GcArray(lltype.Signed), 5), + lltype.malloc(lltype.FixedSizeArray(lltype.Signed, 5), + immortal=True)]: + A = lltype.typeOf(a).TO + SUBARRAY = lltype.FixedSizeArray(lltype.Signed, 1) + a[3] = 132 + adr = cast_ptr_to_adr(a) + itemoffsetof(A, 3) + subarray = cast_adr_to_ptr(adr, lltype.Ptr(SUBARRAY)) + assert subarray[0] == 132 + subarray[0] += 2 + assert a[3] == 134 From tismer at codespeak.net Wed Apr 12 20:28:30 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 12 Apr 2006 20:28:30 +0200 (CEST) Subject: [pypy-svn] r25735 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060412182830.EA38910140@code0.codespeak.net> Author: tismer Date: Wed Apr 12 20:28:29 2006 New Revision: 25735 Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py Log: more simplification Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Wed Apr 12 20:28:29 2006 @@ -636,7 +636,6 @@ return v_inst def rtype_wrap_object_create(hop): - gencapi = hop.llops.gencapicall v_inst, c_spec = hop.inputargs(*hop.args_r) repr = c_spec.value v_res = into_cobject(v_inst, repr, hop.llops) @@ -644,7 +643,7 @@ v_cobj = v_res c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) c_0 = hop.inputconst(Signed, 0) - v_res = gencapi('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyobj_repr) + v_res = hop.llops.gencapicall('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyobj_repr) c_self = hop.inputconst(pyobj_repr, '__self__') hop.genop('setattr', [v_res, c_self, v_cobj], resulttype=pyobj_repr) repr.setfield(v_inst, '_wrapper_', v_res, hop.llops) @@ -697,8 +696,6 @@ if context is init: # saving an extra __new__ method, we create the instance on __init__ v_inst = r_to.new_instance(llops) - gencapi = llops.gencapicall - rtyper = llops.rtyper v_cobj = into_cobject(v_inst, r_to, llops) c_self = inputconst(pyobj_repr, '__self__') llops.genop('setattr', [v, c_self, v_cobj], resulttype=pyobj_repr) From antocuni at codespeak.net Wed Apr 12 21:43:16 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 12 Apr 2006 21:43:16 +0200 (CEST) Subject: [pypy-svn] r25738 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060412194316.BD1B710136@code0.codespeak.net> Author: antocuni Date: Wed Apr 12 21:43:10 2006 New Revision: 25738 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: Added the "extend" method to ootype.List Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Wed Apr 12 21:43:10 2006 @@ -182,7 +182,9 @@ class List(OOType): - ITEMTYPE_T = object() # placeholder, see below + # placeholders + SELFTYPE_T = object() + ITEMTYPE_T = object() def __init__(self, ITEMTYPE): self._ITEMTYPE = ITEMTYPE @@ -191,15 +193,20 @@ # This defines the abstract list interface that backends will # have to map to their native list implementations. # 'ITEMTYPE_T' is used as a placeholder for indicating - # arguments that should have ITEMTYPE type. + # arguments that should have ITEMTYPE type. 'SELFTYPE_T' indicates 'self' + + generic_types = { + self.SELFTYPE_T: self, + self.ITEMTYPE_T: ITEMTYPE, + } - generic_types = {self.ITEMTYPE_T: ITEMTYPE} self._GENERIC_METHODS = frozendict({ # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) "length": Meth([], Signed), "append": Meth([self.ITEMTYPE_T], Void), "getitem": Meth([Signed], self.ITEMTYPE_T), "setitem": Meth([Signed, self.ITEMTYPE_T], Void), + "extend": Meth([self.SELFTYPE_T], Void), }) self._setup_methods(generic_types) @@ -543,6 +550,11 @@ assert typeOf(index) == Signed self._list[index] = item + def extend(self, other): + # NOT_RPYTHON + assert typeOf(other) == typeOf(self) + self._list.extend(other._list) + class _null_list(_null_mixin(_list), _list): def __init__(self, LIST): Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Wed Apr 12 21:43:10 2006 @@ -33,6 +33,7 @@ self.lowleveltype.become(ootype.List(self.item_repr.lowleveltype)) def send_message(self, hop, message, can_raise=False): + print message v_args = hop.inputargs(self, *hop.args_r[1:]) c_name = hop.inputconst(ootype.Void, message) if can_raise: @@ -46,6 +47,9 @@ def rtype_method_append(self, hop): return self.send_message(hop, "append") + def rtype_method_extend(self, hop): + return self.send_message(hop, "extend") + def make_iterator_repr(self): return ListIteratorRepr(self) @@ -65,6 +69,7 @@ # XXX must handle negative indices return r_list.send_message(hop, "setitem", can_raise=True) + def newlist(llops, r_list, items_v): c_1ist = inputconst(ootype.Void, r_list.lowleveltype) v_result = llops.genop("new", [c_1ist], resulttype=r_list.lowleveltype) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Wed Apr 12 21:43:10 2006 @@ -18,6 +18,15 @@ l.append(1) assert l.length() == 1 +def test_extend(): + LT = List(Signed) + l1 = new(LT) + l2 = new(LT) + l1.append(1) + l2.append(2) + l1.extend(l2) + assert l1.length() == 2 + def test_setitem_getitem(): LT = List(Signed) l = new(LT) From arigo at codespeak.net Wed Apr 12 21:57:17 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 21:57:17 +0200 (CEST) Subject: [pypy-svn] r25739 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060412195717.B0EE510175@code0.codespeak.net> Author: arigo Date: Wed Apr 12 21:57:16 2006 New Revision: 25739 Modified: pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/c/primitive.py pypy/dist/pypy/translator/c/test/test_symbolic.py Log: Generally use cdecl(s,'') more systematically instead of s.replace('@','') which might leave behind a broken empty pair of parenthesis. Add a test and a fix for ArrayItemsOffset for FixedSizeArrays. Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Wed Apr 12 21:57:16 2006 @@ -377,7 +377,8 @@ OP_INDIRECT_CALL = OP_DIRECT_CALL def OP_UNSAFE_CALL(self, op): - line = '((%s (*)())(%s))();'%(self.lltypename(op.result).replace('@', ''), self.expr(op.args[0])) + line = '((%s (*)())(%s))();' % (cdecl(self.lltypename(op.result), ''), + self.expr(op.args[0])) if self.lltypemap(op.result) is not Void: r = self.expr(op.result) line = '%s = %s' % (r, line) @@ -586,19 +587,19 @@ #address operations def OP_RAW_STORE(self, op): - addr = self.expr(op.args[0]) - TYPE = op.args[1].value - offset = self.expr(op.args[2]) - value = self.expr(op.args[3]) - typename = self.db.gettype(TYPE).replace("@", "*") #XXX help! is this the way to do it? - return "*(((%(typename)s) %(addr)s ) + %(offset)s) = %(value)s;" % locals() + addr = self.expr(op.args[0]) + TYPE = op.args[1].value + offset = self.expr(op.args[2]) + value = self.expr(op.args[3]) + typename = cdecl(self.db.gettype(TYPE).replace('@', '*@'), '') + return "*(((%(typename)s) %(addr)s ) + %(offset)s) = %(value)s;" % locals() def OP_RAW_LOAD(self, op): addr = self.expr(op.args[0]) TYPE = op.args[1].value offset = self.expr(op.args[2]) result = self.expr(op.result) - typename = self.db.gettype(TYPE).replace("@", "*") #XXX see above + typename = cdecl(self.db.gettype(TYPE).replace('@', '*@'), '') return "%(result)s = *(((%(typename)s) %(addr)s ) + %(offset)s);" % locals() Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Wed Apr 12 21:57:16 2006 @@ -338,7 +338,8 @@ self.name = defnode.access_expr(parentnode.name, parentindex) self.ptrname = '(&%s)' % self.name if self.typename != self.implementationtypename: - self.ptrname = '((%s)(void*)%s)' % (self.typename.replace('@', '*'), + ptrtypename = db.gettype(Ptr(T)) + self.ptrname = '((%s)(void*)%s)' % (cdecl(ptrtypename, ''), self.ptrname) def forward_declaration(self): Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Wed Apr 12 21:57:16 2006 @@ -6,6 +6,7 @@ CompositeOffset, ArrayLengthOffset from pypy.rpython.memory.gc import GCHeaderOffset from pypy.rpython.memory.lladdress import NULL +from pypy.translator.c.support import cdecl # ____________________________________________________________ # @@ -17,19 +18,22 @@ if isinstance(value, FieldOffset): structnode = db.gettypedefnode(value.TYPE) return 'offsetof(%s, %s)'%( - db.gettype(value.TYPE).replace('@', ''), + cdecl(db.gettype(value.TYPE), ''), structnode.c_struct_field_name(value.fldname)) elif isinstance(value, ItemOffset): return '(sizeof(%s) * %s)'%( - db.gettype(value.TYPE).replace('@', ''), value.repeat) + cdecl(db.gettype(value.TYPE), ''), value.repeat) elif isinstance(value, ArrayItemsOffset): - return 'offsetof(%s, items)'%( - db.gettype(value.TYPE).replace('@', '')) + if isinstance(value.TYPE, FixedSizeArray): + return '0' + else: + return 'offsetof(%s, items)'%( + cdecl(db.gettype(value.TYPE), '')) elif isinstance(value, ArrayLengthOffset): return 'offsetof(%s, length)'%( - db.gettype(value.TYPE).replace('@', '')) + cdecl(db.gettype(value.TYPE), '')) elif isinstance(value, CompositeOffset): - return '%s + %s' % (name_signed(value.first, db), name_signed(value.second, db)) + return '(%s + %s)' % (name_signed(value.first, db), name_signed(value.second, db)) elif type(value) == AddressOffset: return '0' elif type(value) == GCHeaderOffset: Modified: pypy/dist/pypy/translator/c/test/test_symbolic.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_symbolic.py (original) +++ pypy/dist/pypy/translator/c/test/test_symbolic.py Wed Apr 12 21:57:16 2006 @@ -56,6 +56,26 @@ res = fn() assert res == 1234501234 +def test_itemoffsetof_fixedsizearray(): + ARRAY = lltype.FixedSizeArray(lltype.Signed, 5) + itemoffsets = [llmemory.itemoffsetof(ARRAY, i) for i in range(5)] + a = lltype.malloc(ARRAY, immortal=True) + def f(): + adr = llmemory.cast_ptr_to_adr(a) + result = 0 + for i in range(5): + a[i] = i + 1 + for i in range(5): + result = result * 10 + (adr + itemoffsets[i]).signed[0] + for i in range(5): + (adr + itemoffsets[i]).signed[0] = i + for i in range(5): + result = 10 * result + a[i] + return result + fn, t = getcompiled(f, []) + res = fn() + assert res == 1234501234 + def test_sizeof_constsize_struct(): # _not_ a GcStruct, since we want to raw_malloc it STRUCT = lltype.Struct("s", ("x", lltype.Signed), ("y", lltype.Signed)) From arigo at codespeak.net Wed Apr 12 21:57:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 21:57:35 +0200 (CEST) Subject: [pypy-svn] r25740 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060412195735.792DD10176@code0.codespeak.net> Author: arigo Date: Wed Apr 12 21:57:34 2006 New Revision: 25740 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: Yay! Progress on rctypes. * don't use addresses directly any more; CCHARP is now a Ptr(FixedSizeArray(Char, 1)). * casting from an array of chars to a CCHARP is based on the new cast_subarray_pointer(). * rarray can be done more naturally as an array of ll_typed items instead of an array of c_data_type boxes. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Wed Apr 12 21:57:34 2006 @@ -1,10 +1,12 @@ from ctypes import ARRAY, c_int from pypy.annotation.model import SomeCTypesObject, SomeBuiltin from pypy.rpython import extregistry +from pypy.rpython.rbuiltin import gen_cast_array_pointer from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype -from pypy.rpython.rctypes.rmodel import CTypesRefRepr, genreccopy, reccopy +from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr +from pypy.rpython.rctypes.rmodel import genreccopy, reccopy ArrayType = type(ARRAY(c_int, 10)) @@ -20,7 +22,7 @@ SomeCTypesObject.MEMORYALIAS)) # Here, self.c_data_type == self.ll_type - c_data_type = lltype.FixedSizeArray(self.r_item.c_data_type, + c_data_type = lltype.FixedSizeArray(self.r_item.ll_type, self.length) super(ArrayRepr, self).__init__(rtyper, s_array, c_data_type) @@ -28,12 +30,38 @@ def initialize_const(self, p, value): for i in range(self.length): llitem = self.r_item.convert_const(value[i]) - reccopy(llitem.c_data, p.c_data[i]) + if isinstance(self.r_item, CTypesRefRepr): + # ByRef case + reccopy(llitem.c_data, p.c_data[i]) + else: + # ByValue case + p.c_data[i] = llitem.c_data[0] def get_c_data_of_item(self, llops, v_array, v_index): v_c_array = self.get_c_data(llops, v_array) - return llops.genop('getarraysubstruct', [v_c_array, v_index], - lltype.Ptr(self.r_item.c_data_type)) + if isinstance(self.r_item, CTypesRefRepr): + # ByRef case + return llops.genop('getarraysubstruct', [v_c_array, v_index], + lltype.Ptr(self.r_item.c_data_type)) + else: + # ByValue case + A = lltype.FixedSizeArray(self.r_item.ll_type, 1) + return gen_cast_array_pointer(llops, lltype.Ptr(A), + v_c_array, v_index) + + def get_item_value(self, llops, v_array, v_index): + # ByValue case only + assert isinstance(self.r_item, CTypesValueRepr) + v_c_array = self.get_c_data(llops, v_array) + return llops.genop('getarrayitem', [v_c_array, v_index], + resulttype = self.r_item.ll_type) + + def set_item_value(self, llops, v_array, v_index, v_newvalue): + # ByValue case only + assert isinstance(self.r_item, CTypesValueRepr) + v_c_array = self.get_c_data(llops, v_array) + llops.genop('setarrayitem', [v_c_array, v_index, v_newvalue]) + class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_setitem((r_array, r_int), hop): Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Wed Apr 12 21:57:34 2006 @@ -1,8 +1,8 @@ from pypy.rpython import extregistry from pypy.rpython.rmodel import inputconst -from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython.lltypesystem import lltype from pypy.rpython.rstr import StringRepr, string_repr -from pypy.rpython.rctypes.rmodel import CTypesValueRepr +from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO from pypy.rpython.rctypes.rarray import ArrayRepr from pypy.annotation import model as annmodel from pypy.annotation.pairtype import pairtype @@ -53,43 +53,35 @@ r_temp.setstring(llops, v_owned_box, v) return llops.convertvar(v_owned_box, r_temp, r_to) -##class __extend__(pairtype(ArrayRepr, CCharPRepr)): -## def convert_from_to((r_from, r_to), v, llops): -## if r_from.r_item.ctype != c_char: -## return NotImplemented -## # warning: no keepalives, only for short-lived conversions like -## # in argument passing -## r_temp = r_to.r_memoryowner -## v_owned_box = r_temp.allocate_instance(llops) -## v_char_array_ptr = r_from.get_c_data(llops, v) -## v_char_array_adr = llops.genop('cast_ptr_to_adr', [v_char_array_ptr], -## resulttype = llmemory.Address) -## cofs = inputconst(lltype.Signed, -## llmemory.itemoffsetof(r_from.ll_type) + -## llmemory.offsetof(r_from.ll_type.OF, 'value')) -## v_first_char_adr = llops.genop('adr_add', [v_char_array_adr, cofs], -## resulttype = llmemory.Address) -## r_temp.setvalue(llops, v_owned_box, v_first_char_adr) -## return llops.convertvar(v_owned_box, r_temp, r_to) +class __extend__(pairtype(ArrayRepr, CCharPRepr)): + def convert_from_to((r_from, r_to), v, llops): + if r_from.r_item.ctype != c_char: + return NotImplemented + # warning: no keepalives, only for short-lived conversions like + # in argument passing + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + v_c_array = r_from.get_c_data_of_item(llops, v, C_ZERO) + r_temp.setvalue(llops, v_owned_box, v_c_array) + return llops.convertvar(v_owned_box, r_temp, r_to) -CCHARP = llmemory.Address # char * -FIRSTITEMOFS = llmemory.ArrayItemsOffset(string_repr.lowleveltype.TO.chars) +CCHARP = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) def ll_strlen(p): i = 0 - while ord(p.char[i]) != 0: + while ord(p[i]) != 0: i += 1 return i def ll_strnlen(p, maxlen): i = 0 - while i < maxlen and ord(p.char[i]) != 0: + while i < maxlen and ord(p[i]) != 0: i += 1 return i def ll_str2charp(s): - return llmemory.cast_ptr_to_adr(s.chars) + FIRSTITEMOFS + return lltype.cast_subarray_pointer(CCHARP, s.chars, 0) def ll_getstring(box): p = box.c_data[0] @@ -104,7 +96,7 @@ length = ll_strlen(p) newstr = lltype.malloc(string_repr.lowleveltype.TO, length) for i in range(length): - newstr.chars[i] = p.char[i] + newstr.chars[i] = p[i] return newstr else: return lltype.nullptr(string_repr.lowleveltype.TO) @@ -113,7 +105,7 @@ if string: box.c_data[0] = ll_str2charp(string) else: - box.c_data[0] = llmemory.NULL + box.c_data[0] = lltype.nullptr(CCHARP.TO) box.keepalive_str = string Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Wed Apr 12 21:57:34 2006 @@ -124,8 +124,8 @@ res = interpret(create_array, []) c_data = res.c_data - assert c_data[0][0] == 0 - assert c_data[9][0] == 0 + assert c_data[0] == 0 + assert c_data[9] == 0 py.test.raises(IndexError, "c_data[10]") assert len(c_data) == 10 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Wed Apr 12 21:57:34 2006 @@ -5,7 +5,7 @@ from pypy.translator.c.test.test_genc import compile from pypy import conftest from pypy.rpython.rstr import string_repr -from pypy.rpython.lltypesystem import lltype, llmemory +from pypy.rpython.lltypesystem import lltype from ctypes import c_int, c_long, c_char_p, c_char @@ -20,11 +20,11 @@ atoi = mylib.atoi atoi.restype = c_int atoi.argtypes = [c_char_p] -def ll_atoi(s_addr): +def ll_atoi(p): "Very approximative ll implementation of atoi(), for testing" i = result = 0 - while '0' <= s_addr.char[i] <= '9': - result = result * 10 + ord(s_addr.char[i]) - ord('0') + while '0' <= p[i] <= '9': + result = result * 10 + ord(p[i]) - ord('0') i += 1 return result atoi.llinterp_friendly_version = ll_atoi @@ -49,15 +49,15 @@ def test_ll_atoi(): keepalive = [] - def str2addr(string): + def str2subarray(string): llstring = string_repr.convert_const(string) keepalive.append(llstring) - return (llmemory.cast_ptr_to_adr(llstring.chars) + - llmemory.ArrayItemsOffset(lltype.typeOf(llstring.chars).TO)) - assert ll_atoi(str2addr("")) == 0 - assert ll_atoi(str2addr("42z7")) == 42 - assert ll_atoi(str2addr("blah")) == 0 - assert ll_atoi(str2addr("18238")) == 18238 + A = lltype.FixedSizeArray(lltype.Char, 1) + return lltype.cast_subarray_pointer(lltype.Ptr(A), llstring.chars, 0) + assert ll_atoi(str2subarray("")) == 0 + assert ll_atoi(str2subarray("42z7")) == 42 + assert ll_atoi(str2subarray("blah")) == 0 + assert ll_atoi(str2subarray("18238")) == 18238 class Test_annotation: def test_annotate_labs(self): @@ -90,7 +90,6 @@ assert res == [0, 42, 0, 18238] def test_specialize_atoi_char_array(self): - py.test.skip("in-progress") A = c_char * 10 choices = [A('\x00'), A('4', '2', 'z', '7', '\x00'), From arigo at codespeak.net Wed Apr 12 22:02:34 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 12 Apr 2006 22:02:34 +0200 (CEST) Subject: [pypy-svn] r25741 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060412200234.A6B9810181@code0.codespeak.net> Author: arigo Date: Wed Apr 12 22:02:33 2006 New Revision: 25741 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py Log: Some optimizations for the common cases of arrays (essentially avoiding the address hackery). Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Wed Apr 12 22:02:33 2006 @@ -7,6 +7,7 @@ from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr from pypy.rpython.rctypes.rmodel import genreccopy, reccopy +from pypy.rpython.rctypes.rprimitive import PrimitiveRepr ArrayType = type(ARRAY(c_int, 10)) @@ -67,13 +68,23 @@ def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item) - v_item_c_data = r_array.r_item.get_c_data(hop.llops, v_item) - v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) - # copy the whole structure's content over - genreccopy(hop.llops, v_item_c_data, v_c_data) + if isinstance(r_array.r_item, CTypesRefRepr): + # ByRef case + v_item_c_data = r_array.r_item.get_c_data(hop.llops, v_item) + v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) + # copy the whole structure's content over + genreccopy(hop.llops, v_item_c_data, v_c_data) + else: + # ByValue case (optimization; the above also works in this case) + v_newvalue = r_array.r_item.getvalue(hop.llops, v_item) + r_array.set_item_value(hop.llops, v_array, v_index, v_newvalue) def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) + if isinstance(r_array.r_item, PrimitiveRepr): + # primitive case (optimization only) + return r_array.get_item_value(hop.llops, v_array, v_index) + # normal case v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) return r_array.r_item.return_c_data(hop.llops, v_c_data) From antocuni at codespeak.net Wed Apr 12 22:15:12 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 12 Apr 2006 22:15:12 +0200 (CEST) Subject: [pypy-svn] r25742 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060412201512.95CE410190@code0.codespeak.net> Author: antocuni Date: Wed Apr 12 22:15:03 2006 New Revision: 25742 Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/runtest.py pypy/dist/pypy/translator/cli/test/test_list.py Log: Added support for list.extend. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Wed Apr 12 22:15:03 2006 @@ -14,6 +14,7 @@ log = py.log.Producer("cli") py.log.setconsumer("cli", ansi_log) +PYPY_LIST = '[pypylib]pypy.runtime.List`1<%s>' _lltype_to_cts = { Void: 'void', @@ -26,6 +27,7 @@ Class: 'class [mscorlib]System.Type', # TODO: it seems a hack + List.SELFTYPE_T: 'class ' + (PYPY_LIST % '!0'), List.ITEMTYPE_T: '!0', } @@ -63,7 +65,7 @@ return 'void' # TODO: is it correct to ignore StaticMethod? elif isinstance(t, List): item_type = self.lltype_to_cts(t._ITEMTYPE) - return self.__class('[pypylib]pypy.runtime.List`1<%s>' % item_type, include_class) + return self.__class(PYPY_LIST % item_type, include_class) return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Wed Apr 12 22:15:03 2006 @@ -12,11 +12,6 @@ public class List: System.Collections.Generic.List { - public void append(T item) - { - this.Add(item); - } - public int length() { return this.Count; @@ -31,5 +26,16 @@ { this[index] = value_; } + + public void append(T item) + { + this.Add(item); + } + + public void extend(List other) + { + this.AddRange(other); + } + } } Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Wed Apr 12 22:15:03 2006 @@ -25,15 +25,14 @@ def bar(x, y): - try: - return ovfcheck(x+y) - except OverflowError: - return 0 + lst = [1,2,3,x,y] + lst.extend([1,2]) + return 0 f = compile_function(bar, [int, int]) try: - check(bar, f, sys.maxint, 13) + check(bar, f, 42, 13) except py.test.Item.Skipped: print 'Test skipped' Modified: pypy/dist/pypy/translator/cli/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/runtest.py (original) +++ pypy/dist/pypy/translator/cli/test/runtest.py Wed Apr 12 22:15:03 2006 @@ -102,7 +102,7 @@ self.graph = t.graphs[0] if getoption('view'): - t.viewcg() + t.view() if getoption('wd'): self.tmpdir = py.path.local('.') Modified: pypy/dist/pypy/translator/cli/test/test_list.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_list.py (original) +++ pypy/dist/pypy/translator/cli/test/test_list.py Wed Apr 12 22:15:03 2006 @@ -28,3 +28,21 @@ lst[0] = 0 lst[1] = 0 return sum_(lst) + +def list_iteration(x, y): + lst = create(x, y) + total = 1 + for item in lst: + total *= item + return total + +def list_concat(x, y): + lst1 = create(x, x) + lst2 = create(y, y) + return sum_(lst1 + lst2) + +def list_extend(x, y): + lst = create(x, y) + lst += [y, y*2] + lst.extend([x, y]) + return sum_(lst) From pedronis at codespeak.net Wed Apr 12 23:16:57 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 12 Apr 2006 23:16:57 +0200 (CEST) Subject: [pypy-svn] r25744 - pypy/dist/pypy/translator Message-ID: <20060412211657.AE11510196@code0.codespeak.net> Author: pedronis Date: Wed Apr 12 23:16:56 2006 New Revision: 25744 Modified: pypy/dist/pypy/translator/unsimplify.py Log: split_block should not mutate space ops in place Modified: pypy/dist/pypy/translator/unsimplify.py ============================================================================== --- pypy/dist/pypy/translator/unsimplify.py (original) +++ pypy/dist/pypy/translator/unsimplify.py Wed Apr 12 23:16:56 2006 @@ -59,11 +59,17 @@ varmap[var] = copyvar(translator, var) return varmap[var] moved_operations = block.operations[index:] + new_moved_ops = [] for op in moved_operations: - for i, arg in enumerate(op.args): - op.args[i] = get_new_name(op.args[i]) + newop = SpaceOperation(op.opname, + [get_new_name(arg) for arg in op.args], + op.result) + new_moved_ops.append(newop) vars_produced_in_new_block[op.result] = True - for link in block.exits: + moved_operations = new_moved_ops + links = block.exits + block.exits = None + for link in links: for i, arg in enumerate(link.args): #last_exception and last_exc_value are considered to be created #when the link is entered @@ -74,7 +80,7 @@ #from block the old block newblock = Block(varmap.values()) newblock.operations = moved_operations - newblock.exits = block.exits + newblock.exits = links newblock.exitswitch = exitswitch newblock.exc_handler = block.exc_handler for link in newblock.exits: From pedronis at codespeak.net Wed Apr 12 23:19:06 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 12 Apr 2006 23:19:06 +0200 (CEST) Subject: [pypy-svn] r25745 - in pypy/dist/pypy: rpython/memory translator/backendopt translator/c Message-ID: <20060412211906.7B22A1019C@code0.codespeak.net> Author: pedronis Date: Wed Apr 12 23:19:04 2006 New Revision: 25745 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/c/exceptiontransform.py Log: avoid to leak variables in annotator.bindings for backendopts and gc/exc tranforms Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Wed Apr 12 23:19:04 2006 @@ -82,6 +82,7 @@ else: insert_empty_block(None, link, newops) + self.links_to_split = None v = Variable('vanishing_exc_value') v.concretetype = self.get_lltype_of_exception_value() graph.exc_cleanup = (v, self.pop_alive(v)) Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Wed Apr 12 23:19:04 2006 @@ -183,13 +183,13 @@ if isinstance(var, Constant): return var if var not in self.varmap: - self.varmap[var] = copyvar(self.translator, var) + self.varmap[var] = copyvar(None, var) return self.varmap[var] def passon_vars(self, cache_key): if cache_key in self._passon_vars: return self._passon_vars[cache_key] - result = [copyvar(self.translator, var) + result = [copyvar(None, var) for var in self.original_passon_vars] self._passon_vars[cache_key] = result return result @@ -312,13 +312,12 @@ exc_match.concretetype = typeOf(exc_match.value) blocks = [] for i, link in enumerate(afterblock.exits[1:]): - etype = copyvar(self.translator, copiedexceptblock.inputargs[0]) - evalue = copyvar(self.translator, copiedexceptblock.inputargs[1]) + etype = copyvar(None, copiedexceptblock.inputargs[0]) + evalue = copyvar(None, copiedexceptblock.inputargs[1]) passon_vars = self.passon_vars(i) block = Block([etype, evalue] + passon_vars) res = Variable() res.concretetype = Bool - self.translator.annotator.bindings[res] = annmodel.SomeBool() cexitcase = Constant(link.llexitcase) cexitcase.concretetype = typeOf(cexitcase.value) args = [exc_match, etype, cexitcase] @@ -350,8 +349,7 @@ def do_inline(self, block, index_operation): - splitlink = split_block_with_keepalive( - self.translator, block, index_operation) + splitlink = split_block_with_keepalive(block, index_operation) afterblock = splitlink.target # these variables have to be passed along all the links in the inlined # graph because the original function needs them in the blocks after @@ -360,7 +358,8 @@ # this copy is created with the method passon_vars self.original_passon_vars = [arg for arg in block.exits[0].args if isinstance(arg, Variable)] - assert afterblock.operations[0] is self.op + assert afterblock.operations[0].opname is self.op.opname + self.op = afterblock.operations[0] #vars that need to be passed through the blocks of the inlined function linktoinlined = splitlink copiedstartblock = self.copy_block(self.graph_to_inline.startblock) Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Wed Apr 12 23:19:04 2006 @@ -62,9 +62,9 @@ keepalive_ops.append(SpaceOperation('keepalive', [v], v_keepalive)) return keepalive_ops -def split_block_with_keepalive(translator, block, index_operation, +def split_block_with_keepalive(block, index_operation, keep_alive_op_args=True): - splitlink = split_block(translator, block, index_operation) + splitlink = split_block(None, block, index_operation) afterblock = splitlink.target conservative_keepalives = needs_conservative_livevar_calculation(block) if conservative_keepalives: @@ -78,7 +78,7 @@ newvar = afterblock.inputargs[index] except ValueError: splitlink.args.append(var) - newvar = copyvar(translator, var) + newvar = copyvar(None, var) afterblock.inputargs.append(newvar) keep_alive_vars[i] = newvar elif keep_alive_op_args and afterblock.operations: @@ -88,8 +88,8 @@ keep_alive_vars = [] if afterblock.exitswitch == c_last_exception: for link in afterblock.exits: - betweenblock = insert_empty_block(translator, link) - fresh_vars = [copyvar(translator, var) for var in keep_alive_vars] + betweenblock = insert_empty_block(None, link) + fresh_vars = [copyvar(None, var) for var in keep_alive_vars] betweenblock.inputargs.extend(fresh_vars) link.args.extend(keep_alive_vars) betweenblock.operations = generate_keepalive(fresh_vars) Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Wed Apr 12 23:19:04 2006 @@ -152,8 +152,7 @@ if not self.raise_analyzer.can_raise(op): continue - splitlink = support.split_block_with_keepalive( - self.translator, block, i+1, False) + splitlink = support.split_block_with_keepalive(block, i+1, False) afterblock = splitlink.target if lastblock is block: lastblock = afterblock From pedronis at codespeak.net Wed Apr 12 23:20:38 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 12 Apr 2006 23:20:38 +0200 (CEST) Subject: [pypy-svn] r25746 - pypy/dist/pypy/translator/c Message-ID: <20060412212038.29A1C101A0@code0.codespeak.net> Author: pedronis Date: Wed Apr 12 23:20:36 2006 New Revision: 25746 Modified: pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py Log: don't leak variables or blocks in the case the original function graph has been copied to inline gc helpers Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Wed Apr 12 23:20:36 2006 @@ -96,6 +96,10 @@ newgraph = copygraph(self.graph, shallow=True) self.oldgraph = self.graph self.graph = newgraph + ## protect + #for block in self.oldgraph.iterblocks(): + # for op in block.operations: + # op.args = tuple(op.args) self.db.gctransformer.inline_helpers(newgraph) self.collect_var_and_types() else: @@ -113,7 +117,9 @@ def implementation_end(self): self.lltypes = None + self.vars = None self.blocknum = None + self.currentblock = None self.graph = self.oldgraph del self.oldgraph Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Wed Apr 12 23:20:36 2006 @@ -309,6 +309,7 @@ continue cost = len(impl) + nbetween yield node, impl + del impl if used + cost > split_criteria: # split if criteria met, unless we would produce nothing. raise StopIteration Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Wed Apr 12 23:20:36 2006 @@ -603,6 +603,7 @@ yield line yield '}' + del bodyiter funcgen.implementation_end() assert not USESLOTS or '__dict__' not in dir(FuncNode) From tismer at codespeak.net Thu Apr 13 00:16:55 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 13 Apr 2006 00:16:55 +0200 (CEST) Subject: [pypy-svn] r25748 - pypy/dist/pypy/translator/c Message-ID: <20060412221655.A8F1A101A9@code0.codespeak.net> Author: tismer Date: Thu Apr 13 00:16:53 2006 New Revision: 25748 Modified: pypy/dist/pypy/translator/c/pyobj.py Log: the annotator does not support asking for a classdef when something is derived from other than object. Avoiding to touch this. Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Thu Apr 13 00:16:53 2006 @@ -335,9 +335,10 @@ return name def nameof_classobj(self, cls): - clsdef = self.translator.annotator.bookkeeper.getuniqueclassdef(cls) - if self.translator.rtyper.needs_wrapper(clsdef): - return self.wrap_exported_class(cls) + if builtin_base(cls) is object: + clsdef = self.translator.annotator.bookkeeper.getuniqueclassdef(cls) + if self.translator.rtyper.needs_wrapper(clsdef): + return self.wrap_exported_class(cls) if cls.__doc__ and cls.__doc__.lstrip().startswith('NOT_RPYTHON'): raise Exception, "%r should never be reached" % (cls,) From pedronis at codespeak.net Thu Apr 13 00:29:44 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 13 Apr 2006 00:29:44 +0200 (CEST) Subject: [pypy-svn] r25749 - pypy/dist/pypy/translator Message-ID: <20060412222944.4CDA0101AD@code0.codespeak.net> Author: pedronis Date: Thu Apr 13 00:29:43 2006 New Revision: 25749 Modified: pypy/dist/pypy/translator/gensupp.py Log: a pure type level version of builtin_base Modified: pypy/dist/pypy/translator/gensupp.py ============================================================================== --- pypy/dist/pypy/translator/gensupp.py (original) +++ pypy/dist/pypy/translator/gensupp.py Thu Apr 13 00:29:43 2006 @@ -54,6 +54,9 @@ def builtin_base(obj): typ = type(obj) + return builtin_type_base(typ) + +def builtin_type_base(typ): from copy_reg import _HEAPTYPE while typ.__flags__&_HEAPTYPE: typ = typ.__base__ From pedronis at codespeak.net Thu Apr 13 00:31:34 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 13 Apr 2006 00:31:34 +0200 (CEST) Subject: [pypy-svn] r25750 - pypy/dist/pypy/translator/c Message-ID: <20060412223134.9771D101B1@code0.codespeak.net> Author: pedronis Date: Thu Apr 13 00:31:33 2006 New Revision: 25750 Modified: pypy/dist/pypy/translator/c/pyobj.py Log: check for old-style class or subclass of object (not another builtin type) Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Thu Apr 13 00:31:33 2006 @@ -3,7 +3,7 @@ from types import FunctionType, CodeType, InstanceType, ClassType from pypy.objspace.flow.model import Variable, Constant -from pypy.translator.gensupp import builtin_base +from pypy.translator.gensupp import builtin_base, builtin_type_base from pypy.translator.c.support import log from pypy.translator.c.wrapper import gen_wrapper @@ -335,7 +335,7 @@ return name def nameof_classobj(self, cls): - if builtin_base(cls) is object: + if isinstance(cls, ClassType) or builtin_type_base(cls) is object: clsdef = self.translator.annotator.bookkeeper.getuniqueclassdef(cls) if self.translator.rtyper.needs_wrapper(clsdef): return self.wrap_exported_class(cls) From pedronis at codespeak.net Thu Apr 13 01:08:09 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 13 Apr 2006 01:08:09 +0200 (CEST) Subject: [pypy-svn] r25752 - pypy/dist/pypy/translator/stackless Message-ID: <20060412230809.EF087101AF@code0.codespeak.net> Author: pedronis Date: Thu Apr 13 01:08:08 2006 New Revision: 25752 Modified: pypy/dist/pypy/translator/stackless/transform.py Log: oops, signature change Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Thu Apr 13 01:08:08 2006 @@ -317,8 +317,7 @@ and block.exitswitch == model.c_last_exception: link = block.exits[0] else: - link = support.split_block_with_keepalive( - self.translator, block, i+1) + link = support.split_block_with_keepalive(block, i+1) block.exitswitch = model.c_last_exception link.llexitcase = None var_unwind_exception = varoftype(evalue) From tismer at codespeak.net Thu Apr 13 08:25:11 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 13 Apr 2006 08:25:11 +0200 (CEST) Subject: [pypy-svn] r25766 - in pypy/dist/pypy/translator/c: . src test Message-ID: <20060413062511.C0F4310082@code0.codespeak.net> Author: tismer Date: Thu Apr 13 08:25:00 2006 New Revision: 25766 Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/src/module.h pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/wrapper.py Log: nicer wrapped function names, support for docstrings everywhere Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Thu Apr 13 08:25:00 2006 @@ -12,7 +12,7 @@ from pypy.tool.udir import udir from pypy.tool import isolate from pypy.translator.locality.calltree import CallTree -from pypy.translator.c.support import log +from pypy.translator.c.support import log, c_string_constant from pypy.rpython.typesystem import getfunctionptr class CBuilder(object): @@ -669,13 +669,14 @@ print >> f, 'static globalfunctiondef_t globalfunctiondefs[] = {' wrappers = pyobjmaker.wrappers.items() wrappers.sort() - for globalobject_name, (base_name, wrapper_name) in wrappers: + for globalobject_name, (base_name, wrapper_name, func_doc) in wrappers: print >> f, ('\t{&%s, "%s", {"%s", (PyCFunction)%s, ' - 'METH_VARARGS|METH_KEYWORDS}},' % ( + 'METH_VARARGS|METH_KEYWORDS, %s}},' % ( globalobject_name, globalobject_name, base_name, - wrapper_name)) + wrapper_name, + c_string_constant(func_doc or ''))) print >> f, '\t{ NULL }\t/* Sentinel */' print >> f, '};' print >> f, 'static globalfunctiondef_t *globalfunctiondefsptr = &globalfunctiondefs[0];' Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Thu Apr 13 08:25:00 2006 @@ -38,6 +38,7 @@ self.wrappers = {} # {'pycfunctionvariable': ('name', 'wrapperfn')} self.import_hints = {} # I don't seem to need it any longer. # leaving the import support intact, doesn't hurt. + self.name_for_meth = {} # get nicer wrapper names def nameof(self, obj, debug=None): if debug: @@ -208,9 +209,9 @@ if self.shouldskipfunc(func): return self.skipped_function(func) - fwrapper = gen_wrapper(func, self.translator) + fwrapper = gen_wrapper(func, self.translator, self.name_for_meth.get(func, func.__name__)) pycfunctionobj = self.uniquename('gfunc_' + func.__name__) - self.wrappers[pycfunctionobj] = func.__name__, self.getvalue(fwrapper) + self.wrappers[pycfunctionobj] = func.__name__, self.getvalue(fwrapper), func.__doc__ return pycfunctionobj def import_function(self, func): @@ -538,14 +539,14 @@ content.sort() for key, value in content: if key.startswith('__'): - if key in ['__module__', '__doc__', '__dict__', + if key in ['__module__', '__dict__', '__doc__', '__weakref__', '__repr__', '__metaclass__']: continue if self.shouldskipfunc(value): log.WARNING("skipped class function: %r" % value) continue -# yield '%s.%s = property(lambda self:%s.__get__(self.__self__))' % ( -# name, key, self.nameof(value)) + if callable(value): + self.name_for_meth[value] = '%s.%s' % (cls.__name__, value.__name__) yield '%s.%s = %s' % (name, key, self.nameof(value)) baseargs = ", ".join(basenames) @@ -554,6 +555,8 @@ a = self.initcode.append a('class %s%s:' % (name, baseargs) ) + if cls.__doc__: + a(' %r' % str(cls.__doc__) ) a(' __metaclass__ = type') a(' __slots__ = ["__self__"] # for PyCObject') self.later(initclassobj()) Modified: pypy/dist/pypy/translator/c/src/module.h ============================================================================== --- pypy/dist/pypy/translator/c/src/module.h (original) +++ pypy/dist/pypy/translator/c/src/module.h Thu Apr 13 08:25:00 2006 @@ -24,6 +24,7 @@ PyModule_AddStringConstant(m, "__sourcefile__", __FILE__); \ this_module_globals = PyModule_GetDict(m); \ PyGenCFunction_Type.tp_base = &PyCFunction_Type; \ + PyGenCFunction_Type.tp_getset = PyCFunction_Type.tp_getset; \ PyType_Ready(&PyGenCFunction_Type); \ RPythonError = PyErr_NewException(#modname ".RPythonError", \ NULL, NULL); \ Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Thu Apr 13 08:25:00 2006 @@ -43,6 +43,11 @@ if isinstance(obj, type): clsdef = bk.getuniqueclassdef(obj) rtyper.add_wrapper(clsdef) + # pull it all out + for name, value in obj.__dict__.items(): + continue # do we want that + if callable(value): + t.annotator.build_types(value, get_annotation(value)) elif callable(obj): t.annotator.build_types(obj, get_annotation(obj)) if view: @@ -231,12 +236,14 @@ # a trivial class to be exposed class DemoClass(DemoBaseNotExposed): + """this is the doc string""" def __init__(self, a, b): self.a = a self.b = b if P:print 'init' def demo(self): + """this is the doc for demo""" if P:print 'demo' return self.a + self.b @@ -353,6 +360,22 @@ res = obj.demo() assert res == DemoClass(2, 3).demo() +def t(a=int, b=int, c=DemoClass): + DemoClass(a, b).demo() + DemoSubclass(a, a, b).demo() + DemoSubclass(a, a, b).demo(6) + DemoSubclass(a, a, b).demo(6, 'hu') + if isinstance(c, DemoSubclass): + print 42 + +# exposing and using classes from a generasted extension module +def test_asd(): + m = get_compiled_module(t, use_boehm=not True, exports=[ + DemoClass, DemoSubclass, DemoNotAnnotated, setup_new_module]) + obj = m.DemoClass(2, 3) + res = obj.demo() + assert res == DemoClass(2, 3).demo() + if __name__=='__main__': test_expose_classes() Modified: pypy/dist/pypy/translator/c/wrapper.py ============================================================================== --- pypy/dist/pypy/translator/c/wrapper.py (original) +++ pypy/dist/pypy/translator/c/wrapper.py Thu Apr 13 08:25:00 2006 @@ -10,7 +10,7 @@ from pypy.rpython.typesystem import getfunctionptr -def gen_wrapper(func, translator): +def gen_wrapper(func, translator, newname=None): """generate a wrapper function for 'func' that can be put in a PyCFunction object. The wrapper has signature @@ -121,7 +121,7 @@ # "return result" block = Block([vself, vargs, vkwds]) - wgraph = FunctionGraph('pyfn_' + func.func_name, block) + wgraph = FunctionGraph('pyfn_' + (newname or func.func_name), block) translator.update_call_graph(wgraph, graph, object()) translator.graphs.append(wgraph) block.operations[:] = newops From arigo at codespeak.net Thu Apr 13 09:03:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 09:03:01 +0200 (CEST) Subject: [pypy-svn] r25767 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060413070301.D7B6510083@code0.codespeak.net> Author: arigo Date: Thu Apr 13 09:02:58 2006 New Revision: 25767 Removed: pypy/dist/pypy/rpython/rctypes/interface.py Modified: pypy/dist/pypy/rpython/rctypes/__init__.py pypy/dist/pypy/rpython/rctypes/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c (contents, props changed) pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Log: Reorganized the rctypes modules to match our coding style. Modified: pypy/dist/pypy/rpython/rctypes/__init__.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/__init__.py (original) +++ pypy/dist/pypy/rpython/rctypes/__init__.py Thu Apr 13 09:02:58 2006 @@ -1 +1 @@ -from pypy.rpython.rctypes.interface import * +#empty Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/ctypes_platform.py Thu Apr 13 09:02:58 2006 @@ -238,19 +238,19 @@ opts, args = getopt.gnu_getopt(sys.argv[1:], 'h:') if not args: print >> sys.stderr, doc - sys.exit(2) - assert len(args) % 2 == 1 - headers = [] - for opt, value in opts: - if opt == '-h': - headers.append('#include <%s>' % (value,)) - name = args[0] - fields = [] - for i in range(1, len(args), 2): - ctype = getattr(ctypes, args[i+1]) - fields.append((args[i], ctype)) + else: + assert len(args) % 2 == 1 + headers = [] + for opt, value in opts: + if opt == '-h': + headers.append('#include <%s>' % (value,)) + name = args[0] + fields = [] + for i in range(1, len(args), 2): + ctype = getattr(ctypes, args[i+1]) + fields.append((args[i], ctype)) - S = getstruct(name, '\n'.join(headers), fields) + S = getstruct(name, '\n'.join(headers), fields) - for key, value in S._fields_: - print key, value + for key, value in S._fields_: + print key, value Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Thu Apr 13 09:02:58 2006 @@ -1,464 +1,21 @@ -""" -The rctypes implementaion is contained here. -""" - -import sys -import ctypes -from ctypes import * -from ctypes import _FUNCFLAG_CDECL -if sys.platform == "win32": - from ctypes import _FUNCFLAG_STDCALL -from pypy.annotation.model import SomeInteger, SomeCTypesObject, \ - SomeString, SomeFloat, SomeBuiltin -from pypy.rpython.lltypesystem.lltype import Signed, SignedLongLong, \ - Unsigned, UnsignedLongLong, Char, Float, Ptr, \ - GcStruct, Struct, \ - Void -from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst -from pypy.rpython.error import TyperError -from pypy.rpython import extregistry -from pypy.annotation.pairtype import pairtype -from pypy.rpython import rint - # Importing for side effect of registering types with extregistry -import pypy.rpython.rctypes.rarray import pypy.rpython.rctypes.rprimitive import pypy.rpython.rctypes.rpointer +import pypy.rpython.rctypes.rarray import pypy.rpython.rctypes.rfunc import pypy.rpython.rctypes.rchar_p -# ctypes_annotation_list contains various attributes that -# are used by the pypy annotation. - -ctypes_annotation_list = [ -# (c_char, Char, None), -# (c_byte, Signed, None), -# (c_ubyte, Unsigned, None), -# (c_short, Signed, None), -# (c_ushort, Unsigned, None), -# (c_int, Signed, None), -# (c_uint, Unsigned, None), -# (c_long, Signed, None), -# (c_ulong, Unsigned, None), -# (c_longlong, SignedLongLong, None), -# (c_ulonglong, UnsignedLongLong, None), -# (c_float, Float, None), -# (c_double, Float, None), -# (c_char_p, None, -# staticmethod(lambda ll_type, arg_name:"RPyString_AsString(%s)" % arg_name)), -# (POINTER(c_char), None, -# staticmethod(lambda ll_type, arg_name:"RPyString_AsString(%s)" % arg_name)), -] - -def create_ctypes_annotations(): - """ - create_ctypes_annotation creates a map between - ctypes, annotation types and low level types. - For convenience, an existing map from low level types to - annotation types is used ('ll_to_annotation_map'). - """ - - from pypy.annotation.model import ll_to_annotation_map - for the_type, ll_type, wrap_arg in ctypes_annotation_list: - the_type.annotator_type = ll_to_annotation_map.get(ll_type) - the_type.ll_type = ll_type - if wrap_arg is not None: - the_type.wrap_arg = wrap_arg - the_type.default_memorystate = SomeCTypesObject.OWNSMEMORY - the_type.external_function_result_memorystate = SomeCTypesObject.MEMORYALIAS - else: - # !!!! attention !!!! - # the basic c_types need some annotation information - # at the moment that are exactly the types that have - # no 'wrap_arg'. This might change in the future - #the_type.compute_result_annotation = classmethod(lambda cls, s_arg:SomeCTypesObject(cls)) - def do_register(the_type): - extregistry.register_value(the_type, - compute_result_annotation=lambda s_arg: SomeCTypesObject(the_type)) - # XXX we need to register the correct repr for each primitive - extregistry.register_type(the_type, - get_repr=lambda rtyper, s_primitive: rint.signed_repr) - do_register(the_type) - the_type.default_memorystate = SomeCTypesObject.NOMEMORY - -create_ctypes_annotations() - - -class FunctionPointerTranslation(object): - - def compute_result_annotation(self, *args_s): - """ - Answer the annotation of the external function's result - """ - # Take 3, Check whether we can get away with the cheap - # precomputed solution and if not it, use a special - # attribute with the memory state - try: - return self.restype.annotator_type - except AttributeError: - return SomeCTypesObject( - self.restype, - self.restype.external_function_result_memorystate ) - # Take 2, looks like we need another level of indirection - # That's to complicated - #o#return self.restype.compute_external_function_result_annotator_type() - # TODO: Check whether the function returns a pointer - # an correct the memory state appropriately - try: - return self.restype.annotator_type - except AttributeError: - return SomeCTypesObject(self.restype) - - def __hash__(self): - return id(self) - - def specialize(self, hop): - return hop.llops.gencapicall( - self.__name__, - hop.args_v, - resulttype = self.restype.ll_type, - _callable=None, - convert_params = self.convert_params ) - - def convert_params(self, backend, param_info_list): - assert "c" == backend.lower() - assert self.argtypes is not None - answer = [] - for ctype_type, (ll_type, arg_name) in zip(self.argtypes, param_info_list): - if ll_type == ctype_type.ll_type: - answer.append(arg_name) - else: - answer.append(ctype_type.wrap_arg(ll_type, arg_name)) - return answer -# -#class CtypesBasicTypeInstantiationTranslation( FunctionPointerTranslation ): -# -# compute_result_annotation = classmethod( -# FunctionPointerTranslation.compute_result_annotation) -# - -class RStructureMeta(type(Structure)): - def __new__(mta,name,bases,clsdict): - _fields = clsdict.get('_fields_',None) - _adict = {} - ll_types = [] - if _fields is not None: - for attr, atype in _fields: - _adict[attr] = atype - ll_types.append( ( attr, atype.ll_type ) ) - clsdict['_fields_def_'] = _adict - # ll_type is just the C-level data part of the structure - clsdict[ "ll_type" ] = Struct( "C-Data_%s" % name, *ll_types ) - #d#print "_fields_def_ s:", _adict - - return super(RStructureMeta,mta).__new__(mta, name, bases, clsdict) - - -class RStructure(Structure): - - __metaclass__ = RStructureMeta - - default_memorystate = SomeCTypesObject.OWNSMEMORY - external_function_result_memorystate = default_memorystate - - def compute_annotation(cls): - return SomeCTypesObject(cls) - compute_annotation = classmethod(compute_annotation) - - def specialize( cls, highLevelOperation ): - ctypesStructureType = highLevelOperation.r_result.lowleveltype - return highLevelOperation.llops.genop( - "malloc", [ inputconst( Void, ctypesStructureType ) ], - highLevelOperation.r_result ) - if False: - def specialize( cls, highLevelOperation ): - ctypesStructureType = highLevelOperation.r_result.lowleveltype - answer = highLevelOperation.llops.genop( - "malloc", [ inputconst( Void, ctypesStructureType ) ], - highLevelOperation.r_result ) - import pdb - pdb.set_trace() - return answer - specialize = classmethod(specialize) - - def compute_result_annotation(cls, *args_s): - """ - Answer the result annotation of calling 'cls'. - """ - return SomeCTypesObject(cls,SomeCTypesObject.OWNSMEMORY) - compute_result_annotation = classmethod(compute_result_annotation) - - def createLowLevelRepresentation( rtyper, annotationObject ): - """ - Answer the correspondending low level object. - """ - if annotationObject.memorystate == annotationObject.OWNSMEMORY: - return CtypesMemoryOwningStructureRepresentation( - rtyper, annotationObject ) - elif annotationObject.memorystate == annotationObject.MEMORYALIAS: - return CtypesMemoryAliasStructureRepresentation( - rtyper, annotationObject ) - else: - raise TyperError( "Unkown memory state in %r" % annotationObject ) - createLowLevelRepresentation = staticmethod( createLowLevelRepresentation ) - - -class RByrefObj(object): - - default_memorystate = SomeCTypesObject.MEMORYALIAS - - def __init__(self): - self.__name__ = 'RByrefObj' - - def compute_result_annotation(cls, s_arg): - """ - Answer the result annotation of calling 'byref'. - """ - return SomeCTypesObject(POINTER(s_arg.knowntype)) - - compute_result_annotation = classmethod(compute_result_annotation) - - def __call__(self,obj): - return byref(obj) - -RByref = RByrefObj() +# Register the correspondance between SomeCTypesObject and the get_repr() +# functions attached to the extregistry to create CTypesReprs -def RPOINTER(cls): - answer = POINTER(cls) - - def compute_result_annotation(cls, s_arg): - """ - Answer the result annotation of calling 'cls'. - """ - assert answer is cls - try: - memorystate = s_arg.memorystate - except AttributeError: - memorystate = None - return SomeCTypesObject(cls, memorystate) - answer.compute_result_annotation = classmethod(compute_result_annotation) - - def createLowLevelRepresentation( rtyper, annotationObject ): - """ - Create a lowlevel representation for the pointer. - """ - if annotationObject.memorystate == annotationObject.OWNSMEMORY: - return CtypesMemoryOwningPointerRepresentation( rtyper, annotationObject ) - elif annotationObject.memorystate == annotationObject.MEMORYALIAS: - return CtypesMemoryAliasPointerRepresentation( rtyper, annotationObject ) - else: - raise TyperError( "Unkown memory state in %r" % annotationObject ) - answer.createLowLevelRepresentation = staticmethod( - createLowLevelRepresentation ) - - def specialize( cls, highLevelOperation ): - #d#print "specialize:", cls, highLevelOperation - ctypesStructureType = highLevelOperation.r_result.lowleveltype - answer = highLevelOperation.llops.genop( - "malloc", [ inputconst( Void, ctypesStructureType ) ], - highLevelOperation.r_result ) - #d#import pdb - #d#pdb.set_trace() - if True: - highLevelOperation.llops.genop( - "setfield", - [ answer, - inputconst( Void, "contents" ), - highLevelOperation.inputarg( highLevelOperation.args_r[ 0 ], 0 ) ] ) - #t#highLevelOperation.inputarg( highLevelOperation.args_r[ 0 ], 0 ) ], - #t#highLevelOperation.r_result ) - return answer - answer.specialize = classmethod( specialize ) - - # We specialcased accessing pointers be getting their contents attribute - # because we can't use the memory state from 'cls'. - # So the obvious way to do it is obsolete (#o#). - answer._fields_def_ = {"contents": cls} - #d#print "p _fields_def_:", answer._fields_def_ - - # XXX Think about that twice and think about obsoleting - # the obsoletion above - answer.default_memorystate = None - answer.external_function_result_memorystate = SomeCTypesObject.MEMORYALIAS - - # Add a low level type attribute, which is only used for computing the - # result of an external function. In other words this is just the non - # gc case - try: - answer.ll_type = Ptr( - Struct( - 'CtypesMemoryAliasPointer_%s' % answer.__name__, - ( "contents", answer._type_.ll_type ) ) ) - except TypeError: - pass - return answer - - -# class RCDLL(CDLL): -# """ -# This is the restricted version of ctypes' CDLL class. -# """ -# -# class _CdeclFuncPtr(FunctionPointerTranslation, CDLL._CdeclFuncPtr): -# """ -# A simple extension of ctypes function pointers that -# implements a simple interface to the anotator. -# """ -# _flags_ = _FUNCFLAG_CDECL -# -# -# -# if sys.platform == "win32": -# class RWinDLL(WinDLL): -# """ -# This is the restricted version of ctypes' WINDLL class -# """ -# -# class _StdcallFuncPtr(FunctionPointerTranslation, WinDLL._StdcallFuncPtr): -# """ -# A simple extension of ctypes function pointers that -# implements a simple interface to the anotator. -# """ -# _flags_ = _FUNCFLAG_STDCALL - -# def RARRAY(typ,length): -# answer = ARRAY(typ,length) -# def compute_result_annotation(cls, *arg_s): -# """ -# Answer the result annotation of calling 'cls'. -# """ -# assert answer is cls -# return SomeCTypesObject(cls, SomeCTypesObject.OWNSMEMORY) -# answer.compute_result_annotation = classmethod(compute_result_annotation) -# return answer - -class AbstractCtypesRepresentation( Repr ): - """ - The abstract base class of all ctypes low level representations. - """ - -class AbstractCtypesStructureRepresentation( AbstractCtypesRepresentation ): - """ - The abstract base class of ctypes structures' low level representation. - """ - - def generateCDataAccess( self, variable, lowLevelOperations ): - """ - Answer the C level data sub structure. - """ - inputargs = [ variable, inputconst( Void, "c_data" ) ] - return lowLevelOperations.genop( - "getsubstruct", - inputargs, - Ptr( self.c_data_lowleveltype ) ) - - def rtype_setattr( self, highLevelOperation ): - c_data = self.generateCDataAccess( - highLevelOperation.inputarg( self, 0 ), - highLevelOperation.llops ) - inputargs = highLevelOperation.inputargs( - *highLevelOperation.args_r[ :3 ] ) - inputargs[ 0 ] = c_data - print "inputargs:", inputargs - print "r_result:", highLevelOperation.r_result - highLevelOperation.genop( "setfield", inputargs ) - - def rtype_getattr( self, highLevelOperation ): - c_data = self.generateCDataAccess( - highLevelOperation.inputarg( self, 0 ), - highLevelOperation.llops ) - inputargs = highLevelOperation.inputargs( - *highLevelOperation.args_r[ :3 ] ) - inputargs[ 0 ] = c_data - return highLevelOperation.genop( - "getfield", - inputargs, - highLevelOperation.r_result ) - - -class CtypesMemoryOwningStructureRepresentation( AbstractCtypesStructureRepresentation ): - """ - The lowlevel representation of a rctypes structure that owns its memory. - """ - - def __init__( self, rtyper, annotationObject ): - # XXX This .ll_type may not work for pointers or structures - # containing structures - fields = [ ( name, ctypesType.ll_type ) - for name, ctypesType in annotationObject.knowntype._fields_ ] - name = annotationObject.knowntype.__name__ - #o#self.c_data_lowleveltype = Struct( "C-Data_%s" % name, *fields ) - self.c_data_lowleveltype = annotationObject.knowntype.ll_type - self.lowleveltype = Ptr( - GcStruct( - "CtypesStructure_%s" % name, - ( "c_data", self.c_data_lowleveltype ) ) ) - - -class CtypesMemoryAliasStructureRepresentation( AbstractCtypesStructureRepresentation ): - """ - The lowlevel representation of a rctypes structure that is an alias to - someone else's memory. - """ - - -class AbstractCtypesPointerRepresentation( AbstractCtypesRepresentation ): - """ - The abstract base class of all rctypes low level representations - of a pointer. - """ - - -class CtypesMemoryOwningPointerRepresentation( AbstractCtypesPointerRepresentation ): - """ - The lowlevel representation of a cytpes pointer that points - to memory owned by rcyptes. - """ - - def __init__( self, rtyper, annotationObject ): - self.lowleveltype = Ptr( - GcStruct( - 'CtypesMemoryOwningPointer_%s' % annotationObject.knowntype.__name__, - ( "contents", - rtyper.getrepr( - annotationObject.knowntype._type_.compute_annotation() - ).lowleveltype ) ) ) - - def rtype_getattr( self, highLevelOperation ): - inputargs = [ - highLevelOperation.inputarg( highLevelOperation.args_r[ 0 ], 0 ), - inputconst( Void, "contents" ) ] - return highLevelOperation.genop( - "getfield", inputargs, highLevelOperation.r_result ) - - -class CtypesMemoryAliasPointerRepresentation( AbstractCtypesPointerRepresentation ): - """ - The lowlevel representation of a cytpes pointer that points - to memory owned by an external library. - """ - - def __init__( self, rtyper, annotationObject ): - self.lowleveltype = annotationObject.knowntype.ll_type +from pypy.annotation.model import SomeCTypesObject +from pypy.rpython import extregistry - class __extend__( SomeCTypesObject ): def rtyper_makerepr( self, rtyper ): - if extregistry.is_registered_type(self.knowntype): - entry = extregistry.lookup_type(self.knowntype) - return entry.get_repr(rtyper, self) - return self.knowntype.createLowLevelRepresentation( rtyper, self ) - + entry = extregistry.lookup_type(self.knowntype) + return entry.get_repr(rtyper, self) + def rtyper_makekey( self ): return self.__class__, self.knowntype, self.memorystate - - -class __extend__( pairtype( CtypesMemoryOwningPointerRepresentation, IntegerRepr ) ): - def rtype_getitem( ( self, integer ), highLevelOperation ): - print "rtype_getitem:", integer - return highLevelOperation.genop( - "getfield", - "contents", - highLevelOperation.r_result ) - Modified: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c (original) +++ pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Thu Apr 13 09:02:58 2006 @@ -1,52 +1,52 @@ -/* simple test, currently only for structures */ -#include -#ifdef MS_WIN32 -#include -#endif -#if defined(MS_WIN32) || defined(__CYGWIN__) -#define EXPORT(x) __declspec(dllexport) x -#else -#define EXPORT(x) x -#endif - -PyMethodDef module_methods[] = { - { NULL, NULL, 0, NULL}, -}; - - -typedef struct tagpoint { - int x; - int y; -} point; - -EXPORT(int) _testfunc_byval(point in, point *pout) -{ - if (pout) - { - pout->x = in.x; - pout->y = in.y; - } - return in.x + in.y; -} - -EXPORT(int) _testfunc_struct(point in) -{ - return in.x + in.y; -} - -EXPORT(point) _testfunc_struct_id(point in) -{ - return in; -} - -EXPORT(point*) _testfunc_struct_pointer_id( point* pin ) -{ - return pin; -} - -DL_EXPORT(void) -init_rctypes_test(void) -{ - Py_InitModule("_rctypes_test", module_methods); -} - +/* simple test, currently only for structures */ +#include +#ifdef MS_WIN32 +#include +#endif +#if defined(MS_WIN32) || defined(__CYGWIN__) +#define EXPORT(x) __declspec(dllexport) x +#else +#define EXPORT(x) x +#endif + +PyMethodDef module_methods[] = { + { NULL, NULL, 0, NULL}, +}; + + +typedef struct tagpoint { + int x; + int y; +} point; + +EXPORT(int) _testfunc_byval(point in, point *pout) +{ + if (pout) + { + pout->x = in.x; + pout->y = in.y; + } + return in.x + in.y; +} + +EXPORT(int) _testfunc_struct(point in) +{ + return in.x + in.y; +} + +EXPORT(point) _testfunc_struct_id(point in) +{ + return in; +} + +EXPORT(point*) _testfunc_struct_pointer_id( point* pin ) +{ + return pin; +} + +DL_EXPORT(void) +init_rctypes_test(void) +{ + Py_InitModule("_rctypes_test", module_methods); +} + Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Thu Apr 13 09:02:58 2006 @@ -3,6 +3,7 @@ """ import py.test +import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy import conftest Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Thu Apr 13 09:02:58 2006 @@ -3,6 +3,7 @@ """ import py.test +import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Thu Apr 13 09:02:58 2006 @@ -1,54 +1,29 @@ """ -Test the rctypes implementation. +Test external function calls using the custom extension module _rctypes_test.c. """ -import py.test +import py from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile, compile_db -#o#from pypy.translator.c import compile from pypy.translator.tool.cbuild import compile_c_module from pypy.annotation.model import SomeCTypesObject, SomeObject from pypy import conftest import sys from pypy.rpython.test.test_llinterp import interpret -thisdir = py.magic.autopath().dirpath() +thisdir = py.path.local(__file__).dirpath() -def compile(fn, argtypes, view=conftest.option.view): - from pypy.translator.c.database import LowLevelDatabase - from pypy.rpython.lltypesystem.lltype import pyobjectptr - t = TranslationContext() - a = t.buildannotator() - a.build_types(fn, argtypes) - t.buildrtyper().specialize() - if view: - t.view() - #t#backend_optimizations(t) - db = LowLevelDatabase(t) - entrypoint = db.get(pyobjectptr(fn)) - db.complete() - module = compile_db(db) - compiled_fn = getattr(module, entrypoint) - def checking_fn(*args, **kwds): - res = compiled_fn(*args, **kwds) - mallocs, frees = module.malloc_counters() - assert mallocs == frees - return res - return checking_fn - try: import ctypes except ImportError: py.test.skip("this test needs ctypes installed") -#py.test.skip("these tests are broken while the ctypes primitive types are ported to use the extregistry") +from ctypes import cdll +from ctypes import POINTER, Structure, c_int -from pypy.rpython.rctypes import cdll, c_char_p, c_int, c_char, \ - c_char, c_byte, c_ubyte, c_short, c_ushort, c_uint,\ - c_long, c_ulong, c_longlong, c_ulonglong, c_float, c_double, \ - POINTER, Structure, byref, ARRAY +# __________ compile and load our local test C file __________ # LoadLibrary is deprecated in ctypes, this should be removed at some point if "load" in dir(cdll): @@ -56,29 +31,6 @@ else: cdll_load = cdll.LoadLibrary -if sys.platform == 'win32': - mylib = cdll_load('msvcrt.dll') -elif sys.platform == 'linux2': - mylib = cdll_load('libc.so.6') -elif sys.platform == 'darwin': - mylib = cdll.c -else: - py.test.skip("don't know how to load the c lib for %s" % - sys.platform) - -atoi = mylib.atoi -atoi.restype = c_int -atoi.argtypes = [c_char_p] -atoi.argtypes = [POINTER(c_char)] - -def o_atoi(a): - return atoi(a) - -class tagpoint(Structure): - _fields_ = [("x", c_int), - ("y", c_int)] - -# compile and load our local test C file # XXX the built module and intermediate files should go to /tmp/usession-*, # see pypy.tool.udir compile_c_module([thisdir.join("_rctypes_test.c")], "_rctypes_test") @@ -88,387 +40,31 @@ else: _rctypes_test = cdll_load(str(thisdir.join("_rctypes_test.so"))) +# struct tagpoint +class tagpoint(Structure): + _fields_ = [("x", c_int), + ("y", c_int)] + # _testfunc_byval testfunc_byval = _rctypes_test._testfunc_byval testfunc_byval.restype = c_int testfunc_byval.argtypes = [tagpoint,POINTER(tagpoint)] -def py_testfunc_byval(inpoint): - opoint = tagpoint() - res = testfunc_byval(inpoint,byref(opoint)) - - return res, opoint - # _test_struct testfunc_struct = _rctypes_test._testfunc_struct testfunc_struct.restype = c_int testfunc_struct.argtypes = [tagpoint] -def py_testfunc_struct(inpoint): - return testfunc_struct(inpoint) - # _test_struct_id testfunc_struct_id = _rctypes_test._testfunc_struct_id testfunc_struct_id.restype = tagpoint testfunc_struct_id.argtypes = [tagpoint] -def py_testfunc_struct_id(inpoint): - return testfunc_struct_id(inpoint) - - -oppoint_type = POINTER(tagpoint) - # _test_struct_id_pointer +tagpointptr = POINTER(tagpoint) testfunc_struct_pointer_id = _rctypes_test._testfunc_struct_pointer_id -testfunc_struct_pointer_id.restype = oppoint_type -testfunc_struct_pointer_id.argtypes = [oppoint_type] - -def py_testfunc_struct_pointer_id(inpoint): - return testfunc_struct_pointer_id(inpoint) +testfunc_struct_pointer_id.restype = tagpointptr +testfunc_struct_pointer_id.argtypes = [tagpointptr] -def py_create_point(): - p = tagpoint() - p.x = 10 - p.y = 20 - return p.x + p.y - -def py_testfunc_POINTER(inpoint): - point = tagpoint() - oppoint = oppoint_type(point) - res = testfunc_byval(inpoint,oppoint) - return res, oppoint - -def py_testfunc_POINTER_dereference(inpoint): - point = tagpoint() - oppoint = oppoint_type(point) - res = testfunc_byval(inpoint,oppoint) - return res, oppoint.contents, oppoint[0] - -def py_test_mixed_memory_state( randomboolean ): - if randomboolean: - return tagpoint() - else: - return oppoint_type(tagpoint()).contents - -def py_test_simple_cint(): - return c_int(10) - -def py_test_simple_ctypes(): - return ( - c_char('a'), - c_byte(1), - c_ubyte(1), - c_short(1), - c_ushort(1), - c_int(1), - c_uint(1), - c_long(1), - c_ulong(1), - c_longlong(1), - c_ulonglong(1), - c_float(1.0), - c_double(1.0) - ) - -def py_test_simple_ctypes_non_const(): - a = 10 - return c_float( a + 10 ) - -c_int_10 = ARRAY(c_int,10) -c_int_p_test = POINTER(c_int) - -def py_test_annotate_array(): - return c_int_10() - -def py_test_annotate_array_content(): - my_array = c_int_10() - my_array[0] = c_int(1) - my_array[1] = 2 - - return my_array[0] - -def py_test_annotate_pointer_content(): - # Never run this function! - # See test_annotate_pointer_access_as_array_or_whatever - # for the weird reasons why this gets annotated - my_pointer = c_int_p_test(10) - my_pointer[0] = c_int(1) - my_pointer[1] = 2 - - return my_pointer[0] - -def py_test_annotate_array_slice_content(): - my_array = c_int_10() - #f#my_array[0:7] = c_int(1) * 7 - my_array[0:5] = range(5) - - return my_array[0:5] - -def py_test_annotate_array_content_variable_index(): - my_array = c_int_10() - my_array[2] = 2 - sum = 0 - for idx in range(10): - sum += my_array[idx] - - return sum - -def py_test_annotate_array_content_index_error_on_positive_index(): - my_array = c_int_10() - return my_array[10] - -def py_test_annotate_array_content_index_error_on_negative_index(): - my_array = c_int_10() - return my_array[-11] - -def py_test_specialize_struct(): - p = tagpoint() - p.x = 1 - p.y = 2 - return p.x - -def _py_test_compile_struct( p, x, y ): - p.x = x - p.y = y - return p - -def py_test_compile_struct( x, y ): - return _py_test_compile_struct( tagpoint(), x, y ).x - -def py_test_compile_pointer( x, y ): - return _py_test_compile_pointer( oppoint_type( tagpoint() ), x, y ).x - -def _py_test_compile_pointer( p, x, y ): - s = p.contents - s.x = x - s.y = y - return s - -class Test_rctypes: - - def test_simple(self): - res = o_atoi('42') - assert res == 42 - - def test_annotate_simple(self): - a = RPythonAnnotator() - s = a.build_types(o_atoi, [str]) - # result should be an integer - assert s.knowntype == int - - def x_test_specialize_simple(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(o_atoi, [str]) - # result should be an integer - assert s.knowntype == int - t.buildrtyper().specialize() - - if conftest.option.view: - a.translator.view() - - def x_test_compile_simple(self): - fn = compile(o_atoi, [str]) - res = fn("42") - assert res == 42 - - -class Test_structure: - - def test_simple_as_extension_module(self): - # Full path names follow because of strange behavior in the presence - # of an __init__.py in this test directory. When there is an - # __init__.py then the full path names appear in sys.modules - import pypy.rpython.rctypes.test._rctypes_test as t0 - import pypy.rpython.rctypes.test._rctypes_test as t1 - assert t1 is t0 - assert "pypy.rpython.rctypes.test._rctypes_test" in sys.modules - - def test_simple(self): - if sys.platform == "win32": - dll = cdll_load("_rctypes_test.pyd") - else: - dll = cdll_load(str(thisdir.join("_rctypes_test.so"))) - in_point = tagpoint() - in_point.x = 42 - in_point.y = 17 - out_point = tagpoint() - assert in_point.x + in_point.y == dll._testfunc_byval(in_point, byref(out_point)) - assert out_point.x == 42 - assert out_point.y == 17 - - def test_structure(self): - in_point = tagpoint() - in_point.x = 10 - in_point.y = 20 - res = py_testfunc_struct(in_point) - assert res == 30 - - def x_test_annotate_struct(self): - a = RPythonAnnotator() - s = a.build_types(py_testfunc_struct, [tagpoint]) - assert s.knowntype == int - - if conftest.option.view: - a.translator.view() - - def x_test_annotate_struct2(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_testfunc_struct_id, [tagpoint]) - assert s.knowntype == tagpoint - assert s.memorystate == SomeCTypesObject.OWNSMEMORY - - def x_test_annotate_pointer_to_struct(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_testfunc_struct_pointer_id, [oppoint_type]) - assert s.knowntype == oppoint_type - assert s.memorystate == SomeCTypesObject.MEMORYALIAS - return t - - def x_test_create_point(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_create_point,[]) - assert s.knowntype == int - - if conftest.option.view: - a.translator.view() - - def x_test_annotate_byval(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_testfunc_byval,[tagpoint]) - assert s.knowntype == tuple - assert len(s.items) == 2 - assert s.items[0].knowntype == int - assert s.items[1].knowntype == tagpoint - assert s.items[1].memorystate == SomeCTypesObject.OWNSMEMORY - - def x_test_annotate_POINTER(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_testfunc_POINTER,[tagpoint]) - assert s.knowntype == tuple - assert len(s.items) == 2 - assert s.items[0].knowntype == int - assert s.items[1].knowntype == POINTER(tagpoint) - assert s.items[1].memorystate == SomeCTypesObject.OWNSMEMORY - #d#t.view() - - def x_test_annotate_POINTER_dereference(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_testfunc_POINTER_dereference, [tagpoint]) - assert s.knowntype == tuple - assert len(s.items) == 3 - assert s.items[0].knowntype == int - assert s.items[1].knowntype == tagpoint - assert s.items[1].memorystate == SomeCTypesObject.OWNSMEMORY - assert s.items[2].knowntype == tagpoint - assert s.items[2].memorystate == SomeCTypesObject.OWNSMEMORY - #d#t.view() - - def x_test_annotate_mixed_memorystate(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_test_mixed_memory_state, [int]) - #d#t.view() - assert s.knowntype == tagpoint - # This memory state will be supported in the future (#f#) - # Obviously the test is wrong for now - #f#assert s.memorystate == SomeCTypesObject.MIXEDMEMORYOWNERSHIP - assert isinstance(s, SomeObject) - - def test_annotate_simple_cint(self): - a = RPythonAnnotator() - s = a.build_types(py_test_simple_cint,[]) - assert s.knowntype == c_int - - def test_annotate_simple_types(self): - a = RPythonAnnotator() - s = a.build_types(py_test_simple_ctypes,[]) - assert s.knowntype == tuple - assert len(s.items) == 13 - assert s.items[0].knowntype == c_char - assert s.items[1].knowntype == c_byte - assert s.items[2].knowntype == c_ubyte - assert s.items[3].knowntype == c_short - assert s.items[4].knowntype == c_ushort - assert s.items[5].knowntype == c_int - assert s.items[6].knowntype == c_uint - assert s.items[7].knowntype == c_long - assert s.items[8].knowntype == c_ulong - assert s.items[9].knowntype == c_longlong - assert s.items[10].knowntype == c_ulonglong - assert s.items[11].knowntype == c_float - assert s.items[12].knowntype == c_double - - def test_annotate_simple_types_non_const(self): - a = RPythonAnnotator() - s = a.build_types(py_test_simple_ctypes_non_const,[]) - assert s.knowntype == c_float - - def x_test_specialize_struct(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_test_specialize_struct, []) - # result should be an integer - assert s.knowntype == int - try: - t.buildrtyper().specialize() - finally: - if conftest.option.view: - t.view() - - def x_test_specialize_struct_1(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_test_compile_struct, [int, int]) - #d#t.view() - try: - t.buildrtyper().specialize() - finally: - #d#t.view() - pass - - def x_test_specialize_pointer_to_struct(self): - t = self.test_annotate_pointer_to_struct() - t.buildrtyper().specialize() - if conftest.option.view: - t.view() - - def x_test_compile_pointer_to_struct(self): - fn = compile( py_testfunc_struct_pointer_id, [ oppoint_type ] ) - - def x_test_compile_struct(self): - fn = compile( py_test_compile_struct, [ int, int ] ) - res = fn( 42, -42 ) - assert res == 42 - - def x_test_specialize_POINTER_dereference(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(py_testfunc_POINTER_dereference, [tagpoint]) - assert s.knowntype == tuple - try: - t.buildrtyper().specialize() - finally: - #d#t.view() - pass - - def x_test_specialize_pointer(self): - t = TranslationContext() - a = t.buildannotator() - s = a.build_types( py_test_compile_pointer, [ int, int ] ) - assert s.knowntype == int - #d#t.view() - t.buildrtyper().specialize() - #d#t.view() - - def x_test_compile_pointer(self): - fn = compile( py_test_compile_pointer, [ int, int ] ) - res = fn( -42, 42 ) - assert res == -42 +# XXX no test here so far Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Thu Apr 13 09:02:58 2006 @@ -1,14 +1,34 @@ import py +import sys +import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator -from pypy.rpython.rctypes.test.test_rctypes import mylib from pypy.rpython.test.test_llinterp import interpret from pypy.translator.c.test.test_genc import compile from pypy import conftest from pypy.rpython.rstr import string_repr from pypy.rpython.lltypesystem import lltype +from ctypes import cdll from ctypes import c_int, c_long, c_char_p, c_char +# __________ the standard C library __________ + +# LoadLibrary is deprecated in ctypes, this should be removed at some point +if "load" in dir(cdll): + cdll_load = cdll.load +else: + cdll_load = cdll.LoadLibrary + +if sys.platform == 'win32': + mylib = cdll_load('msvcrt.dll') +elif sys.platform == 'linux2': + mylib = cdll_load('libc.so.6') +elif sys.platform == 'darwin': + mylib = cdll.c +else: + py.test.skip("don't know how to load the c lib for %s" % + sys.platform) +# ____________________________________________ labs = mylib.labs labs.restype = c_long Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Thu Apr 13 09:02:58 2006 @@ -3,14 +3,13 @@ """ import py.test +import pypy.rpython.rctypes.implementation from pypy.translator.translator import TranslationContext from pypy import conftest from pypy.rpython.test.test_llinterp import interpret from ctypes import c_int, c_float, POINTER, pointer -import pypy.rpython.rctypes.implementation - class Test_annotation: def test_simple(self): res = c_int(42) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Thu Apr 13 09:02:58 2006 @@ -3,6 +3,7 @@ """ import py.test +import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile From antocuni at codespeak.net Thu Apr 13 11:38:33 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 13 Apr 2006 11:38:33 +0200 (CEST) Subject: [pypy-svn] r25773 - in pypy/dist/pypy/translator/cli: src test Message-ID: <20060413093833.3708F1007A@code0.codespeak.net> Author: antocuni Date: Thu Apr 13 11:38:25 2006 New Revision: 25773 Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py Log: Added support for negative indexes in lists. Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Thu Apr 13 11:38:25 2006 @@ -17,12 +17,12 @@ return this.Count; } - public T getitem(int index) + public T getitem_nonneg(int index) { return this[index]; } - public void setitem(int index, T value_) + public void setitem_nonneg(int index, T value_) { this[index] = value_; } Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Thu Apr 13 11:38:25 2006 @@ -26,8 +26,8 @@ def bar(x, y): lst = [1,2,3,x,y] - lst.extend([1,2]) - return 0 + lst[-1] = 42 + return lst[-1] f = compile_function(bar, [int, int]) From antocuni at codespeak.net Thu Apr 13 11:39:04 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 13 Apr 2006 11:39:04 +0200 (CEST) Subject: [pypy-svn] r25774 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060413093904.BBE961007D@code0.codespeak.net> Author: antocuni Date: Thu Apr 13 11:38:58 2006 New Revision: 25774 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: Added support for negative indexes in list. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Thu Apr 13 11:38:58 2006 @@ -204,8 +204,8 @@ # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) "length": Meth([], Signed), "append": Meth([self.ITEMTYPE_T], Void), - "getitem": Meth([Signed], self.ITEMTYPE_T), - "setitem": Meth([Signed, self.ITEMTYPE_T], Void), + "getitem_nonneg": Meth([Signed], self.ITEMTYPE_T), + "setitem_nonneg": Meth([Signed, self.ITEMTYPE_T], Void), "extend": Meth([self.SELFTYPE_T], Void), }) @@ -539,15 +539,17 @@ assert typeOf(item) == self._TYPE._ITEMTYPE self._list.append(item) - def getitem(self, index): + def getitem_nonneg(self, index): # NOT_RPYTHON assert typeOf(index) == Signed + assert index >= 0 return self._list[index] - def setitem(self, index, item): + def setitem_nonneg(self, index, item): # NOT_RPYTHON assert typeOf(item) == self._TYPE._ITEMTYPE assert typeOf(index) == Signed + assert index >= 0 self._list[index] = item def extend(self, other): Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Thu Apr 13 11:38:58 2006 @@ -5,6 +5,7 @@ from pypy.rpython.rmodel import inputconst, externalvsinternal from pypy.rpython.ootypesystem import ootype from pypy.rpython.ootypesystem.riterable import iterator_type +from pypy.rpython.lltypesystem.lltype import Signed class BaseListRepr(AbstractBaseListRepr): @@ -33,7 +34,6 @@ self.lowleveltype.become(ootype.List(self.item_repr.lowleveltype)) def send_message(self, hop, message, can_raise=False): - print message v_args = hop.inputargs(self, *hop.args_r[1:]) c_name = hop.inputconst(ootype.Void, message) if can_raise: @@ -62,13 +62,33 @@ class __extend__(pairtype(BaseListRepr, IntegerRepr)): def rtype_getitem((r_list, r_int), hop): - # XXX must handle negative indices - return r_list.send_message(hop, "getitem", can_raise=True) + if hop.args_s[1].nonneg: + return r_list.send_message(hop, "getitem_nonneg", can_raise=True) + else: + v_list, v_index = hop.inputargs(r_list, Signed) + hop.exception_is_here() + v_res = hop.gendirectcall(ll_getitem, v_list, v_index) + return r_list.recast(hop.llops, v_res) def rtype_setitem((r_list, r_int), hop): - # XXX must handle negative indices - return r_list.send_message(hop, "setitem", can_raise=True) - + if hop.args_s[1].nonneg: + return r_list.send_message(hop, "setitem_nonneg", can_raise=True) + else: + v_list, v_index, v_item = hop.inputargs(r_list, Signed, r_list.item_repr) + hop.exception_is_here() + return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) + + + +def ll_getitem(lst, index): + if index < 0: + index += lst.length() + return lst.getitem_nonneg(index) + +def ll_setitem(lst, index, item): + if index < 0: + index += lst.length() + return lst.setitem_nonneg(index, item) def newlist(llops, r_list, items_v): c_1ist = inputconst(ootype.Void, r_list.lowleveltype) @@ -93,7 +113,7 @@ len2 = l2.length() i = 0 while i < len2: - l1.append(l2.getitem(i)) + l1.append(l2.getitem_nonneg(i)) i += 1 def ll_concat(RESLIST, l1, l2): @@ -102,11 +122,11 @@ l = ootype.new(RESLIST) i = 0 while i < len1: - l.append(l1.getitem(i)) + l.append(l1.getitem_nonneg(i)) i += 1 i = 0 while i < len2: - l.append(l2.getitem(i)) + l.append(l2.getitem_nonneg(i)) i += 1 return l @@ -134,5 +154,5 @@ if index >= l.length(): raise StopIteration iter.index = index + 1 - return l.getitem(index) + return l.getitem_nonneg(index) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Thu Apr 13 11:38:58 2006 @@ -31,15 +31,15 @@ LT = List(Signed) l = new(LT) l.append(2) - assert l.getitem(0) == 2 - l.setitem(0, 3) - assert l.getitem(0) == 3 + assert l.getitem_nonneg(0) == 2 + l.setitem_nonneg(0, 3) + assert l.getitem_nonneg(0) == 3 def test_setitem_indexerror(): LT = List(Signed) l = new(LT) - py.test.raises(IndexError, l.getitem, 0) - py.test.raises(IndexError, l.setitem, 0, 1) + py.test.raises(IndexError, l.getitem_nonneg, 0) + py.test.raises(IndexError, l.setitem_nonneg, 0, 1) def test_null(): LT = List(Signed) From arigo at codespeak.net Thu Apr 13 12:10:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 12:10:35 +0200 (CEST) Subject: [pypy-svn] r25775 - pypy/dist/pypy/annotation Message-ID: <20060413101035.335C21007C@code0.codespeak.net> Author: arigo Date: Thu Apr 13 12:10:34 2006 New Revision: 25775 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/unaryop.py Log: Clean up old code. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Thu Apr 13 12:10:34 2006 @@ -775,6 +775,7 @@ return s_result.return_annotation() class __extend__(pairtype(SomeCTypesObject, SomeSlice)): + # XXX ctypes array slicing not really supported for now def setitem((s_cto, s_slice), s_iterable): pass Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Thu Apr 13 12:10:34 2006 @@ -638,30 +638,9 @@ def getattr(cto, s_attr): if s_attr.is_constant() and isinstance(s_attr.const, str): attr = s_attr.const - # We reactivate the old contents field hack - if False: - try: - atype = cto.knowntype._fields_def_[attr] - except AttributeError: - # We are dereferencing a pointer by accessing its contents attribute - if s_attr.const == "contents": - return SomeCTypesObject( - cto.knowntype._type_, cto.MEMORYALIAS) - else: - raise AttributeError( - "%r object has no attribute %r" % ( - cto.knowntype, s_attr.const)) - else: - if extregistry.is_registered_type(cto.knowntype): - entry = extregistry.lookup_type(cto.knowntype) - s_value = entry.get_field_annotation(cto, attr) - return s_value - else: - atype = cto.knowntype._fields_def_[attr] - try: - return atype.annotator_type - except AttributeError: - return SomeCTypesObject(atype) + entry = extregistry.lookup_type(cto.knowntype) + s_value = entry.get_field_annotation(cto, attr) + return s_value else: return SomeObject() From arigo at codespeak.net Thu Apr 13 12:15:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 12:15:47 +0200 (CEST) Subject: [pypy-svn] r25776 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060413101547.423171007C@code0.codespeak.net> Author: arigo Date: Thu Apr 13 12:15:45 2006 New Revision: 25776 Added: pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Modified: pypy/dist/pypy/rpython/rctypes/implementation.py Log: Starting to reimplement ctypes structures. Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Thu Apr 13 12:15:45 2006 @@ -4,6 +4,7 @@ import pypy.rpython.rctypes.rarray import pypy.rpython.rctypes.rfunc import pypy.rpython.rctypes.rchar_p +import pypy.rpython.rctypes.rstruct # Register the correspondance between SomeCTypesObject and the get_repr() Added: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Thu Apr 13 12:15:45 2006 @@ -0,0 +1,44 @@ +from ctypes import Structure +from pypy.annotation.model import SomeCTypesObject, SomeBuiltin +from pypy.rpython import extregistry +from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr + +StructType = type(Structure) + + + + +def structtype_specialize_call(hop): + r_struct = hop.r_result + return hop.genop("malloc", [ + hop.inputconst(lltype.Void, r_struct.lowleveltype.TO), + ], resulttype=r_struct.lowleveltype, + ) + +def structtype_compute_annotation(metatype, type): + def compute_result_annotation(*arg_s): + return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) + return SomeBuiltin(compute_result_annotation, methodname=type.__name__) + +extregistry.register_type(StructType, + compute_annotation=structtype_compute_annotation, + specialize_call=structtype_specialize_call) + +def struct_instance_compute_annotation(type, instance): + return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) + +def struct_instance_field_annotation(s_struct, fieldname): + structtype = s_struct.knowntype + for name, ctype in structtype._fields_: + if name == fieldname: + s_result = SomeCTypesObject(ctype, SomeCTypesObject.MEMORYALIAS) + return s_result.return_annotation() + raise AttributeError('%r has no field %r' % (structtype, fieldname)) + +def structtype_get_repr(rtyper, s_struct): + return StructRepr(rtyper, s_struct) + +entry = extregistry.register_metatype(StructType, + compute_annotation=struct_instance_compute_annotation, + get_repr=structtype_get_repr) +entry.get_field_annotation = struct_instance_field_annotation Added: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Thu Apr 13 12:15:45 2006 @@ -0,0 +1,52 @@ +""" +Test the rctypes implementation. +""" + +import py.test +import pypy.rpython.rctypes.implementation +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy import conftest +from pypy.translator.c.test.test_genc import compile +import sys +from pypy.rpython.test.test_llinterp import interpret + +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + +from ctypes import c_int, c_short, Structure, POINTER, pointer + +class tagpoint(Structure): + _fields_ = [("x", c_int), + ("y", c_int)] + +class Test_annotation: + def test_annotate_struct(self): + def create_struct(): + return tagpoint() + + a = RPythonAnnotator() + s = a.build_types(create_struct, []) + assert s.knowntype == tagpoint + + if conftest.option.view: + a.translator.view() + + def test_annotate_struct_access(self): + def access_struct(n): + my_point = tagpoint() + my_point.x = c_int(1) + my_point.y = 2 + my_point.x += n + + return my_point.x + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(access_struct, [int]) + assert s.knowntype == int + + if conftest.option.view: + t.view() From stephan at codespeak.net Thu Apr 13 12:21:02 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Thu, 13 Apr 2006 12:21:02 +0200 (CEST) Subject: [pypy-svn] r25777 - pypy/dist/pypy/objspace/std Message-ID: <20060413102102.DA7B41007C@code0.codespeak.net> Author: stephan Date: Thu Apr 13 12:20:55 2006 New Revision: 25777 Modified: pypy/dist/pypy/objspace/std/frozensettype.py pypy/dist/pypy/objspace/std/objspace.py pypy/dist/pypy/objspace/std/setobject.py pypy/dist/pypy/objspace/std/settype.py Log: somewhat finished set/frozenset implementation. Needs some more debugging because translation fails in "backendopt:inlining". Checkin is for documentation purposes only. Modified: pypy/dist/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/dist/pypy/objspace/std/frozensettype.py (original) +++ pypy/dist/pypy/objspace/std/frozensettype.py Thu Apr 13 12:20:55 2006 @@ -20,12 +20,10 @@ def descr__frozenset__new__(space, w_frozensettype, w_iterable=NoneNotWrapped): from pypy.objspace.std.setobject import W_FrozensetObject from pypy.objspace.std.setobject import _is_frozenset_exact - if w_iterable is None: - w_iterable = space.newtuple([]) - if _is_frozenset_exact(space, w_iterable): + if _is_frozenset_exact(w_iterable): return w_iterable w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype) - W_FrozensetObject.__init__(w_obj, space, w_iterable) + W_FrozensetObject.__init__(w_obj, space, None) return w_obj Modified: pypy/dist/pypy/objspace/std/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/std/objspace.py (original) +++ pypy/dist/pypy/objspace/std/objspace.py Thu Apr 13 12:20:55 2006 @@ -349,11 +349,11 @@ return W_ComplexObject(self, realval, imagval) if WITHSET: - def newset(self, list_w): - return W_SetObject(self, list_w) + def newset(self, rdict_w): + return W_SetObject(self, rdict_w) - def newfrozenset(self, list_w): - return W_FrozensetObject(self, list_w) + def newfrozenset(self, rdict_w): + return W_FrozensetObject(self, rdict_w) def newlong(self, val): # val is an int return W_LongObject.fromint(self, val) Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Thu Apr 13 12:20:55 2006 @@ -3,33 +3,45 @@ from pypy.objspace.std.model import WITHSET from pypy.objspace.std.stdtypedef import StdObjSpaceMultiMethod from pypy.rpython.objectmodel import r_dict -from pypy.rpython.rarithmetic import intmask +from pypy.rpython.rarithmetic import intmask, r_uint from pypy.interpreter import gateway class W_BaseSetObject(W_Object): - def __init__(w_self, space, wrappeditems=None): + def __init__(w_self, space, setdata=None): W_Object.__init__(w_self, space) - w_self.setdata = setdata = r_dict(space.eq_w, space.hash_w) - _initialize_set(space, w_self, wrappeditems) - w_self.in_init = True + if setdata is None: + w_self.setdata = r_dict(space.eq_w, space.hash_w) + else: + w_self.setdata = setdata.copy() def __repr__(w_self): """representation for debugging purposes""" reprlist = [repr(w_item) for w_item in w_self.setdata.keys()] return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist)) - def _newobj(w_self, space): - return space.call(space.type(w_self),None) + def _newobj(w_self, space, rdict_w=None): + #return space.call(space.type(w_self),W_SetIterObject(space,rdict_w)) + objtype = type(w_self) + if objtype is W_SetObject: + obj = W_SetObject(space, rdict_w) + elif objtype is W_FrozensetObject: + obj = W_FrozensetObject(space, rdict_w) + else: + obj = space.call(space.type(w_self),W_SetIterObject(space,rdict_w)) + return obj class W_SetObject(W_BaseSetObject): from pypy.objspace.std.settype import set_typedef as typedef + def __init__(w_self, space, setdata=None): + W_BaseSetObject.__init__(w_self, space, setdata) + class W_FrozensetObject(W_BaseSetObject): from pypy.objspace.std.frozensettype import frozenset_typedef as typedef - def __init__(w_self, space, wrappeditems): - W_BaseSetObject.__init__(w_self, space, wrappeditems) + def __init__(w_self, space, setdata): + W_BaseSetObject.__init__(w_self, space, setdata) w_self.hash = -1 registerimplementation(W_SetObject) @@ -80,18 +92,10 @@ # some helper functions -def _iter_to_dict(space, w_iterable): +def make_setdata_from_w_iterable(space, w_iterable=None): data = r_dict(space.eq_w, space.hash_w) - iterable_w = space.unpackiterable(w_iterable) - for w_item in iterable_w: - data[w_item] = None - - return data - -def _initialize_set(space, w_obj, wrappeditems=None): - w_obj.setdata.clear() - if wrappeditems is not None: - w_iterator = space.iter(wrappeditems) + if w_iterable is not None: + w_iterator = space.iter(w_iterable) while True: try: w_item = space.next(w_iterator) @@ -99,24 +103,31 @@ if not e.match(space, space.w_StopIteration): raise break - w_obj.setdata[w_item] = None + data[w_item] = None + return data + +def _initialize_set(space, w_obj, w_iterable=None): + w_obj.setdata.clear() + if w_iterable is not None: + w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable)) # helper functions for set operation on dicts -def _is_setlike(space, w_obj): - if space.is_true(space.isinstance(w_obj, space.w_set)) or \ - space.is_true(space.isinstance(w_obj, space.w_frozenset)): +def _is_frozenset_exact(w_obj): + if (w_obj is not None) and (type(w_obj) is W_FrozensetObject): return True else: return False -def _is_frozenset_exact(space, w_obj): - if space.eq_w(space.type(w_obj), space.w_frozenset): - return True - else: +def _is_eq(w_left, w_right): + if len(w_left.setdata) != len(w_right.setdata): return False + for w_key in w_left.setdata.iterkeys(): + if w_key not in w_right.setdata: + return False + return True -def _union_dict(space, ldict, rdict, isupdate): +def _union_dict(ldict, rdict, isupdate): if isupdate: ld = ldict else: @@ -124,7 +135,7 @@ ld.update(rdict) return ld, rdict -def _difference_dict(space, ldict, rdict, isupdate): +def _difference_dict(ldict, rdict, isupdate): if isupdate: ld = ldict else: @@ -138,7 +149,7 @@ return ld, rdict -def _intersection_dict(space, ldict, rdict, isupdate): +def _intersection_dict(ldict, rdict, isupdate): if isupdate: ld = ldict else: @@ -154,7 +165,7 @@ return ld, rdict -def _symmetric_difference_dict(space, ldict, rdict, isupdate): +def _symmetric_difference_dict(ldict, rdict, isupdate): if isupdate: ld = ldict else: @@ -179,14 +190,21 @@ #end helper functions +def set_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _union_dict(ld, rd, True) + return space.w_None + +set_update__Set_Frozenset = set_update__Set_Set + def set_update__Set_ANY(space, w_left, w_other): """Update a set with the union of itself and another.""" - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _union_dict(space, ld, rd, True) + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _union_dict(ld, rd, True) return space.w_None def inplace_or__Set_Set(space, w_left, w_other): - set_update__Set_ANY(space, w_left, w_other) + set_update__Set_Set(space, w_left, w_other) return w_left inplace_or__Set_Frozenset = inplace_or__Set_Set @@ -199,13 +217,11 @@ w_left.setdata[w_other] = None return space.w_None -def set_copy__Set(space, w_left): - w_obj = w_left._newobj(space) - w_obj.setdata.update(w_left.setdata) - return w_obj +def set_copy__Set(space, w_set): + return w_set._newobj(space,w_set.setdata) def frozenset_copy__Frozenset(space, w_left): - if _is_frozenset_exact(space, w_left): + if _is_frozenset_exact(w_left): return w_left else: return set_copy__Set(space,w_left) @@ -214,66 +230,89 @@ w_left.setdata.clear() return space.w_None +def set_difference__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _difference_dict(ld, rd, False) + return w_left._newobj(space, new_ld) + +set_difference__Set_Frozenset = set_difference__Set_Set +frozenset_difference__Frozenset_Set = set_difference__Set_Set +frozenset_difference__Frozenset_Frozenset = set_difference__Set_Set +sub__Set_Set = set_difference__Set_Set +sub__Set_Frozenset = set_difference__Set_Set +sub__Frozenset_Set = set_difference__Set_Set +sub__Frozenset_Frozenset = set_difference__Set_Set + def set_difference__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _difference_dict(space, ld, rd, False) - w_obj = w_left._newobj(space) - w_obj.setdata.update(new_ld) - return w_obj + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _difference_dict(ld, rd, False) + return w_left._newobj(space, new_ld) -sub__Set_Set = set_difference__Set_ANY -sub__Set_Frozenset = set_difference__Set_ANY frozenset_difference__Frozenset_ANY = set_difference__Set_ANY -sub__Frozenset_Set = set_difference__Set_ANY -sub__Frozenset_Frozenset = set_difference__Set_ANY +def set_difference_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _difference_dict(ld, rd, True) + return space.w_None + +set_difference_update__Set_Frozenset = set_difference_update__Set_Set + def set_difference_update__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _difference_dict(space, ld, rd, True) + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _difference_dict(ld, rd, True) return space.w_None def inplace_sub__Set_Set(space, w_left, w_other): - set_difference_update__Set_ANY(space, w_left, w_other) + set_difference_update__Set_Set(space, w_left, w_other) return w_left inplace_sub__Set_Frozenset = inplace_sub__Set_Set -def eq__Set_ANY(space, w_left, w_other): - if not _is_setlike(space, w_other): - return space.w_False - if space.is_w(w_left, w_other): - return space.w_True +def eq__Set_Set(space, w_left, w_other): + return space.wrap(_is_eq(w_left, w_other)) - if len(w_left.setdata) != len(w_other.setdata): - return space.w_False +eq__Set_Frozenset = eq__Set_Set +eq__Frozenset_Frozenset = eq__Set_Set +eq__Frozenset_Set = eq__Set_Set - for w_key in w_left.setdata.iterkeys(): - if w_key not in w_other.setdata: - return space.w_False - return space.w_True +def eq__Set_ANY(space, w_left, w_other): + return space.w_False eq__Frozenset_ANY = eq__Set_ANY +def contains__Set_Set(space, w_left, w_other): + w_f = space.newfrozenset(w_other.setdata) + return space.newbool(w_f in w_left.setdata) + +contains__Frozenset_Set = contains__Set_Set + def contains__Set_ANY(space, w_left, w_other): - try: - r = w_other in w_left.setdata - return space.newbool(r) - except Exception, exp: - if _is_setlike(space, w_other): - w_f = space.newfrozenset(w_other) - r = w_f in w_left.setdata - return space.newbool(r) - else: - raise exp + return space.newbool(w_other in w_left.setdata) contains__Frozenset_ANY = contains__Set_ANY +def set_issubset__Set_Set(space, w_left, w_other): + if space.is_w(w_left, w_other): + return space.w_True + ld, rd = w_left.setdata, w_other.setdata + if len(ld) > len(rd): + return space.w_False + + for w_key in ld: + if w_key not in rd: + return space.w_False + return space.w_True + +set_issubset__Set_Frozenset = set_issubset__Set_Set +frozenset_issubset__Frozenset_Set = set_issubset__Set_Set +frozenset_issubset__Frozenset_Frozenset = set_issubset__Set_Set + def set_issubset__Set_ANY(space, w_left, w_other): if space.is_w(w_left, w_other): return space.w_True - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) if len(ld) > len(rd): return space.w_False @@ -284,15 +323,32 @@ frozenset_issubset__Frozenset_ANY = set_issubset__Set_ANY -le__Set_Set = set_issubset__Set_ANY -le__Set_Frozenset = set_issubset__Set_ANY -le__Frozenset_Frozenset = set_issubset__Set_ANY +le__Set_Set = set_issubset__Set_Set +le__Set_Frozenset = set_issubset__Set_Set +le__Frozenset_Frozenset = set_issubset__Set_Set + +def set_issuperset__Set_Set(space, w_left, w_other): + if space.is_w(w_left, w_other): + return space.w_True + + ld, rd = w_left.setdata, w_other.setdata + if len(ld) < len(rd): + return space.w_False + + for w_key in rd: + if w_key not in ld: + return space.w_False + return space.w_True + +set_issuperset__Set_Frozenset = set_issuperset__Set_Set +set_issuperset__Frozenset_Set = set_issuperset__Set_Set +set_issuperset__Frozenset_Frozenset = set_issuperset__Set_Set def set_issuperset__Set_ANY(space, w_left, w_other): if space.is_w(w_left, w_other): return space.w_True - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) if len(ld) < len(rd): return space.w_False @@ -303,51 +359,48 @@ frozenset_issuperset__Frozenset_ANY = set_issuperset__Set_ANY -ge__Set_Set = set_issuperset__Set_ANY -ge__Set_Frozenset = set_issuperset__Set_ANY -ge__Frozenset_Frozenset = set_issuperset__Set_ANY +ge__Set_Set = set_issuperset__Set_Set +ge__Set_Frozenset = set_issuperset__Set_Set +ge__Frozenset_Frozenset = set_issuperset__Set_Set + +def set_discard__Set_Set(space, w_left, w_item): + w_f = space.newfrozenset(w_item.setdata) + if w_f in w_left.setdata: + del w_left.setdata[w_f] def set_discard__Set_ANY(space, w_left, w_item): + if w_item in w_left.setdata: + del w_left.setdata[w_item] + +def set_remove__Set_Set(space, w_left, w_item): + w_f = space.newfrozenset(w_item.setdata) try: - if w_item in w_left.setdata: - del w_left.setdata[w_item] - except Exception, exp: - if _is_setlike(space, w_item): - w_f = space.newfrozenset(w_item) - if w_f in w_left.setdata: - del w_left.setdata[w_f] - else: - raise exp - + del w_left.setdata[w_f] + except KeyError: + raise OperationError(space.w_KeyError, + space.call_method(w_item,'__repr__')) + def set_remove__Set_ANY(space, w_left, w_item): try: del w_left.setdata[w_item] except KeyError: raise OperationError(space.w_KeyError, space.call_method(w_item,'__repr__')) - except Exception, exp: - if _is_setlike(space, w_item): - w_f = space.newfrozenset(w_item) - try: - del w_left.setdata[w_f] - except KeyError: - raise OperationError(space.w_KeyError, - space.call_method(w_item,'__repr__')) - else: - raise exp def hash__Set(space, w_set): raise OperationError(space.w_TypeError, space.wrap('set objects are unhashable')) def hash__Frozenset(space, w_set): + multi = r_uint(1822399083) + r_uint(1822399083) + 1 if w_set.hash != -1: return space.wrap(w_set.hash) hash = 1927868237 hash *= (len(w_set.setdata) + 1) for w_item in w_set.setdata.iterkeys(): - h = space.int_w(space.hash(w_item)) - hash ^= ((h ^ (h << 16) ^ 89869747) * 3644798167) + h = space.hash_w(w_item) + value = ((h ^ (h << 16) ^ 89869747) * multi) + hash = intmask(hash ^ value) hash = hash * 69069 + 907133923 if hash == -1: hash = 590923713 @@ -366,22 +419,37 @@ return w_value +def set_intersection__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _intersection_dict(ld, rd, False) + return w_left._newobj(space,new_ld) + +set_intersection__Set_Frozenset = set_intersection__Set_Set +set_intersection__Frozenset_Frozenset = set_intersection__Set_Set +set_intersection__Frozenset_Set = set_intersection__Set_Set + def set_intersection__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _intersection_dict(space, ld, rd, False) - w_obj = w_left._newobj(space) - w_obj.setdata.update(new_ld) - return w_obj + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _intersection_dict(ld, rd, False) + return w_left._newobj(space,new_ld) -and__Set_Set = set_intersection__Set_ANY -and__Set_Frozenset = set_intersection__Set_ANY frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY -and__Frozenset_Set = set_intersection__Set_ANY -and__Frozenset_Frozenset = set_intersection__Set_ANY + +and__Set_Set = set_intersection__Set_Set +and__Set_Frozenset = set_intersection__Set_Set +and__Frozenset_Set = set_intersection__Set_Set +and__Frozenset_Frozenset = set_intersection__Set_Set + +def set_intersection_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _intersection_dict(ld, rd, True) + return space.w_None + +set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set def set_intersection_update__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _intersection_dict(space, ld, rd, True) + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _intersection_dict(ld, rd, True) return space.w_None def inplace_and__Set_Set(space, w_left, w_other): @@ -390,44 +458,69 @@ inplace_and__Set_Frozenset = inplace_and__Set_Set -def set_symmetric_difference__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _symmetric_difference_dict(space, ld, rd, False) - w_obj = w_left._newobj(space) - w_obj.setdata.update(new_ld) - return w_obj +def set_symmetric_difference__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _symmetric_difference_dict(ld, rd, False) + return w_left._newobj(space, new_ld) + +set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set +set_symmetric_difference__Frozenset_Set = set_symmetric_difference__Set_Set +set_symmetric_difference__Frozenset_Frozenset = \ + set_symmetric_difference__Set_Set + +xor__Set_Set = set_symmetric_difference__Set_Set +xor__Set_Frozenset = set_symmetric_difference__Set_Set +xor__Frozenset_Set = set_symmetric_difference__Set_Set +xor__Frozenset_Frozenset = set_symmetric_difference__Set_Set + -xor__Set_Set = set_symmetric_difference__Set_ANY -xor__Set_Frozenset = set_symmetric_difference__Set_ANY +def set_symmetric_difference__Set_ANY(space, w_left, w_other): + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _symmetric_difference_dict(ld, rd, False) + return w_left._newobj(space, new_ld) frozenset_symmetric_difference__Frozenset_ANY = \ set_symmetric_difference__Set_ANY -xor__Frozenset_Set = set_symmetric_difference__Set_ANY -xor__Frozenset_Frozenset = set_symmetric_difference__Set_ANY + +def set_symmetric_difference_update__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _symmetric_difference_dict(ld, rd, True) + return space.w_None + +set_symmetric_difference_update__Set_Frozenset = \ + set_symmetric_difference_update__Set_Set def set_symmetric_difference_update__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _symmetric_difference_dict(space, ld, rd, True) + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _symmetric_difference_dict(ld, rd, True) return space.w_None def inplace_xor__Set_Set(space, w_left, w_other): - set_symmetric_difference_update__Set_ANY(space, w_left, w_other) + set_symmetric_difference_update__Set_Set(space, w_left, w_other) return w_left inplace_xor__Set_Frozenset = inplace_xor__Set_Set +def set_union__Set_Set(space, w_left, w_other): + ld, rd = w_left.setdata, w_other.setdata + new_ld, rd = _union_dict(ld, rd, False) + return w_left._newobj(space, new_ld) + +set_union__Set_Frozenset = set_union__Set_Set +set_union__Frozenset_Set = set_union__Set_Set +set_union__Frozenset_Frozenset = set_union__Set_Set +or__Set_Set = set_union__Set_Set +or__Set_Frozenset = set_union__Set_Set +or__Frozenset_Set = set_union__Set_Set +or__Frozenset_Frozenset = set_union__Set_Set + + def set_union__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, _iter_to_dict(space, w_other) - new_ld, rd = _union_dict(space, ld, rd, False) - w_obj = w_left._newobj(space) - w_obj.setdata.update(new_ld) - return w_obj + ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) + new_ld, rd = _union_dict(ld, rd, False) + return w_left._newobj(space, new_ld) -or__Set_Set = set_union__Set_ANY -or__Set_Frozenset = set_union__Set_ANY frozenset_union__Frozenset_ANY = set_union__Set_ANY -or__Frozenset_Set = set_union__Set_ANY -or__Frozenset_Frozenset = set_union__Set_ANY def len__Set(space, w_left): return space.newint(len(w_left.setdata)) @@ -450,11 +543,16 @@ def init__Set(space, w_set, __args__): w_iterable, = __args__.parse('set', (['some_iterable'], None, None), - #[W_SetObject(space,None)]) [space.newtuple([])]) - if not w_set.in_init: + _initialize_set(space, w_set, w_iterable) + +def init__Frozenset(space, w_set, __args__): + w_iterable, = __args__.parse('set', + (['some_iterable'], None, None), + [space.newtuple([])]) + if w_set.hash == -1: _initialize_set(space, w_set, w_iterable) - w_set.in_init = False + hash__Frozenset(space, w_set) app = gateway.applevel(""" def ne__Set_ANY(s, o): Modified: pypy/dist/pypy/objspace/std/settype.py ============================================================================== --- pypy/dist/pypy/objspace/std/settype.py (original) +++ pypy/dist/pypy/objspace/std/settype.py Thu Apr 13 12:20:55 2006 @@ -26,20 +26,19 @@ register_all(vars(), globals()) -def descr__set__new__(space, w_settype, w_iterable=NoneNotWrapped): +def descr__new__(space, w_settype, __args__): from pypy.objspace.std.setobject import W_SetObject - if w_iterable is None: - w_iterable = space.newtuple([]) w_obj = space.allocate_instance(W_SetObject, w_settype) - W_SetObject.__init__(w_obj, space, w_iterable) - + W_SetObject.__init__(w_obj, space, None) return w_obj set_typedef = StdTypeDef("set", __doc__ = """set(iterable) --> set object Build an unordered collection.""", - __new__ = newmethod(descr__set__new__), + __new__ = newmethod(descr__new__, unwrap_spec=[gateway.ObjSpace, + gateway.W_Root, + gateway.Arguments]), ) set_typedef.registermethods(globals()) From auc at codespeak.net Thu Apr 13 12:34:09 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 13 Apr 2006 12:34:09 +0200 (CEST) Subject: [pypy-svn] r25778 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060413103409.BEF001007C@code0.codespeak.net> Author: auc Date: Thu Apr 13 12:34:07 2006 New Revision: 25778 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: merge Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Thu Apr 13 12:34:07 2006 @@ -100,7 +100,6 @@ self.status = self._space.newint(1) return self.status self.status = self._space.newint(self.distributor.fanout) - print "duh ?" return self.status def w_clone(self): @@ -118,6 +117,7 @@ def w_commit(self, w_choice): self.distributor.w_distribute(self, w_choice) + self.status = None def w_set_distributor(self, w_distributor): self.distributor = w_distributor @@ -153,11 +153,19 @@ return self._space.newlist(self.dependant_constraints(w_var)) def w_define_problem(self, w_problem): - print "there" - self.sol_set = self._space.call(w_problem, self) + self.sol_set = self._space.call(w_problem, self._space.newlist([self])) - def w_set_root(self, w_solution_list): - self.sol_set = w_solution_list + def w_merge(self): + res = [] + for var in self.sol_set.wrappeditems: + res.append(self.var_dom[var].get_values()[0]) + return self._space.newtuple(res) + + def w_print_state(self): + print "VARS :", self.name_var.keys() + print "CONST :", self.var_const.values() + print "DOMS :", self.var_dom.values() + print "CHK :", self.to_check #-- everything else --------------- @@ -197,7 +205,7 @@ # we should also remove the constraint from # the set of satifiable constraints of the space if const in affected_constraints: - affected_constraints.remove(const) + del affected_constraints[const] W_ComputationSpace.typedef = typedef.TypeDef( @@ -209,10 +217,10 @@ ask = interp2app(W_ComputationSpace.w_ask), clone = interp2app(W_ComputationSpace.w_clone), commit = interp2app(W_ComputationSpace.w_commit), + merge = interp2app(W_ComputationSpace.w_merge), dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints), define_problem = interp2app(W_ComputationSpace.w_define_problem), - set_root = interp2app(W_ComputationSpace.w_set_root)) - + print_state = interp2app(W_ComputationSpace.w_print_state)) def newspace(object_space): return W_ComputationSpace(object_space) Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Thu Apr 13 12:34:07 2006 @@ -104,3 +104,17 @@ csp.tell(make_expression([y, z], 'y 1 + + def test_merge(self): + csp = newspace() + def problem(csp): + x = csp.var('x', FiniteDomain([1])) + y = csp.var('y', FiniteDomain([1, 2])) + z = csp.var('z', FiniteDomain([1, 2, 3])) + csp.tell(make_expression([x, y], 'x Author: auc Date: Thu Apr 13 14:00:12 2006 New Revision: 25779 Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Log: fix clone Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Thu Apr 13 14:00:12 2006 @@ -98,6 +98,7 @@ self.distributor.find_distribution_variable(self) except: # FIXME: indexError ? self.status = self._space.newint(1) + self.w_print_state() return self.status self.status = self._space.newint(self.distributor.fanout) return self.status @@ -105,14 +106,19 @@ def w_clone(self): new = newspace(self._space) new.distributor = self.distributor + # copy the domains only for var, dom in self.var_dom.items(): new.var_dom[var] = dom.w_copy() - # !! be sure to not put state in constraints - new.constraints = self.constraints + # keep track of constraint check-list for const in self.to_check: new.to_check[const] = True + # copy the var->const mapping for var, const in self.var_const.items(): new.var_const[var] = const + # share other stateless stuff + new.constraints = self.constraints + new.name_var = self.name_var + new.sol_set = self.sol_set return new def w_commit(self, w_choice): Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Thu Apr 13 14:00:12 2006 @@ -118,3 +118,17 @@ csp.define_problem(problem) assert csp.ask() == 1 assert csp.merge() == (1, 2, 3) + + def test_clone_and_merge(self): + csp = newspace() + def problem(csp): + x = csp.var('x', FiniteDomain([1])) + y = csp.var('y', FiniteDomain([1, 2])) + z = csp.var('z', FiniteDomain([1, 2, 3])) + csp.tell(AllDistinct([x, y, z])) + return [x, y, z] + csp.define_problem(problem) + csp.print_state() + csp = csp.clone() + assert csp.ask() == 1 + assert csp.merge() == (1, 2, 3) From stephan at codespeak.net Thu Apr 13 14:09:16 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Thu, 13 Apr 2006 14:09:16 +0200 (CEST) Subject: [pypy-svn] r25780 - pypy/dist/pypy/objspace/std Message-ID: <20060413120916.13D321007C@code0.codespeak.net> Author: stephan Date: Thu Apr 13 14:09:14 2006 New Revision: 25780 Modified: pypy/dist/pypy/objspace/std/setobject.py Log: replaced a 'space.call' call by an 'space.call_function' call in _newobj Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Thu Apr 13 14:09:14 2006 @@ -28,7 +28,8 @@ elif objtype is W_FrozensetObject: obj = W_FrozensetObject(space, rdict_w) else: - obj = space.call(space.type(w_self),W_SetIterObject(space,rdict_w)) + itemiterator = space.iter(W_SetIterObject(space,rdict_w)) + obj = space.call_function(space.type(w_self),itemiterator) return obj class W_SetObject(W_BaseSetObject): From cfbolz at codespeak.net Thu Apr 13 14:53:25 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 13 Apr 2006 14:53:25 +0200 (CEST) Subject: [pypy-svn] r25786 - in pypy/dist/pypy/translator: . backendopt test Message-ID: <20060413125325.974B91007C@code0.codespeak.net> Author: cfbolz Date: Thu Apr 13 14:53:24 2006 New Revision: 25786 Modified: pypy/dist/pypy/translator/backendopt/removenoops.py pypy/dist/pypy/translator/simplify.py pypy/dist/pypy/translator/test/test_simplify.py Log: fix a rarely occuring bug in join_blocks: if the exitswitch of the second block was a constant in eh link coming from the first block, the exits of the second block would not be properly pruned. Also remove some op.cleanup leftovers as well as some visit usages in removenoops. Modified: pypy/dist/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/removenoops.py Thu Apr 13 14:53:24 2006 @@ -9,12 +9,10 @@ """Removes unary low-level ops with a name appearing in the opnames list. """ positions = [] - def visit(node): - if isinstance(node, Block): - for i, op in enumerate(node.operations): - if op.opname in opnames: - positions.append((node, i)) - traverse(visit, graph) + for block in graph.iterblocks(): + for i, op in enumerate(block.operations): + if op.opname in opnames: + positions.append((block, i)) while positions: block, index = positions.pop() op_result = block.operations[index].result @@ -34,22 +32,13 @@ if isinstance(op_arg, Variable): block.exitswitch = op_arg else: - assert isinstance(op_arg, Constant) - newexits = [link for link in block.exits - if link.exitcase == op_arg.value] - assert len(newexits) == 1 - newexits[0].exitcase = None - if hasattr(newexits[0], 'llexitcase'): - newexits[0].llexitcase = None - block.exitswitch = None - block.recloseblock(*newexits) + simplify.replace_exitswitch_by_constant(block, op_arg) block.operations[index] = None # remove all operations - def visit(node): - if isinstance(node, Block) and node.operations: - node.operations[:] = filter(None, node.operations) - traverse(visit, graph) + for block in graph.iterblocks(): + if block.operations: + block.operations[:] = filter(None, block.operations) def remove_same_as(graph): remove_unaryops(graph, ["same_as"]) Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Thu Apr 13 14:53:24 2006 @@ -37,6 +37,18 @@ return None +def replace_exitswitch_by_constant(block, const): + assert isinstance(const, Constant) + assert const != c_last_exception + newexits = [link for link in block.exits + if link.exitcase == const.value] + assert len(newexits) == 1 + newexits[0].exitcase = None + if hasattr(newexits[0], 'llexitcase'): + newexits[0].llexitcase = None + block.exitswitch = None + block.recloseblock(*newexits) + # ____________________________________________________________ def eliminate_empty_blocks(graph): @@ -303,26 +315,13 @@ len(entrymap[link.target]) == 1 and link.target.exits): # stop at the returnblock renaming = {} - cache_cleanups = {} for vprev, vtarg in zip(link.args, link.target.inputargs): renaming[vtarg] = vprev def rename(v): return renaming.get(v, v) def rename_op(op): args = [rename(a) for a in op.args] - if hasattr(op, "cleanup"): - if op.cleanup is None: - cleanup = None - elif op.cleanup not in cache_cleanups: - finallyops, exceptops = op.cleanup - cleanup = (tuple([rename_op(fo) for fo in finallyops]), - tuple([rename_op(eo) for eo in exceptops])) - cache_cleanups[op.cleanup] = cleanup - else: - cleanup = cache_cleanups[op.cleanup] - op = SpaceOperation(op.opname, args, rename(op.result), cleanup=cleanup) - else: - op = SpaceOperation(op.opname, args, rename(op.result)) + op = SpaceOperation(op.opname, args, rename(op.result)) return op for op in link.target.operations: link.prevblock.operations.append(rename_op(op)) @@ -330,8 +329,11 @@ for exit in link.target.exits: newexit = exit.copy(rename) exits.append(newexit) - link.prevblock.exitswitch = rename(link.target.exitswitch) + newexitswitch = rename(link.target.exitswitch) + link.prevblock.exitswitch = newexitswitch link.prevblock.recloseblock(*exits) + if isinstance(newexitswitch, Constant) and newexitswitch != c_last_exception: + replace_exitswitch_by_constant(link.prevblock, newexitswitch) stack.extend(exits) else: if link.target not in seen: Modified: pypy/dist/pypy/translator/test/test_simplify.py ============================================================================== --- pypy/dist/pypy/translator/test/test_simplify.py (original) +++ pypy/dist/pypy/translator/test/test_simplify.py Thu Apr 13 14:53:24 2006 @@ -154,3 +154,21 @@ assert g(1) == 1001 # does not crash: previously join_blocks would barf on this graph, t = translate(g, [int]) + +def test_join_blocks_cleans_links(): + from pypy.rpython.lltypesystem import lltype + from pypy.objspace.flow.model import Constant + from pypy.translator.backendopt.removenoops import remove_same_as + def f(x): + return bool(x + 2) + def g(x): + if f(x): + return 1 + else: + return 2 + graph, t = translate(g, [int], backend_optimize=False) + fgraph = graphof(t, f) + fgraph.startblock.exits[0].args = [Constant(True, lltype.Bool)] + # does not crash: previously join_blocks would barf on this + remove_same_as(graph) + backend_optimizations(t) From antocuni at codespeak.net Thu Apr 13 14:59:21 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 13 Apr 2006 14:59:21 +0200 (CEST) Subject: [pypy-svn] r25789 - in pypy/dist/pypy/rpython: . lltypesystem ootypesystem test Message-ID: <20060413125921.DF59F1007C@code0.codespeak.net> Author: antocuni Date: Thu Apr 13 14:59:07 2006 New Revision: 25789 Added: pypy/dist/pypy/rpython/lltypesystem/rslice.py - copied, changed from r25772, pypy/dist/pypy/rpython/rslice.py pypy/dist/pypy/rpython/ootypesystem/rslice.py (contents, props changed) Removed: pypy/dist/pypy/rpython/rslice.py Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/rstr.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/rpython/typesystem.py Log: rpython.rslice has been made typesystem-specific; the old rslice implementation has been moved to lltypesystem, and a new one (based on classes) has been placed in ootypesystem. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Thu Apr 13 14:59:07 2006 @@ -7,9 +7,9 @@ from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ AbstractListIteratorRepr, rtype_newlist from pypy.rpython.rlist import dum_nocheck, dum_checkidx -from pypy.rpython.rslice import SliceRepr -from pypy.rpython.rslice import startstop_slice_repr, startonly_slice_repr -from pypy.rpython.rslice import minusone_slice_repr +from pypy.rpython.lltypesystem.rslice import SliceRepr +from pypy.rpython.lltypesystem.rslice import startstop_slice_repr, startonly_slice_repr +from pypy.rpython.lltypesystem.rslice import minusone_slice_repr from pypy.rpython.lltypesystem. lltype import \ GcForwardReference, Ptr, GcArray, GcStruct, \ Void, Signed, malloc, typeOf, Primitive, \ Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Thu Apr 13 14:59:07 2006 @@ -3,9 +3,12 @@ AbstractListIteratorRepr, rtype_newlist from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.rmodel import inputconst, externalvsinternal +from pypy.rpython.lltypesystem.lltype import Signed from pypy.rpython.ootypesystem import ootype from pypy.rpython.ootypesystem.riterable import iterator_type -from pypy.rpython.lltypesystem.lltype import Signed +from pypy.rpython.ootypesystem.rslice import SliceRepr, \ + startstop_slice_repr, startonly_slice_repr, minusone_slice_repr + class BaseListRepr(AbstractBaseListRepr): @@ -78,6 +81,22 @@ hop.exception_is_here() return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) + +class __extend__(pairtype(BaseListRepr, SliceRepr)): + + def rtype_getitem((r_list, r_slic), hop): + raise NotImplementedError # TODO +## cRESLIST = hop.inputconst(Void, hop.r_result.LIST) +## if r_slic == startonly_slice_repr: +## v_lst, v_start = hop.inputargs(r_lst, startonly_slice_repr) +## return hop.gendirectcall(ll_listslice_startonly, cRESLIST, v_lst, v_start) +## if r_slic == startstop_slice_repr: +## v_lst, v_slice = hop.inputargs(r_lst, startstop_slice_repr) +## return hop.gendirectcall(ll_listslice, cRESLIST, v_lst, v_slice) +## if r_slic == minusone_slice_repr: +## v_lst, v_ignored = hop.inputargs(r_lst, minusone_slice_repr) +## return hop.gendirectcall(ll_listslice_minusone, cRESLIST, v_lst) +## raise TyperError('getitem does not support slices with %r' % (r_slic,)) def ll_getitem(lst, index): Added: pypy/dist/pypy/rpython/ootypesystem/rslice.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/rslice.py Thu Apr 13 14:59:07 2006 @@ -0,0 +1,24 @@ +from pypy.rpython.rmodel import Repr +from pypy.rpython.lltypesystem.lltype import Void, Signed +from pypy.rpython.ootypesystem import ootype + +SLICE = ootype.Instance('Slice', ootype.ROOT, {'start': Signed, 'stop': Signed}) + +class SliceRepr(Repr): + pass + +startstop_slice_repr = SliceRepr() +startstop_slice_repr.lowleveltype = SLICE +startonly_slice_repr = SliceRepr() +startonly_slice_repr.lowleveltype = Signed +minusone_slice_repr = SliceRepr() +minusone_slice_repr.lowleveltype = Void # only for [:-1] + +# ____________________________________________________________ + +def ll_newslice(start, stop): + s = ootype.new(SLICE) + s.start = start + s.stop = stop + return s + Modified: pypy/dist/pypy/rpython/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/rstr.py (original) +++ pypy/dist/pypy/rpython/rstr.py Thu Apr 13 14:59:07 2006 @@ -8,9 +8,9 @@ from pypy.rpython.robject import PyObjRepr, pyobj_repr from pypy.rpython.lltypesystem.rtuple import TupleRepr # XXX type system! from pypy.rpython import rint -from pypy.rpython.rslice import SliceRepr -from pypy.rpython.rslice import startstop_slice_repr, startonly_slice_repr -from pypy.rpython.rslice import minusone_slice_repr +from pypy.rpython.lltypesystem.rslice import SliceRepr # XXX type system! +from pypy.rpython.lltypesystem.rslice import startstop_slice_repr, startonly_slice_repr +from pypy.rpython.lltypesystem.rslice import minusone_slice_repr from pypy.rpython.lltypesystem.lltype import \ GcStruct, Signed, Array, Char, Ptr, malloc, \ Bool, Void, GcArray, nullptr, pyobjectptr Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Thu Apr 13 14:59:07 2006 @@ -2,7 +2,7 @@ from pypy.translator.translator import TranslationContext from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.lltypesystem.rlist import * -from pypy.rpython.rslice import ll_newslice +from pypy.rpython.lltypesystem.rslice import ll_newslice from pypy.rpython.rint import signed_repr from pypy.rpython.test.test_llinterp import interpret, interpret_raises from pypy.translator.translator import TranslationContext Modified: pypy/dist/pypy/rpython/typesystem.py ============================================================================== --- pypy/dist/pypy/rpython/typesystem.py (original) +++ pypy/dist/pypy/rpython/typesystem.py Thu Apr 13 14:59:07 2006 @@ -20,7 +20,7 @@ None, None, ['__doc__']) except ImportError: return None - if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', + if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', 'rslice', 'exceptiondata'): mod = load(name) if mod is not None: From auc at codespeak.net Thu Apr 13 15:43:50 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 13 Apr 2006 15:43:50 +0200 (CEST) Subject: [pypy-svn] r25792 - in pypy/dist/pypy/objspace/constraint: . applevel test Message-ID: <20060413134350.F2BB41007B@code0.codespeak.net> Author: auc Date: Thu Apr 13 15:43:48 2006 New Revision: 25792 Modified: pypy/dist/pypy/objspace/constraint/applevel/problems.py pypy/dist/pypy/objspace/constraint/applevel/solver.py pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_solver.py Log: solver basically working - but beware AllDistinct weirdness Modified: pypy/dist/pypy/objspace/constraint/applevel/problems.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/applevel/problems.py (original) +++ pypy/dist/pypy/objspace/constraint/applevel/problems.py Thu Apr 13 15:43:48 2006 @@ -5,6 +5,8 @@ return (ret,) def send_more_money(computation_space): + #FIXME: this problem needs propagators for integer finite domains + # performance is terrible without it cs = computation_space variables = (s, e, n, d, m, o, r, y) = cs.make_vars('s', 'e', 'n', 'd', 'm', 'o', 'r', 'y') @@ -40,7 +42,6 @@ variables = [cs.var(v, FiniteDomain(dom_values)) for v in ('c01','c02','c03','c04','c05', 'c06','c07','c08','c09','c10')] - for conf in ('c03','c04','c05','c06'): v = cs.find_var(conf) cs.tell(make_expression([v], "%s[0] == 'room C'" % conf)) @@ -58,11 +59,18 @@ ('c03','c05','c06','c07'), ('c01','c03','c07','c08')) - for group in groups: - cs.tell(AllDistinct([cs.find_var(v) for v in group])) +## for group in groups: +## cs.tell(AllDistinct([cs.find_var(v) for v in group])) ## cs.add_expression(AllDistinct(cs, tuple([cs.find_var(v) ## for v in group]))) + for group in groups: + for conf1 in group: + for conf2 in group: + v1, v2 = cs.find_vars((conf1, conf2)) + if conf2 > conf1: + cs.tell(make_expression([v1, v2], '%s[1] != %s[1]'% (v1.name(),v2.name()))) + ## for g in groups: ## for conf1 in g: ## for conf2 in g: Modified: pypy/dist/pypy/objspace/constraint/applevel/solver.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/applevel/solver.py (original) +++ pypy/dist/pypy/objspace/constraint/applevel/solver.py Thu Apr 13 15:43:48 2006 @@ -22,20 +22,21 @@ def collect(space): sp_stack.appendleft(space) + print "ready to find solution ..." while len(sp_stack): space = sp_stack.pop() - print ' '*len(sp_stack), "ask ..." + print ' '*len(sp_stack), "ask [depth = %s]" % len(sp_stack) status = space.ask() if status == 1: print ' '*len(sp_stack), "solution !" yield space.merge() elif status > 1: - print ' '*len(sp_stack), "branches ..." - sp1 = space.clone() - sp1.commit(1) - collect(sp1) - sp2 = space.clone() - sp2.commit(2) - collect(sp2) + print ' '*len(sp_stack), "%s branches ..." % status + for i in range(status): + clone = space.clone() + clone.commit(status-i) + collect(clone) + elif status == 0: + print ' '*len(sp_stack), "dead-end" solve = lazily_iter_solve_all Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Thu Apr 13 15:43:48 2006 @@ -98,7 +98,6 @@ self.distributor.find_distribution_variable(self) except: # FIXME: indexError ? self.status = self._space.newint(1) - self.w_print_state() return self.status self.status = self._space.newint(self.distributor.fanout) return self.status @@ -112,11 +111,10 @@ # keep track of constraint check-list for const in self.to_check: new.to_check[const] = True - # copy the var->const mapping - for var, const in self.var_const.items(): - new.var_const[var] = const # share other stateless stuff + # this sharing will pose problems ... new.constraints = self.constraints + new.var_const = self.var_const new.name_var = self.name_var new.sol_set = self.sol_set return new @@ -143,6 +141,12 @@ def w_find_var(self, w_name): return self.name_var[self._space.str_w(w_name)] + def w_find_vars(self, w_vars): + res = [] + for var in w_vars.wrappeditems: + res.append(self.w_find_var(var)) + return self._space.newlist(res) + def w_dom(self, w_variable): assert isinstance(w_variable, W_Variable) return self.var_dom[w_variable] @@ -167,6 +171,18 @@ res.append(self.var_dom[var].get_values()[0]) return self._space.newtuple(res) + def w_test_solution(self, w_sol): + varset = {} + for var, val in zip(self.sol_set.wrappeditems, + w_sol.wrappeditems): + varset[var.w_name] = val + for _const in self.constraints: + if not _const.test_solution(varset): + print "Solution", sol, "doesn't satisfy", _const + return self._space.newbool(False) + return self._space.newbool(True) + + def w_print_state(self): print "VARS :", self.name_var.keys() print "CONST :", self.var_const.values() @@ -218,6 +234,7 @@ "W_ComputationSpace", var = interp2app(W_ComputationSpace.w_var), find_var = interp2app(W_ComputationSpace.w_find_var), + find_vars = interp2app(W_ComputationSpace.w_find_vars), dom = interp2app(W_ComputationSpace.w_dom), tell = interp2app(W_ComputationSpace.w_tell), ask = interp2app(W_ComputationSpace.w_ask), @@ -226,6 +243,7 @@ merge = interp2app(W_ComputationSpace.w_merge), dependant_constraints = interp2app(W_ComputationSpace.w_dependant_constraints), define_problem = interp2app(W_ComputationSpace.w_define_problem), + test_solution = interp2app(W_ComputationSpace.w_test_solution), print_state = interp2app(W_ComputationSpace.w_print_state)) def newspace(object_space): Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Thu Apr 13 15:43:48 2006 @@ -85,7 +85,7 @@ assert isinstance(w_cs, W_ComputationSpace) return self.__cost - def test_solution(self, sol): + def test_solution(self, w_sol): """test a solution against this constraint accept a mapping of variable names to value""" values = sol.items() @@ -103,7 +103,7 @@ # then the value must be removed from the other domains for size, var, dom in variables: if self._space.eq_w(dom.w_size(), self._space.newint(1)): - print "AllDistinct removes values" + #print "AllDistinct removes values" for _siz, _var, _dom in variables: if not self._space.eq_w(_var, var): try: @@ -119,7 +119,7 @@ for val in dom.w_get_values().wrappeditems: values[val] = 0 if len(values) < len(variables): - print "AllDistinct failed" + #print "AllDistinct failed" raise OperationError(self._space.w_RuntimeError, self._space.wrap("Consistency Failure")) @@ -163,13 +163,13 @@ self.formula = self._space.str_w(w_formula) self.filter_func = make_filter(self._space, w_variables, w_formula) - def test_solution(self, sol ): + def test_solution(self, sol_dict): """test a solution against this constraint accept a mapping of variable names to value""" args = [] for var in self._variables: - args.append( sol[var.w_name] ) - return self.filterFunc( *args ) + args.append(sol_dict[var.w_name]) + return self.filter_func(*args) def _init_result_cache(self): """key = (variable,value), value = [has_success,has_failure]""" Modified: pypy/dist/pypy/objspace/constraint/test/test_solver.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_solver.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_solver.py Thu Apr 13 15:43:48 2006 @@ -9,9 +9,19 @@ from pypy.objspace.constraint.applevel import solver, problems spc = newspace() - spc.set_root(problems.conference_scheduling(spc)) - #FIXME: that 'interation over non-sequence' kills me ... - #spc.define_problem(problems.conference_scheduling) + spc.define_problem(problems.conference_scheduling) sols = solver.solve(spc) assert str(type(sols)) == "" + + def test_solve(self): + from pypy.objspace.constraint.applevel import solver, problems + spc = newspace() + + spc.define_problem(problems.conference_scheduling) + + sols = solver.solve(spc) + count = 0 + for sol in sols: + count += 1 + assert count == 64 From cfbolz at codespeak.net Thu Apr 13 16:05:57 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 13 Apr 2006 16:05:57 +0200 (CEST) Subject: [pypy-svn] r25793 - in pypy/dist/pypy: rpython/memory translator/c Message-ID: <20060413140557.F194710080@code0.codespeak.net> Author: cfbolz Date: Thu Apr 13 16:05:57 2006 New Revision: 25793 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/c/gc.py Log: remove the really strange inheritance of the Framework classes from the boehm classes Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Thu Apr 13 16:05:57 2006 @@ -664,7 +664,7 @@ yield a -class FrameworkGCTransformer(BoehmGCTransformer): +class FrameworkGCTransformer(GCTransformer): def __init__(self, translator): from pypy.rpython.memory.support import get_address_linked_list Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Thu Apr 13 16:05:57 2006 @@ -263,11 +263,22 @@ def pre_pre_gc_code(self): yield '#define USING_NO_GC' -# the framework GC policy -- we are very optimistic tonight -class FrameworkGcPolicy(NoneGcPolicy): +class FrameworkGcPolicy(BasicGcPolicy): transformerclass = gctransform.FrameworkGCTransformer + def struct_setup(self, structdefnode, rtti): + pass + + def array_setup(self, arraydefnode): + pass + + def rtti_type(self): + return BoehmGcRuntimeTypeInfo_OpaqueNode.typename + + def rtti_node_factory(self): + return BoehmGcRuntimeTypeInfo_OpaqueNode + def gc_startup_code(self): fnptr = self.db.gctransformer.frameworkgc_setup_ptr.value yield '%s();' % (self.db.get(fnptr),) @@ -292,3 +303,9 @@ break o = n return [defnode.db.gctransformer.id_of_type[typeOf(o)] << 1] + + def zero_malloc(self, TYPE, esize, eresult): + assert TYPE._gcstatus() # we don't really support this + return 'OP_ZERO_MALLOC(%s, %s);' % (esize, + eresult) + From arigo at codespeak.net Thu Apr 13 16:22:08 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 16:22:08 +0200 (CEST) Subject: [pypy-svn] r25795 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060413142208.EC37F10080@code0.codespeak.net> Author: arigo Date: Thu Apr 13 16:22:05 2006 New Revision: 25795 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Log: Yet another strange lltype operation: cast_structfield_pointer(). It allows a pointer to a structure field to be taken. This pointer gets a type of Ptr(FixedSizeArray(FIELD_TYPE, 1)). As with cast_subarray_pointer(), the actual implementation will use address manipulations. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Thu Apr 13 16:22:05 2006 @@ -129,7 +129,7 @@ def set(self, value): setattr(self.struct, self.fieldname, value) def type(self): - return getattr(lltype.typeOf(self.struct), self.fieldname) + return getattr(lltype.typeOf(self.struct).TO, self.fieldname) class _obref(object): def __init__(self, ob): @@ -218,12 +218,18 @@ ref = self.ref() if (isinstance(ref, _arrayitemref) and isinstance(EXPECTED_TYPE.TO, lltype.FixedSizeArray) and - isinstance(lltype.typeOf(ref.array).TO, (lltype.FixedSizeArray, - lltype.Array))): + ref.type() == EXPECTED_TYPE.TO.OF): # special case that requires cast_subarray_pointer return lltype.cast_subarray_pointer(EXPECTED_TYPE, ref.array, ref.index) + elif (isinstance(ref, _structfieldref) and + isinstance(EXPECTED_TYPE.TO, lltype.FixedSizeArray) and + ref.type() == EXPECTED_TYPE.TO.OF): + # special case that requires cast_structfield_pointer + return lltype.cast_structfield_pointer(EXPECTED_TYPE, + ref.struct, + ref.fieldname) else: # regular case assert isinstance(ref.type(), lltype.ContainerType) Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Thu Apr 13 16:22:05 2006 @@ -636,6 +636,31 @@ subarray = cache[key] = _subarray(ARRAYTYPE, arrayptr._obj, baseoffset) return _ptr(ARRAYPTRTYPE, subarray) +def cast_structfield_pointer(ARRAYPTRTYPE, structptr, fieldname): + CURPTRTYPE = typeOf(structptr) + if not isinstance(CURPTRTYPE, Ptr) or not isinstance(ARRAYPTRTYPE, Ptr): + raise TypeError, "can only cast pointers to other pointers" + ARRAYTYPE = ARRAYPTRTYPE.TO + if (not isinstance(CURPTRTYPE.TO, Struct) or + not isinstance(ARRAYTYPE, FixedSizeArray)): + raise TypeError, "can only cast from (Gc)Struct to FixedSizeArray" + if ARRAYTYPE.length != 1: + raise TypeError, "can only cast to FixedSizeArray of length 1" + if getattr(CURPTRTYPE.TO, fieldname) != ARRAYTYPE.OF: + raise TypeError, "mismatching array item type" + if not structptr: + raise RuntimeError("cast_structfield_pointer: NULL argument") + try: + cache = _subarray._cache[structptr._obj] + except KeyError: + cache = _subarray._cache[structptr._obj] = {} + try: + subarray = cache[fieldname] + except KeyError: + subarray = cache[fieldname] = _subarray(ARRAYTYPE, structptr._obj, + fieldname) + return _ptr(ARRAYPTRTYPE, subarray) + def _expose(val, solid=False): """XXX A nice docstring here""" T = typeOf(val) @@ -1066,12 +1091,13 @@ class _subarray(_parentable): # only for cast_subarray_pointer() + # and cast_structfield_pointer() _kind = "subarray" _cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays} - def __init__(self, TYPE, arrayobj, baseoffset): + def __init__(self, TYPE, parent, baseoffset_or_fieldname): _parentable.__init__(self, TYPE) - self._setparentstructure(arrayobj, baseoffset) + self._setparentstructure(parent, baseoffset_or_fieldname) def getlength(self): assert isinstance(self._TYPE, FixedSizeArray) @@ -1079,16 +1105,28 @@ def getbounds(self): baseoffset = self._parent_index + if isinstance(baseoffset, str): + return 0, 1 # structfield case start, stop = self._parentstructure().getbounds() return start - baseoffset, stop - baseoffset def getitem(self, index): baseoffset = self._parent_index - return self._parentstructure().getitem(baseoffset + index) + if isinstance(baseoffset, str): + assert index == 0 + fieldname = baseoffset # structfield case + return getattr(self._parentstructure(), fieldname) + else: + return self._parentstructure().getitem(baseoffset + index) def setitem(self, index, value): baseoffset = self._parent_index - self._parentstructure().setitem(baseoffset + index, value) + if isinstance(baseoffset, str): + assert index == 0 + fieldname = baseoffset # structfield case + setattr(self._parentstructure(), fieldname, value) + else: + self._parentstructure().setitem(baseoffset + index, value) class _func(object): Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Thu Apr 13 16:22:05 2006 @@ -173,3 +173,12 @@ assert subarray[0] == 132 subarray[0] += 2 assert a[3] == 134 + +def test_cast_structfield_pointer(): + S = lltype.GcStruct('S', ('x', lltype.Signed), ('y', lltype.Signed)) + s = lltype.malloc(S) + SUBARRAY = lltype.FixedSizeArray(lltype.Signed, 1) + adr = cast_ptr_to_adr(s) + offsetof(S, 'y') + subarray = cast_adr_to_ptr(adr, lltype.Ptr(SUBARRAY)) + subarray[0] = 121 + assert s.y == 121 Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Thu Apr 13 16:22:05 2006 @@ -610,3 +610,12 @@ py.test.raises(IndexError, "b01[-1]") py.test.raises(IndexError, "b34[2]") py.test.raises(IndexError, "b12[4]") + +def test_cast_structfield_pointer(): + S = GcStruct('S', ('x', Signed), ('y', Signed)) + A = FixedSizeArray(Signed, 1) + s = malloc(S) + a = cast_structfield_pointer(Ptr(A), s, 'y') + a[0] = 34 + assert s.y == 34 + py.test.raises(IndexError, "a[1]") From auc at codespeak.net Thu Apr 13 16:23:49 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 13 Apr 2006 16:23:49 +0200 (CEST) Subject: [pypy-svn] r25796 - pypy/dist/pypy/objspace/constraint/applevel Message-ID: <20060413142349.6B84010081@code0.codespeak.net> Author: auc Date: Thu Apr 13 16:23:48 2006 New Revision: 25796 Added: pypy/dist/pypy/objspace/constraint/applevel/__init__.py Log: missing init Added: pypy/dist/pypy/objspace/constraint/applevel/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/applevel/__init__.py Thu Apr 13 16:23:48 2006 @@ -0,0 +1 @@ +#void From arigo at codespeak.net Thu Apr 13 16:37:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 16:37:36 +0200 (CEST) Subject: [pypy-svn] r25798 - in pypy/dist/pypy: annotation rpython rpython/rctypes rpython/rctypes/test rpython/test Message-ID: <20060413143736.282A610085@code0.codespeak.net> Author: arigo Date: Thu Apr 13 16:37:28 2006 New Revision: 25798 Modified: pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/test/test_rbuiltin.py Log: RTyping of cast_structfield_pointer(), with fixes to rctypes.rarray and a couple extra tests. Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Thu Apr 13 16:37:28 2006 @@ -429,6 +429,15 @@ 0) return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) +def cast_structfield_pointer(PtrT, s_p, s_fieldname): + assert isinstance(s_p, SomePtr), "casting of non-pointer: %r" % s_p + assert PtrT.is_constant() + assert s_fieldname.is_constant() + cast_p = lltype.cast_structfield_pointer(PtrT.const, + s_p.ll_ptrtype._example(), + s_fieldname.const) + return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) + def cast_ptr_to_int(s_ptr): # xxx return SomeInteger() @@ -446,6 +455,7 @@ BUILTIN_ANALYZERS[lltype.nullptr] = nullptr BUILTIN_ANALYZERS[lltype.cast_pointer] = cast_pointer BUILTIN_ANALYZERS[lltype.cast_subarray_pointer] = cast_subarray_pointer +BUILTIN_ANALYZERS[lltype.cast_structfield_pointer] = cast_structfield_pointer BUILTIN_ANALYZERS[lltype.cast_ptr_to_int] = cast_ptr_to_int BUILTIN_ANALYZERS[lltype.getRuntimeTypeInfo] = getRuntimeTypeInfo BUILTIN_ANALYZERS[lltype.runtime_type_info] = runtime_type_info Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Thu Apr 13 16:37:28 2006 @@ -315,10 +315,10 @@ hop.exception_cannot_occur() # instead of adding a 'cast_subarray_pointer' ll operation, # we do it with address manipulations - return gen_cast_array_pointer(hop.llops, hop.r_result.lowleveltype, - v_input, v_baseoffset) + return gen_cast_subarray_pointer(hop.llops, hop.r_result.lowleveltype, + v_input, v_baseoffset) -def gen_cast_array_pointer(llops, ARRAYPTRTYPE, v_array, v_baseoffset): +def gen_cast_subarray_pointer(llops, ARRAYPTRTYPE, v_array, v_baseoffset): "Helper to generate the equivalent of a 'cast_subarray_pointer' operation." from pypy.rpython.lltypesystem import llmemory SRCTYPE = v_array.concretetype.TO @@ -335,6 +335,32 @@ return llops.genop('cast_adr_to_ptr', [v_base_adr], resulttype = ARRAYPTRTYPE) +def rtype_cast_structfield_pointer(hop): + assert hop.args_s[0].is_constant() + assert hop.args_s[2].is_constant() + fieldname = hop.args_s[2].const + assert isinstance(hop.args_r[1], rptr.PtrRepr) + v_type, v_input, v_fieldname = hop.inputargs(lltype.Void, hop.args_r[1], + lltype.Void) + hop.exception_cannot_occur() + # instead of adding a 'cast_structfield_pointer' ll operation, + # we do it with address manipulations + return gen_cast_structfield_pointer(hop.llops, hop.r_result.lowleveltype, + v_input, fieldname) + +def gen_cast_structfield_pointer(llops, ARRAYPTRTYPE, v_struct, fieldname): + "Helper to generate the equivalent of a 'cast_structfield_pointer' op." + from pypy.rpython.lltypesystem import llmemory + SRCTYPE = v_struct.concretetype.TO + v_struct_adr = llops.genop('cast_ptr_to_adr', [v_struct], + resulttype = llmemory.Address) + c_ofs1 = inputconst(lltype.Signed, + llmemory.FieldOffset(SRCTYPE, fieldname)) + v_base_adr = llops.genop('adr_add', [v_struct_adr, c_ofs1], + resulttype = llmemory.Address) + return llops.genop('cast_adr_to_ptr', [v_base_adr], + resulttype = ARRAYPTRTYPE) + def rtype_cast_primitive(hop): assert hop.args_s[0].is_constant() TGT = hop.args_s[0].const @@ -370,6 +396,7 @@ BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_subarray_pointer] = rtype_cast_subarray_pointer +BUILTIN_TYPER[lltype.cast_structfield_pointer] = rtype_cast_structfield_pointer BUILTIN_TYPER[lltype.cast_ptr_to_int] = rtype_cast_ptr_to_int BUILTIN_TYPER[lltype.typeOf] = rtype_const_result BUILTIN_TYPER[lltype.nullptr] = rtype_const_result Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Thu Apr 13 16:37:28 2006 @@ -1,7 +1,7 @@ from ctypes import ARRAY, c_int from pypy.annotation.model import SomeCTypesObject, SomeBuiltin from pypy.rpython import extregistry -from pypy.rpython.rbuiltin import gen_cast_array_pointer +from pypy.rpython.rbuiltin import gen_cast_subarray_pointer from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype @@ -47,8 +47,8 @@ else: # ByValue case A = lltype.FixedSizeArray(self.r_item.ll_type, 1) - return gen_cast_array_pointer(llops, lltype.Ptr(A), - v_c_array, v_index) + return gen_cast_subarray_pointer(llops, lltype.Ptr(A), + v_c_array, v_index) def get_item_value(self, llops, v_array, v_index): # ByValue case only @@ -88,6 +88,7 @@ v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) return r_array.r_item.return_c_data(hop.llops, v_c_data) +# ____________________________________________________________ def arraytype_specialize_call(hop): r_array = hop.r_result Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Thu Apr 13 16:37:28 2006 @@ -169,3 +169,17 @@ fn = compile(access_array, []) assert fn() == 2 + + def test_compile_prebuilt(self): + my_array_2 = (c_short*10)(0, 1, 4, 9, 16, 25, 36, 49, 64, 81) + my_array_3 = (c_short*10)(0, 1, 8, 27, 64, 125, 216, 343, 512, 729) + def func(i, n): + if i == 2: + array = my_array_2 + else: + array = my_array_3 + return array[n] + + fn = compile(func, [int, int]) + assert fn(2, 7) == 49 + assert fn(3, 6) == 216 Modified: pypy/dist/pypy/rpython/test/test_rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rbuiltin.py (original) +++ pypy/dist/pypy/rpython/test/test_rbuiltin.py Thu Apr 13 16:37:28 2006 @@ -335,3 +335,50 @@ return lltype.cast_primitive(lltype.UniChar, v) res = interpret(llf, [ord('x')], policy=LowLevelAnnotatorPolicy()) assert res == u'x' + +def test_cast_subarray_pointer(): + from pypy.rpython.lltypesystem.lltype import malloc, GcArray, Signed + from pypy.rpython.lltypesystem.lltype import FixedSizeArray, Ptr + for a in [malloc(GcArray(Signed), 5), + malloc(FixedSizeArray(Signed, 5), immortal=True)]: + a[0] = 0 + a[1] = 10 + a[2] = 20 + a[3] = 30 + a[4] = 40 + BOX = Ptr(FixedSizeArray(Signed, 2)) + b01 = lltype.cast_subarray_pointer(BOX, a, 0) + b12 = lltype.cast_subarray_pointer(BOX, a, 1) + b23 = lltype.cast_subarray_pointer(BOX, a, 2) + b34 = lltype.cast_subarray_pointer(BOX, a, 3) + def llf(n): + saved = a[n] + a[n] = 1000 + try: + return b01[0] + b12[0] + b23[1] + b34[1] + finally: + a[n] = saved + + res = interpret(llf, [0]) + assert res == 1000 + 10 + 30 + 40 + res = interpret(llf, [1]) + assert res == 0 + 1000 + 30 + 40 + res = interpret(llf, [2]) + assert res == 0 + 10 + 30 + 40 + res = interpret(llf, [3]) + assert res == 0 + 10 + 1000 + 40 + res = interpret(llf, [4]) + assert res == 0 + 10 + 30 + 1000 + +def test_cast_structfield_pointer(): + S = lltype.GcStruct('S', ('x', lltype.Signed), ('y', lltype.Signed)) + SUBARRAY = lltype.FixedSizeArray(lltype.Signed, 1) + P = lltype.Ptr(SUBARRAY) + def llf(n): + s = lltype.malloc(S) + a = lltype.cast_structfield_pointer(P, s, 'y') + a[0] = n + return s.y + + res = interpret(llf, [34]) + assert res == 34 From arigo at codespeak.net Thu Apr 13 16:44:50 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 16:44:50 +0200 (CEST) Subject: [pypy-svn] r25799 - pypy/dist/pypy/translator/c/test Message-ID: <20060413144450.BCBD810086@code0.codespeak.net> Author: arigo Date: Thu Apr 13 16:44:48 2006 New Revision: 25799 Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py Log: Two more tests for cast_*_pointer() translation. Passing out of the box. The generated code looks ok. Don't ask me exactly how. Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Thu Apr 13 16:44:48 2006 @@ -160,3 +160,41 @@ assert res == 0 + 10 + 1000 + 40 res = fn(4) assert res == 0 + 10 + 30 + 1000 + + def test_cast_structfield_pointer(self): + S = GcStruct('S', ('x', Signed), ('y', Signed)) + SUBARRAY = FixedSizeArray(Signed, 1) + P = Ptr(SUBARRAY) + def llf(n=int): + s = malloc(S) + a = cast_structfield_pointer(P, s, 'y') + a[0] = n + return s.y + + fn = self.getcompiled(llf) + res = fn(34) + assert res == 34 + + def test_prebuilt_subarrays(self): + a1 = malloc(GcArray(Signed), 5) + a2 = malloc(FixedSizeArray(Signed, 5), immortal=True) + s = malloc(GcStruct('S', ('x', Signed), ('y', Signed))) + a1[3] = 7000 + a2[1] = 600 + s.x = 50 + s.y = 4 + P = Ptr(FixedSizeArray(Signed, 1)) + p1 = cast_subarray_pointer(P, a1, 3) + p2 = cast_subarray_pointer(P, a2, 1) + p3 = cast_structfield_pointer(P, s, 'x') + p4 = cast_structfield_pointer(P, s, 'y') + def llf(): + a1[3] += 1000 + a2[1] += 100 + s.x += 10 + s.y += 1 + return p1[0] + p2[0] + p3[0] + p4[0] + + fn = self.getcompiled(llf) + res = fn() + assert res == 8765 From cfbolz at codespeak.net Thu Apr 13 16:57:06 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 13 Apr 2006 16:57:06 +0200 (CEST) Subject: [pypy-svn] r25800 - pypy/dist/pypy/translator/backendopt Message-ID: <20060413145706.E1F6410082@code0.codespeak.net> Author: cfbolz Date: Thu Apr 13 16:57:06 2006 New Revision: 25800 Modified: pypy/dist/pypy/translator/backendopt/canraise.py Log: remove strange junk code Modified: pypy/dist/pypy/translator/backendopt/canraise.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/canraise.py (original) +++ pypy/dist/pypy/translator/backendopt/canraise.py Thu Apr 13 16:57:06 2006 @@ -2,15 +2,6 @@ from pypy.rpython.lltypesystem.lloperation import llop, LL_OPERATIONS from pypy.rpython.lltypesystem import lltype -class ExceptionInfo(object): - def __init__(self, translator, can_raise, cannot_raise=None): - self.can_raise = can_raise - self.cannot_raise = cannot_raise - self.translator = translator - - def exception_match(self, etype): - pass - class RaiseAnalyzer(object): def __init__(self, translator): self.translator = translator From stephan at codespeak.net Thu Apr 13 17:02:11 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Thu, 13 Apr 2006 17:02:11 +0200 (CEST) Subject: [pypy-svn] r25801 - pypy/dist/pypy/translator/goal Message-ID: <20060413150211.74D4610083@code0.codespeak.net> Author: stephan Date: Thu Apr 13 17:02:10 2006 New Revision: 25801 Added: pypy/dist/pypy/translator/goal/targetrdicttest.py Log: This simple test program shows bad rdict behaviour. Added: pypy/dist/pypy/translator/goal/targetrdicttest.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetrdicttest.py Thu Apr 13 17:02:10 2006 @@ -0,0 +1,67 @@ +from pypy.rpython.objectmodel import r_dict +import os, sys +import operator + +# __________ Entry point __________ + +def eq(arg1, arg2): + return arg1 == arg2 + +def myhash(arg): + return arg.__hash__() + +class hashable(object): + def __hash__(self): + return 1 + +class unhashable(hashable): + def __hash__(self): + raise TypeError + return 1 + +def entry_point(argv): + mydict = r_dict(eq,myhash) + os.write(1,'test for exceptions raised by r_dict on unhashable objects\n') + hobj = hashable() + uhobj = unhashable() + mydict[hobj] = 'a' + os.write(1,'test 1: hashing an unhashable object\n') + try: + myhash(uhobj) + except KeyError: + os.write(1,'\tKeyError\n') + except TypeError: + os.write(1,'\tTypeError\n') + else: + os.write(1,'\tno exception\n') + + os.write(1,'test 2: getitem with unhashable key:\n') + try: + mydict[uhobj] + except KeyError: + os.write(1,'\tKeyError\n') + except TypeError: + os.write(1,'\tTypeError\n') + else: + os.write(1,'\tno exception\n') + +# The Following can't be translated (for reasons I don't understand: +# it has to do with the assignment. +# +# os.write(1,'test 1: setitem with unhashable key:\n') +# try: +# mydict[uhobj] = 'b' +# except KeyError: +# os.write(1,'\tKeyError\n') +# except TypeError: +# os.write(1,'\tTypeError\n') +# else: +# os.write(1,'\tno exception\n') +# + return 0 + +# _____ Define and setup target ___ + +def target(*args): + return entry_point, None + From arigo at codespeak.net Thu Apr 13 17:14:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 17:14:35 +0200 (CEST) Subject: [pypy-svn] r25802 - pypy/dist/pypy/rpython Message-ID: <20060413151435.A1C8510087@code0.codespeak.net> Author: arigo Date: Thu Apr 13 17:14:34 2006 New Revision: 25802 Added: pypy/dist/pypy/rpython/rslice.py Log: Antonio: did you forget to add this file, or were just tricked by CPython continuing to import rslice.pyc after the .py file was deleted? BTW two ootypesystem tests fail, looking for 'getitem' when there is only 'getitem_nonneg' on List. Added: pypy/dist/pypy/rpython/rslice.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rslice.py Thu Apr 13 17:14:34 2006 @@ -0,0 +1,56 @@ +from pypy.rpython.lltypesystem.lltype import Signed, Void +from pypy.objspace.flow.model import Constant +from pypy.annotation import model as annmodel + + +def select_slice_repr(self): + # Select which one of the three prebuilt reprs to use. + # Return a name. + if not self.step.is_constant() or self.step.const not in (None, 1): + raise TyperError("only supports slices with step 1") + if (self.start.is_constant() and self.start.const in (None, 0) and + self.stop.is_constant() and self.stop.const == -1): + return "minusone_slice_repr" # [:-1] + if isinstance(self.start, annmodel.SomeInteger): + if not self.start.nonneg: + raise TyperError("slice start must be proved non-negative") + if isinstance(self.stop, annmodel.SomeInteger): + if not self.stop.nonneg: + raise TyperError("slice stop must be proved non-negative") + if self.stop.is_constant() and self.stop.const is None: + return "startonly_slice_repr" + else: + return "startstop_slice_repr" + +class __extend__(annmodel.SomeSlice): + def rtyper_makerepr(self, rtyper): + return getattr(rtyper.type_system.rslice, select_slice_repr(self)) + + def rtyper_makekey(self): + return self.__class__, select_slice_repr(self) + + +def rtype_newslice(hop): + sig = [] + for s in hop.args_s: + if s.is_constant() and s.const is None: + sig.append(Void) + else: + sig.append(Signed) + v_start, v_stop, v_step = hop.inputargs(*sig) + assert isinstance(v_step, Constant) and v_step.value in (None, 1) + if isinstance(v_start, Constant) and v_start.value is None: + v_start = hop.inputconst(Signed, 0) + if (isinstance(v_start, Constant) and v_start.value == 0 and + isinstance(v_stop, Constant) and v_stop.value == -1): + # [:-1] slice + return hop.inputconst(Void, slice(None,-1)) + if isinstance(v_stop, Constant) and v_stop.value is None: + # start-only slice + # NB. cannot just return v_start in case it is a constant + return hop.genop('same_as', [v_start], + resulttype=hop.rtyper.type_system.rslice.startonly_slice_repr) + else: + # start-stop slice + return hop.gendirectcall(hop.rtyper.type_system.rslice.ll_newslice, + v_start, v_stop) From stephan at codespeak.net Thu Apr 13 17:17:22 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Thu, 13 Apr 2006 17:17:22 +0200 (CEST) Subject: [pypy-svn] r25803 - pypy/dist/pypy/translator/goal Message-ID: <20060413151722.50DAD1008B@code0.codespeak.net> Author: stephan Date: Thu Apr 13 17:17:20 2006 New Revision: 25803 Modified: pypy/dist/pypy/translator/goal/targetrdicttest.py Log: added a third case to targetrdicttest which shows a strange behaviour: when catching a general 'Exception' and printing it with 'str(e)', the former KeyError suddenly turns into a . Modified: pypy/dist/pypy/translator/goal/targetrdicttest.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetrdicttest.py (original) +++ pypy/dist/pypy/translator/goal/targetrdicttest.py Thu Apr 13 17:17:20 2006 @@ -45,6 +45,13 @@ else: os.write(1,'\tno exception\n') + os.write(1,'test 3: getitem with unhashable key: (and catching general Exception)\n') + try: + mydict[uhobj] + except Exception, e: + os.write(1,'\t' + str(e) + '\n') + else: + os.write(1,'\tno exception\n') # The Following can't be translated (for reasons I don't understand: # it has to do with the assignment. # From arigo at codespeak.net Thu Apr 13 17:25:09 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 17:25:09 +0200 (CEST) Subject: [pypy-svn] r25804 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060413152509.1FE701008D@code0.codespeak.net> Author: arigo Date: Thu Apr 13 17:25:07 2006 New Revision: 25804 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Implemented structures in rctypes, almost exactly copied from rarray. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Thu Apr 13 17:25:07 2006 @@ -65,6 +65,15 @@ class __extend__(pairtype(ArrayRepr, IntegerRepr)): + def rtype_getitem((r_array, r_int), hop): + v_array, v_index = hop.inputargs(r_array, lltype.Signed) + if isinstance(r_array.r_item, PrimitiveRepr): + # primitive case (optimization only) + return r_array.get_item_value(hop.llops, v_array, v_index) + # normal case + v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) + return r_array.r_item.return_c_data(hop.llops, v_c_data) + def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item) @@ -79,15 +88,6 @@ v_newvalue = r_array.r_item.getvalue(hop.llops, v_item) r_array.set_item_value(hop.llops, v_array, v_index, v_newvalue) - def rtype_getitem((r_array, r_int), hop): - v_array, v_index = hop.inputargs(r_array, lltype.Signed) - if isinstance(r_array.r_item, PrimitiveRepr): - # primitive case (optimization only) - return r_array.get_item_value(hop.llops, v_array, v_index) - # normal case - v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) - return r_array.r_item.return_c_data(hop.llops, v_c_data) - # ____________________________________________________________ def arraytype_specialize_call(hop): Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Thu Apr 13 17:25:07 2006 @@ -1,12 +1,95 @@ from ctypes import Structure from pypy.annotation.model import SomeCTypesObject, SomeBuiltin from pypy.rpython import extregistry +from pypy.rpython.rmodel import inputconst +from pypy.rpython.rbuiltin import gen_cast_structfield_pointer +from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr +from pypy.rpython.rctypes.rmodel import genreccopy +from pypy.rpython.rctypes.rprimitive import PrimitiveRepr StructType = type(Structure) +class StructRepr(CTypesRefRepr): + def __init__(self, rtyper, s_struct): + struct_ctype = s_struct.knowntype + + # Find the repr and low-level type of the fields from their ctype + self.r_fields = {} + llfields = [] + for name, field_ctype in struct_ctype._fields_: + r_field = rtyper.getrepr(SomeCTypesObject(field_ctype, + SomeCTypesObject.MEMORYALIAS)) + self.r_fields[name] = r_field + llfields.append((name, r_field.ll_type)) + + # Here, self.c_data_type == self.ll_type + c_data_type = lltype.Struct(struct_ctype.__name__, *llfields) + + super(StructRepr, self).__init__(rtyper, s_struct, c_data_type) + + def get_c_data_of_field(self, llops, v_struct, fieldname): + v_c_struct = self.get_c_data(llops, v_struct) + r_field = self.r_fields[fieldname] + if isinstance(r_field, CTypesRefRepr): + # ByRef case + c_fieldname = inputconst(lltype.Void, fieldname) + return llops.genop('getsubstruct', [v_c_struct, c_fieldname], + lltype.Ptr(r_field.c_data_type)) + else: + # ByValue case + A = lltype.FixedSizeArray(r_field.ll_type, 1) + return gen_cast_structfield_pointer(llops, lltype.Ptr(A), + v_c_struct, fieldname) + + def get_field_value(self, llops, v_struct, fieldname): + # ByValue case only + r_field = self.r_fields[fieldname] + assert isinstance(r_field, CTypesValueRepr) + v_c_struct = self.get_c_data(llops, v_struct) + c_fieldname = inputconst(lltype.Void, fieldname) + return llops.genop('getfield', [v_c_struct, c_fieldname], + resulttype = r_field.ll_type) + + def set_field_value(self, llops, v_struct, fieldname, v_newvalue): + # ByValue case only + r_field = self.r_fields[fieldname] + assert isinstance(r_field, CTypesValueRepr) + v_c_struct = self.get_c_data(llops, v_struct) + c_fieldname = inputconst(lltype.Void, fieldname) + llops.genop('setfield', [v_c_struct, c_fieldname, v_newvalue]) + + def rtype_getattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + name = s_attr.const + r_field = self.r_fields[name] + v_struct, v_attr = hop.inputargs(self, lltype.Void) + if isinstance(r_field, PrimitiveRepr): + # primitive case (optimization only) + return self.get_field_value(hop.llops, v_struct, name) + # normal case + v_c_data = self.get_c_data_of_item(hop.llops, v_struct, name) + return r_field.return_c_data(hop.llops, v_c_data) + + def rtype_setattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + name = s_attr.const + r_field = self.r_fields[name] + v_struct, v_attr, v_item = hop.inputargs(self, lltype.Void, r_field) + if isinstance(r_field, CTypesRefRepr): + # ByRef case + v_new_c_data = r_field.get_c_data(hop.llops, v_item) + v_c_data = self.get_c_data_of_item(hop.llops, v_struct, name) + # copy the whole structure's content over + genreccopy(hop.llops, v_new_c_data, v_c_data) + else: + # ByValue case (optimization; the above also works in this case) + v_newvalue = r_field.getvalue(hop.llops, v_item) + self.set_field_value(hop.llops, v_struct, name, v_newvalue) - +# ____________________________________________________________ def structtype_specialize_call(hop): r_struct = hop.r_result Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Thu Apr 13 17:25:07 2006 @@ -63,3 +63,18 @@ assert p[0] == x.value == "world" p[0] = "other" assert x.value == p.contents.value == p[0] == "other" + +def test_struct(): + class tagpoint(Structure): + _fields_ = [('x', c_int), + ('p', POINTER(c_short))] + + y = c_short(123) + z = c_short(-33) + s = tagpoint() + s.p.contents = z + assert s.p.contents.value == -33 + s.p = pointer(y) + assert s.p.contents.value == 123 + s.p.contents.value = 124 + assert y.value == 124 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Thu Apr 13 17:25:07 2006 @@ -50,3 +50,42 @@ if conftest.option.view: t.view() + + def test_annotate_prebuilt(self): + my_struct_2 = tagpoint(5, 7) + my_struct_3 = tagpoint(x=6, y=11) + def func(i): + if i == 2: + struct = my_struct_2 + else: + struct = my_struct_3 + return struct.y + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, [int]) + if conftest.option.view: + a.translator.view() + assert s.knowntype == int + +class Test_specialization: + def test_specialize_struct(self): + def create_struct(): + return tagpoint() + + res = interpret(create_struct, []) + c_data = res.c_data + assert c_data.x == 0 + assert c_data.y == 0 + + def test_specialize_struct_access(self): + def access_struct(n): + my_struct = tagpoint() + my_struct.x = c_int(1) + my_struct.y = 2 + my_struct.x += n + + return my_struct.x * my_struct.y + + res = interpret(access_struct, [44]) + assert res == 90 From afayolle at codespeak.net Thu Apr 13 17:27:37 2006 From: afayolle at codespeak.net (afayolle at codespeak.net) Date: Thu, 13 Apr 2006 17:27:37 +0200 (CEST) Subject: [pypy-svn] r25805 - in pypy/dist/pypy: lib/logic/computation_space objspace/constraint objspace/constraint/applevel objspace/constraint/test Message-ID: <20060413152737.33A111008D@code0.codespeak.net> Author: afayolle Date: Thu Apr 13 17:27:33 2006 New Revision: 25805 Modified: pypy/dist/pypy/lib/logic/computation_space/computationspace.py pypy/dist/pypy/lib/logic/computation_space/variable.py pypy/dist/pypy/objspace/constraint/applevel/problems.py pypy/dist/pypy/objspace/constraint/applevel/solver.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/test/test_solver.py Log: small enhancements Modified: pypy/dist/pypy/lib/logic/computation_space/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/computationspace.py Thu Apr 13 17:27:33 2006 @@ -180,7 +180,7 @@ def commit(self, choice): """if self is distributable, causes the Choose call in the space to complete and return some_number as a result. This - may cause the spzce to resume execution. + may cause the space to resume execution. some_number must satisfy 1= conf1: + v1, v2 = cs.find_vars((conf1, conf2)) cs.tell(make_expression([v1, v2], '%s[1] != %s[1]'% (v1.name(),v2.name()))) + cs.tell(AllDistinct(variables)) -## for g in groups: -## for conf1 in g: -## for conf2 in g: -## v1, v2 = cs.find_vars(conf1, conf2) -## if conf2 > conf1: -## cs.add_constraint([v1,v2], '%s[1] != %s[1]'% (v1.name,v2.name)) - - for conf1 in variables: - for conf2 in variables: - if conf2 > conf1: - cs.tell(make_expression([conf1,conf2], - '%s != %s'%(conf1.name(),conf2.name()))) return variables def sudoku(computation_space): Modified: pypy/dist/pypy/objspace/constraint/applevel/solver.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/applevel/solver.py (original) +++ pypy/dist/pypy/objspace/constraint/applevel/solver.py Thu Apr 13 17:27:33 2006 @@ -23,7 +23,7 @@ sp_stack.appendleft(space) print "ready to find solution ..." - while len(sp_stack): + while sp_stack: space = sp_stack.pop() print ' '*len(sp_stack), "ask [depth = %s]" % len(sp_stack) status = space.ask() @@ -32,7 +32,7 @@ yield space.merge() elif status > 1: print ' '*len(sp_stack), "%s branches ..." % status - for i in range(status): + for i in xrange(status): clone = space.clone() clone.commit(status-i) collect(clone) Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Thu Apr 13 17:27:33 2006 @@ -93,8 +93,9 @@ return len(value_set) == len(sol) def revise(self, w_cs): + _spc = self._space assert isinstance(w_cs, W_ComputationSpace) - variables = [(self._space.int_w(w_cs.w_dom(variable).w_size()), + variables = [(_spc.int_w(w_cs.w_dom(variable).w_size()), variable, w_cs.w_dom(variable)) for variable in self._variables] @@ -102,10 +103,10 @@ # if a domain has a size of 1, # then the value must be removed from the other domains for size, var, dom in variables: - if self._space.eq_w(dom.w_size(), self._space.newint(1)): + if _spc.eq_w(dom.w_size(), _spc.newint(1)): #print "AllDistinct removes values" for _siz, _var, _dom in variables: - if not self._space.eq_w(_var, var): + if not _spc.eq_w(_var, var): try: _dom.w_remove_value(dom.w_get_values().wrappeditems[0]) except KeyError: @@ -118,19 +119,21 @@ for size, var, dom in variables: for val in dom.w_get_values().wrappeditems: values[val] = 0 + if len(values) < len(variables): #print "AllDistinct failed" - raise OperationError(self._space.w_RuntimeError, - self._space.wrap("Consistency Failure")) + raise OperationError(_spc.w_RuntimeError, + _spc.wrap("ConsistencyFailure")) # the constraint is entailed if all domains have a size of 1 for variable in variables: - if self._space.is_true(self._space.ne(variable[2].w_size(), self._space.newint(1))): + if not _spc.eq_w(variable[2].w_size(), + _spc.newint(1)): return False # Question : did we *really* completely check # our own alldistinctness predicate ? - + #print "All distinct entailed" return True W_AllDistinct.typedef = typedef.TypeDef( Modified: pypy/dist/pypy/objspace/constraint/test/test_solver.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_solver.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_solver.py Thu Apr 13 17:27:33 2006 @@ -22,6 +22,9 @@ sols = solver.solve(spc) count = 0 + solutions = set() for sol in sols: + assert sol not in solutions + solutions.add(sol) count += 1 assert count == 64 From arigo at codespeak.net Thu Apr 13 17:59:23 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 17:59:23 +0200 (CEST) Subject: [pypy-svn] r25806 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060413155923.6ACC610085@code0.codespeak.net> Author: arigo Date: Thu Apr 13 17:59:22 2006 New Revision: 25806 Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Prebuilt structs. Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Thu Apr 13 17:59:22 2006 @@ -5,7 +5,7 @@ from pypy.rpython.rbuiltin import gen_cast_structfield_pointer from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr -from pypy.rpython.rctypes.rmodel import genreccopy +from pypy.rpython.rctypes.rmodel import genreccopy, reccopy from pypy.rpython.rctypes.rprimitive import PrimitiveRepr StructType = type(Structure) @@ -28,6 +28,16 @@ super(StructRepr, self).__init__(rtyper, s_struct, c_data_type) + def initialize_const(self, p, value): + for name, r_field in self.r_fields.items(): + llitem = r_field.convert_const(getattr(value, name)) + if isinstance(r_field, CTypesRefRepr): + # ByRef case + reccopy(llitem.c_data, getattr(p.c_data, name)) + else: + # ByValue case + setattr(p.c_data, name, llitem.c_data[0]) + def get_c_data_of_field(self, llops, v_struct, fieldname): v_c_struct = self.get_c_data(llops, v_struct) r_field = self.r_fields[fieldname] Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Thu Apr 13 17:59:22 2006 @@ -89,3 +89,45 @@ res = interpret(access_struct, [44]) assert res == 90 + + def test_specialize_prebuilt(self): + my_struct_2 = tagpoint(5, 7) + my_struct_3 = tagpoint(x=6, y=11) + def func(i): + if i == 2: + struct = my_struct_2 + else: + struct = my_struct_3 + return struct.y + + res = interpret(func, [2]) + assert res == 7 + res = interpret(func, [3]) + assert res == 11 + +class Test_compilation: + def test_compile_struct_access(self): + def access_struct(n): + my_struct = tagpoint() + my_struct.x = c_int(1) + my_struct.y = 2 + my_struct.x += n + + return my_struct.x * my_struct.y + + fn = compile(access_struct, [int]) + assert fn(44) == 90 + + def test_compile_prebuilt(self): + my_struct_2 = tagpoint(5, 7) + my_struct_3 = tagpoint(x=6, y=11) + def func(i): + if i == 2: + struct = my_struct_2 + else: + struct = my_struct_3 + return struct.y + + fn = compile(func, [int]) + assert fn(2) == 7 + assert fn(3) == 11 From nik at codespeak.net Thu Apr 13 17:59:28 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 13 Apr 2006 17:59:28 +0200 (CEST) Subject: [pypy-svn] r25807 - pypy/dist/pypy/translator/squeak/test Message-ID: <20060413155928.098F51008B@code0.codespeak.net> Author: nik Date: Thu Apr 13 17:59:26 2006 New Revision: 25807 Modified: pypy/dist/pypy/translator/squeak/test/runtest.py pypy/dist/pypy/translator/squeak/test/test_llops.py Log: make squeak llops tests skip earlier (thanks for the hint, hpk). it seemed to me like the overhead was mostly due to the generative tests not really the relatively late skips. Modified: pypy/dist/pypy/translator/squeak/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/squeak/test/runtest.py (original) +++ pypy/dist/pypy/translator/squeak/test/runtest.py Thu Apr 13 17:59:26 2006 @@ -8,6 +8,18 @@ def compile_function(func, annotation=[], graph=None): return SqueakFunction(func, annotation, graph) +def squeak_checks(): + try: + import posix + except ImportError: + py.test.skip("Squeak tests only work on Unix right now.") + try: + py.path.local.sysfind("squeak") + except py.error.ENOENT: + py.test.skip("Squeak is not on your path.") + if os.getenv("SQUEAK_IMAGE") is None: + py.test.skip("Squeak tests expect the SQUEAK_IMAGE environment " + "variable to point to an image.") # For now use pipes to communicate with squeak. This is very flaky # and only works for posix systems. At some later point we'll @@ -32,6 +44,7 @@ class SqueakFunction: def __init__(self, func, annotation, graph=None): + squeak_checks() self._func = func self._gen = self._build(func, annotation, graph) @@ -78,17 +91,6 @@ def __call__(self, *args): # NB: only integers arguments are supported currently - try: - import posix - except ImportError: - py.test.skip("Squeak tests only work on Unix right now.") - try: - py.path.local.sysfind("squeak") - except py.error.ENOENT: - py.test.skip("Squeak is not on your path.") - if os.getenv("SQUEAK_IMAGE") is None: - py.test.skip("Squeak tests expect the SQUEAK_IMAGE environment " - "variable to point to an image.") startup_st = self._write_startup() options = "-headless" if hasattr(conftest, "showsqueak") and conftest.option.showsqueak: Modified: pypy/dist/pypy/translator/squeak/test/test_llops.py ============================================================================== --- pypy/dist/pypy/translator/squeak/test/test_llops.py (original) +++ pypy/dist/pypy/translator/squeak/test/test_llops.py Thu Apr 13 17:59:26 2006 @@ -1,5 +1,5 @@ import sys -from pypy.translator.squeak.test.runtest import compile_function +from pypy.translator.squeak.test.runtest import compile_function, squeak_checks from pypy.rpython.rarithmetic import r_uint, r_longlong, r_ulonglong from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy from pypy.rpython.lltypesystem.lloperation import llop @@ -7,6 +7,9 @@ from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong, Float from pypy.rpython.test.test_llinterp import interpret +def setup_module(mod): + squeak_checks() + def optest(testcase): llopname = testcase[0] RESTYPE = testcase[1] From arigo at codespeak.net Thu Apr 13 18:43:56 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 18:43:56 +0200 (CEST) Subject: [pypy-svn] r25809 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060413164356.1950810081@code0.codespeak.net> Author: arigo Date: Thu Apr 13 18:43:54 2006 New Revision: 25809 Modified: pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Log: Automatic c_char_p-to-str conversion when reads fields of a struct or getting a function's result. Typos, tests. Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Thu Apr 13 18:43:54 2006 @@ -451,11 +451,8 @@ from pypy.rpython import extregistry assert extregistry.is_registered_type(self.knowntype) entry = extregistry.lookup_type(self.knowntype) - if hasattr(entry, 'lowleveltype'): - # special case for returning primitives - return lltype_to_annotation(entry.lowleveltype) - else: - return self + # special case for returning primitives or c_char_p + return getattr(entry, 's_return_trick', self) class SomeImpossibleValue(SomeObject): Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Thu Apr 13 18:43:54 2006 @@ -12,6 +12,13 @@ class CCharPRepr(CTypesValueRepr): + def return_c_data(self, llops, v_c_data): + """Read out the RPython string from a raw C pointer. + Used when the data is returned from an operation or C function call. + """ + v_char_p = self.getvalue_from_c_data(llops, v_c_data) + return llops.gendirectcall(ll_charp2str, v_char_p) + def get_content_keepalives(self): "Return an extra keepalive field used for the RPython string." return [('keepalive_str', string_repr.lowleveltype)] @@ -83,6 +90,13 @@ def ll_str2charp(s): return lltype.cast_subarray_pointer(CCHARP, s.chars, 0) +def ll_charp2str(p): + length = ll_strlen(p) + newstr = lltype.malloc(string_repr.lowleveltype.TO, length) + for i in range(length): + newstr.chars[i] = p[i] + return newstr + def ll_getstring(box): p = box.c_data[0] if p: @@ -137,7 +151,9 @@ compute_annotation = c_char_compute_annotation, get_repr = c_char_p_get_repr, ) +s_value_annotation = annmodel.SomeString(can_be_None=True) def c_char_p_get_field_annotation(s_char_p, fieldname): assert fieldname == 'value' - return annmodel.SomeString(can_be_None=True) + return s_value_annotation entry.get_field_annotation = c_char_p_get_field_annotation +entry.s_return_trick = s_value_annotation Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Thu Apr 13 18:43:54 2006 @@ -95,11 +95,12 @@ compute_annotation=compute_prebuilt_instance_annotation, get_repr=primitive_get_repr, ) + s_value_annotation = annmodel.lltype_to_annotation(ll_type) def primitive_get_field_annotation(s_primitive, fieldname): assert fieldname == 'value' - return annmodel.lltype_to_annotation(ll_type) + return s_value_annotation entry.get_field_annotation = primitive_get_field_annotation - entry.lowleveltype = ll_type + entry.s_return_trick = s_value_annotation for the_type, ll_type in ctypes_annotation_list: do_register(the_type, ll_type) Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Thu Apr 13 18:43:54 2006 @@ -79,7 +79,7 @@ # primitive case (optimization only) return self.get_field_value(hop.llops, v_struct, name) # normal case - v_c_data = self.get_c_data_of_item(hop.llops, v_struct, name) + v_c_data = self.get_c_data_of_field(hop.llops, v_struct, name) return r_field.return_c_data(hop.llops, v_c_data) def rtype_setattr(self, hop): @@ -91,7 +91,7 @@ if isinstance(r_field, CTypesRefRepr): # ByRef case v_new_c_data = r_field.get_c_data(hop.llops, v_item) - v_c_data = self.get_c_data_of_item(hop.llops, v_struct, name) + v_c_data = self.get_c_data_of_field(hop.llops, v_struct, name) # copy the whole structure's content over genreccopy(hop.llops, v_new_c_data, v_c_data) else: Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Thu Apr 13 18:43:54 2006 @@ -64,6 +64,11 @@ p[0] = "other" assert x.value == p.contents.value == p[0] == "other" + myarray = (c_char_p * 10)() + myarray[7] = "hello" + assert isinstance(myarray[7], str) + assert myarray[7] == "hello" + def test_struct(): class tagpoint(Structure): _fields_ = [('x', c_int), Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Thu Apr 13 18:43:54 2006 @@ -16,7 +16,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int, c_short, ARRAY, POINTER, pointer +from ctypes import c_int, c_short, ARRAY, POINTER, pointer, c_char_p c_int_10 = ARRAY(c_int,10) @@ -118,6 +118,30 @@ a.translator.view() assert s.knowntype == int + def test_annotate_variants(self): + A1 = c_int * 10 + A2 = POINTER(c_int) * 10 + A3 = A1 * 10 + A4 = POINTER(A1) * 10 + A5 = c_char_p * 10 + def func(): + a1 = A1(); a1[4] = 1000 + a2 = A2(); a2[5] = pointer(c_int(200)) + a3 = A3(); a3[2][9] = 30 + a4 = A4(); a4[3] = pointer(a1); a1[1] = 4 + a5 = A5(); a5[7] = "hello" + res = a1[4] + a2[5].contents.value + a3[2][9] + a4[3].contents[1] + res *= ord(a5[7][1]) + return res + assert func() == 1234 * ord('e') + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + if conftest.option.view: + a.translator.view() + assert s.knowntype == int + class Test_specialization: def test_specialize_array(self): def create_array(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Thu Apr 13 18:43:54 2006 @@ -17,7 +17,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_char_p, pointer +from ctypes import c_char_p, pointer, Structure class Test_annotation: def test_annotate_c_char_p(self): @@ -44,6 +44,21 @@ if conftest.option.view: t.view() + def test_annotate_return(self): + class S(Structure): + _fields_ = [('string', c_char_p)] + def func(): + s = S() + s.string = "hello" + return s.string + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + assert s.knowntype == str + if conftest.option.view: + t.view() + class Test_specialization: def test_specialize_c_char_p(self): def func(): @@ -82,6 +97,18 @@ res = interpret(func, []) assert ''.join(res.chars) == 'ka' + def test_specialize_return(self): + class S(Structure): + _fields_ = [('string', c_char_p)] + def func(): + s = S() + s.string = "hello" + return s.string + + assert func() == "hello" + res = interpret(func, []) + assert ''.join(res.chars) == "hello" + class Test_compilation: def test_compile_c_char_p(self): From arigo at codespeak.net Thu Apr 13 18:52:12 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 18:52:12 +0200 (CEST) Subject: [pypy-svn] r25810 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060413165212.85EB810081@code0.codespeak.net> Author: arigo Date: Thu Apr 13 18:52:11 2006 New Revision: 25810 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Some more tests. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Thu Apr 13 18:52:11 2006 @@ -20,6 +20,24 @@ c_int_10 = ARRAY(c_int,10) +def maketest(): + A1 = c_int * 10 + A2 = POINTER(c_int) * 10 + A3 = A1 * 10 + A4 = POINTER(A1) * 10 + A5 = c_char_p * 10 + def func(): + a1 = A1(); a1[4] = 1000 + a2 = A2(); a2[5] = pointer(c_int(200)) + a3 = A3(); a3[2][9] = 30 + a4 = A4(); a4[3] = pointer(a1); a1[1] = 4 + a5 = A5(); a5[7] = "hello" + res = a1[4] + a2[5].contents.value + a3[2][9] + a4[3].contents[1] + res *= ord(a5[7][1]) + return res + return func, 1234 * ord('e') + + class Test_annotation: def test_annotate_array(self): def create_array(): @@ -119,22 +137,8 @@ assert s.knowntype == int def test_annotate_variants(self): - A1 = c_int * 10 - A2 = POINTER(c_int) * 10 - A3 = A1 * 10 - A4 = POINTER(A1) * 10 - A5 = c_char_p * 10 - def func(): - a1 = A1(); a1[4] = 1000 - a2 = A2(); a2[5] = pointer(c_int(200)) - a3 = A3(); a3[2][9] = 30 - a4 = A4(); a4[3] = pointer(a1); a1[1] = 4 - a5 = A5(); a5[7] = "hello" - res = a1[4] + a2[5].contents.value + a3[2][9] + a4[3].contents[1] - res *= ord(a5[7][1]) - return res - assert func() == 1234 * ord('e') - + func, expected = maketest() + assert func() == expected t = TranslationContext() a = t.buildannotator() s = a.build_types(func, []) @@ -181,6 +185,11 @@ res = interpret(func, [3, 7]) assert res == 343 + def test_specialize_variants(self): + func, expected = maketest() + res = interpret(func, []) + assert res == expected + class Test_compilation: def test_compile_array_access(self): def access_array(): @@ -207,3 +216,8 @@ fn = compile(func, [int, int]) assert fn(2, 7) == 49 assert fn(3, 6) == 216 + + def test_compile_variants(self): + func, expected = maketest() + fn = compile(func, []) + assert fn() == expected Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Thu Apr 13 18:52:11 2006 @@ -16,12 +16,30 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int, c_short, Structure, POINTER, pointer +from ctypes import c_int, c_short, Structure, POINTER, pointer, c_char_p class tagpoint(Structure): _fields_ = [("x", c_int), ("y", c_int)] +def maketest(): + class S1(Structure): _fields_ = [('x', c_int)] + class S2(Structure): _fields_ = [('x', POINTER(c_int))] + class S3(Structure): _fields_ = [('x', S1)] + class S4(Structure): _fields_ = [('x', POINTER(S1))] + class S5(Structure): _fields_ = [('x', c_char_p)] + def func(): + s1 = S1(); s1.x = 500 + s2 = S2(); s2.x = pointer(c_int(200)) + s3 = S3(); s3.x.x = 30 + s4 = S4(); s4.x = pointer(s1) + s5 = S5(); s5.x = "hello" + res = s1.x + s2.x.contents.value + s3.x.x + s4.x.contents.x + res *= ord(s5.x[4]) + return res + return func, 1230 * ord('o') + + class Test_annotation: def test_annotate_struct(self): def create_struct(): @@ -68,6 +86,16 @@ a.translator.view() assert s.knowntype == int + def test_annotate_variants(self): + func, expected = maketest() + assert func() == expected + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + if conftest.option.view: + a.translator.view() + assert s.knowntype == int + class Test_specialization: def test_specialize_struct(self): def create_struct(): @@ -105,6 +133,11 @@ res = interpret(func, [3]) assert res == 11 + def test_specialize_variants(self): + func, expected = maketest() + res = interpret(func, []) + assert res == expected + class Test_compilation: def test_compile_struct_access(self): def access_struct(n): @@ -131,3 +164,8 @@ fn = compile(func, [int]) assert fn(2) == 7 assert fn(3) == 11 + + def test_compile_variants(self): + func, expected = maketest() + fn = compile(func, []) + assert fn() == expected From arigo at codespeak.net Thu Apr 13 19:07:32 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 19:07:32 +0200 (CEST) Subject: [pypy-svn] r25811 - pypy/dist/pypy/annotation Message-ID: <20060413170732.64B8110081@code0.codespeak.net> Author: arigo Date: Thu Apr 13 19:07:31 2006 New Revision: 25811 Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: /me got tired of having to run --pdb to figure out what this unexpected 'pyobj' prebuilt constant could be. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Thu Apr 13 19:07:31 2006 @@ -456,7 +456,11 @@ name) else: # must be a frozen pre-built constant, but let's check - assert pyobj._freeze_() + try: + assert pyobj._freeze_() + except AttributeError: + raise Exception("unexpected prebuilt constant: %r" % ( + pyobj,)) result = self.getfrozen(pyobj) self.descs[pyobj] = result return result From antocuni at codespeak.net Thu Apr 13 21:31:07 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 13 Apr 2006 21:31:07 +0200 (CEST) Subject: [pypy-svn] r25817 - pypy/dist/pypy/rpython/ootypesystem/test Message-ID: <20060413193107.2DA9C10080@code0.codespeak.net> Author: antocuni Date: Thu Apr 13 21:30:56 2006 New Revision: 25817 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: Fixed two tests in test_oortype that was broken by my editing of ootypesystem/rlist.py. Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Thu Apr 13 21:30:56 2006 @@ -128,8 +128,8 @@ def oof(): l = new(LT) l.append(1) - l.setitem(0, 2) - return l.getitem(0) + l.setitem_nonneg(0, 2) + return l.getitem_nonneg(0) g = gengraph(oof, []) rettype = g.getreturnvar().concretetype @@ -141,7 +141,7 @@ def oof(): l = new(LT) try: - l.getitem(0) + l.getitem_nonneg(0) except IndexError: return -1 return 0 From antocuni at codespeak.net Thu Apr 13 21:39:34 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 13 Apr 2006 21:39:34 +0200 (CEST) Subject: [pypy-svn] r25818 - pypy/dist/pypy/rpython Message-ID: <20060413193934.E2ABA1007D@code0.codespeak.net> Author: antocuni Date: Thu Apr 13 21:39:20 2006 New Revision: 25818 Modified: pypy/dist/pypy/rpython/rslice.py Log: Added a missing import (TyperError). Modified: pypy/dist/pypy/rpython/rslice.py ============================================================================== --- pypy/dist/pypy/rpython/rslice.py (original) +++ pypy/dist/pypy/rpython/rslice.py Thu Apr 13 21:39:20 2006 @@ -1,7 +1,7 @@ from pypy.rpython.lltypesystem.lltype import Signed, Void from pypy.objspace.flow.model import Constant from pypy.annotation import model as annmodel - +from pypy.rpython.error import TyperError def select_slice_repr(self): # Select which one of the three prebuilt reprs to use. From arigo at codespeak.net Thu Apr 13 22:36:11 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 13 Apr 2006 22:36:11 +0200 (CEST) Subject: [pypy-svn] r25819 - in pypy/dist/pypy/rpython: . lltypesystem rctypes rctypes/test Message-ID: <20060413203611.65CA410081@code0.codespeak.net> Author: arigo Date: Thu Apr 13 22:36:09 2006 New Revision: 25819 Modified: pypy/dist/pypy/rpython/extregistry.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: extregistry support for things that cannot be weakly referenced, like built-in functions (duh). In rctypes, byref() and functions taking 'struct' arguments. Modified: pypy/dist/pypy/rpython/extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/extregistry.py (original) +++ pypy/dist/pypy/rpython/extregistry.py Thu Apr 13 22:36:09 2006 @@ -21,6 +21,8 @@ return self.compute_annotation(type, instance) EXT_REGISTRY_BY_VALUE = weakref.WeakKeyDictionary() +EXT_REGISTRY_BY_VALUE_NONWEAK = {} # for things that cannot be weakly ref'ed + # like built-in functions EXT_REGISTRY_BY_TYPE = weakref.WeakKeyDictionary() EXT_REGISTRY_BY_METATYPE = weakref.WeakKeyDictionary() @@ -46,9 +48,15 @@ get_repr) def register_value(value, **kwargs): - assert value not in EXT_REGISTRY_BY_VALUE - EXT_REGISTRY_BY_VALUE[value] = create_entry(**kwargs) - return EXT_REGISTRY_BY_VALUE[value] + try: + weakref.ref(value) + except TypeError: + REG = EXT_REGISTRY_BY_VALUE_NONWEAK + else: + REG = EXT_REGISTRY_BY_VALUE + assert value not in REG + REG[value] = create_entry(**kwargs) + return REG[value] def register_type(t, **kwargs): assert t not in EXT_REGISTRY_BY_TYPE @@ -75,7 +83,13 @@ def lookup(instance): try: - return EXT_REGISTRY_BY_VALUE[instance] + weakref.ref(instance) + except TypeError: + REG = EXT_REGISTRY_BY_VALUE_NONWEAK + else: + REG = EXT_REGISTRY_BY_VALUE + try: + return REG[instance] except (KeyError, TypeError): return lookup_type(type(instance)) Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Thu Apr 13 22:36:09 2006 @@ -379,8 +379,9 @@ def __init__(self, args, result): for arg in args: assert isinstance(arg, LowLevelType) - if isinstance(arg, ContainerType): - raise TypeError, "function arguments can only be primitives or pointers" + # -- disabled the following check for the benefits of rctypes -- + #if isinstance(arg, ContainerType): + # raise TypeError, "function arguments can only be primitives or pointers" self.ARGS = tuple(args) assert isinstance(result, LowLevelType) if isinstance(result, ContainerType): Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Thu Apr 13 22:36:09 2006 @@ -1,5 +1,6 @@ from pypy.annotation import model as annmodel from pypy.rpython import extregistry +from pypy.rpython.rctypes.rmodel import CTypesValueRepr import ctypes @@ -35,8 +36,17 @@ args_r.append(r_arg) vlist = hop.inputargs(*args_r) - unwrapped_args_v = [r_arg.getvalue(hop.llops, v) - for r_arg, v in zip(args_r, vlist)] + unwrapped_args_v = [] + for r_arg, v in zip(args_r, vlist): + if isinstance(r_arg, CTypesValueRepr): + # ValueRepr case + unwrapped_args_v.append(r_arg.getvalue(hop.llops, v)) + else: + # RefRepr case -- i.e. the function argument that we pass by + # value is e.g. a complete struct; we pass a pointer to it + # in the low-level graphs and it's up to the back-end to + # generate the correct dereferencing + unwrapped_args_v.append(r_arg.get_c_data(hop.llops, v)) s_res = annmodel.SomeCTypesObject(cfuncptr.restype, annmodel.SomeCTypesObject.OWNSMEMORY) r_res = hop.rtyper.getrepr(s_res) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Thu Apr 13 22:36:09 2006 @@ -6,7 +6,7 @@ from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesValueRepr, genreccopy -from ctypes import POINTER, pointer, c_int +from ctypes import POINTER, pointer, byref, c_int class PointerRepr(CTypesValueRepr): def __init__(self, rtyper, s_pointer): @@ -140,3 +140,9 @@ # same rtyping for calling pointer() or calling a specific instance # of PointerType: specialize_call=pointertype_specialize_call) + +# byref() is equivalent to pointer() -- the difference is only an +# optimization that is useful in ctypes but not in rctypes. +extregistry.register_value(byref, + compute_result_annotation=pointerfn_compute_annotation, + specialize_call=pointertype_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Thu Apr 13 22:36:09 2006 @@ -3,6 +3,7 @@ """ import py +import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile, compile_db @@ -21,7 +22,7 @@ from ctypes import cdll -from ctypes import POINTER, Structure, c_int +from ctypes import POINTER, Structure, c_int, byref # __________ compile and load our local test C file __________ @@ -48,7 +49,14 @@ # _testfunc_byval testfunc_byval = _rctypes_test._testfunc_byval testfunc_byval.restype = c_int -testfunc_byval.argtypes = [tagpoint,POINTER(tagpoint)] +testfunc_byval.argtypes = [tagpoint, POINTER(tagpoint)] + +def ll_testfunc_byval(in_, pout): + if pout: + pout.x = in_.x + pout.y = in_.y + return in_.x + in_.y +testfunc_byval.llinterp_friendly_version = ll_testfunc_byval # _test_struct testfunc_struct = _rctypes_test._testfunc_struct @@ -67,4 +75,24 @@ testfunc_struct_pointer_id.argtypes = [tagpointptr] -# XXX no test here so far +def test_rctypes_dll(): # this test is also annotated/specialized below + in_point = tagpoint() + in_point.x = 42 + in_point.y = 17 + out_point = tagpoint() + res = testfunc_byval(in_point, byref(out_point)) + assert res == in_point.x + in_point.y + assert out_point.x == 42 + assert out_point.y == 17 + +class Test_annotation: + def test_annotate_byval(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(test_rctypes_dll, []) + if conftest.option.view: + t.view() + +class Test_specialization: + def test_specialize_byval(self): + interpret(test_rctypes_dll, []) From arigo at codespeak.net Fri Apr 14 00:13:31 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 14 Apr 2006 00:13:31 +0200 (CEST) Subject: [pypy-svn] r25821 - in pypy/dist/pypy: rpython/rctypes rpython/rctypes/test translator/c Message-ID: <20060413221331.A19EC10081@code0.codespeak.net> Author: arigo Date: Fri Apr 14 00:13:29 2006 New Revision: 25821 Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/gc.py pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py Log: Supports compilation of function calls with 'struct' arguments. For now, this is a collection of short XXX hacks all around rctypes and genc. Hopefully things can be improved at some point... Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Fri Apr 14 00:13:29 2006 @@ -1,5 +1,6 @@ from pypy.annotation import model as annmodel from pypy.rpython import extregistry +from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesValueRepr import ctypes @@ -37,24 +38,38 @@ vlist = hop.inputargs(*args_r) unwrapped_args_v = [] + ARGTYPES = [] for r_arg, v in zip(args_r, vlist): if isinstance(r_arg, CTypesValueRepr): # ValueRepr case unwrapped_args_v.append(r_arg.getvalue(hop.llops, v)) + ARGTYPES.append(r_arg.ll_type) else: # RefRepr case -- i.e. the function argument that we pass by # value is e.g. a complete struct; we pass a pointer to it # in the low-level graphs and it's up to the back-end to # generate the correct dereferencing unwrapped_args_v.append(r_arg.get_c_data(hop.llops, v)) + ARGTYPES.append(r_arg.c_data_type) s_res = annmodel.SomeCTypesObject(cfuncptr.restype, annmodel.SomeCTypesObject.OWNSMEMORY) r_res = hop.rtyper.getrepr(s_res) ll_func = getattr(cfuncptr, 'llinterp_friendly_version', None) + includes = getattr(cfuncptr, 'includes', ()) v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, resulttype = r_res.ll_type, - _callable = ll_func) + _callable = ll_func, + includes = includes) + # XXX hack! hack! temporary! I promize! + FUNCTYPE = lltype.FuncType(ARGTYPES, r_res.ll_type) + last_op = hop.llops[-1] + assert last_op.opname == 'direct_call' + last_op.args[0].concretetype = lltype.Ptr(FUNCTYPE) + last_op.args[0].value._set_TYPE(last_op.args[0].concretetype) + last_op.args[0].value._set_T(FUNCTYPE) + last_op.args[0].value._obj._TYPE = FUNCTYPE + return v_result extregistry.register_metatype(CFuncPtrType, Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Fri Apr 14 00:13:29 2006 @@ -24,7 +24,10 @@ llfields.append((name, r_field.ll_type)) # Here, self.c_data_type == self.ll_type - c_data_type = lltype.Struct(struct_ctype.__name__, *llfields) + external = getattr(struct_ctype, '_external_', False) + extras = {'hints': {'c_name': struct_ctype.__name__, + 'external': external}} + c_data_type = lltype.Struct(struct_ctype.__name__, *llfields, **extras) super(StructRepr, self).__init__(rtyper, s_struct, c_data_type) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Apr 14 00:13:29 2006 @@ -34,7 +34,10 @@ # XXX the built module and intermediate files should go to /tmp/usession-*, # see pypy.tool.udir -compile_c_module([thisdir.join("_rctypes_test.c")], "_rctypes_test") +c_source = thisdir.join("_rctypes_test.c") +compile_c_module([c_source], "_rctypes_test") +includes = (str(c_source),) # in the sequel, we #include the whole .c file +del c_source # into the generated C sources if sys.platform == "win32": _rctypes_test = cdll_load("_rctypes_test.pyd") @@ -45,6 +48,7 @@ class tagpoint(Structure): _fields_ = [("x", c_int), ("y", c_int)] + _external_ = True # hack to avoid redeclaration of the struct in C # _testfunc_byval testfunc_byval = _rctypes_test._testfunc_byval @@ -57,6 +61,7 @@ pout.y = in_.y return in_.x + in_.y testfunc_byval.llinterp_friendly_version = ll_testfunc_byval +testfunc_byval.includes = includes # _test_struct testfunc_struct = _rctypes_test._testfunc_struct @@ -75,7 +80,7 @@ testfunc_struct_pointer_id.argtypes = [tagpointptr] -def test_rctypes_dll(): # this test is also annotated/specialized below +def test_rctypes_dll(): in_point = tagpoint() in_point.x = 42 in_point.y = 17 @@ -84,6 +89,7 @@ assert res == in_point.x + in_point.y assert out_point.x == 42 assert out_point.y == 17 + return out_point.x - out_point.y # this test function is reused below class Test_annotation: def test_annotate_byval(self): @@ -92,7 +98,14 @@ s = a.build_types(test_rctypes_dll, []) if conftest.option.view: t.view() + assert s.knowntype == int class Test_specialization: def test_specialize_byval(self): - interpret(test_rctypes_dll, []) + res = interpret(test_rctypes_dll, []) + assert res == 42 - 17 + +class Test_compile: + def test_compile_byval(self): + fn = compile(test_rctypes_dll, []) + assert fn() == 42 - 17 Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Apr 14 00:13:29 2006 @@ -7,7 +7,7 @@ from pypy.rpython.lltypesystem.lltype import Ptr, PyObject, Void, Bool, Signed from pypy.rpython.lltypesystem.lltype import Unsigned, SignedLongLong from pypy.rpython.lltypesystem.lltype import UnsignedLongLong, Char, UniChar -from pypy.rpython.lltypesystem.lltype import pyobjectptr +from pypy.rpython.lltypesystem.lltype import pyobjectptr, ContainerType from pypy.rpython.lltypesystem.lltype import Struct, Array, FixedSizeArray PyObjPtr = Ptr(PyObject) @@ -366,9 +366,17 @@ return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r) def OP_DIRECT_CALL(self, op): - # skip 'void' arguments - args = [self.expr(v) for v in op.args if self.lltypemap(v) is not Void] - line = '%s(%s);' % (args[0], ', '.join(args[1:])) + args = [] + fn = op.args[0] + for v, ARGTYPE in zip(op.args[1:], fn.concretetype.TO.ARGS): + if ARGTYPE is Void: + continue # skip 'void' argument + args.append(self.expr(v)) + # special case for rctypes: by-value container args: + if isinstance(ARGTYPE, ContainerType): + args[-1] = '*%s' % (args[-1],) + + line = '%s(%s);' % (self.expr(fn), ', '.join(args)) if self.lltypemap(op.result) is not Void: # skip assignment of 'void' return value r = self.expr(op.result) @@ -379,7 +387,7 @@ return line # the following works since the extra arguments that indirect_call has - # is of type Void, which is removed by OP_DIRECT_CALL + # is removed by zip() OP_INDIRECT_CALL = OP_DIRECT_CALL def OP_UNSAFE_CALL(self, op): Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Fri Apr 14 00:13:29 2006 @@ -131,7 +131,6 @@ class RefcountingRuntimeTypeInfo_OpaqueNode(ContainerNode): nodekind = 'refcnt rtti' globalcontainer = True - includes = () typename = 'void (@)(void *)' def __init__(self, db, T, obj): @@ -232,7 +231,6 @@ class BoehmGcRuntimeTypeInfo_OpaqueNode(ContainerNode): nodekind = 'boehm rtti' globalcontainer = True - includes = () typename = 'char @' def __init__(self, db, T, obj): Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Fri Apr 14 00:13:29 2006 @@ -544,8 +544,9 @@ includes = {} for node in database.globalcontainers(): - for include in node.includes: - includes[include] = True + if hasattr(node, 'includes'): + for include in node.includes: + includes[include] = True includes = includes.keys() includes.sort() for include in includes: @@ -604,8 +605,9 @@ includes = {} for node in database.globalcontainers(): - for include in node.includes: - includes[include] = True + if hasattr(node, 'includes'): + for include in node.includes: + includes[include] = True includes = includes.keys() includes.sort() for include in includes: Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Fri Apr 14 00:13:29 2006 @@ -44,11 +44,16 @@ basename = db.gettypedefnode(STRUCT).barename basename = '%s_len%d' % (basename, varlength) with_number = False - (self.barename, - self.name) = db.namespace.uniquename(basename, with_number=with_number, - bare=True) + if STRUCT._hints.get('c_name'): + self.barename = self.name = STRUCT._hints['c_name'] + self.prefix = '' + else: + (self.barename, + self.name) = db.namespace.uniquename(basename, + with_number=with_number, + bare=True) + self.prefix = somelettersfrom(STRUCT._name) + '_' self.dependencies = {} - self.prefix = somelettersfrom(STRUCT._name) + '_' def setup(self): # this computes self.fields @@ -98,6 +103,8 @@ return '%s.%s' % (baseexpr, fldname) def definition(self): + if self.STRUCT._hints.get('external'): # XXX hack + return yield 'struct %s {' % self.name is_empty = True @@ -320,7 +327,6 @@ includes""".split() def __init__(self, db, T, obj): - self.includes = () self.db = db self.T = T self.obj = obj @@ -530,7 +536,6 @@ self.includes = obj.includes self.name = self.basename() else: - self.includes = () self.name = db.namespace.uniquename('g_' + self.basename()) if not getattr(obj, 'isgchelper', False): self.make_funcgens() @@ -634,7 +639,7 @@ return [FunctionCodeGenerator(fnobj.graph, db, cpython_exc, functionname)] elif getattr(fnobj, 'external', None) == 'C': # deprecated case - if getattr(fnobj, 'includes', None): + if hasattr(fnobj, 'includes'): return [] # assume no wrapper needed else: return [CExternalFunctionCodeGenerator(fnobj, db)] @@ -675,7 +680,6 @@ globalcontainer = True typename = 'PyObject @' implementationtypename = 'PyObject *@' - includes = () def __init__(self, db, T, obj): # obj is a _pyobject here; obj.value is the underlying CPython object From antocuni at codespeak.net Fri Apr 14 00:14:52 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 14 Apr 2006 00:14:52 +0200 (CEST) Subject: [pypy-svn] r25822 - in pypy/dist/pypy: rpython rpython/lltypesystem rpython/ootypesystem translator/cli/src translator/cli/test Message-ID: <20060413221452.3259B10081@code0.codespeak.net> Author: antocuni Date: Fri Apr 14 00:14:12 2006 New Revision: 25822 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/lltypesystem/rslice.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rslice.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/rslice.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/test_list.py Log: - Added support for basic slice operations on ootypesystem.rlist - Added some tests for basic slice operations in the CLI backend Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Fri Apr 14 00:14:12 2006 @@ -51,6 +51,10 @@ # setup() needs to be called to finish this initialization self.ll_concat = ll_concat self.ll_extend = ll_extend + self.ll_listslice_startonly = ll_listslice_startonly + self.ll_listslice = ll_listslice + self.ll_listslice_minusone = ll_listslice_minusone + self.ll_listsetslice = ll_listsetslice self.list_builder = ListBuilder() def _setup_repr_final(self): @@ -263,7 +267,7 @@ v_res = hop.gendirectcall(llfn, v_func, *args) return self.recast(hop.llops, v_res) -class FixedSizeListRepr(BaseListRepr): +class FixedSizeListRepr(AbstractListRepr, BaseListRepr): def _setup_repr(self): if 'item_repr' not in self.__dict__: @@ -351,30 +355,6 @@ v_lst, v_factor = hop.inputargs(r_lst, Signed) return hop.gendirectcall(ll_inplace_mul, v_lst, v_factor) -class __extend__(pairtype(BaseListRepr, SliceRepr)): - - def rtype_getitem((r_lst, r_slic), hop): - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - if r_slic == startonly_slice_repr: - v_lst, v_start = hop.inputargs(r_lst, startonly_slice_repr) - return hop.gendirectcall(ll_listslice_startonly, cRESLIST, v_lst, v_start) - if r_slic == startstop_slice_repr: - v_lst, v_slice = hop.inputargs(r_lst, startstop_slice_repr) - return hop.gendirectcall(ll_listslice, cRESLIST, v_lst, v_slice) - if r_slic == minusone_slice_repr: - v_lst, v_ignored = hop.inputargs(r_lst, minusone_slice_repr) - return hop.gendirectcall(ll_listslice_minusone, cRESLIST, v_lst) - raise TyperError('getitem does not support slices with %r' % (r_slic,)) - - def rtype_setitem((r_lst, r_slic), hop): - #if r_slic == startonly_slice_repr: - # not implemented - if r_slic == startstop_slice_repr: - v_lst, v_slice, v_lst2 = hop.inputargs(r_lst, startstop_slice_repr, - hop.args_r[2]) - hop.gendirectcall(ll_listsetslice, v_lst, v_slice, v_lst2) - return - raise TyperError('setitem does not support slices with %r' % (r_slic,)) class __extend__(pairtype(ListRepr, SliceRepr)): Modified: pypy/dist/pypy/rpython/lltypesystem/rslice.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rslice.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rslice.py Fri Apr 14 00:14:12 2006 @@ -1,4 +1,4 @@ -from pypy.rpython.rmodel import Repr +from pypy.rpython.rslice import AbstractSliceRepr from pypy.rpython.lltypesystem.lltype import \ GcStruct, Signed, Ptr, Void, malloc @@ -18,7 +18,7 @@ SLICE = GcStruct("slice", ("start", Signed), ("stop", Signed)) -class SliceRepr(Repr): +class SliceRepr(AbstractSliceRepr): pass startstop_slice_repr = SliceRepr() Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 14 00:14:12 2006 @@ -206,6 +206,9 @@ "append": Meth([self.ITEMTYPE_T], Void), "getitem_nonneg": Meth([Signed], self.ITEMTYPE_T), "setitem_nonneg": Meth([Signed, self.ITEMTYPE_T], Void), +## "getslice_startonly": Meth([Signed], self.SELFTYPE_T), +## "getslice_startstop": Meth([Signed, Signed], self.SELFTYPE_T), +## "reverse": Meth([], self.SELFTYPE_T), "extend": Meth([self.SELFTYPE_T], Void), }) Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Fri Apr 14 00:14:12 2006 @@ -3,7 +3,7 @@ AbstractListIteratorRepr, rtype_newlist from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.rmodel import inputconst, externalvsinternal -from pypy.rpython.lltypesystem.lltype import Signed +from pypy.rpython.lltypesystem.lltype import Signed, Void from pypy.rpython.ootypesystem import ootype from pypy.rpython.ootypesystem.riterable import iterator_type from pypy.rpython.ootypesystem.rslice import SliceRepr, \ @@ -27,6 +27,10 @@ self.list_cache = {} self.ll_concat = ll_concat self.ll_extend = ll_extend + self.ll_listslice_startonly = ll_listslice_startonly + self.ll_listslice = ll_listslice + self.ll_listslice_minusone = ll_listslice_minusone + self.ll_listsetslice = ll_listsetslice # setup() needs to be called to finish this initialization def _setup_repr(self): @@ -36,8 +40,9 @@ if isinstance(self.lowleveltype, ootype.ForwardReference): self.lowleveltype.become(ootype.List(self.item_repr.lowleveltype)) - def send_message(self, hop, message, can_raise=False): - v_args = hop.inputargs(self, *hop.args_r[1:]) + def send_message(self, hop, message, can_raise=False, v_args=None): + if v_args is None: + v_args = hop.inputargs(self, *hop.args_r[1:]) c_name = hop.inputconst(ootype.Void, message) if can_raise: hop.exception_is_here() @@ -80,34 +85,7 @@ v_list, v_index, v_item = hop.inputargs(r_list, Signed, r_list.item_repr) hop.exception_is_here() return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) - -class __extend__(pairtype(BaseListRepr, SliceRepr)): - - def rtype_getitem((r_list, r_slic), hop): - raise NotImplementedError # TODO -## cRESLIST = hop.inputconst(Void, hop.r_result.LIST) -## if r_slic == startonly_slice_repr: -## v_lst, v_start = hop.inputargs(r_lst, startonly_slice_repr) -## return hop.gendirectcall(ll_listslice_startonly, cRESLIST, v_lst, v_start) -## if r_slic == startstop_slice_repr: -## v_lst, v_slice = hop.inputargs(r_lst, startstop_slice_repr) -## return hop.gendirectcall(ll_listslice, cRESLIST, v_lst, v_slice) -## if r_slic == minusone_slice_repr: -## v_lst, v_ignored = hop.inputargs(r_lst, minusone_slice_repr) -## return hop.gendirectcall(ll_listslice_minusone, cRESLIST, v_lst) -## raise TyperError('getitem does not support slices with %r' % (r_slic,)) - - -def ll_getitem(lst, index): - if index < 0: - index += lst.length() - return lst.getitem_nonneg(index) - -def ll_setitem(lst, index, item): - if index < 0: - index += lst.length() - return lst.setitem_nonneg(index, item) def newlist(llops, r_list, items_v): c_1ist = inputconst(ootype.Void, r_list.lowleveltype) @@ -124,6 +102,16 @@ def ll_newlist(LIST): return ootype.new(LIST) +def ll_getitem(lst, index): + if index < 0: + index += lst.length() + return lst.getitem_nonneg(index) + +def ll_setitem(lst, index, item): + if index < 0: + index += lst.length() + return lst.setitem_nonneg(index, item) + def ll_append(lst, item): lst.append(item) @@ -149,6 +137,54 @@ i += 1 return l +def ll_listslice_startonly(RESLIST, lst, start): + len1 = lst.length() + #newlength = len1 - start + res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements + i = start + while i < len1: + res.append(lst.getitem_nonneg(i)) + i += 1 + return res + +def ll_listslice(RESLIST, lst, slice): + start = slice.start + stop = slice.stop + length = lst.length() + if stop > length: + stop = length + #newlength = stop - start + res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements + i = start + while i < stop: + res.append(lst.getitem_nonneg(i)) + i += 1 + return res + +def ll_listslice_minusone(RESLIST, lst): + newlength = lst.length() - 1 + #assert newlength >= 0 # TODO: asserts seems to have problems with ootypesystem + res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements + i = 0 + while i < newlength: + res.append(lst.getitem_nonneg(i)) + i += 1 + return res + +def ll_listsetslice(l1, slice, l2): + count = l2.length() +## assert count == slice.stop - slice.start, ( # TODO: see above +## "setslice cannot resize lists in RPython") + # XXX but it should be easy enough to support, soon + start = slice.start + j = start + i = 0 + while i < count: + l1.setitem_nonneg(j, l2.getitem_nonneg(i)) + i += 1 + j += 1 + + # ____________________________________________________________ # # Iteration. @@ -161,6 +197,7 @@ self.ll_listiter = ll_listiter self.ll_listnext = ll_listnext + def ll_listiter(ITER, lst): iter = ootype.new(ITER) iter.iterable = lst Modified: pypy/dist/pypy/rpython/ootypesystem/rslice.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rslice.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rslice.py Fri Apr 14 00:14:12 2006 @@ -1,10 +1,10 @@ -from pypy.rpython.rmodel import Repr +from pypy.rpython.rslice import AbstractSliceRepr from pypy.rpython.lltypesystem.lltype import Void, Signed from pypy.rpython.ootypesystem import ootype SLICE = ootype.Instance('Slice', ootype.ROOT, {'start': Signed, 'stop': Signed}) -class SliceRepr(Repr): +class SliceRepr(AbstractSliceRepr): pass startstop_slice_repr = SliceRepr() Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Fri Apr 14 00:14:12 2006 @@ -1,6 +1,7 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.rpython.rmodel import Repr, IteratorRepr, inputconst +from pypy.rpython.rslice import AbstractSliceRepr from pypy.rpython.lltypesystem import lltype from pypy.rpython import robject @@ -73,6 +74,35 @@ hop.gendirectcall(r_lst1.ll_extend, v_lst1, v_lst2) return v_lst1 + +class __extend__(pairtype(AbstractListRepr, AbstractSliceRepr)): + + def rtype_getitem((r_lst, r_slic), hop): + rs = r_lst.rtyper.type_system.rslice + cRESLIST = hop.inputconst(lltype.Void, hop.r_result.LIST) + if r_slic == rs.startonly_slice_repr: + v_lst, v_start = hop.inputargs(r_lst, rs.startonly_slice_repr) + return hop.gendirectcall(r_lst.ll_listslice_startonly, cRESLIST, v_lst, v_start) + if r_slic == rs.startstop_slice_repr: + v_lst, v_slice = hop.inputargs(r_lst, rs.startstop_slice_repr) + return hop.gendirectcall(r_lst.ll_listslice, cRESLIST, v_lst, v_slice) + if r_slic == rs.minusone_slice_repr: + v_lst, v_ignored = hop.inputargs(r_lst, rs.minusone_slice_repr) + return hop.gendirectcall(r_lst.ll_listslice_minusone, cRESLIST, v_lst) + raise TyperError('getitem does not support slices with %r' % (r_slic,)) + + def rtype_setitem((r_lst, r_slic), hop): + #if r_slic == startonly_slice_repr: + # not implemented + rs = r_lst.rtyper.type_system.rslice + if r_slic == rs.startstop_slice_repr: + v_lst, v_slice, v_lst2 = hop.inputargs(r_lst, rs.startstop_slice_repr, + hop.args_r[2]) + hop.gendirectcall(r_lst.ll_listsetslice, v_lst, v_slice, v_lst2) + return + raise TyperError('setitem does not support slices with %r' % (r_slic,)) + + # ____________________________________________________________ # # Iteration. Modified: pypy/dist/pypy/rpython/rslice.py ============================================================================== --- pypy/dist/pypy/rpython/rslice.py (original) +++ pypy/dist/pypy/rpython/rslice.py Fri Apr 14 00:14:12 2006 @@ -1,8 +1,13 @@ +from pypy.rpython.rmodel import Repr from pypy.rpython.lltypesystem.lltype import Signed, Void from pypy.objspace.flow.model import Constant from pypy.annotation import model as annmodel from pypy.rpython.error import TyperError +class AbstractSliceRepr(Repr): + pass + + def select_slice_repr(self): # Select which one of the three prebuilt reprs to use. # Return a name. Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Fri Apr 14 00:14:12 2006 @@ -10,6 +10,8 @@ } } + //The public interface List must implement is defined in + // rpython.ootypesystem.ootype.List.GENERIC_METHODS public class List: System.Collections.Generic.List { public int length() @@ -17,6 +19,11 @@ return this.Count; } + public void append(T item) + { + this.Add(item); + } + public T getitem_nonneg(int index) { return this[index]; @@ -27,11 +34,6 @@ this[index] = value_; } - public void append(T item) - { - this.Add(item); - } - public void extend(List other) { this.AddRange(other); Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Fri Apr 14 00:14:12 2006 @@ -24,10 +24,17 @@ print 'OK' +def sum_(lst): + total = 0 + i = 0 + while i < len(lst): + total += lst[i] + i += 1 + return total + def bar(x, y): lst = [1,2,3,x,y] - lst[-1] = 42 - return lst[-1] + return sum_(lst[:-1]) f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/test_list.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_list.py (original) +++ pypy/dist/pypy/translator/cli/test/test_list.py Fri Apr 14 00:14:12 2006 @@ -46,3 +46,18 @@ lst += [y, y*2] lst.extend([x, y]) return sum_(lst) + +def list_negative_index(x, y): + lst = create(x, y) + lst[-1] = 4321 + lst[-2] = lst[-1] + return sum_(lst) + +def list_getslice(x, y): + lst = create(x, y) + return sum_(lst[1:3]) * sum_(lst[3:]) * sum_(lst[:-1]) + +def list_setslice(x, y): + lst = create(x, y) + lst[1:3] = [1234, 5678] + return sum_(lst) From tismer at codespeak.net Fri Apr 14 00:28:00 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 14 Apr 2006 00:28:00 +0200 (CEST) Subject: [pypy-svn] r25823 - in pypy/dist/pypy/translator/c: . test winproj/extension Message-ID: <20060413222800.5374110083@code0.codespeak.net> Author: tismer Date: Fri Apr 14 00:27:57 2006 New Revision: 25823 Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Log: small fixes, thanks armin Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Fri Apr 14 00:27:57 2006 @@ -678,7 +678,7 @@ globalobject_name, base_name, wrapper_name, - c_string_constant(func_doc or ''))) + func_doc and c_string_constant(func_doc) or 'NULL')) print >> f, '\t{ NULL }\t/* Sentinel */' print >> f, '};' print >> f, 'static globalfunctiondef_t *globalfunctiondefsptr = &globalfunctiondefs[0];' Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Fri Apr 14 00:27:57 2006 @@ -29,8 +29,8 @@ from pypy.translator.c import gc from pypy.translator.c.genc import CExtModuleBuilder - global t # allow us to view later - t = TranslationContext(do_imports_immediately=False) + global _t # allow us to view later + _t = t = TranslationContext(do_imports_immediately=False) t.buildannotator() rtyper = t.buildrtyper() bk = rtyper.annotator.bookkeeper Modified: pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj ============================================================================== --- pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj (original) +++ pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Fri Apr 14 00:27:57 2006 @@ -208,7 +208,7 @@ + RelativePath="..\..\..\..\..\..\..\Documents and Settings\ctismer\Local Settings\Temp\usession-2024\testing_1\testing_1.c"> From tismer at codespeak.net Fri Apr 14 04:15:41 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 14 Apr 2006 04:15:41 +0200 (CEST) Subject: [pypy-svn] r25824 - pypy/dist/pypy/translator/c/test Message-ID: <20060414021541.4A6101007A@code0.codespeak.net> Author: tismer Date: Fri Apr 14 04:15:38 2006 New Revision: 25824 Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py Log: got rid of internal rtyping. Did wrap/unwrap by using convertvar. Next thing to come is tunneling annotations across someobjectness. This allows to specify annotations without havin to be explicit. Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Fri Apr 14 04:15:38 2006 @@ -83,137 +83,45 @@ # stubs for special annotation/rtyping -def wrap_obj(thing): - RaiseNameError +def wrap(thing): + return thing # untranslated case -def unwrap_obj(pyobj, typ): - RaiseNameError -unwrap_obj._annspecialcase_ = 'specialize:arg(1)' - -def call_destructor(thing, savedrepr): - ll_call_destructor(thing, savedrepr) - -def ll_call_destructor(thang, savedtrpr): - return 42 # really not relevant - -""" -creating a wrapper object with its destructor. -Note that we need annotate_helper_fn, because -the destructor is never explicitly called. -Note also the "hand specialization" which passes the repr through! -This was only possible with Samuele's hints. -""" - -def rtype_destruct_object(hop): - v_any, c_spec = hop.inputargs(*hop.args_r) - repr = c_spec.value - if repr.has_wrapper: - null = hop.inputconst(lltype.Ptr(lltype.PyObject), lltype.nullptr(lltype.PyObject)) - # XXX this is a hack! We need an operation to remove a broken PyObject - repr.setfield(v_any, '_wrapper_', null, hop.llops, opname='bare_setfield') - hop.genop('gc_unprotect', [v_any]) - -def rtype_unwrap_object(hop): - pyptr = hop.args_r[0] - klass = hop.args_s[1].const - classdef = hop.rtyper.annotator.bookkeeper.getuniqueclassdef(klass) - repr = rclass.getinstancerepr(hop.rtyper, classdef, True) - v_pyobj, v_type = hop.inputargs(*hop.args_r) - if repr.has_wrapper: - c_self = hop.inputconst(robject.pyobj_repr, '__self__') - v_pyobj = hop.genop('getattr', [v_pyobj, c_self], resulttype=pyptr) - v_adr = hop.llops.gencapicall('PyCObject_AsVoidPtr', [v_pyobj], resulttype=hop.r_result) - hop.genop('gc_protect', [v_adr]) - return v_adr - -def rtype_wrap_object(hop): - v_any, = hop.inputargs(*hop.args_r) - repr = hop.args_r[0] - c_repr = hop.inputconst(lltype.Void, repr) - if repr.has_wrapper: - return hop.gendirectcall(ll_wrap_object, v_any, c_repr) - else: - return hop.gendirectcall(create_pywrapper, v_any, c_repr) - -def ll_wrap_object(obj, repr): - ret = fetch_pywrapper(obj, repr) - if not ret: - ret = create_pywrapper(obj, repr) - return ret - -def create_pywrapper(thing, repr): - return ll_create_pywrapper(thing, repr) - -def ll_create_pywrapper(thing, repr): - return 42 - -def fetch_pywrapper(thing, repr): - return ll_fetch_pywrapper(thing, repr) - -def ll_fetch_pywrapper(thing, repr): - return 42 - -def rtype_wrap_object_create(hop): - gencapi = hop.llops.gencapicall - pyptr = hop.r_result - v_any, c_spec = hop.inputargs(*hop.args_r) - repr = c_spec.value - f = call_destructor - hop.genop('gc_protect', [v_any]) - ARG = repr.lowleveltype - reprPBC = hop.rtyper.annotator.bookkeeper.immutablevalue(repr) - fp_dtor = hop.rtyper.annotate_helper_fn(f, [ARG, reprPBC]) - FUNC = lltype.FuncType([ARG, lltype.Void], lltype.Void) - c_dtor = hop.inputconst(lltype.Ptr(FUNC), fp_dtor) - res = gencapi('PyCObject_FromVoidPtr', [v_any, c_dtor], resulttype=pyptr) - if repr.has_wrapper: - cobj = res - c_cls = hop.inputconst(robject.pyobj_repr, repr.classdef.classdesc.pyobj) - c_0 = hop.inputconst(lltype.Signed, 0) - res = gencapi('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyptr) - c_self = hop.inputconst(robject.pyobj_repr, '__self__') - hop.genop('setattr', [res, c_self, cobj], resulttype=pyptr) - repr.setfield(v_any, '_wrapper_', res, hop.llops) - hop.genop('gc_unprotect', [res]) # yes a weak ref - return res - -def rtype_wrap_object_fetch(hop): - v_any, c_spec = hop.inputargs(*hop.args_r) - repr = c_spec.value - if repr.has_wrapper: - return repr.getfield(v_any, '_wrapper_', hop.llops) - else: - null = hop.inputconst(lltype.Ptr(lltype.PyObject), lltype.nullptr(lltype.PyObject)) - return null - - -def compute_annotation_unwrap(s_wrapper, s_class): - assert hasattr(s_class, 'descriptions'), 'need a class in unwrap 2nd arg' - descs = s_class.descriptions +def unwrap(pyobj, typ): + assert isinstance(pyobj, typ) + return pyobj # untranslated case +unwrap._annspecialcase_ = 'specialize:arg(1)' + + +def rtype_unwrap(hop): + v_obj = hop.inputarg(hop.args_r[0], 0) + return hop.llops.convertvar(v_obj, hop.args_r[0], hop.r_result) + +def rtype_wrap(hop): + assert len(hop.args_r) == 1, 'wrap() takes exactly one argument' + v_obj, = hop.inputargs(*hop.args_r) + return hop.llops.convertvar(v_obj, hop.args_r[0], robject.pyobj_repr) + +def compute_annotation_unwrap(s_wrapped, s_spec): + # this will go away, much better way found! + assert hasattr(s_spec, 'descriptions'), 'need a class in unwrap 2nd arg' + descs = s_spec.descriptions assert len(descs) == 1, 'missing specialisation, classdesc not unique!' for desc in descs.keys(): classdef = desc.getuniqueclassdef() return annmodel.SomeInstance(classdef) -extregistry.register_value(ll_create_pywrapper, - compute_result_annotation=annmodel.SomePtr(lltype.Ptr(lltype.PyObject)), - specialize_call=rtype_wrap_object_create) - -extregistry.register_value(ll_fetch_pywrapper, - compute_result_annotation=annmodel.SomePtr(lltype.Ptr(lltype.PyObject)), - specialize_call=rtype_wrap_object_fetch) - -extregistry.register_value(ll_call_destructor, - compute_result_annotation=lambda *args: None, - specialize_call=rtype_destruct_object) +# XXX +# wrapping/unwrapping should be annotatable. +# Idea: create tunnel objects which share +# annotation across SomeObjectness, sharing a key! -extregistry.register_value(wrap_obj, +extregistry.register_value(wrap, compute_result_annotation=annmodel.SomeObject(), - specialize_call=rtype_wrap_object) + specialize_call=rtype_wrap) -extregistry.register_value(unwrap_obj, +extregistry.register_value(unwrap, compute_result_annotation=compute_annotation_unwrap, - specialize_call=rtype_unwrap_object) + specialize_call=rtype_unwrap) # _______________________________________________- # the actual tests @@ -278,28 +186,17 @@ def retrieve(self): return self.hugo -# we have more helper functions here than needed. -# this was to make the debugging easier. - -def call_wrap_obj(inst): - return wrap_obj(inst) -call_wrap_obj._annspecialcase_ = 'specialize:argtype(0)' - -def call_unwrap_obj(pyobj, klass): - return unwrap_obj(pyobj, klass) -call_unwrap_obj._annspecialcase_ = 'specialize:arg(1)' - def democlass_helper_sub(a, b): # prevend inlining if a == -42: return democlass_helper_sub(a-1, b) inst = DemoClass(a, b) - pyobj = call_wrap_obj(inst) - obj = call_unwrap_obj(pyobj, DemoClass) + pyobj = wrap(inst) + obj = unwrap(pyobj, DemoClass) ret = obj.demo() inst = DemoSubclass(a, b, 42) - pyobj = call_wrap_obj(inst) - obj = call_unwrap_obj(pyobj, DemoSubclass) + pyobj = wrap(inst) + obj = unwrap(pyobj, DemoSubclass) ret = obj.demo() return ret @@ -319,7 +216,6 @@ def setup_new_module(mod, modname): # note the name clash with py.test on setup_module - return from types import module m = module(modname) allobjs = mod.__dict__.values() @@ -333,15 +229,6 @@ funcs.append( (obj.func_name, obj) ) print 'funcs=', funcs funcs.sort() - for name, func in funcs: - print name, func - for name in mod.__all__: - obj = getattr(mod, name) - #if isinstance(obj, type): - # careful, this gives a class! - if hasattr(mod, '__bases__'): - print name - setattr(m, name, obj) return m From arigo at codespeak.net Fri Apr 14 11:34:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 14 Apr 2006 11:34:47 +0200 (CEST) Subject: [pypy-svn] r25825 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060414093447.D08C31007D@code0.codespeak.net> Author: arigo Date: Fri Apr 14 11:34:45 2006 New Revision: 25825 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: Returning pointers from rctypes functions. More tests, some uniformization. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Fri Apr 14 11:34:45 2006 @@ -67,12 +67,14 @@ class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) - if isinstance(r_array.r_item, PrimitiveRepr): - # primitive case (optimization only) - return r_array.get_item_value(hop.llops, v_array, v_index) - # normal case - v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) - return r_array.r_item.return_c_data(hop.llops, v_c_data) + if isinstance(r_array.r_item, CTypesRefRepr): + # ByRef case + v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) + return r_array.r_item.return_c_data(hop.llops, v_c_data) + else: + # ByValue case (optimization; the above also works in this case) + v_value = r_array.get_item_value(hop.llops, v_array, v_index) + return r_array.r_item.return_value(hop.llops, v_value) def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Fri Apr 14 11:34:45 2006 @@ -19,6 +19,11 @@ v_char_p = self.getvalue_from_c_data(llops, v_c_data) return llops.gendirectcall(ll_charp2str, v_char_p) + def return_value(self, llops, v_value): + # like return_c_data(), but when the input is only the value + # field instead of the c_data pointer + return llops.gendirectcall(ll_charp2str, v_value) + def get_content_keepalives(self): "Return an extra keepalive field used for the RPython string." return [('keepalive_str', string_repr.lowleveltype)] Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Fri Apr 14 11:34:45 2006 @@ -70,7 +70,7 @@ last_op.args[0].value._set_T(FUNCTYPE) last_op.args[0].value._obj._TYPE = FUNCTYPE - return v_result + return r_res.return_value(hop.llops, v_result) extregistry.register_metatype(CFuncPtrType, compute_annotation=cfuncptrtype_compute_annotation, Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Fri Apr 14 11:34:45 2006 @@ -201,6 +201,14 @@ value = value.value p.c_data[0] = value + def return_value(self, llops, v_value): + # like return_c_data(), but when the input is only the value + # field instead of the c_data pointer + r_temp = self.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + r_temp.setvalue(llops, v_owned_box, v_value) + return llops.convertvar(v_owned_box, r_temp, self) + # ____________________________________________________________ C_ZERO = inputconst(lltype.Signed, 0) Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Fri Apr 14 11:34:45 2006 @@ -34,6 +34,11 @@ """ return self.getvalue_from_c_data(llops, v_c_data) + def return_value(self, llops, v_value): + # like return_c_data(), but when the input is only the value + # field instead of the c_data pointer + return v_value + def rtype_getattr(self, hop): s_attr = hop.args_s[1] assert s_attr.is_constant() Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Fri Apr 14 11:34:45 2006 @@ -78,12 +78,14 @@ name = s_attr.const r_field = self.r_fields[name] v_struct, v_attr = hop.inputargs(self, lltype.Void) - if isinstance(r_field, PrimitiveRepr): - # primitive case (optimization only) - return self.get_field_value(hop.llops, v_struct, name) - # normal case - v_c_data = self.get_c_data_of_field(hop.llops, v_struct, name) - return r_field.return_c_data(hop.llops, v_c_data) + if isinstance(r_field, CTypesRefRepr): + # ByRef case + v_c_data = self.get_c_data_of_field(hop.llops, v_struct, name) + return r_field.return_c_data(hop.llops, v_c_data) + else: + # ByValue case (optimization; the above also works in this case) + v_value = self.get_field_value(hop.llops, v_struct, name) + return r_field.return_value(hop.llops, v_value) def rtype_setattr(self, hop): s_attr = hop.args_s[1] Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Apr 14 11:34:45 2006 @@ -22,7 +22,7 @@ from ctypes import cdll -from ctypes import POINTER, Structure, c_int, byref +from ctypes import POINTER, Structure, c_int, byref, pointer # __________ compile and load our local test C file __________ @@ -50,6 +50,16 @@ ("y", c_int)] _external_ = True # hack to avoid redeclaration of the struct in C +# _test_struct +testfunc_struct = _rctypes_test._testfunc_struct +testfunc_struct.restype = c_int +testfunc_struct.argtypes = [tagpoint] + +def ll_testfunc_struct(in_): + return in_.x + in_.y +testfunc_struct.llinterp_friendly_version = ll_testfunc_struct +testfunc_struct.includes = includes + # _testfunc_byval testfunc_byval = _rctypes_test._testfunc_byval testfunc_byval.restype = c_int @@ -63,15 +73,11 @@ testfunc_byval.llinterp_friendly_version = ll_testfunc_byval testfunc_byval.includes = includes -# _test_struct -testfunc_struct = _rctypes_test._testfunc_struct -testfunc_struct.restype = c_int -testfunc_struct.argtypes = [tagpoint] - # _test_struct_id -testfunc_struct_id = _rctypes_test._testfunc_struct_id -testfunc_struct_id.restype = tagpoint -testfunc_struct_id.argtypes = [tagpoint] +# XXX no support for returning structs +#testfunc_struct_id = _rctypes_test._testfunc_struct_id +#testfunc_struct_id.restype = tagpoint +#testfunc_struct_id.argtypes = [tagpoint] # _test_struct_id_pointer tagpointptr = POINTER(tagpoint) @@ -79,8 +85,22 @@ testfunc_struct_pointer_id.restype = tagpointptr testfunc_struct_pointer_id.argtypes = [tagpointptr] +def ll_testfunc_struct_pointer_id(pin): + return pin +testfunc_struct_pointer_id.llinterp_friendly_version = ( + ll_testfunc_struct_pointer_id) +testfunc_struct_pointer_id.includes = includes + + +def test_testfunc_struct(): + in_point = tagpoint() + in_point.x = 42 + in_point.y = 17 + res = testfunc_struct(in_point) + assert res == in_point.x + in_point.y + return in_point.x - in_point.y # this test function is reused below -def test_rctypes_dll(): +def test_testfunc_byval(): in_point = tagpoint() in_point.x = 42 in_point.y = 17 @@ -91,21 +111,50 @@ assert out_point.y == 17 return out_point.x - out_point.y # this test function is reused below +def test_testfunc_struct_pointer_id(): + in_point = tagpoint() + in_point.x = 42 + in_point.y = 17 + res = testfunc_struct_pointer_id(byref(in_point)) + res.contents.x //= 2 + assert in_point.x == 21 + return in_point.x - in_point.y # this test function is reused below + class Test_annotation: + def test_annotate_struct(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(test_testfunc_struct, []) + if conftest.option.view: + t.view() + assert s.knowntype == int + def test_annotate_byval(self): t = TranslationContext() a = t.buildannotator() - s = a.build_types(test_rctypes_dll, []) + s = a.build_types(test_testfunc_byval, []) if conftest.option.view: t.view() assert s.knowntype == int class Test_specialization: + def test_specialize_struct(self): + res = interpret(test_testfunc_struct, []) + assert res == 42 - 17 + def test_specialize_byval(self): - res = interpret(test_rctypes_dll, []) + res = interpret(test_testfunc_byval, []) assert res == 42 - 17 + def test_specialize_struct_pointer_id(self): + res = interpret(test_testfunc_struct_pointer_id, []) + assert res == 21 - 17 + class Test_compile: def test_compile_byval(self): - fn = compile(test_rctypes_dll, []) + fn = compile(test_testfunc_byval, []) assert fn() == 42 - 17 + + def test_compile_struct_pointer_id(self): + fn = compile(test_testfunc_struct_pointer_id, []) + assert fn() == 21 - 17 From antocuni at codespeak.net Fri Apr 14 13:42:55 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 14 Apr 2006 13:42:55 +0200 (CEST) Subject: [pypy-svn] r25828 - in pypy/dist/pypy: rpython rpython/lltypesystem rpython/ootypesystem rpython/test translator/cli translator/cli/src translator/cli/test Message-ID: <20060414114255.EBA801007D@code0.codespeak.net> Author: antocuni Date: Fri Apr 14 13:42:40 2006 New Revision: 25828 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/translator/cli/rte.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/test_list.py Log: - Added support for deleting items and slices in ootypesystem rlist - Added some ootypesystem tests in rpython/test/test_rlist.py - Improved support for lists in the CLI backend. At the moment two tests in test_rlist skip because they need to return a tuple of lists and ootypesystem doesn't support that, yet. TODO: Remove the skip as soon as ootypesystem is fixed. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Fri Apr 14 13:42:40 2006 @@ -5,7 +5,7 @@ from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.rmodel import externalvsinternal from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ - AbstractListIteratorRepr, rtype_newlist + AbstractFixedSizeListRepr, AbstractListIteratorRepr, rtype_newlist from pypy.rpython.rlist import dum_nocheck, dum_checkidx from pypy.rpython.lltypesystem.rslice import SliceRepr from pypy.rpython.lltypesystem.rslice import startstop_slice_repr, startonly_slice_repr @@ -55,6 +55,8 @@ self.ll_listslice = ll_listslice self.ll_listslice_minusone = ll_listslice_minusone self.ll_listsetslice = ll_listsetslice + self.ll_listdelslice_startonly = ll_listdelslice_startonly + self.ll_listdelslice = ll_listdelslice self.list_builder = ListBuilder() def _setup_repr_final(self): @@ -267,7 +269,7 @@ v_res = hop.gendirectcall(llfn, v_func, *args) return self.recast(hop.llops, v_res) -class FixedSizeListRepr(AbstractListRepr, BaseListRepr): +class FixedSizeListRepr(AbstractFixedSizeListRepr, BaseListRepr): def _setup_repr(self): if 'item_repr' not in self.__dict__: @@ -356,19 +358,6 @@ return hop.gendirectcall(ll_inplace_mul, v_lst, v_factor) -class __extend__(pairtype(ListRepr, SliceRepr)): - - def rtype_delitem((r_lst, r_slic), hop): - if r_slic == startonly_slice_repr: - v_lst, v_start = hop.inputargs(r_lst, startonly_slice_repr) - hop.gendirectcall(ll_listdelslice_startonly, v_lst, v_start) - return - if r_slic == startstop_slice_repr: - v_lst, v_slice = hop.inputargs(r_lst, startstop_slice_repr) - hop.gendirectcall(ll_listdelslice, v_lst, v_slice) - return - raise TyperError('delitem does not support slices with %r' % (r_slic,)) - class __extend__(pairtype(BaseListRepr, BaseListRepr)): def convert_from_to((r_lst1, r_lst2), v, llops): if r_lst1.listitem is None or r_lst2.listitem is None: Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 14 13:42:40 2006 @@ -206,10 +206,8 @@ "append": Meth([self.ITEMTYPE_T], Void), "getitem_nonneg": Meth([Signed], self.ITEMTYPE_T), "setitem_nonneg": Meth([Signed, self.ITEMTYPE_T], Void), -## "getslice_startonly": Meth([Signed], self.SELFTYPE_T), -## "getslice_startstop": Meth([Signed, Signed], self.SELFTYPE_T), -## "reverse": Meth([], self.SELFTYPE_T), "extend": Meth([self.SELFTYPE_T], Void), + "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) }) self._setup_methods(generic_types) @@ -560,6 +558,9 @@ assert typeOf(other) == typeOf(self) self._list.extend(other._list) + def remove_range(self, start, count): + del self._list[start:start+count] + class _null_list(_null_mixin(_list), _list): def __init__(self, LIST): Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Fri Apr 14 13:42:40 2006 @@ -30,7 +30,9 @@ self.ll_listslice_startonly = ll_listslice_startonly self.ll_listslice = ll_listslice self.ll_listslice_minusone = ll_listslice_minusone - self.ll_listsetslice = ll_listsetslice + self.ll_listsetslice = ll_listsetslice + self.ll_listdelslice_startonly = ll_listdelslice_startonly + self.ll_listdelslice = ll_listdelslice # setup() needs to be called to finish this initialization def _setup_repr(self): @@ -87,6 +89,19 @@ return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) +class __extend__(pairtype(ListRepr, IntegerRepr)): + + def rtype_delitem((r_list, r_int), hop): + v_list, v_index = hop.inputargs(r_list, Signed) + if hop.args_s[1].nonneg: + v_count = hop.inputconst(Signed, 1) + return r_list.send_message(hop, "remove_range",can_raise=True, + v_args=[v_list, v_index, v_count]) + else: + hop.exception_is_here() + return hop.gendirectcall(ll_delitem, v_list, v_index) + + def newlist(llops, r_list, items_v): c_1ist = inputconst(ootype.Void, r_list.lowleveltype) v_result = llops.genop("new", [c_1ist], resulttype=r_list.lowleveltype) @@ -112,6 +127,11 @@ index += lst.length() return lst.setitem_nonneg(index, item) +def ll_delitem(lst, index): + if index < 0: + index += lst.length() + return lst.remove_range(index, 1) + def ll_append(lst, item): lst.append(item) @@ -184,6 +204,20 @@ i += 1 j += 1 +def ll_listdelslice_startonly(lst, start): + count = lst.length() - start + if count > 0: + lst.remove_range(start, count) + +def ll_listdelslice(lst, slice): + start = slice.start + stop = slice.stop + length = lst.length() + if stop > length: + stop = length + count = stop - start + if count > 0: + lst.remove_range(start, count) # ____________________________________________________________ # Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Fri Apr 14 13:42:40 2006 @@ -36,9 +36,12 @@ return llops.convertvar(v, self.item_repr, self.external_item_repr) class AbstractListRepr(AbstractBaseListRepr): - pass +class AbstractFixedSizeListRepr(AbstractBaseListRepr): + pass + + def rtype_newlist(hop): nb_args = hop.nb_args r_list = hop.r_result @@ -75,7 +78,7 @@ return v_lst1 -class __extend__(pairtype(AbstractListRepr, AbstractSliceRepr)): +class __extend__(pairtype(AbstractBaseListRepr, AbstractSliceRepr)): def rtype_getitem((r_lst, r_slic), hop): rs = r_lst.rtyper.type_system.rslice @@ -103,6 +106,21 @@ raise TyperError('setitem does not support slices with %r' % (r_slic,)) +class __extend__(pairtype(AbstractListRepr, AbstractSliceRepr)): + + def rtype_delitem((r_lst, r_slic), hop): + rs = r_lst.rtyper.type_system.rslice + if r_slic == rs.startonly_slice_repr: + v_lst, v_start = hop.inputargs(r_lst, rs.startonly_slice_repr) + hop.gendirectcall(r_lst.ll_listdelslice_startonly, v_lst, v_start) + return + if r_slic == rs.startstop_slice_repr: + v_lst, v_slice = hop.inputargs(r_lst, rs.startstop_slice_repr) + hop.gendirectcall(r_lst.ll_listdelslice, v_lst, v_slice) + return + raise TyperError('delitem does not support slices with %r' % (r_slic,)) + + # ____________________________________________________________ # # Iteration. Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Fri Apr 14 13:42:40 2006 @@ -251,79 +251,80 @@ res = interpret(dummyfn, [], type_system=self.ts) assert self.ll_to_list(res) == [5, 6, 7, 8, 9] -def test_slice(): - def dummyfn(): - l = [5, 6, 7, 8, 9] - return l[:2], l[1:4], l[3:] - res = interpret(dummyfn, []) - assert tolst(res.item0) == [5, 6] - assert tolst(res.item1) == [6, 7, 8] - assert tolst(res.item2) == [8, 9] - - def dummyfn(): - l = [5, 6, 7, 8] - l.append(9) - return l[:2], l[1:4], l[3:] - res = interpret(dummyfn, []) - assert tolst(res.item0) == [5, 6] - assert tolst(res.item1) == [6, 7, 8] - assert tolst(res.item2) == [8, 9] - -def test_set_del_item(): - def dummyfn(): - l = [5, 6, 7] - l[1] = 55 - l[-1] = 66 - return l - res = interpret(dummyfn, []) - assert tolst(res) == [5, 55, 66] - - def dummyfn(): - l = [] - l.append(5) - l.append(6) - l.append(7) - l[1] = 55 - l[-1] = 66 - return l - res = interpret(dummyfn, []) - assert tolst(res) == [5, 55, 66] - - def dummyfn(): - l = [5, 6, 7] - l[1] = 55 - l[-1] = 66 - del l[0] - del l[-1] - del l[:] - return len(l) - res = interpret(dummyfn, []) - assert res == 0 + def test_slice(self): + if self.ts == 'ootype': + py.test.skip("ootypesystem doesn't support returning tuples of lists, yet") + def dummyfn(): + l = [5, 6, 7, 8, 9] + return l[:2], l[1:4], l[3:] + res = interpret(dummyfn, [], type_system=self.ts) + assert self.ll_to_list(res.item0) == [5, 6] + assert self.ll_to_list(res.item1) == [6, 7, 8] + assert self.ll_to_list(res.item2) == [8, 9] + + def dummyfn(): + l = [5, 6, 7, 8] + l.append(9) + return l[:2], l[1:4], l[3:] + res = interpret(dummyfn, [], type_system=self.ts) + assert self.ll_to_list(res.item0) == [5, 6] + assert self.ll_to_list(res.item1) == [6, 7, 8] + assert self.ll_to_list(res.item2) == [8, 9] + + def test_set_del_item(self): + def dummyfn(): + l = [5, 6, 7] + l[1] = 55 + l[-1] = 66 + return l + res = interpret(dummyfn, [], type_system=self.ts) + assert self.ll_to_list(res) == [5, 55, 66] -def tolst(l): - return map(None, l.ll_items())[:l.ll_length()] + def dummyfn(): + l = [] + l.append(5) + l.append(6) + l.append(7) + l[1] = 55 + l[-1] = 66 + return l + res = interpret(dummyfn, [], type_system=self.ts) + assert self.ll_to_list(res) == [5, 55, 66] -def test_setslice(): - def dummyfn(): - l = [10, 9, 8, 7] - l[:2] = [6, 5] - return l[0], l[1], l[2], l[3] - res = interpret(dummyfn, ()) - assert res.item0 == 6 - assert res.item1 == 5 - assert res.item2 == 8 - assert res.item3 == 7 + def dummyfn(): + l = [5, 6, 7] + l[1] = 55 + l[-1] = 66 + del l[0] + del l[-1] + del l[:] + return len(l) + res = interpret(dummyfn, [], type_system=self.ts) + assert res == 0 - def dummyfn(): - l = [10, 9, 8] - l.append(7) - l[:2] = [6, 5] - return l[0], l[1], l[2], l[3] - res = interpret(dummyfn, ()) - assert res.item0 == 6 - assert res.item1 == 5 - assert res.item2 == 8 - assert res.item3 == 7 + def test_setslice(self): + if self.ts == 'ootype': + py.test.skip("ootypesystem doesn't support returning tuples of lists, yet") + def dummyfn(): + l = [10, 9, 8, 7] + l[:2] = [6, 5] + return l[0], l[1], l[2], l[3] + res = interpret(dummyfn, (), type_system=self.ts) + assert res.item0 == 6 + assert res.item1 == 5 + assert res.item2 == 8 + assert res.item3 == 7 + + def dummyfn(): + l = [10, 9, 8] + l.append(7) + l[:2] = [6, 5] + return l[0], l[1], l[2], l[3] + res = interpret(dummyfn, (), type_system=self.ts) + assert res.item0 == 6 + assert res.item1 == 5 + assert res.item2 == 8 + assert res.item3 == 7 def test_insert_pop(): def dummyfn(): Modified: pypy/dist/pypy/translator/cli/rte.py ============================================================================== --- pypy/dist/pypy/translator/cli/rte.py (original) +++ pypy/dist/pypy/translator/cli/rte.py Fri Apr 14 13:42:40 2006 @@ -9,6 +9,12 @@ from pypy.translator.cli.sdk import SDK +from pypy.tool.ansi_print import ansi_log +import py +log = py.log.Producer("cli") +py.log.setconsumer("cli", ansi_log) + + SRC = 'pypylib.cs' DLL = 'pypylib.dll' @@ -35,6 +41,7 @@ return dll def compile(source, dll): + log.red("Compiling pypylib.dll") csc = SDK.csc() compiler = subprocess.Popen([csc, '/t:library', '/out:%s' % dll, source], stdout=subprocess.PIPE, stderr=subprocess.PIPE) Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Fri Apr 14 13:42:40 2006 @@ -39,5 +39,10 @@ this.AddRange(other); } + public void remove_range(int start, int count) + { + this.RemoveRange(start, count); + } + } } Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Fri Apr 14 13:42:40 2006 @@ -34,7 +34,8 @@ def bar(x, y): lst = [1,2,3,x,y] - return sum_(lst[:-1]) + del lst[:] + return sum_(lst) f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/test_list.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_list.py (original) +++ pypy/dist/pypy/translator/cli/test/test_list.py Fri Apr 14 13:42:40 2006 @@ -9,7 +9,7 @@ def create(x, y): - return [x, y, x+y, x*y] + return [1, 2, 3, x, y, x+y, x*y] def sum_(lst): total = 0 @@ -61,3 +61,11 @@ lst = create(x, y) lst[1:3] = [1234, 5678] return sum_(lst) + +##def list_del_item_slice(x, y): +## lst = create(x, y) +## del lst[0] +## del lst[2:4] +## del lst[5:] +## return sum_(lst) + From antocuni at codespeak.net Fri Apr 14 16:45:32 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 14 Apr 2006 16:45:32 +0200 (CEST) Subject: [pypy-svn] r25829 - in pypy/dist/pypy: rpython rpython/lltypesystem rpython/ootypesystem rpython/test translator/cli/src translator/cli/test Message-ID: <20060414144532.B3BC11007D@code0.codespeak.net> Author: antocuni Date: Fri Apr 14 16:45:17 2006 New Revision: 25829 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/test_list.py Log: Added support for the following operations in ootypesystem/rlist.py: bltn_list, is_true, method_index. Added some tests both in rpython/test/test_rlist.py and translator/cli/test/test_list.py. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Fri Apr 14 16:45:17 2006 @@ -57,6 +57,7 @@ self.ll_listsetslice = ll_listsetslice self.ll_listdelslice_startonly = ll_listdelslice_startonly self.ll_listdelslice = ll_listdelslice + self.ll_listindex = ll_listindex self.list_builder = ListBuilder() def _setup_repr_final(self): @@ -103,12 +104,6 @@ def rtype_is_true(self, hop): v_lst, = hop.inputargs(self) return hop.gendirectcall(ll_list_is_true, v_lst) - - def rtype_method_index(self, hop): - v_lst, v_value = hop.inputargs(self, self.item_repr) - hop.has_implicit_exception(ValueError) # record that we know about it - hop.exception_is_here() - return hop.gendirectcall(ll_listindex, v_lst, v_value, self.get_eqfunc()) def rtype_method_reverse(self, hop): v_lst, = hop.inputargs(self) Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 14 16:45:17 2006 @@ -559,6 +559,7 @@ self._list.extend(other._list) def remove_range(self, start, count): + # NOT_RPYTHON del self._list[start:start+count] class _null_list(_null_mixin(_list), _list): Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Fri Apr 14 16:45:17 2006 @@ -33,6 +33,7 @@ self.ll_listsetslice = ll_listsetslice self.ll_listdelslice_startonly = ll_listdelslice_startonly self.ll_listdelslice = ll_listdelslice + self.ll_listindex = ll_listindex # setup() needs to be called to finish this initialization def _setup_repr(self): @@ -51,9 +52,22 @@ return hop.genop("oosend", [c_name] + v_args, resulttype=hop.r_result.lowleveltype) + def get_eqfunc(self): + return inputconst(Void, self.item_repr.get_ll_eq_function()) + def rtype_len(self, hop): return self.send_message(hop, "length") + def rtype_is_true(self, hop): + v_lst, = hop.inputargs(self) + return hop.gendirectcall(ll_list_is_true, v_lst) + + def rtype_bltn_list(self, hop): + v_lst = hop.inputarg(self, 0) + c_start = hop.inputconst(Signed, 0) + cRESLIST = hop.inputconst(Void, hop.r_result.LIST) + return hop.gendirectcall(self.ll_listslice_startonly, cRESLIST, v_lst, c_start) + def rtype_method_append(self, hop): return self.send_message(hop, "append") @@ -94,9 +108,9 @@ def rtype_delitem((r_list, r_int), hop): v_list, v_index = hop.inputargs(r_list, Signed) if hop.args_s[1].nonneg: - v_count = hop.inputconst(Signed, 1) + c_count = hop.inputconst(Signed, 1) return r_list.send_message(hop, "remove_range",can_raise=True, - v_args=[v_list, v_index, v_count]) + v_args=[v_list, v_index, c_count]) else: hop.exception_is_here() return hop.gendirectcall(ll_delitem, v_list, v_index) @@ -132,6 +146,9 @@ index += lst.length() return lst.remove_range(index, 1) +def ll_list_is_true(lst): + return bool(lst) and lst.length() != 0 + def ll_append(lst, item): lst.append(item) @@ -219,6 +236,21 @@ if count > 0: lst.remove_range(start, count) +def ll_listindex(lst, obj, eqfn): + lng = lst.length() + j = 0 + while j < lng: + if eqfn is None: + if lst.getitem_nonneg(j) == obj: + return j + else: + if eqfn(lst.getitem_nonneg(j), obj): + return j + j += 1 + raise ValueError # can't say 'list.index(x): x not in list' + + + # ____________________________________________________________ # # Iteration. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Fri Apr 14 16:45:17 2006 @@ -35,6 +35,13 @@ def recast(self, llops, v): return llops.convertvar(v, self.item_repr, self.external_item_repr) + def rtype_method_index(self, hop): + v_lst, v_value = hop.inputargs(self, self.item_repr) + hop.has_implicit_exception(ValueError) # record that we know about it + hop.exception_is_here() + return hop.gendirectcall(self.ll_listindex, v_lst, v_value, self.get_eqfunc()) + + class AbstractListRepr(AbstractBaseListRepr): pass Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Fri Apr 14 16:45:17 2006 @@ -326,6 +326,41 @@ assert res.item2 == 8 assert res.item3 == 7 + def test_bltn_list(self): + def dummyfn(): + l1 = [42] + l2 = list(l1) + l2[0] = 0 + return l1[0] + res = interpret(dummyfn, (), type_system=self.ts) + assert res == 42 + + def test_is_true(self): + def is_true(lst): + if lst: + return True + else: + return False + def dummyfn1(): + return is_true(None) + def dummyfn2(): + return is_true([]) + def dummyfn3(): + return is_true([0]) + assert interpret(dummyfn1, (), type_system=self.ts) == False + assert interpret(dummyfn2, (), type_system=self.ts) == False + assert interpret(dummyfn3, (), type_system=self.ts) == True + + def test_list_index_simple(self): + def dummyfn(i): + l = [5,6,7,8] + return l.index(i) + + res = interpret(dummyfn, (6,), type_system=self.ts) + assert res == 1 + interpret_raises(ValueError, dummyfn, [42], type_system=self.ts) + + def test_insert_pop(): def dummyfn(): l = [6, 7, 8] Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Fri Apr 14 16:45:17 2006 @@ -29,9 +29,9 @@ return this[index]; } - public void setitem_nonneg(int index, T value_) + public void setitem_nonneg(int index, T item) { - this[index] = value_; + this[index] = item; } public void extend(List other) @@ -44,5 +44,9 @@ this.RemoveRange(start, count); } + public int index(T item) + { + return this.IndexOf(item); + } } } Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Fri Apr 14 16:45:17 2006 @@ -34,8 +34,8 @@ def bar(x, y): lst = [1,2,3,x,y] - del lst[:] - return sum_(lst) + #return sum_(list(lst)) + return sum_(lst[:]) f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/test_list.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_list.py (original) +++ pypy/dist/pypy/translator/cli/test/test_list.py Fri Apr 14 16:45:17 2006 @@ -62,10 +62,25 @@ lst[1:3] = [1234, 5678] return sum_(lst) -##def list_del_item_slice(x, y): -## lst = create(x, y) -## del lst[0] -## del lst[2:4] -## del lst[5:] -## return sum_(lst) +def list_bltn_list(x, y): + lst = create(x, y) + lst2 = list(lst) + del lst2[:] + return sum_(lst) + +def list_del_item_slice(x, y): + lst = create(x, y) + del lst[0] + del lst[2:4] + del lst[5:] + return sum_(lst) + +def list_index(x, y): + lst = create(x, y) + res = lst.index(x) + try: + lst.index(x*y+1) + except ValueError: + res += 1 + return res From arigo at codespeak.net Fri Apr 14 18:13:40 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 14 Apr 2006 18:13:40 +0200 (CEST) Subject: [pypy-svn] r25830 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060414161340.95DC810091@code0.codespeak.net> Author: arigo Date: Fri Apr 14 18:13:38 2006 New Revision: 25830 Modified: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py (props changed) pypy/dist/pypy/rpython/rctypes/rchar_p.py (props changed) pypy/dist/pypy/rpython/rctypes/rfunc.py (props changed) pypy/dist/pypy/rpython/rctypes/rmodel.py (props changed) pypy/dist/pypy/rpython/rctypes/rstruct.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (props changed) pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (props changed) Log: fixeol From arigo at codespeak.net Fri Apr 14 18:14:25 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 14 Apr 2006 18:14:25 +0200 (CEST) Subject: [pypy-svn] r25831 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060414161425.5EAE610092@code0.codespeak.net> Author: arigo Date: Fri Apr 14 18:14:23 2006 New Revision: 25831 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/ (props changed) pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (contents, props changed) Log: A draft start at a ctypes-based socket module. Added: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Fri Apr 14 18:14:23 2006 @@ -0,0 +1,41 @@ +import ctypes_socket as _c + + +globals().update(_c.constants) + + +class socket(object): + + def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0): + self.family = family + self.type = type + self.proto = proto + self._fd = _c.socket(family, type, proto) + if self._fd == -1: + XXX + + def __del__(self): + if self._fd != -1: + _c.closesocket(self._fd) + + def close(self): + fd = self._fd + if fd != -1: + self._fd = -1 + _c.closesocket(fd) + + def bind(self, addr): + caddr, caddrlen = self._getsockaddr(addr) + res = _c.bind(self._fd, caddr, caddrlen) + if res < 0: + XXX + + def _getsockaddr(self, addr): + if self.family == AF_INET: + (host, port) = addr + caddr = sockaddr_in() + caddr.sin_family = AF_INET + caddr.sin_port = port + caddr.sin_addr.s_addr = getaddrinfo(host) + else: + XXX Added: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Fri Apr 14 18:14:23 2006 @@ -0,0 +1,55 @@ +import os +from pypy.rpython.rctypes import ctypes_platform +from ctypes import * + +includes = ('sys/types.h', 'sys/socket.h', 'netinet/in.h', 'netdb.h') +HEADER = ''.join(['#include <%s>\n' % filename for filename in includes]) +constants = {} + +# constants +for name in ['AF_INET', + 'SOCK_STREAM', + 'SOCK_DGRAM', + ]: + constants[name] = ctypes_platform.getconstantinteger(name, HEADER) + +# types +sockaddr = ctypes_platform.getstruct('struct sockaddr', HEADER, + [('sa_family', c_int), + # unknown and variable fields follow + ]) +in_addr = ctypes_platform.getstruct('struct in_addr', HEADER, + [('s_addr', c_uint)]) +sockaddr_in = ctypes_platform.getstruct('struct sockaddr_in', HEADER, + [('sin_family', c_int), + ('sin_port', c_ushort), + ('sin_addr', in_addr)]) +addrinfo_ptr = POINTER("addrinfo") +addrinfo = ctypes_platform.getstruct('struct addrinfo', HEADER, + [('ai_flags', c_int), + ('ai_family', c_int), + ('ai_socktype', c_int), + ('ai_protocol', c_int), + ('ai_addrlen', c_int), + ('ai_addr', POINTER(sockaddr)), + ('ai_canonname', c_char_p), + ('ai_next', addrinfo_ptr)]) +SetPointerType(addrinfo_ptr, addrinfo) + +# functions +socketdll = cdll.load('libc.so.6') + +socket = socketdll.socket +socket.argtypes = [c_int, c_int, c_int] +socket.restype = c_int + +socketclose = os.close + +getaddrinfo = socketdll.getaddrinfo +getaddrinfo.argtypes = [c_char_p, c_char_p, POINTER(addrinfo), + POINTER(POINTER(addrinfo))] +getaddrinfo.restype = c_int + +freeaddrinfo = socketdll.freeaddrinfo +freeaddrinfo.argtypes = [POINTER(addrinfo)] +freeaddrinfo.restype = None From arigo at codespeak.net Fri Apr 14 18:37:29 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 14 Apr 2006 18:37:29 +0200 (CEST) Subject: [pypy-svn] r25832 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060414163729.EA5441008E@code0.codespeak.net> Author: arigo Date: Fri Apr 14 18:37:28 2006 New Revision: 25832 Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: Support for rctypes functions returning void (restype==None). Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Fri Apr 14 18:37:28 2006 @@ -15,6 +15,8 @@ Answer the annotation of the external function's result """ result_ctype = instance.restype + if result_ctype is None: + return None s_result = annmodel.SomeCTypesObject(result_ctype, annmodel.SomeCTypesObject.OWNSMEMORY) return s_result.return_annotation() @@ -51,18 +53,22 @@ # generate the correct dereferencing unwrapped_args_v.append(r_arg.get_c_data(hop.llops, v)) ARGTYPES.append(r_arg.c_data_type) - s_res = annmodel.SomeCTypesObject(cfuncptr.restype, - annmodel.SomeCTypesObject.OWNSMEMORY) - r_res = hop.rtyper.getrepr(s_res) + if cfuncptr.restype is not None: + s_res = annmodel.SomeCTypesObject(cfuncptr.restype, + annmodel.SomeCTypesObject.OWNSMEMORY) + r_res = hop.rtyper.getrepr(s_res) + RESTYPE = r_res.ll_type + else: + RESTYPE = lltype.Void ll_func = getattr(cfuncptr, 'llinterp_friendly_version', None) includes = getattr(cfuncptr, 'includes', ()) v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, - resulttype = r_res.ll_type, + resulttype = RESTYPE, _callable = ll_func, includes = includes) # XXX hack! hack! temporary! I promize! - FUNCTYPE = lltype.FuncType(ARGTYPES, r_res.ll_type) + FUNCTYPE = lltype.FuncType(ARGTYPES, RESTYPE) last_op = hop.llops[-1] assert last_op.opname == 'direct_call' last_op.args[0].concretetype = lltype.Ptr(FUNCTYPE) @@ -70,7 +76,10 @@ last_op.args[0].value._set_T(FUNCTYPE) last_op.args[0].value._obj._TYPE = FUNCTYPE - return r_res.return_value(hop.llops, v_result) + if RESTYPE is lltype.Void: + return None + else: + return r_res.return_value(hop.llops, v_result) extregistry.register_metatype(CFuncPtrType, compute_annotation=cfuncptrtype_compute_annotation, Modified: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c (original) +++ pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Fri Apr 14 18:37:28 2006 @@ -29,6 +29,13 @@ return in.x + in.y; } +EXPORT(void) _testfunc_swap(point *p) +{ + int tmp = p->x; + p->x = p->y; + p->y = tmp; +} + EXPORT(int) _testfunc_struct(point in) { return in.x + in.y; Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Apr 14 18:37:28 2006 @@ -91,6 +91,16 @@ ll_testfunc_struct_pointer_id) testfunc_struct_pointer_id.includes = includes +# _testfunc_swap +testfunc_swap = _rctypes_test._testfunc_swap +testfunc_swap.restype = None +testfunc_swap.argtypes = [tagpointptr] + +def ll_testfunc_swap(p): + p.x, p.y = p.y, p.x +testfunc_swap.llinterp_friendly_version = ll_testfunc_swap +testfunc_swap.includes = includes + def test_testfunc_struct(): in_point = tagpoint() @@ -120,6 +130,15 @@ assert in_point.x == 21 return in_point.x - in_point.y # this test function is reused below +def test_testfunc_swap(): + pt = tagpoint() + pt.x = 5 + pt.y = 9 + testfunc_swap(pointer(pt)) + assert pt.x == 9 + assert pt.y == 5 + return pt.x - pt.y # this test function is reused below + class Test_annotation: def test_annotate_struct(self): t = TranslationContext() @@ -150,6 +169,10 @@ res = interpret(test_testfunc_struct_pointer_id, []) assert res == 21 - 17 + def test_specialize_swap(self): + res = interpret(test_testfunc_swap, []) + assert res == 4 + class Test_compile: def test_compile_byval(self): fn = compile(test_testfunc_byval, []) @@ -158,3 +181,7 @@ def test_compile_struct_pointer_id(self): fn = compile(test_testfunc_struct_pointer_id, []) assert fn() == 21 - 17 + + def test_compile_swap(self): + fn = compile(test_testfunc_swap, []) + assert fn() == 4 From cfbolz at codespeak.net Fri Apr 14 21:40:36 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 14 Apr 2006 21:40:36 +0200 (CEST) Subject: [pypy-svn] r25833 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060414194036.9643210093@code0.codespeak.net> Author: cfbolz Date: Fri Apr 14 21:40:35 2006 New Revision: 25833 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: remove chunk Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Fri Apr 14 21:40:35 2006 @@ -1,4 +1,4 @@ -from pypy.rpython.memory import gctransform, support +from pypy.rpython.memory import gctransform from pypy.objspace.flow.model import c_last_exception, Variable from pypy.rpython.memory.gctransform import var_needsgc, var_ispyobj from pypy.translator.translator import TranslationContext, graphof @@ -646,7 +646,6 @@ # tests for FrameworkGCTransformer def DONT_test_framework_simple(): - support.AddressLinkedList.unused_chunks = support.FreeList(support.CHUNK_SIZE + 2) # 'leaks' but well def g(x): return x + 1 class A(object): From cfbolz at codespeak.net Fri Apr 14 21:49:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 14 Apr 2006 21:49:17 +0200 (CEST) Subject: [pypy-svn] r25834 - pypy/dist/pypy/translator/backendopt Message-ID: <20060414194917.1DD6D10093@code0.codespeak.net> Author: cfbolz Date: Fri Apr 14 21:49:15 2006 New Revision: 25834 Added: pypy/dist/pypy/translator/backendopt/graphanalyze.py Modified: pypy/dist/pypy/translator/backendopt/canraise.py Log: factor out class to analyze graphs in a generic way Modified: pypy/dist/pypy/translator/backendopt/canraise.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/canraise.py (original) +++ pypy/dist/pypy/translator/backendopt/canraise.py Fri Apr 14 21:49:15 2006 @@ -1,48 +1,18 @@ from pypy.translator.simplify import get_graph from pypy.rpython.lltypesystem.lloperation import llop, LL_OPERATIONS from pypy.rpython.lltypesystem import lltype +from pypy.translator.backendopt import graphanalyze -class RaiseAnalyzer(object): - def __init__(self, translator): - self.translator = translator - self.call_can_raise = {} - - def can_raise(self, op, seen=None): - if op.opname == "direct_call": - graph = get_graph(op.args[0], self.translator) - ##print "graph", graph - if graph is None: - return True - return self.direct_call_can_raise(graph, seen) - elif op.opname == "indirect_call": - return self.indirect_call_can_raise(op.args[-1].value, seen) - else: +class RaiseAnalyzer(graphanalyze.GraphAnalyzer): + def operation_is_true(self, op): + try: return bool(LL_OPERATIONS[op.opname].canraise) + except KeyError: + return True - def direct_call_can_raise(self, graph, seen=None): - if graph in self.call_can_raise: - return self.call_can_raise[graph] - if seen is None: - seen = {} - if graph in seen: - self.call_can_raise[graph] = False - return False - else: - seen[graph] = True - for block in graph.iterblocks(): - if block is graph.exceptblock: - return True # the except block is reached - for op in block.operations: - if self.can_raise(op, seen): - self.call_can_raise[graph] = True - return True - self.call_can_raise[graph] = False - return False + def analyze_exceptblock(self, block, seen=None): + return True - def indirect_call_can_raise(self, graphs, seen=None): - if graphs is None: - return True - for graph in graphs: - if self.direct_call_can_raise(graph, seen): - return True - return False + # backward compatible interface + def can_raise(self, op, seen=None): + return self.analyze(op, seen) Added: pypy/dist/pypy/translator/backendopt/graphanalyze.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/backendopt/graphanalyze.py Fri Apr 14 21:49:15 2006 @@ -0,0 +1,75 @@ +from pypy.translator.simplify import get_graph +from pypy.rpython.lltypesystem.lloperation import llop, LL_OPERATIONS +from pypy.rpython.lltypesystem import lltype + +class GraphAnalyzer(object): + """generic way to analyze graphs: recursively follow it until the first + operation is found on which self.bad_op returns True""" + def __init__(self, translator): + self.translator = translator + self.analyzed_calls = {} + + # methods to be overridden by subclass + + def operation_is_true(self, op): + raise NotImplementedError("abstract base class") + + def analyze_exceptblock(self, block, seen=None): + return False + + def analyze_startblock(self, block, seen=None): + return False + + # general methods + + def analyze(self, op, seen=None): + if op.opname == "direct_call": + graph = get_graph(op.args[0], self.translator) + if graph is None: + return True + return self.analyze_direct_call(graph, seen) + elif op.opname == "indirect_call": + if op.args[-1].value is None: + return True + return self.analyze_indirect_call(op.args[-1].value, seen) + if self.operation_is_true(op): + return True + + def analyze_direct_call(self, graph, seen=None): + if graph in self.analyzed_calls: + return self.analyzed_calls[graph] + if seen is None: + seen = {} + if graph in seen: + self.analyzed_calls[graph] = False + return False + else: + seen[graph] = True + for block in graph.iterblocks(): + if block is graph.startblock: + if self.analyze_startblock(block, seen): + self.analyzed_calls[graph] = True + return True + if block is graph.exceptblock: + if self.analyze_exceptblock(block, seen): + self.analyzed_calls[graph] = True + return True + for op in block.operations: + if self.analyze(op, seen): + self.analyzed_calls[graph] = True + return True + self.analyzed_calls[graph] = False + return False + + def analyze_indirect_call(self, graphs, seen=None): + for graph in graphs: + if self.analyze_direct_call(graph, seen): + return True + return False + + def analyze_all(self, graphs=None): + if graphs is None: + graphs = self.translator.graphs + for graph in graphs: + for block, op in graph.iterblockops(): + self.analyze(op) From cfbolz at codespeak.net Fri Apr 14 21:55:09 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 14 Apr 2006 21:55:09 +0200 (CEST) Subject: [pypy-svn] r25835 - in pypy/dist/pypy: rpython/memory translator/c/test Message-ID: <20060414195509.2EF0B10093@code0.codespeak.net> Author: cfbolz Date: Fri Apr 14 21:55:06 2006 New Revision: 25835 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/c/test/test_boehm.py pypy/dist/pypy/translator/c/test/test_newgc.py Log: only protect calls that actually can collect in the framework Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 14 21:55:06 2006 @@ -6,6 +6,7 @@ from pypy.translator.unsimplify import insert_empty_block from pypy.translator.translator import graphof from pypy.translator.backendopt.support import var_needsgc, needs_conservative_livevar_calculation +from pypy.translator.backendopt import graphanalyze from pypy.annotation import model as annmodel from pypy.rpython import rmodel, rptr, annlowlevel from pypy.rpython.memory import gc, lladdress @@ -664,6 +665,10 @@ yield a +class CollectAnalyzer(graphanalyze.GraphAnalyzer): + def operation_is_true(self, op): + return op.opname in ("malloc", "malloc_varsize") + class FrameworkGCTransformer(GCTransformer): def __init__(self, translator): @@ -822,6 +827,9 @@ self.malloc_varsize_ptr = self.graph2funcptr(malloc_varsize_graph) self.graphs_to_inline[malloc_fixedsize_graph] = True + self.collect_analyzer = CollectAnalyzer(self.translator) + self.collect_analyzer.analyze_all() + def graph2funcptr(self, graph): self.need_minimal_transform(graph) return const_funcptr_fromgraph(graph) @@ -971,8 +979,13 @@ newops.extend(self.pop_roots(livevars)) return newops, index - replace_direct_call = protect_roots - replace_indirect_call = protect_roots + def replace_direct_call(self, op, livevars, block): + if self.collect_analyzer.analyze(op): + return self.protect_roots(op, livevars, block) + else: + return [op], 0 + + replace_indirect_call = replace_direct_call def replace_malloc(self, op, livevars, block): TYPE = op.args[0].value Modified: pypy/dist/pypy/translator/c/test/test_boehm.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_boehm.py (original) +++ pypy/dist/pypy/translator/c/test/test_boehm.py Fri Apr 14 21:55:06 2006 @@ -20,6 +20,7 @@ def getcompiled(self, func): t = TranslationContext(simplifying=True) + self.t = t # builds starting-types from func_defs argstypelist = [] if func.func_defaults: Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Fri Apr 14 21:55:06 2006 @@ -4,6 +4,7 @@ from py.test import raises from pypy.translator.translator import TranslationContext +from pypy.translator.backendopt.stat import print_statistics from pypy.translator.c import genc, gc from pypy.rpython.lltypesystem import lltype @@ -193,7 +194,7 @@ from pypy.translator.c.gc import FrameworkGcPolicy as gcpolicy def test_framework_simple(self): - def g(x): + def g(x): # cannot cause a collect return x + 1 class A(object): pass @@ -208,6 +209,7 @@ fn = self.getcompiled(f) res = fn() assert res == 2 + assert len(self.t.graphs[0].startblock.exits[False].target.operations) == 10 def test_framework_varsized(self): S = lltype.GcStruct("S", ('x', lltype.Signed)) From cfbolz at codespeak.net Sat Apr 15 01:06:01 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 01:06:01 +0200 (CEST) Subject: [pypy-svn] r25836 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060414230601.F25021009C@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 01:06:00 2006 New Revision: 25836 Modified: pypy/dist/pypy/translator/backendopt/propagate.py pypy/dist/pypy/translator/backendopt/test/test_propagate.py Log: catch TypeError if comparing instances of different type Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Sat Apr 15 01:06:00 2006 @@ -280,7 +280,10 @@ def can_be_same(val1, val2): if isinstance(val1, Constant) and isinstance(val2, Constant): - return val1.value == val2.value + try: + return val1.value == val2.value + except TypeError: + return False return val1.concretetype == val2.concretetype def remove_getfield(graph, translator): @@ -290,8 +293,11 @@ def equality(op1, op2): if isinstance(op1.args[0], Constant): if isinstance(op2.args[0], Constant): - return (op1.args[0].value == op2.args[0].value and - op1.args[1].value == op2.args[1].value) + try: + return (op1.args[0].value == op2.args[0].value and + op1.args[1].value == op2.args[1].value) + except TypeError: + return False return False return (op1.args[0] == op2.args[0] and op1.args[1].value == op2.args[1].value) Modified: pypy/dist/pypy/translator/backendopt/test/test_propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_propagate.py Sat Apr 15 01:06:00 2006 @@ -289,3 +289,21 @@ assert len(graph.startblock.operations) == 20 check_graph(graph, [42], 42, t) +def test_getfield_nonmatching_types(): + class A(object): + pass + class B(object): + pass + global_a = A() + global_b = B() + def f(x): + global_a.x = x + global_b.x = 2 * x + return global_a.x + global_b.x + graph, t = get_graph(f, [int]) + count = remove_all_getfields(graph, t) + assert count == 2 + if conftest.option.view: + t.view() + check_graph(graph, [42], 3 * 42, t) + From cfbolz at codespeak.net Sat Apr 15 01:10:30 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 01:10:30 +0200 (CEST) Subject: [pypy-svn] r25837 - pypy/dist/pypy/translator/backendopt Message-ID: <20060414231030.245EE1009D@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 01:10:27 2006 New Revision: 25837 Modified: pypy/dist/pypy/translator/backendopt/propagate.py Log: don't crash if the operation is not in the table of all operations Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Sat Apr 15 01:10:27 2006 @@ -134,7 +134,10 @@ return super(CountingLLFrame, self).eval_operation(operation) def op_can_be_folded(op): - return not lloperation.LL_OPERATIONS[op.opname].canfold + try: + return not lloperation.LL_OPERATIONS[op.opname].canfold + except KeyError: + return True def constant_folding(graph, translator): """do constant folding if the arguments of an operations are constants""" @@ -145,7 +148,7 @@ for i, op in enumerate(block.operations): if sum([isinstance(arg, Variable) for arg in op.args]): continue - if lloperation.LL_OPERATIONS[op.opname].canfold: + if op.opname in lloperation.LL_OPERATIONS and lloperation.LL_OPERATIONS[op.opname].canfold: if op.opname in ("getsubstruct", "getarraysubstruct"): if not var_needsgc(op.result): continue @@ -201,7 +204,8 @@ return usedvars = {} for op in block.operations: - if not lloperation.LL_OPERATIONS[op.opname].canfold: + if (op.opname not in lloperation.LL_OPERATIONS or + not lloperation.LL_OPERATIONS[op.opname].canfold): return for arg in op.args: if (isinstance(arg, Variable) and arg in block.inputargs): @@ -239,6 +243,8 @@ else: assert 0, "this should not occur" unchanged = link.target == nextblock and link.args == newargs + if not unchanged: + print "doing partial folding in graph", graph.name link.target = nextblock link.args = newargs checkgraph(graph) @@ -361,6 +367,7 @@ translator) or changed changed = partial_folding(graph, translator) or changed changed = remove_all_getfields(graph, translator) or changed + checkgraph(graph) return changed do_atmost(10, prop) From cfbolz at codespeak.net Sat Apr 15 02:06:57 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 02:06:57 +0200 (CEST) Subject: [pypy-svn] r25838 - pypy/dist/pypy/rpython/memory Message-ID: <20060415000657.4069E1009D@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 02:06:54 2006 New Revision: 25838 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: try to safe some push/popping of stack roots: after a cast_pointer or same_as, don't push/pop both variables. After a getfield or getarrayitem don't push/pop the resulting variable if the parent is already known to be part of the root set. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Apr 15 02:06:54 2006 @@ -956,15 +956,25 @@ return newgcdependencies def protect_roots(self, op, livevars, block, index=-1): - livevars = [var for var in livevars if not var_ispyobj(var)] + livevars = dict.fromkeys( + [var for var in livevars if not var_ispyobj(var)], True) if not needs_conservative_livevar_calculation(block): if index == -1: index = block.operations.index(op) # XXX hum needed = {} - for other_op in block.operations[index:]: - for arg in other_op.args: + for before_op in block.operations[:index]: + if before_op.result not in livevars: + continue + if before_op.opname in ("cast_pointer", "same_as"): + del livevars[before_op.result] + elif before_op.opname in ("getfield", "getarrayitem"): + if (before_op.args[0] in livevars or + isinstance(before_op.args[0], Constant)): + del livevars[before_op.result] + for after_op in block.operations[index:]: + for arg in after_op.args: needed[arg] = True - needed[other_op.result] = True + needed[after_op.result] = True for exit in block.exits: for arg in exit.args: needed[arg] = True @@ -973,6 +983,8 @@ if var in needed: newlivevars.append(var) livevars = newlivevars + else: + livevars = livevars.keys() newops = list(self.push_roots(livevars)) index = len(newops) newops.append(op) From antocuni at codespeak.net Sat Apr 15 10:22:45 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 15 Apr 2006 10:22:45 +0200 (CEST) Subject: [pypy-svn] r25840 - in pypy/dist/pypy: rpython/lltypesystem translator/goal Message-ID: <20060415082245.DEF24100A7@code0.codespeak.net> Author: antocuni Date: Sat Apr 15 10:22:17 2006 New Revision: 25840 Added: pypy/dist/pypy/translator/goal/targetrlisttest.py (contents, props changed) Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py Log: Experimental "ll_getelement" interface in rlist.py. ll_getelement is intended to do a fast element lookup, with no index checking. There is a commented "ll_listindex" function that uses that interfaces, too. The new translator/goal/targetrslisttest.py does some performance test for the ll_listindex function. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Sat Apr 15 10:22:17 2006 @@ -204,6 +204,7 @@ "ll_items": ll_items, "list_builder": self.list_builder, "ITEM": ITEM, + "ll_getelement": ll_getelement # XXX: experimental }) ) @@ -278,6 +279,7 @@ "ll_items": ll_fixed_items, "list_builder": self.list_builder, "ITEM": ITEM, + "ll_getelement": ll_getelement # XXX: experimental }) self.LIST.become(ITEMARRAY) @@ -819,6 +821,24 @@ j += 1 raise ValueError # can't say 'list.index(x): x not in list' +# XXX experimental: this version uses the getelement interface +##def ll_listindex(lst, obj, eqfn): +## lng = lst.ll_length() +## j = 0 +## while j < lng: +## if eqfn is None: +## if lst.ll_getelement(j) == obj: +## return j +## else: +## if eqfn(lst.ll_getelement(j), obj): +## return j +## j += 1 +## raise ValueError # can't say 'list.index(x): x not in list' + +# XXX experimental +def ll_getelement(lst, index): + return lst.ll_items()[index] + TEMP = GcArray(Ptr(rstr.STR)) def ll_inplace_mul(l, factor): Added: pypy/dist/pypy/translator/goal/targetrlisttest.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetrlisttest.py Sat Apr 15 10:22:17 2006 @@ -0,0 +1,23 @@ +from pypy.rpython.objectmodel import r_dict +import os, sys +import operator +import time + +# __________ Entry point __________ + +N_ITEMS = 100000000 + +def entry_point(argv): + lst = range(N_ITEMS) + lst[0] = 1 + start = time.time() + lst.index(N_ITEMS-1) + end = time.time() + os.write(1, 'Time elapsed: %s\n' % (end-start)) + return 0 + +# _____ Define and setup target ___ + +def target(*args): + return entry_point, None + From arigo at codespeak.net Sat Apr 15 11:05:04 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 11:05:04 +0200 (CEST) Subject: [pypy-svn] r25845 - in pypy/dist/pypy/translator: . goal Message-ID: <20060415090504.0CBD4100A3@code0.codespeak.net> Author: arigo Date: Sat Apr 15 11:05:02 2006 New Revision: 25845 Modified: pypy/dist/pypy/translator/driver.py pypy/dist/pypy/translator/goal/ (props changed) pypy/dist/pypy/translator/goal/targetpypystandalone.py pypy/dist/pypy/translator/goal/translate.py Log: Convince translate.py to produce executables that are not all called pypy-c or pypy-llvm no matter what the target is. Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Sat Apr 15 11:05:02 2006 @@ -41,7 +41,8 @@ class TranslationDriver(SimpleTaskEngine): - def __init__(self, options=None, default_goal=None, disable=[]): + def __init__(self, options=None, default_goal=None, disable=[], + exe_name = 'target-%(backend)s'): SimpleTaskEngine.__init__(self) self.log = log @@ -49,7 +50,8 @@ if options is None: options = DEFAULT_OPTIONS self.options = options - + self.exe_name = exe_name + self.done = {} self.disable(disable) @@ -261,7 +263,10 @@ def create_exe(self): import shutil exename = mkexename(self.c_entryp) - newexename = mkexename('./pypy-%s' % self.options.backend) + newexename = self.exe_name % self.options.__dict__ + if '/' not in newexename and '\\' not in newexename: + newexename = './' + newexename + newexename = mkexename(newexename) shutil.copy(exename, newexename) self.c_entryp = newexename self.log.info("created: %s" % (self.c_entryp,)) @@ -335,7 +340,8 @@ def task_compile_llvm(self): gen = self.llvmgen if self.standalone: - self.c_entryp = gen.compile_llvm_source(exe_name='pypy-llvm') + exe_name = self.exe_name % self.options.__dict__ + self.c_entryp = gen.compile_llvm_source(exe_name=exe_name) self.create_exe() else: self.c_entryp = gen.compile_llvm_source(return_fn=True) @@ -372,7 +378,9 @@ options = DEFAULT_OPTIONS driver = TranslationDriver(options, default_goal, disable) - + if '__name__' in targetspec_dic: + driver.exe_name = targetspec_dic['__name__'] + '-%(backend)s' + target = targetspec_dic['target'] spec = target(driver, args) Modified: pypy/dist/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/dist/pypy/translator/goal/targetpypystandalone.py Sat Apr 15 11:05:02 2006 @@ -63,6 +63,7 @@ def target(driver, args): + driver.exe_name = 'pypy-%(backend)s' options = driver.options tgt_options, _ = opt_parser().parse_args(args) Modified: pypy/dist/pypy/translator/goal/translate.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate.py (original) +++ pypy/dist/pypy/translator/goal/translate.py Sat Apr 15 11:05:02 2006 @@ -154,8 +154,9 @@ if not targetspec.endswith('.py'): targetspec += '.py' thismod = sys.modules[__name__] - targetspec_dic = {'__name__':'__rpythonmain__', - 'translate': thismod} + targetspec_dic = { + '__name__': os.path.splitext(os.path.basename(targetspec))[0], + 'translate': thismod} sys.path.insert(0, os.path.dirname(targetspec)) execfile(targetspec, targetspec_dic) return targetspec_dic From arigo at codespeak.net Sat Apr 15 11:49:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 11:49:48 +0200 (CEST) Subject: [pypy-svn] r25846 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060415094948.AC436100A3@code0.codespeak.net> Author: arigo Date: Sat Apr 15 11:49:47 2006 New Revision: 25846 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: Some more lines in there. Doesn't run, no tests, bad me. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sat Apr 15 11:49:47 2006 @@ -1,3 +1,4 @@ +from ctypes import c_char_p, POINTER, byref import ctypes_socket as _c @@ -36,6 +37,46 @@ caddr = sockaddr_in() caddr.sin_family = AF_INET caddr.sin_port = port - caddr.sin_addr.s_addr = getaddrinfo(host) + caddr.sin_addr.s_addr = XXX(host) else: XXX + + +def makesockaddr(caddr, caddrlen, proto): + if addrlen == 0: + # No address -- may be recvfrom() from known socket + return None + if caddr.sa_family == AF_INET: + return makeipaddr(caddr), _c.ntohs( + else: + XXX + +def getaddrinfo(host, port, family=AF_UNSPEC, socktype=0, proto=0, flags=0): + if isinstance(port, (int, long)): + port = str(port) + hptr = c_char_p(host) # string or None + pptr = c_char_p(port) # int or string or None + hints = _c.addrinfo() + hints.ai_family = family + hints.ai_socktype = socktype + hints.ai_protocol = proto + hints.ai_flags = flags + res0 = POINTER(_c.addrinfo)() + error = _c.getaddrinfo(hptr, pptr, byref(hints), byref(res0)) + if error: + XXX + try: + return _getaddrinfo_chainedlist(res0) + finally: + if res0: + _c.freeaddrinfo(res0) + +def _getaddrinfo_chainedlist(res): + result = [] + while res: + res = res.contents + addr = makesockaddr(res.ai_addr, res.ai_addrlen, proto) + result.append((res.ai_family, res.ai_socktype, res.ai_protocol, + res.ai_canonname or "", addr)) + res = res.ai_next + return result Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sat Apr 15 11:49:47 2006 @@ -8,6 +8,7 @@ # constants for name in ['AF_INET', + 'AF_UNSPEC', 'SOCK_STREAM', 'SOCK_DGRAM', ]: From arigo at codespeak.net Sat Apr 15 11:50:21 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 11:50:21 +0200 (CEST) Subject: [pypy-svn] r25847 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060415095021.A951D100A9@code0.codespeak.net> Author: arigo Date: Sat Apr 15 11:50:20 2006 New Revision: 25847 Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: Support for in-function POINTER() calls -- with a constant argument only, as usual. Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Sat Apr 15 11:50:20 2006 @@ -146,3 +146,19 @@ extregistry.register_value(byref, compute_result_annotation=pointerfn_compute_annotation, specialize_call=pointertype_specialize_call) + +# constant-fold POINTER(CONSTANT_CTYPE) calls +def POINTER_compute_annotation(s_arg): + from pypy.annotation.bookkeeper import getbookkeeper + assert s_arg.is_constant(), "POINTER(%r): argument must be constant" % ( + s_arg,) + RESTYPE = POINTER(s_arg.const) + return getbookkeeper().immutablevalue(RESTYPE) + +def POINTER_specialize_call(hop): + assert hop.s_result.is_constant() + return hop.inputconst(lltype.Void, hop.s_result.const) + +extregistry.register_value(POINTER, + compute_result_annotation=POINTER_compute_annotation, + specialize_call=POINTER_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Sat Apr 15 11:50:20 2006 @@ -114,6 +114,20 @@ if conftest.option.view: t.view() + def test_annotate_POINTER(self): + def fn(): + p = POINTER(c_float)() + p.contents = c_float(6.1) + return p + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(fn, []) + assert s.knowntype == POINTER(c_float) + + if conftest.option.view: + t.view() + class Test_specialization: def test_specialize_c_int_ptr(self): @@ -227,3 +241,13 @@ res = interpret(fn, []) assert res == 12 + + def test_specialize_POINTER(self): + def fn(): + p = POINTER(c_float)() + p.contents = c_float(6.25) + return p + + res = interpret(fn, []) + float_c_data = res.c_data[0] + assert float_c_data[0] == 6.25 From arigo at codespeak.net Sat Apr 15 12:23:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 12:23:59 +0200 (CEST) Subject: [pypy-svn] r25848 - in pypy/dist/pypy: annotation rpython rpython/rctypes rpython/rctypes/test rpython/test Message-ID: <20060415102359.CBD50100A7@code0.codespeak.net> Author: arigo Date: Sat Apr 15 12:23:56 2006 New Revision: 25848 Added: pypy/dist/pypy/rpython/rctypes/rvoid_p.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py (contents, props changed) Modified: pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/extregistry.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/rpython/test/test_extregistry.py Log: * annotating c_void_p and cast(). * 'ctypes.cast' has no __name__, fix a few places to default to None... * 'ctypes.cast' is not even hashable! Not easy to put it into the extregistry. Added a FlexibleWeakDict that accepts more or less any kind of keys... Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Sat Apr 15 12:23:56 2006 @@ -508,7 +508,8 @@ if bltn.s_self is not None: return bltn.analyser(bltn.s_self, *args) else: - getbookkeeper().count(bltn.methodname.replace('.', '_'), *args) + if bltn.methodname: + getbookkeeper().count(bltn.methodname.replace('.', '_'), *args) return bltn.analyser(*args) def call(bltn, args, implicit_init=False): Modified: pypy/dist/pypy/rpython/extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/extregistry.py (original) +++ pypy/dist/pypy/rpython/extregistry.py Sat Apr 15 12:23:56 2006 @@ -1,4 +1,7 @@ import weakref +import UserDict +from pypy.tool.uid import Hashable + class ExtRegistryFunc(object): def __init__(self, compute_result_annotation, specialize_call): @@ -9,7 +12,7 @@ assert func is not None from pypy.annotation import model as annmodel return annmodel.SomeBuiltin(self.compute_result_annotation, - methodname=func.__name__) + methodname=getattr(func, '__name__', None)) class ExtRegistryInstance(object): def __init__(self, compute_annotation, specialize_call, get_repr): @@ -19,10 +22,49 @@ def get_annotation(self, type, instance=None): return self.compute_annotation(type, instance) - -EXT_REGISTRY_BY_VALUE = weakref.WeakKeyDictionary() -EXT_REGISTRY_BY_VALUE_NONWEAK = {} # for things that cannot be weakly ref'ed - # like built-in functions + +# ____________________________________________________________ + +class FlexibleWeakDict(UserDict.DictMixin): + """A WeakKeyDictionary that accepts more or less anything as keys: + weakly referenceable objects or not, hashable objects or not. + """ + def __init__(self): + self._regdict = {} + self._weakdict = weakref.WeakKeyDictionary() + self._iddict = {} + + def ref(self, key): + try: + hash(key) + except TypeError: + return self._iddict, Hashable(key) # key is not hashable + try: + weakref.ref(key) + except TypeError: + return self._regdict, key # key cannot be weakly ref'ed + else: + return self._weakdict, key # normal case + + def __getitem__(self, key): + d, key = self.ref(key) + return d[key] + + def __setitem__(self, key, value): + d, key = self.ref(key) + d[key] = value + + def __delitem__(self, key): + d, key = self.ref(key) + del d[key] + + def keys(self): + return (self._regdict.keys() + + self._weakdict.keys() + + [hashable.value for hashable in self._iddict]) + + +EXT_REGISTRY_BY_VALUE = FlexibleWeakDict() EXT_REGISTRY_BY_TYPE = weakref.WeakKeyDictionary() EXT_REGISTRY_BY_METATYPE = weakref.WeakKeyDictionary() @@ -48,15 +90,9 @@ get_repr) def register_value(value, **kwargs): - try: - weakref.ref(value) - except TypeError: - REG = EXT_REGISTRY_BY_VALUE_NONWEAK - else: - REG = EXT_REGISTRY_BY_VALUE - assert value not in REG - REG[value] = create_entry(**kwargs) - return REG[value] + assert value not in EXT_REGISTRY_BY_VALUE + EXT_REGISTRY_BY_VALUE[value] = create_entry(**kwargs) + return EXT_REGISTRY_BY_VALUE[value] def register_type(t, **kwargs): assert t not in EXT_REGISTRY_BY_TYPE @@ -83,13 +119,7 @@ def lookup(instance): try: - weakref.ref(instance) - except TypeError: - REG = EXT_REGISTRY_BY_VALUE_NONWEAK - else: - REG = EXT_REGISTRY_BY_VALUE - try: - return REG[instance] + return EXT_REGISTRY_BY_VALUE[instance] except (KeyError, TypeError): return lookup_type(type(instance)) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Sat Apr 15 12:23:56 2006 @@ -5,6 +5,7 @@ import pypy.rpython.rctypes.rfunc import pypy.rpython.rctypes.rchar_p import pypy.rpython.rctypes.rstruct +import pypy.rpython.rctypes.rvoid_p # Register the correspondance between SomeCTypesObject and the get_repr() Modified: pypy/dist/pypy/rpython/rctypes/rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/rfunc.py Sat Apr 15 12:23:56 2006 @@ -22,7 +22,7 @@ return s_result.return_annotation() return annmodel.SomeBuiltin(compute_result_annotation, - methodname=instance.__name__) + methodname=getattr(instance, '__name__', None)) def cfuncptrtype_specialize_call(hop): # this is necessary to get the original function pointer when specializing Added: pypy/dist/pypy/rpython/rctypes/rvoid_p.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rvoid_p.py Sat Apr 15 12:23:56 2006 @@ -0,0 +1,43 @@ +from pypy.rpython import extregistry +from pypy.rpython.lltypesystem import llmemory +from pypy.annotation import model as annmodel + +from ctypes import c_void_p, c_int, POINTER, cast + +PointerType = type(POINTER(c_int)) + + +# c_void_p() as a function +def c_void_p_compute_result_annotation(s_arg=None): + raise NotImplementedError("XXX calling c_void_p()") + +extregistry.register_value(c_void_p, + compute_result_annotation=c_void_p_compute_result_annotation, + ) + +# c_void_p instances +def c_void_compute_annotation(the_type, instance): + return annmodel.SomeCTypesObject(c_void_p, + annmodel.SomeCTypesObject.OWNSMEMORY) + +def c_void_p_get_repr(rtyper, s_void_p): + return CVoidPRepr(rtyper, s_void_p, llmemory.Address) + +extregistry.register_type(c_void_p, + compute_annotation = c_void_compute_annotation, + get_repr = c_void_p_get_repr, + ) + +# cast() support +def cast_compute_result_annotation(s_arg, s_type): + assert s_type.is_constant(), "cast(p, %r): argument 2 must be constant" % ( + s_type,) + type = s_type.const + assert isinstance(type, PointerType) or type == c_void_p, ( + "cast(p, %r): XXX can only cast between pointer types so far" % (type,)) + return annmodel.SomeCTypesObject(type, + annmodel.SomeCTypesObject.OWNSMEMORY) + +extregistry.register_value(cast, + compute_result_annotation=cast_compute_result_annotation, + ) Modified: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c (original) +++ pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Sat Apr 15 12:23:56 2006 @@ -51,6 +51,14 @@ return pin; } +EXPORT(void *) _testfunc_erase_type(void) +{ + static point p; + p.x = 'x'; + p.y = 'y'; + return (void *)&p; +} + DL_EXPORT(void) init_rctypes_test(void) { Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Sat Apr 15 12:23:56 2006 @@ -83,3 +83,9 @@ assert s.p.contents.value == 123 s.p.contents.value = 124 assert y.value == 124 + +def test_void_p(): + x = c_int(12) + p1 = cast(pointer(x), c_void_p) + p2 = cast(p1, POINTER(c_int)) + assert p2.contents.value == 12 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Sat Apr 15 12:23:56 2006 @@ -22,7 +22,7 @@ from ctypes import cdll -from ctypes import POINTER, Structure, c_int, byref, pointer +from ctypes import POINTER, Structure, c_int, byref, pointer, c_void_p # __________ compile and load our local test C file __________ @@ -101,6 +101,11 @@ testfunc_swap.llinterp_friendly_version = ll_testfunc_swap testfunc_swap.includes = includes +# _testfunc_erase_type +testfunc_erase_type = _rctypes_test._testfunc_erase_type +testfunc_erase_type.restype = c_void_p +testfunc_erase_type.argtypes = [] + def test_testfunc_struct(): in_point = tagpoint() Added: pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Sat Apr 15 12:23:56 2006 @@ -0,0 +1,37 @@ +""" +Test the rctypes implementation. +""" + +import py.test +import pypy.rpython.rctypes.implementation +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy.translator.c.test.test_genc import compile +from pypy import conftest +import sys +from pypy.rpython.test.test_llinterp import interpret + +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + +from ctypes import c_void_p, c_int, cast, pointer, POINTER + +class Test_annotation: + def test_annotate_c_char_p(self): + def fn(): + x = c_int(12) + p1 = cast(pointer(x), c_void_p) + p2 = cast(p1, POINTER(c_int)) + assert p2.contents.value == 12 + return p1, p2 + + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(fn, []) + assert s.items[0].knowntype == c_void_p + assert s.items[1].knowntype == POINTER(c_int) + + if conftest.option.view: + t.view() Modified: pypy/dist/pypy/rpython/test/test_extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_extregistry.py (original) +++ pypy/dist/pypy/rpython/test/test_extregistry.py Sat Apr 15 12:23:56 2006 @@ -163,3 +163,16 @@ assert res == 42 +def test_register_unhashable(): + lst1 = [5, 6] + lst2 = [5, 6] + entry = extregistry.register_value(lst1) + assert extregistry.lookup(lst1) is entry + py.test.raises(KeyError, "extregistry.lookup(lst2)") + +def test_register_non_weakly_refable(n=6): + tup1 = (5, 6) + tup2 = (5, n) + entry = extregistry.register_value(tup1) + assert extregistry.lookup(tup1) is entry + assert extregistry.lookup(tup2) is entry From antocuni at codespeak.net Sat Apr 15 12:27:28 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 15 Apr 2006 12:27:28 +0200 (CEST) Subject: [pypy-svn] r25849 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060415102728.4845A100A2@code0.codespeak.net> Author: antocuni Date: Sat Apr 15 12:27:17 2006 New Revision: 25849 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py Log: The interface of ListRepr and FixedSizeListRepr has changed: two accessor methods has been added: ll_getitem_fast and ll_setitem_fast. They should be used instead of the ll_items()[index] idiom: that way when ootypesystem's list will support that interface we will able to write function useable with both typesystem with no modification. The various ll_* helper function has been adapted to use the new interface. Moreover function that accessed directly to the "l.length" field has been changed to call the "ll_length()" method instead, for the same reasons as above. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Sat Apr 15 12:27:17 2006 @@ -204,7 +204,8 @@ "ll_items": ll_items, "list_builder": self.list_builder, "ITEM": ITEM, - "ll_getelement": ll_getelement # XXX: experimental + "ll_getitem_fast": ll_getitem_fast, + "ll_setitem_fast": ll_setitem_fast, }) ) @@ -279,7 +280,8 @@ "ll_items": ll_fixed_items, "list_builder": self.list_builder, "ITEM": ITEM, - "ll_getelement": ll_getelement # XXX: experimental + "ll_getitem_fast": ll_fixed_getitem_fast, + "ll_setitem_fast": ll_fixed_setitem_fast }) self.LIST.become(ITEMARRAY) @@ -467,13 +469,11 @@ def ll_copy(RESLIST, l): - items = l.ll_items() length = l.ll_length() new_lst = RESLIST.ll_newlist(length) i = 0 - new_items = new_lst.ll_items() while i < length: - new_items[i] = items[i] + new_lst.ll_setitem_fast(i, l.ll_getitem_fast(i)) i += 1 return new_lst ll_copy.oopspec = 'list.copy(l)' @@ -488,86 +488,82 @@ ll_list_is_true.oopspec = 'list.nonzero(l)' def ll_append(l, newitem): - length = l.length + length = l.ll_length() _ll_list_resize_ge(l, length+1) - l.items[length] = newitem + l.ll_setitem_fast(length, newitem) ll_append.oopspec = 'list.append(l, newitem)' # this one is for the special case of insert(0, x) def ll_prepend(l, newitem): - length = l.length + length = l.ll_length() _ll_list_resize_ge(l, length+1) - items = l.items dst = length while dst > 0: src = dst - 1 - items[dst] = items[src] + l.ll_setitem_fast(dst, l.ll_getitem_fast(src)) dst = src - items[0] = newitem + l.ll_setitem_fast(0, newitem) ll_prepend.oopspec = 'list.insert(l, 0, newitem)' def ll_insert_nonneg(l, index, newitem): - length = l.length + length = l.ll_length() _ll_list_resize_ge(l, length+1) - items = l.items dst = length while dst > index: src = dst - 1 - items[dst] = items[src] + l.ll_setitem_fast(dst, l.ll_getitem_fast(src)) dst = src - items[index] = newitem + l.ll_setitem_fast(index, newitem) ll_insert_nonneg.oopspec = 'list.insert(l, index, newitem)' def ll_pop_nonneg(func, l, index): - if func is dum_checkidx and (index >= l.length): + if func is dum_checkidx and (index >= l.ll_length()): raise IndexError - res = l.items[index] + res = l.ll_getitem_fast(index) ll_delitem_nonneg(dum_nocheck, l, index) return res ll_pop_nonneg.oopspec = 'list.pop(l, index)' def ll_pop_default(func, l): - length = l.length + length = l.ll_length() if func is dum_checkidx and (length == 0): raise IndexError index = length - 1 newlength = index - items = l.items - res = items[index] + res = l.ll_getitem_fast(index) ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): - items[index] = nullptr(ITEM.TO) + l.ll_setitem_fast(index, nullptr(ITEM.TO)) _ll_list_resize_le(l, newlength) return res ll_pop_default.oopspec = 'list.pop(l)' def ll_pop_zero(func, l): - length = l.length + length = l.ll_length() if func is dum_checkidx and (length == 0): raise IndexError newlength = length - 1 - res = l.items[0] + res = l.ll_getitem_fast(0) j = 0 - items = l.items j1 = j+1 while j < newlength: - items[j] = items[j1] + l.ll_setitem_fast(j, l.ll_getitem_fast(j1)) j = j1 j1 += 1 ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): - items[newlength] = nullptr(ITEM.TO) + l.ll_setitem_fast(newlength, nullptr(ITEM.TO)) _ll_list_resize_le(l, newlength) return res ll_pop_zero.oopspec = 'list.pop(l, 0)' def ll_pop(func, l, index): - length = l.length + length = l.ll_length() if index < 0: index += length if func is dum_checkidx and (index < 0 or index >= length): raise IndexError - res = l.items[index] + res = l.ll_getitem_fast(index) ll_delitem_nonneg(dum_nocheck, l, index) return res ll_pop.oopspec = 'list.pop(l, index)' @@ -575,12 +571,11 @@ def ll_reverse(l): length = l.ll_length() i = 0 - items = l.ll_items() length_1_i = length-1-i while i < length_1_i: - tmp = items[i] - items[i] = items[length_1_i] - items[length_1_i] = tmp + tmp = l.ll_getitem_fast(i) + l.ll_setitem_fast(i, l.ll_getitem_fast(length_1_i)) + l.ll_setitem_fast(length_1_i, tmp) i += 1 length_1_i -= 1 ll_reverse.oopspec = 'list.reverse(l)' @@ -588,7 +583,7 @@ def ll_getitem_nonneg(func, l, index): if func is dum_checkidx and (index >= l.ll_length()): raise IndexError - return l.ll_items()[index] + return l.ll_getitem_fast(index) ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' def ll_getitem(func, l, index): @@ -597,13 +592,13 @@ index += length if func is dum_checkidx and (index < 0 or index >= length): raise IndexError - return l.ll_items()[index] + return l.ll_getitem_fast(index) ll_getitem.oopspec = 'list.getitem(l, index)' def ll_setitem_nonneg(func, l, index, newitem): if func is dum_checkidx and (index >= l.ll_length()): raise IndexError - l.ll_items()[index] = newitem + l.ll_setitem_fast(index, newitem) ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' def ll_setitem(func, l, index, newitem): @@ -612,29 +607,28 @@ index += length if func is dum_checkidx and (index < 0 or index >= length): raise IndexError - l.ll_items()[index] = newitem + l.ll_setitem_fast(index, newitem) ll_setitem.oopspec = 'list.setitem(l, index, newitem)' def ll_delitem_nonneg(func, l, index): - length = l.length + length = l.ll_length() if func is dum_checkidx and (index >= length): raise IndexError newlength = length - 1 j = index - items = l.items j1 = j+1 while j < newlength: - items[j] = items[j1] + l.ll_setitem_fast(j, l.ll_getitem_fast(j1)) j = j1 j1 += 1 ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): - items[newlength] = nullptr(ITEM.TO) + l.ll_setitem_fast(newlength, nullptr(ITEM.TO)) _ll_list_resize_le(l, newlength) ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' def ll_delitem(func, l, i): - length = l.length + length = l.ll_length() if i < 0: i += length if func is dum_checkidx and (i < 0 or i >= length): @@ -647,32 +641,27 @@ len2 = l2.ll_length() newlength = len1 + len2 l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() j = 0 - source = l1.ll_items() while j < len1: - newitems[j] = source[j] + l.ll_setitem_fast(j, l1.ll_getitem_fast(j)) j += 1 i = 0 - source = l2.ll_items() while i < len2: - newitems[j] = source[i] + l.ll_setitem_fast(j, l2.ll_getitem_fast(i)) i += 1 j += 1 return l ll_concat.oopspec = 'list.concat(l1, l2)' def ll_extend(l1, l2): - len1 = l1.length + len1 = l1.ll_length() len2 = l2.ll_length() newlength = len1 + len2 _ll_list_resize_ge(l1, newlength) - items = l1.items - source = l2.ll_items() i = 0 j = len1 while i < len2: - items[j] = source[i] + l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) i += 1 j += 1 @@ -680,12 +669,10 @@ len1 = l1.ll_length() newlength = len1 - start l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() j = 0 - source = l1.ll_items() i = start while i < len1: - newitems[j] = source[i] + l.ll_setitem_fast(j, l1.ll_getitem_fast(i)) i += 1 j += 1 return l @@ -698,12 +685,10 @@ stop = length newlength = stop - start l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() j = 0 - source = l1.ll_items() i = start while i < stop: - newitems[j] = source[i] + l.ll_setitem_fast(j, l1.ll_getitem_fast(i)) i += 1 j += 1 return l @@ -712,11 +697,9 @@ newlength = l1.ll_length() - 1 assert newlength >= 0 l = RESLIST.ll_newlist(newlength) - newitems = l.ll_items() j = 0 - source = l1.ll_items() while j < newlength: - newitems[j] = source[j] + l.ll_setitem_fast(j, l1.ll_getitem_fast(j)) j += 1 return l @@ -724,31 +707,30 @@ newlength = start ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): - j = l.length - 1 - items = l.items + j = l.ll_length() - 1 while j >= newlength: - items[j] = nullptr(ITEM.TO) + l.ll_setitem_fast(j, nullptr(ITEM.TO)) j -= 1 _ll_list_resize_le(l, newlength) def ll_listdelslice(l, slice): start = slice.start stop = slice.stop - if stop > l.length: - stop = l.length - newlength = l.length - (stop-start) + length = l.ll_length() + if stop > length: + stop = length + newlength = length - (stop-start) j = start - items = l.items i = stop while j < newlength: - items[j] = items[i] + l.ll_setitem_fast(j, l.ll_getitem_fast(i)) i += 1 j += 1 ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): - j = l.length - 1 + j = length - 1 while j >= newlength: - items[j] = nullptr(ITEM.TO) + l.ll_setitem_fast(j, nullptr(ITEM.TO)) j -= 1 _ll_list_resize_le(l, newlength) @@ -759,11 +741,9 @@ # XXX but it should be easy enough to support, soon start = slice.start j = start - items1 = l1.ll_items() - items2 = l2.ll_items() i = 0 while i < count: - items1[j] = items2[i] + l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) i += 1 j += 1 @@ -781,64 +761,42 @@ if len1 != len2: return False j = 0 - items1 = l1.ll_items() - items2 = l2.ll_items() while j < len1: if eqfn is None: - if items1[j] != items2[j]: + if l1.ll_getitem_fast(j) != l2.ll_getitem_fast(j): return False else: - if not eqfn(items1[j], items2[j]): + if not eqfn(l1.ll_getitem_fast(j), l2.ll_getitem_fast(j)): return False j += 1 return True def ll_listcontains(lst, obj, eqfn): - items = lst.ll_items() lng = lst.ll_length() j = 0 while j < lng: if eqfn is None: - if items[j] == obj: + if lst.ll_getitem_fast(j) == obj: return True else: - if eqfn(items[j], obj): + if eqfn(lst.ll_getitem_fast(j), obj): return True j += 1 return False def ll_listindex(lst, obj, eqfn): - items = lst.ll_items() lng = lst.ll_length() j = 0 while j < lng: if eqfn is None: - if items[j] == obj: + if lst.ll_getitem_fast(j) == obj: return j else: - if eqfn(items[j], obj): + if eqfn(lst.ll_getitem_fast(j), obj): return j j += 1 raise ValueError # can't say 'list.index(x): x not in list' -# XXX experimental: this version uses the getelement interface -##def ll_listindex(lst, obj, eqfn): -## lng = lst.ll_length() -## j = 0 -## while j < lng: -## if eqfn is None: -## if lst.ll_getelement(j) == obj: -## return j -## else: -## if eqfn(lst.ll_getelement(j), obj): -## return j -## j += 1 -## raise ValueError # can't say 'list.index(x): x not in list' - -# XXX experimental -def ll_getelement(lst, index): - return lst.ll_items()[index] - TEMP = GcArray(Ptr(rstr.STR)) def ll_inplace_mul(l, factor): @@ -849,13 +807,11 @@ res = l _ll_list_resize(res, resultlen) j = length - source = l.ll_items() - target = res.ll_items() while j < resultlen: i = 0 while i < length: p = j + i - target[p] = source[i] + res.ll_setitem_fast(p, l.ll_getitem_fast(i)) i += 1 j += length return res @@ -868,13 +824,11 @@ resultlen = length * factor res = RESLIST.ll_newlist(resultlen) j = 0 - source = l.ll_items() - target = res.ll_items() while j < resultlen: i = 0 while i < length: p = j + i - target[p] = source[i] + res.ll_setitem_fast(p, l.ll_getitem_fast(i)) i += 1 j += length return res @@ -882,7 +836,7 @@ # ____________________________________________________________ # -# Irregular operations. +# Accessor methods def ll_newlist(LIST, length): l = malloc(LIST) @@ -898,6 +852,12 @@ def ll_items(l): return l.items +def ll_getitem_fast(l, index): + return l.ll_items()[index] + +def ll_setitem_fast(l, index, item): + l.ll_items()[index] = item + # fixed size versions def ll_fixed_newlist(LIST, length): @@ -912,6 +872,12 @@ def ll_fixed_items(l): return l +def ll_fixed_getitem_fast(l, index): + return l[index] + +def ll_fixed_setitem_fast(l, index, item): + l[index] = item + def newlist(llops, r_list, items_v): LIST = r_list.LIST cno = inputconst(Signed, len(items_v)) @@ -967,4 +933,4 @@ if index >= l.ll_length(): raise StopIteration iter.index = index + 1 - return l.ll_items()[index] + return l.ll_getitem_fast(index) From arigo at codespeak.net Sat Apr 15 12:42:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 12:42:16 +0200 (CEST) Subject: [pypy-svn] r25850 - in pypy/dist/pypy/rpython: lltypesystem rctypes rctypes/test Message-ID: <20060415104216.25B66100A6@code0.codespeak.net> Author: arigo Date: Sat Apr 15 12:42:14 2006 New Revision: 25850 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/rctypes/rvoid_p.py pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Log: Specialize and compile cast() between pointers. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Sat Apr 15 12:42:14 2006 @@ -232,7 +232,6 @@ ref.fieldname) else: # regular case - assert isinstance(ref.type(), lltype.ContainerType) return lltype.cast_pointer(EXPECTED_TYPE, ref.get()) def _cast_to_int(self): Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rvoid_p.py Sat Apr 15 12:42:14 2006 @@ -1,12 +1,18 @@ from pypy.rpython import extregistry -from pypy.rpython.lltypesystem import llmemory +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.annotation import model as annmodel +from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO +from pypy.rpython.rctypes.rpointer import PointerRepr from ctypes import c_void_p, c_int, POINTER, cast PointerType = type(POINTER(c_int)) +class CVoidPRepr(CTypesValueRepr): + pass # No operations supported on c_void_p instances so far + + # c_void_p() as a function def c_void_p_compute_result_annotation(s_arg=None): raise NotImplementedError("XXX calling c_void_p()") @@ -33,11 +39,34 @@ assert s_type.is_constant(), "cast(p, %r): argument 2 must be constant" % ( s_type,) type = s_type.const - assert isinstance(type, PointerType) or type == c_void_p, ( - "cast(p, %r): XXX can only cast between pointer types so far" % (type,)) + + def checkptr(ctype): + assert isinstance(ctype, PointerType) or ctype == c_void_p, ( + "cast(): can only cast between pointers so far, not %r" % (ctype,)) + checkptr(s_arg.knowntype) + checkptr(type) return annmodel.SomeCTypesObject(type, annmodel.SomeCTypesObject.OWNSMEMORY) +def cast_specialize_call(hop): + assert isinstance(hop.args_r[0], (PointerRepr, CVoidPRepr)) + targetctype = hop.args_s[1].const + v_box, c_targetctype = hop.inputargs(hop.args_r[0], lltype.Void) + v_adr = hop.args_r[0].getvalue(hop.llops, v_box) + if v_adr.concretetype != llmemory.Address: + v_adr = hop.genop('cast_ptr_to_adr', [v_adr], + resulttype = llmemory.Address) + + if targetctype == c_void_p: + # cast to void + v_result = v_adr + else: + # cast to pointer + v_result = hop.genop('cast_adr_to_ptr', [v_adr], + resulttype = hop.r_result.ll_type) + return hop.r_result.return_value(hop.llops, v_result) + extregistry.register_value(cast, compute_result_annotation=cast_compute_result_annotation, + specialize_call=cast_specialize_call, ) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Sat Apr 15 12:42:14 2006 @@ -8,7 +8,7 @@ from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile from pypy import conftest -import sys +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.test.test_llinterp import interpret try: @@ -19,7 +19,7 @@ from ctypes import c_void_p, c_int, cast, pointer, POINTER class Test_annotation: - def test_annotate_c_char_p(self): + def test_annotate_c_void_p(self): def fn(): x = c_int(12) p1 = cast(pointer(x), c_void_p) @@ -35,3 +35,26 @@ if conftest.option.view: t.view() + +class Test_specialization: + def test_specialize_c_void_p(self): + def func(): + x = c_int(12) + p1 = cast(pointer(x), c_void_p) + p2 = cast(p1, POINTER(c_int)) + return p1, p2.contents.value + + res = interpret(func, []) + assert lltype.typeOf(res.item0.c_data[0]) == llmemory.Address + assert res.item1 == 12 + +class Test_compilation: + def test_compile_c_char_p(self): + def func(): + x = c_int(12) + p1 = cast(pointer(x), c_void_p) + p2 = cast(p1, POINTER(c_int)) + return p2.contents.value + + fn = compile(func, []) + assert fn() == 12 From cfbolz at codespeak.net Sat Apr 15 13:01:38 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 13:01:38 +0200 (CEST) Subject: [pypy-svn] r25851 - pypy/dist/pypy/translator/c Message-ID: <20060415110138.4360A100A6@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 13:01:36 2006 New Revision: 25851 Modified: pypy/dist/pypy/translator/c/gc.py Log: for now never reload the protected object since mark and sweep never moves anyway and since this makes it a few percent faster. Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Sat Apr 15 13:01:36 2006 @@ -286,7 +286,8 @@ def OP_GC_RELOAD_POSSIBLY_MOVED(self, funcgen, op): args = [funcgen.expr(v) for v in op.args] - return '%s = %s; /* for moving GCs */' % (args[1], args[0]) + # XXX this more or less assumes mark-and-sweep gc + return '/* %s = %s; */ /* for moving GCs */' % (args[1], args[0]) def common_gcheader_definition(self, defnode): # XXX assumes mark and sweep From arigo at codespeak.net Sat Apr 15 13:02:15 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 13:02:15 +0200 (CEST) Subject: [pypy-svn] r25852 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060415110215.DF0CF100AC@code0.codespeak.net> Author: arigo Date: Sat Apr 15 13:02:14 2006 New Revision: 25852 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test_addr.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: Very incomplete implementation of getaddrinfo(), with a test (only works on on-line machines so far). The idea is that rctypes should now support all ctypes constructions that were necessary. I will start a regular mixed-module _socket based on this, but first we need to figure out how to best test mixed-modules based on ctypes -- ideally, they should be testable and compilable without the rest of the PyPy interpreter... Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sat Apr 15 13:02:14 2006 @@ -1,4 +1,4 @@ -from ctypes import c_char_p, POINTER, byref +from ctypes import c_char_p, POINTER, byref, cast, create_string_buffer import ctypes_socket as _c @@ -42,12 +42,21 @@ XXX +def makeipaddr(caddr, caddrlen): + buf = create_string_buffer(NI_MAXHOST) + error = _c.getnameinfo(caddr, caddrlen, buf, NI_MAXHOST, + c_char_p(), 0, NI_NUMERICHOST) + if error: + XXX + return buf.value + def makesockaddr(caddr, caddrlen, proto): - if addrlen == 0: + if caddrlen == 0: # No address -- may be recvfrom() from known socket return None - if caddr.sa_family == AF_INET: - return makeipaddr(caddr), _c.ntohs( + if caddr.contents.sa_family == AF_INET: + a = cast(caddr, POINTER(_c.sockaddr_in)) + return makeipaddr(caddr, caddrlen), _c.ntohs(a.contents.sin_port) else: XXX @@ -66,12 +75,12 @@ if error: XXX try: - return _getaddrinfo_chainedlist(res0) + return _getaddrinfo_chainedlist(res0, proto) finally: if res0: _c.freeaddrinfo(res0) -def _getaddrinfo_chainedlist(res): +def _getaddrinfo_chainedlist(res, proto): result = [] while res: res = res.contents Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sat Apr 15 13:02:14 2006 @@ -2,7 +2,12 @@ from pypy.rpython.rctypes import ctypes_platform from ctypes import * -includes = ('sys/types.h', 'sys/socket.h', 'netinet/in.h', 'netdb.h') +includes = ('sys/types.h', + 'sys/socket.h', + 'netinet/in.h', + 'netdb.h', + 'arpa/inet.h', + ) HEADER = ''.join(['#include <%s>\n' % filename for filename in includes]) constants = {} @@ -11,10 +16,19 @@ 'AF_UNSPEC', 'SOCK_STREAM', 'SOCK_DGRAM', + 'NI_MAXHOST', + 'NI_NUMERICHOST', ]: constants[name] = ctypes_platform.getconstantinteger(name, HEADER) # types +uint16_t = ctypes_platform.getsimpletype('uint16_t', HEADER, c_ushort) +uint32_t = ctypes_platform.getsimpletype('uint32_t', HEADER, c_uint) +size_t = ctypes_platform.getsimpletype('size_t', HEADER, c_int) +size_t = ctypes_platform.getsimpletype('size_t', HEADER, c_int) +socklen_t = ctypes_platform.getsimpletype('socklen_t', HEADER, c_int) + +# struct types sockaddr = ctypes_platform.getstruct('struct sockaddr', HEADER, [('sa_family', c_int), # unknown and variable fields follow @@ -54,3 +68,25 @@ freeaddrinfo = socketdll.freeaddrinfo freeaddrinfo.argtypes = [POINTER(addrinfo)] freeaddrinfo.restype = None + +getnameinfo = socketdll.getnameinfo +getnameinfo.argtypes = [POINTER(sockaddr), socklen_t, + c_char_p, size_t, + c_char_p, size_t, c_int] +getnameinfo.restype = c_int + +htonl = socketdll.htonl +htonl.argtypes = [uint32_t] +htonl.restype = uint32_t + +htons = socketdll.htonl +htons.argtypes = [uint16_t] +htons.restype = uint16_t + +ntohl = socketdll.htonl +ntohl.argtypes = [uint32_t] +ntohl.restype = uint32_t + +ntohs = socketdll.htonl +ntohs.argtypes = [uint16_t] +ntohs.restype = uint16_t Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test_addr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test_addr.py Sat Apr 15 13:02:14 2006 @@ -0,0 +1,10 @@ +from _socket import * + +def test_getaddrinfo(): + lst = getaddrinfo('snake.cs.uni-duesseldorf.de', None) + assert isinstance(lst, list) + found = False + for family, socktype, protocol, canonname, (host, port) in lst: + if host == '134.99.112.214': + found = True + assert found, lst From cfbolz at codespeak.net Sat Apr 15 13:33:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 13:33:17 +0200 (CEST) Subject: [pypy-svn] r25853 - in pypy/dist/pypy: rpython/memory translator/c/test Message-ID: <20060415113317.10032100A7@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 13:33:08 2006 New Revision: 25853 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/translator/c/test/test_newgc.py Log: test for saving of push/pop pairs plus a bit more tweaking Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Apr 15 13:33:08 2006 @@ -959,6 +959,7 @@ livevars = dict.fromkeys( [var for var in livevars if not var_ispyobj(var)], True) if not needs_conservative_livevar_calculation(block): + not_needed = {} if index == -1: index = block.operations.index(op) # XXX hum needed = {} @@ -966,11 +967,11 @@ if before_op.result not in livevars: continue if before_op.opname in ("cast_pointer", "same_as"): - del livevars[before_op.result] + not_needed[before_op.result] = True elif before_op.opname in ("getfield", "getarrayitem"): if (before_op.args[0] in livevars or isinstance(before_op.args[0], Constant)): - del livevars[before_op.result] + not_needed[before_op.result] = True for after_op in block.operations[index:]: for arg in after_op.args: needed[arg] = True @@ -980,7 +981,7 @@ needed[arg] = True newlivevars = [] for var in livevars: - if var in needed: + if var in needed and var not in not_needed: newlivevars.append(var) livevars = newlivevars else: Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Sat Apr 15 13:33:08 2006 @@ -210,6 +210,36 @@ res = fn() assert res == 2 assert len(self.t.graphs[0].startblock.exits[False].target.operations) == 10 + def test_framework_safe_pushpop(self): + class A(object): + pass + class B(object): + pass + def g(x): # can cause a collect + return B() + global_a = A() + global_a.b = B() + global_a.b.a = A() + global_a.b.a.b = B() + global_a.b.a.b.c = 1 + def f(): + global_a.b.a.b.c = 40 + a = global_a.b.a + b = a.b + b.c = 41 + g(1) + b0 = a.b + b0.c = b.c = 42 + # this should trigger a couple of collections + # XXX make sure it triggers at least one somehow! + for i in range(100000): + [A()] * 1000 + return global_a.b.a.b.c + fn = self.getcompiled(f) + startblock = self.t.graphs[0].startblock + res = fn() + assert res == 42 + assert len([op for op in startblock.operations if op.opname == "gc_reload_possibly_moved"]) == 0 def test_framework_varsized(self): S = lltype.GcStruct("S", ('x', lltype.Signed)) From cfbolz at codespeak.net Sat Apr 15 14:03:44 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 14:03:44 +0200 (CEST) Subject: [pypy-svn] r25854 - pypy/dist/pypy/translator/c Message-ID: <20060415120344.F32C9100A8@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 14:03:39 2006 New Revision: 25854 Modified: pypy/dist/pypy/translator/c/gc.py Log: completely remove this line, instead of commenting it out: sometimes our variables can contain comments Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Sat Apr 15 14:03:39 2006 @@ -287,7 +287,9 @@ def OP_GC_RELOAD_POSSIBLY_MOVED(self, funcgen, op): args = [funcgen.expr(v) for v in op.args] # XXX this more or less assumes mark-and-sweep gc - return '/* %s = %s; */ /* for moving GCs */' % (args[1], args[0]) + return '' + # proper return value for moving GCs: + # %s = %s; /* for moving GCs */' % (args[1], args[0]) def common_gcheader_definition(self, defnode): # XXX assumes mark and sweep From arigo at codespeak.net Sat Apr 15 14:35:15 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 14:35:15 +0200 (CEST) Subject: [pypy-svn] r25855 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060415123515.B7AE1100A8@code0.codespeak.net> Author: arigo Date: Sat Apr 15 14:35:14 2006 New Revision: 25855 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/_test_addr.py - copied unchanged from r25852, pypy/dist/pypy/rpython/rctypes/socketmodule/test_addr.py Removed: pypy/dist/pypy/rpython/rctypes/socketmodule/test_addr.py Log: Disable this test, it leaves a broken _socket module imported behind. From cfbolz at codespeak.net Sat Apr 15 14:43:29 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 14:43:29 +0200 (CEST) Subject: [pypy-svn] r25856 - pypy/dist/pypy/rpython/memory Message-ID: <20060415124329.EE929100A3@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 14:43:28 2006 New Revision: 25856 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: simplify pop_root since the result is anyway not used for a non-moving GC Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Apr 15 14:43:28 2006 @@ -774,11 +774,15 @@ top.address[0] = addr gcdata.root_stack_top = top + sizeofaddr + # XXX specific to mark and sweep def pop_root(): - top = gcdata.root_stack_top - sizeofaddr - result = top.address[0] - gcdata.root_stack_top = top - return result + gcdata.root_stack_top -= sizeofaddr + # this should really be: + # def pop_root(): + # top = gcdata.root_stack_top - sizeofaddr + # result = top.address[0] + # gcdata.root_stack_top = top + # return result bk = self.translator.annotator.bookkeeper @@ -805,7 +809,7 @@ [annmodel.SomeAddress()], annmodel.s_None) pop_root_graph = annhelper.getgraph(pop_root, [], - annmodel.SomeAddress()) + annmodel.s_None) classdef = bk.getuniqueclassdef(GCData.GCClass) s_gcdata = annmodel.SomeInstance(classdef) From arigo at codespeak.net Sat Apr 15 14:44:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 15 Apr 2006 14:44:48 +0200 (CEST) Subject: [pypy-svn] r25857 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060415124448.D5532100A8@code0.codespeak.net> Author: arigo Date: Sat Apr 15 14:44:44 2006 New Revision: 25857 Added: pypy/dist/pypy/rpython/rctypes/aarray.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/achar_p.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/afunc.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rfunc.py pypy/dist/pypy/rpython/rctypes/apointer.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/aprimitive.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/astruct.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/avoid_p.py - copied, changed from r25852, pypy/dist/pypy/rpython/rctypes/rvoid_p.py pypy/dist/pypy/rpython/rctypes/test/test_import.py (contents, props changed) Removed: pypy/dist/pypy/rpython/rctypes/rfunc.py Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/rvoid_p.py Log: Split all r* modules in two files, the a* part doing the extregistering and annotating, and the r* part containing the Repr. This allows us to annotate rpython programs using rctypes without importing the complete rtyper. Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Sat Apr 15 14:44:44 2006 @@ -1,11 +1,11 @@ # Importing for side effect of registering types with extregistry -import pypy.rpython.rctypes.rprimitive -import pypy.rpython.rctypes.rpointer -import pypy.rpython.rctypes.rarray -import pypy.rpython.rctypes.rfunc -import pypy.rpython.rctypes.rchar_p -import pypy.rpython.rctypes.rstruct -import pypy.rpython.rctypes.rvoid_p +import pypy.rpython.rctypes.aprimitive +import pypy.rpython.rctypes.apointer +import pypy.rpython.rctypes.aarray +import pypy.rpython.rctypes.afunc +import pypy.rpython.rctypes.achar_p +import pypy.rpython.rctypes.astruct +import pypy.rpython.rctypes.avoid_p # Register the correspondance between SomeCTypesObject and the get_repr() Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Sat Apr 15 14:44:44 2006 @@ -1,16 +1,16 @@ from ctypes import ARRAY, c_int -from pypy.annotation.model import SomeCTypesObject, SomeBuiltin -from pypy.rpython import extregistry from pypy.rpython.rbuiltin import gen_cast_subarray_pointer -from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst +from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr from pypy.rpython.rctypes.rmodel import genreccopy, reccopy from pypy.rpython.rctypes.rprimitive import PrimitiveRepr +from pypy.annotation.model import SomeCTypesObject ArrayType = type(ARRAY(c_int, 10)) + class ArrayRepr(CTypesRefRepr): def __init__(self, rtyper, s_array): array_ctype = s_array.knowntype @@ -89,31 +89,3 @@ # ByValue case (optimization; the above also works in this case) v_newvalue = r_array.r_item.getvalue(hop.llops, v_item) r_array.set_item_value(hop.llops, v_array, v_index, v_newvalue) - -# ____________________________________________________________ - -def arraytype_specialize_call(hop): - r_array = hop.r_result - return hop.genop("malloc", [ - hop.inputconst(lltype.Void, r_array.lowleveltype.TO), - ], resulttype=r_array.lowleveltype, - ) - -def arraytype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - return SomeBuiltin(compute_result_annotation, methodname=type.__name__) - -extregistry.register_type(ArrayType, - compute_annotation=arraytype_compute_annotation, - specialize_call=arraytype_specialize_call) - -def array_instance_compute_annotation(type, instance): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - -def arraytype_get_repr(rtyper, s_array): - return ArrayRepr(rtyper, s_array) - -extregistry.register_metatype(ArrayType, - compute_annotation=array_instance_compute_annotation, - get_repr=arraytype_get_repr) Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Sat Apr 15 14:44:44 2006 @@ -1,10 +1,8 @@ -from pypy.rpython import extregistry from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype from pypy.rpython.rstr import StringRepr, string_repr from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO from pypy.rpython.rctypes.rarray import ArrayRepr -from pypy.annotation import model as annmodel from pypy.annotation.pairtype import pairtype from ctypes import c_char, c_char_p @@ -126,39 +124,3 @@ else: box.c_data[0] = lltype.nullptr(CCHARP.TO) box.keepalive_str = string - - -def c_char_p_compute_result_annotation(s_arg=None): - return annmodel.SomeCTypesObject(c_char_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_char_p_specialize_call(hop): - r_char_p = hop.r_result - v_result = r_char_p.allocate_instance(hop.llops) - if len(hop.args_s): - v_value, = hop.inputargs(string_repr) - r_char_p.setstring(hop.llops, v_result, v_value) - return v_result - -extregistry.register_value(c_char_p, - compute_result_annotation=c_char_p_compute_result_annotation, - specialize_call=c_char_p_specialize_call - ) - -def c_char_compute_annotation(the_type, instance): - return annmodel.SomeCTypesObject(c_char_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_char_p_get_repr(rtyper, s_char_p): - return CCharPRepr(rtyper, s_char_p, CCHARP) - -entry = extregistry.register_type(c_char_p, - compute_annotation = c_char_compute_annotation, - get_repr = c_char_p_get_repr, - ) -s_value_annotation = annmodel.SomeString(can_be_None=True) -def c_char_p_get_field_annotation(s_char_p, fieldname): - assert fieldname == 'value' - return s_value_annotation -entry.get_field_annotation = c_char_p_get_field_annotation -entry.s_return_trick = s_value_annotation Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Sat Apr 15 14:44:44 2006 @@ -1,12 +1,10 @@ -from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst +from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.error import TyperError -from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype -from pypy.annotation import model as annmodel from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesValueRepr, genreccopy +from pypy.annotation.model import SomeCTypesObject -from ctypes import POINTER, pointer, byref, c_int class PointerRepr(CTypesValueRepr): def __init__(self, rtyper, s_pointer): @@ -14,8 +12,8 @@ ref_ctype = ptr_ctype._type_ # Find the repr and low-level type of the contents from its ctype - self.r_contents = rtyper.getrepr(annmodel.SomeCTypesObject(ref_ctype, - annmodel.SomeCTypesObject.MEMORYALIAS)) + self.r_contents = rtyper.getrepr(SomeCTypesObject(ref_ctype, + SomeCTypesObject.MEMORYALIAS)) ll_contents = lltype.Ptr(self.r_contents.c_data_type) @@ -87,78 +85,3 @@ raise TyperError("assignment to pointer[x] with x != 0") # copy the whole structure's content over genreccopy(hop.llops, v_new_c_data, v_target) - -# ____________________________________________________________ - -def pointertype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s): - return annmodel.SomeCTypesObject(type, - annmodel.SomeCTypesObject.OWNSMEMORY) - return annmodel.SomeBuiltin(compute_result_annotation, - methodname=type.__name__) - -def pointertype_specialize_call(hop): - r_ptr = hop.r_result - v_result = r_ptr.allocate_instance(hop.llops) - if len(hop.args_s): - v_contentsbox, = hop.inputargs(r_ptr.r_contents) - r_ptr.setcontents(hop.llops, v_result, v_contentsbox) - return v_result - -def pointerinstance_compute_annotation(type, instance): - return annmodel.SomeCTypesObject(type, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def pointerinstance_field_annotation(s_pointer, fieldname): - assert fieldname == "contents" - ptrtype = s_pointer.knowntype - return annmodel.SomeCTypesObject(ptrtype._type_, - annmodel.SomeCTypesObject.MEMORYALIAS) - -def pointerinstance_get_repr(rtyper, s_pointer): - return PointerRepr(rtyper, s_pointer) - -PointerType = type(POINTER(c_int)) -extregistry.register_type(PointerType, - compute_annotation=pointertype_compute_annotation, - specialize_call=pointertype_specialize_call) - -entry = extregistry.register_metatype(PointerType, - compute_annotation=pointerinstance_compute_annotation, - get_repr=pointerinstance_get_repr) -entry.get_field_annotation = pointerinstance_field_annotation - -def pointerfn_compute_annotation(s_arg): - assert isinstance(s_arg, annmodel.SomeCTypesObject) - ctype = s_arg.knowntype - result_ctype = POINTER(ctype) - return annmodel.SomeCTypesObject(result_ctype, - annmodel.SomeCTypesObject.OWNSMEMORY) - -extregistry.register_value(pointer, - compute_result_annotation=pointerfn_compute_annotation, - # same rtyping for calling pointer() or calling a specific instance - # of PointerType: - specialize_call=pointertype_specialize_call) - -# byref() is equivalent to pointer() -- the difference is only an -# optimization that is useful in ctypes but not in rctypes. -extregistry.register_value(byref, - compute_result_annotation=pointerfn_compute_annotation, - specialize_call=pointertype_specialize_call) - -# constant-fold POINTER(CONSTANT_CTYPE) calls -def POINTER_compute_annotation(s_arg): - from pypy.annotation.bookkeeper import getbookkeeper - assert s_arg.is_constant(), "POINTER(%r): argument must be constant" % ( - s_arg,) - RESTYPE = POINTER(s_arg.const) - return getbookkeeper().immutablevalue(RESTYPE) - -def POINTER_specialize_call(hop): - assert hop.s_result.is_constant() - return hop.inputconst(lltype.Void, hop.s_result.const) - -extregistry.register_value(POINTER, - compute_result_annotation=POINTER_compute_annotation, - specialize_call=POINTER_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Sat Apr 15 14:44:44 2006 @@ -1,30 +1,10 @@ -from ctypes import c_char, c_byte, c_ubyte, c_short, c_ushort, c_int, c_uint -from ctypes import c_long, c_ulong, c_longlong, c_ulonglong, c_float -from ctypes import c_double, c_char_p -from pypy.annotation import model as annmodel -from pypy.rpython import extregistry -from pypy.rpython.rmodel import Repr, inputconst +from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import IntegerRepr, FloatRepr, CharRepr from pypy.rpython.error import TyperError from pypy.rpython.rctypes.rmodel import CTypesValueRepr -ctypes_annotation_list = { - c_char: lltype.Char, - c_byte: lltype.Signed, - c_ubyte: lltype.Unsigned, - c_short: lltype.Signed, - c_ushort: lltype.Unsigned, - c_int: lltype.Signed, - c_uint: lltype.Unsigned, - c_long: lltype.Signed, - c_ulong: lltype.Unsigned, - c_longlong: lltype.SignedLongLong, - c_ulonglong: lltype.UnsignedLongLong, - c_float: lltype.Float, - c_double: lltype.Float, -}.items() # nb. platform-dependent duplicate ctypes are removed class PrimitiveRepr(CTypesValueRepr): @@ -69,43 +49,3 @@ # return this box possibly converted to the expected output repr, # which might be a memory-aliasing box return llops.convertvar(v_owned_box, r_temp, r_to) - - -def primitive_specialize_call(hop): - r_primitive = hop.r_result - v_result = r_primitive.allocate_instance(hop.llops) - if len(hop.args_s): - v_value, = hop.inputargs(r_primitive.ll_type) - r_primitive.setvalue(hop.llops, v_result, v_value) - return v_result - -def do_register(the_type, ll_type): - def compute_result_annotation_function(s_arg=None): - return annmodel.SomeCTypesObject(the_type, - annmodel.SomeCTypesObject.OWNSMEMORY) - - extregistry.register_value(the_type, - compute_result_annotation=compute_result_annotation_function, - specialize_call=primitive_specialize_call - ) - - def compute_prebuilt_instance_annotation(the_type, instance): - return annmodel.SomeCTypesObject(the_type, - annmodel.SomeCTypesObject.OWNSMEMORY) - - def primitive_get_repr(rtyper, s_primitive): - return PrimitiveRepr(rtyper, s_primitive, ll_type) - - entry = extregistry.register_type(the_type, - compute_annotation=compute_prebuilt_instance_annotation, - get_repr=primitive_get_repr, - ) - s_value_annotation = annmodel.lltype_to_annotation(ll_type) - def primitive_get_field_annotation(s_primitive, fieldname): - assert fieldname == 'value' - return s_value_annotation - entry.get_field_annotation = primitive_get_field_annotation - entry.s_return_trick = s_value_annotation - -for the_type, ll_type in ctypes_annotation_list: - do_register(the_type, ll_type) Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Sat Apr 15 14:44:44 2006 @@ -1,14 +1,11 @@ -from ctypes import Structure -from pypy.annotation.model import SomeCTypesObject, SomeBuiltin -from pypy.rpython import extregistry from pypy.rpython.rmodel import inputconst from pypy.rpython.rbuiltin import gen_cast_structfield_pointer from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr from pypy.rpython.rctypes.rmodel import genreccopy, reccopy from pypy.rpython.rctypes.rprimitive import PrimitiveRepr +from pypy.annotation.model import SomeCTypesObject -StructType = type(Structure) class StructRepr(CTypesRefRepr): def __init__(self, rtyper, s_struct): @@ -103,40 +100,3 @@ # ByValue case (optimization; the above also works in this case) v_newvalue = r_field.getvalue(hop.llops, v_item) self.set_field_value(hop.llops, v_struct, name, v_newvalue) - -# ____________________________________________________________ - -def structtype_specialize_call(hop): - r_struct = hop.r_result - return hop.genop("malloc", [ - hop.inputconst(lltype.Void, r_struct.lowleveltype.TO), - ], resulttype=r_struct.lowleveltype, - ) - -def structtype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - return SomeBuiltin(compute_result_annotation, methodname=type.__name__) - -extregistry.register_type(StructType, - compute_annotation=structtype_compute_annotation, - specialize_call=structtype_specialize_call) - -def struct_instance_compute_annotation(type, instance): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - -def struct_instance_field_annotation(s_struct, fieldname): - structtype = s_struct.knowntype - for name, ctype in structtype._fields_: - if name == fieldname: - s_result = SomeCTypesObject(ctype, SomeCTypesObject.MEMORYALIAS) - return s_result.return_annotation() - raise AttributeError('%r has no field %r' % (structtype, fieldname)) - -def structtype_get_repr(rtyper, s_struct): - return StructRepr(rtyper, s_struct) - -entry = extregistry.register_metatype(StructType, - compute_annotation=struct_instance_compute_annotation, - get_repr=structtype_get_repr) -entry.get_field_annotation = struct_instance_field_annotation Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rvoid_p.py Sat Apr 15 14:44:44 2006 @@ -1,72 +1,5 @@ -from pypy.rpython import extregistry -from pypy.rpython.lltypesystem import lltype, llmemory -from pypy.annotation import model as annmodel -from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO -from pypy.rpython.rctypes.rpointer import PointerRepr - -from ctypes import c_void_p, c_int, POINTER, cast - -PointerType = type(POINTER(c_int)) +from pypy.rpython.rctypes.rmodel import CTypesValueRepr class CVoidPRepr(CTypesValueRepr): pass # No operations supported on c_void_p instances so far - - -# c_void_p() as a function -def c_void_p_compute_result_annotation(s_arg=None): - raise NotImplementedError("XXX calling c_void_p()") - -extregistry.register_value(c_void_p, - compute_result_annotation=c_void_p_compute_result_annotation, - ) - -# c_void_p instances -def c_void_compute_annotation(the_type, instance): - return annmodel.SomeCTypesObject(c_void_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_void_p_get_repr(rtyper, s_void_p): - return CVoidPRepr(rtyper, s_void_p, llmemory.Address) - -extregistry.register_type(c_void_p, - compute_annotation = c_void_compute_annotation, - get_repr = c_void_p_get_repr, - ) - -# cast() support -def cast_compute_result_annotation(s_arg, s_type): - assert s_type.is_constant(), "cast(p, %r): argument 2 must be constant" % ( - s_type,) - type = s_type.const - - def checkptr(ctype): - assert isinstance(ctype, PointerType) or ctype == c_void_p, ( - "cast(): can only cast between pointers so far, not %r" % (ctype,)) - checkptr(s_arg.knowntype) - checkptr(type) - return annmodel.SomeCTypesObject(type, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def cast_specialize_call(hop): - assert isinstance(hop.args_r[0], (PointerRepr, CVoidPRepr)) - targetctype = hop.args_s[1].const - v_box, c_targetctype = hop.inputargs(hop.args_r[0], lltype.Void) - v_adr = hop.args_r[0].getvalue(hop.llops, v_box) - if v_adr.concretetype != llmemory.Address: - v_adr = hop.genop('cast_ptr_to_adr', [v_adr], - resulttype = llmemory.Address) - - if targetctype == c_void_p: - # cast to void - v_result = v_adr - else: - # cast to pointer - v_result = hop.genop('cast_adr_to_ptr', [v_adr], - resulttype = hop.r_result.ll_type) - return hop.r_result.return_value(hop.llops, v_result) - -extregistry.register_value(cast, - compute_result_annotation=cast_compute_result_annotation, - specialize_call=cast_specialize_call, - ) Added: pypy/dist/pypy/rpython/rctypes/test/test_import.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_import.py Sat Apr 15 14:44:44 2006 @@ -0,0 +1,18 @@ +""" +Checks that importing and registering the annotations for rctypes +doesn't bring in the whole rtyper. +""" + +import py, sys + +def test_import(): + gw = py.execnet.PopenGateway() + channel = gw.remote_exec(''' + import sys + sys.path = channel.receive() + import pypy.rpython.rctypes.implementation + channel.send(sys.modules.keys()) + ''') + channel.send(sys.path) + modules = channel.receive() + assert 'pypy.rpython.rmodel' not in modules From cfbolz at codespeak.net Sat Apr 15 18:42:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 18:42:28 +0200 (CEST) Subject: [pypy-svn] r25863 - pypy/dist/pypy/rpython/memory Message-ID: <20060415164228.7A91B100BC@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 18:42:26 2006 New Revision: 25863 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: humpf. this works as long as you inline the helpers :-( Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Apr 15 18:42:26 2006 @@ -1058,11 +1058,12 @@ def pop_roots(self, vars): for var in vars[::-1]: - v = varoftype(llmemory.Address) + v = varoftype(lltype.Void) + # XXX specific to non-moving collectors yield SpaceOperation("direct_call", [self.pop_root_ptr], v) - yield SpaceOperation("gc_reload_possibly_moved", [v, var], - varoftype(lltype.Void)) + #yield SpaceOperation("gc_reload_possibly_moved", [v, var], + # varoftype(lltype.Void)) # XXX copied and modified from lltypelayout.py def offsets_to_gc_pointers(TYPE): From cfbolz at codespeak.net Sat Apr 15 19:02:41 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 15 Apr 2006 19:02:41 +0200 (CEST) Subject: [pypy-svn] r25864 - pypy/dist/pypy/translator/backendopt Message-ID: <20060415170241.0B863100CA@code0.codespeak.net> Author: cfbolz Date: Sat Apr 15 19:02:40 2006 New Revision: 25864 Modified: pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/backendopt/propagate.py Log: enable the more sensible propagate optimizations by default Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Sat Apr 15 19:02:40 2006 @@ -18,7 +18,7 @@ mallocs=True, ssa_form=True, merge_if_blocks_to_switch=True, - propagate=False, + propagate=True, heap2stack=False, clever_malloc_removal=False): Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Sat Apr 15 19:02:40 2006 @@ -243,8 +243,8 @@ else: assert 0, "this should not occur" unchanged = link.target == nextblock and link.args == newargs - if not unchanged: - print "doing partial folding in graph", graph.name +# if not unchanged: +# print "doing partial folding in graph", graph.name link.target = nextblock link.args = newargs checkgraph(graph) @@ -362,10 +362,10 @@ changed = False changed = rewire_links(graph) or changed changed = propagate_consts(graph) or changed - changed = coalesce_links(graph) or changed - changed = do_atmost(100, constant_folding, graph, - translator) or changed - changed = partial_folding(graph, translator) or changed +# changed = coalesce_links(graph) or changed +# changed = do_atmost(100, constant_folding, graph, +# translator) or changed +# changed = partial_folding(graph, translator) or changed changed = remove_all_getfields(graph, translator) or changed checkgraph(graph) return changed From antocuni at codespeak.net Sun Apr 16 10:23:25 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sun, 16 Apr 2006 10:23:25 +0200 (CEST) Subject: [pypy-svn] r25867 - in pypy/dist/pypy: rpython/ootypesystem rpython/ootypesystem/test translator/cli/src translator/cli/test Message-ID: <20060416082325.57B5610098@code0.codespeak.net> Author: antocuni Date: Sun Apr 16 10:22:57 2006 New Revision: 25867 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py Log: The interface of ootypesystem.ootype.List has been modified to reflect the changes in lltypesystem's list. In particular the 'length', 'getitem_nonneg' and 'setitem_nonneg' have been renamed to 'll_length', 'll_getitem_fast' and 'll_setitem_fast'. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sun Apr 16 10:22:57 2006 @@ -200,18 +200,24 @@ self.ITEMTYPE_T: ITEMTYPE, } + # the methods are named after the ADT methods of lltypesystem's lists self._GENERIC_METHODS = frozendict({ # "name": Meth([ARGUMENT1_TYPE, ARGUMENT2_TYPE, ...], RESULT_TYPE) - "length": Meth([], Signed), - "append": Meth([self.ITEMTYPE_T], Void), - "getitem_nonneg": Meth([Signed], self.ITEMTYPE_T), - "setitem_nonneg": Meth([Signed, self.ITEMTYPE_T], Void), + "ll_length": Meth([], Signed), + "ll_getitem_fast": Meth([Signed], self.ITEMTYPE_T), + "ll_setitem_fast": Meth([Signed, self.ITEMTYPE_T], Void), + "append": Meth([self.ITEMTYPE_T], Void), "extend": Meth([self.SELFTYPE_T], Void), "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) }) self._setup_methods(generic_types) + # this is the equivalent of the lltypesystem ll_newlist that is + # marked as typeMethod. + def ll_newlist(self): + return new(self) + def _setup_methods(self, generic_types): methods = {} for name, meth in self._GENERIC_METHODS.iteritems(): @@ -531,7 +537,7 @@ # use by the llinterpreter and ootype tests. There are NOT_RPYTHON # because the annotator is not supposed to follow them. - def length(self): + def ll_length(self): # NOT_RPYTHON return len(self._list) @@ -540,13 +546,13 @@ assert typeOf(item) == self._TYPE._ITEMTYPE self._list.append(item) - def getitem_nonneg(self, index): + def ll_getitem_fast(self, index): # NOT_RPYTHON assert typeOf(index) == Signed assert index >= 0 return self._list[index] - def setitem_nonneg(self, index, item): + def ll_setitem_fast(self, index, item): # NOT_RPYTHON assert typeOf(item) == self._TYPE._ITEMTYPE assert typeOf(index) == Signed Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Sun Apr 16 10:22:57 2006 @@ -56,7 +56,7 @@ return inputconst(Void, self.item_repr.get_ll_eq_function()) def rtype_len(self, hop): - return self.send_message(hop, "length") + return self.send_message(hop, "ll_length") def rtype_is_true(self, hop): v_lst, = hop.inputargs(self) @@ -87,7 +87,7 @@ def rtype_getitem((r_list, r_int), hop): if hop.args_s[1].nonneg: - return r_list.send_message(hop, "getitem_nonneg", can_raise=True) + return r_list.send_message(hop, "ll_getitem_fast", can_raise=True) else: v_list, v_index = hop.inputargs(r_list, Signed) hop.exception_is_here() @@ -96,7 +96,7 @@ def rtype_setitem((r_list, r_int), hop): if hop.args_s[1].nonneg: - return r_list.send_message(hop, "setitem_nonneg", can_raise=True) + return r_list.send_message(hop, "ll_setitem_fast", can_raise=True) else: v_list, v_index, v_item = hop.inputargs(r_list, Signed, r_list.item_repr) hop.exception_is_here() @@ -133,83 +133,84 @@ def ll_getitem(lst, index): if index < 0: - index += lst.length() - return lst.getitem_nonneg(index) + index += lst.ll_length() + return lst.ll_getitem_fast(index) def ll_setitem(lst, index, item): if index < 0: - index += lst.length() - return lst.setitem_nonneg(index, item) + index += lst.ll_length() + return lst.ll_setitem_fast(index, item) def ll_delitem(lst, index): if index < 0: - index += lst.length() + index += lst.ll_length() return lst.remove_range(index, 1) def ll_list_is_true(lst): - return bool(lst) and lst.length() != 0 + return bool(lst) and lst.ll_length() != 0 def ll_append(lst, item): lst.append(item) def ll_extend(l1, l2): # This is a bit inefficient, could also add extend to the list interface - len2 = l2.length() + len2 = l2.ll_length() i = 0 while i < len2: - l1.append(l2.getitem_nonneg(i)) + l1.append(l2.ll_getitem_fast(i)) i += 1 def ll_concat(RESLIST, l1, l2): - len1 = l1.length() - len2 = l2.length() - l = ootype.new(RESLIST) + len1 = l1.ll_length() + len2 = l2.ll_length() + #l = ootype.new(RESLIST) + l = RESLIST.ll_newlist() i = 0 while i < len1: - l.append(l1.getitem_nonneg(i)) + l.append(l1.ll_getitem_fast(i)) i += 1 i = 0 while i < len2: - l.append(l2.getitem_nonneg(i)) + l.append(l2.ll_getitem_fast(i)) i += 1 return l def ll_listslice_startonly(RESLIST, lst, start): - len1 = lst.length() + len1 = lst.ll_length() #newlength = len1 - start res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements i = start while i < len1: - res.append(lst.getitem_nonneg(i)) + res.append(lst.ll_getitem_fast(i)) i += 1 return res def ll_listslice(RESLIST, lst, slice): start = slice.start stop = slice.stop - length = lst.length() + length = lst.ll_length() if stop > length: stop = length #newlength = stop - start res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements i = start while i < stop: - res.append(lst.getitem_nonneg(i)) + res.append(lst.ll_getitem_fast(i)) i += 1 return res def ll_listslice_minusone(RESLIST, lst): - newlength = lst.length() - 1 + newlength = lst.ll_length() - 1 #assert newlength >= 0 # TODO: asserts seems to have problems with ootypesystem res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements i = 0 while i < newlength: - res.append(lst.getitem_nonneg(i)) + res.append(lst.ll_getitem_fast(i)) i += 1 return res def ll_listsetslice(l1, slice, l2): - count = l2.length() + count = l2.ll_length() ## assert count == slice.stop - slice.start, ( # TODO: see above ## "setslice cannot resize lists in RPython") # XXX but it should be easy enough to support, soon @@ -217,19 +218,19 @@ j = start i = 0 while i < count: - l1.setitem_nonneg(j, l2.getitem_nonneg(i)) + l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) i += 1 j += 1 def ll_listdelslice_startonly(lst, start): - count = lst.length() - start + count = lst.ll_length() - start if count > 0: lst.remove_range(start, count) def ll_listdelslice(lst, slice): start = slice.start stop = slice.stop - length = lst.length() + length = lst.ll_length() if stop > length: stop = length count = stop - start @@ -237,14 +238,14 @@ lst.remove_range(start, count) def ll_listindex(lst, obj, eqfn): - lng = lst.length() + lng = lst.ll_length() j = 0 while j < lng: if eqfn is None: - if lst.getitem_nonneg(j) == obj: + if lst.ll_getitem_fast(j) == obj: return j else: - if eqfn(lst.getitem_nonneg(j), obj): + if eqfn(lst.ll_getitem_fast(j), obj): return j j += 1 raise ValueError # can't say 'list.index(x): x not in list' @@ -273,8 +274,8 @@ def ll_listnext(iter): l = iter.iterable index = iter.index - if index >= l.length(): + if index >= l.ll_length(): raise StopIteration iter.index = index + 1 - return l.getitem_nonneg(index) + return l.ll_getitem_fast(index) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Sun Apr 16 10:22:57 2006 @@ -10,13 +10,13 @@ def test_len(): LT = List(Signed) l = new(LT) - assert l.length() == 0 + assert l.ll_length() == 0 def test_append(): LT = List(Signed) l = new(LT) l.append(1) - assert l.length() == 1 + assert l.ll_length() == 1 def test_extend(): LT = List(Signed) @@ -25,21 +25,21 @@ l1.append(1) l2.append(2) l1.extend(l2) - assert l1.length() == 2 + assert l1.ll_length() == 2 def test_setitem_getitem(): LT = List(Signed) l = new(LT) l.append(2) - assert l.getitem_nonneg(0) == 2 - l.setitem_nonneg(0, 3) - assert l.getitem_nonneg(0) == 3 + assert l.ll_getitem_fast(0) == 2 + l.ll_setitem_fast(0, 3) + assert l.ll_getitem_fast(0) == 3 def test_setitem_indexerror(): LT = List(Signed) l = new(LT) - py.test.raises(IndexError, l.getitem_nonneg, 0) - py.test.raises(IndexError, l.setitem_nonneg, 0, 1) + py.test.raises(IndexError, l.ll_getitem_fast, 0) + py.test.raises(IndexError, l.ll_setitem_fast, 0, 1) def test_null(): LT = List(Signed) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Sun Apr 16 10:22:57 2006 @@ -104,7 +104,7 @@ def oof(): l = new(LT) - return l.length() + return l.ll_length() g = gengraph(oof, []) rettype = g.getreturnvar().concretetype @@ -116,7 +116,7 @@ def oof(): l = new(LT) l.append(1) - return l.length() + return l.ll_length() g = gengraph(oof, []) rettype = g.getreturnvar().concretetype @@ -128,8 +128,8 @@ def oof(): l = new(LT) l.append(1) - l.setitem_nonneg(0, 2) - return l.getitem_nonneg(0) + l.ll_setitem_fast(0, 2) + return l.ll_getitem_fast(0) g = gengraph(oof, []) rettype = g.getreturnvar().concretetype @@ -141,7 +141,7 @@ def oof(): l = new(LT) try: - l.getitem_nonneg(0) + l.ll_getitem_fast(0) except IndexError: return -1 return 0 @@ -162,7 +162,7 @@ LIST = List(Signed) def oof(lst): - return lst.length() + return lst.ll_length() lst = new(LIST) lst.append(1) Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Sun Apr 16 10:22:57 2006 @@ -14,24 +14,24 @@ // rpython.ootypesystem.ootype.List.GENERIC_METHODS public class List: System.Collections.Generic.List { - public int length() + public int ll_length() { return this.Count; } - public void append(T item) + public T ll_getitem_fast(int index) { - this.Add(item); + return this[index]; } - public T getitem_nonneg(int index) + public void ll_setitem_fast(int index, T item) { - return this[index]; + this[index] = item; } - public void setitem_nonneg(int index, T item) + public void append(T item) { - this[index] = item; + this.Add(item); } public void extend(List other) Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Sun Apr 16 10:22:57 2006 @@ -33,9 +33,9 @@ return total def bar(x, y): - lst = [1,2,3,x,y] + lst = [1,2,3,x,y] + [1,2] #return sum_(list(lst)) - return sum_(lst[:]) + return lst[4] f = compile_function(bar, [int, int]) From arigo at codespeak.net Sun Apr 16 18:13:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 16 Apr 2006 18:13:36 +0200 (CEST) Subject: [pypy-svn] r25871 - in pypy/dist/pypy: . bin objspace objspace/test Message-ID: <20060416161336.CD17510095@code0.codespeak.net> Author: arigo Date: Sun Apr 16 18:13:35 2006 New Revision: 25871 Added: pypy/dist/pypy/objspace/test/test_trivialobjspace.py (contents, props changed) pypy/dist/pypy/objspace/trivial.py (contents, props changed) Modified: pypy/dist/pypy/bin/py.py pypy/dist/pypy/conftest.py Log: A new trivial object space. This one is based on "unwrapping" interpreter-level W_Xxx objects into a simulated CPython object whose type is built from the typedef. A similar type-building approach will be needed for the CPyObjSpace. Modified: pypy/dist/pypy/bin/py.py ============================================================================== --- pypy/dist/pypy/bin/py.py (original) +++ pypy/dist/pypy/bin/py.py Sun Apr 16 18:13:35 2006 @@ -54,14 +54,9 @@ return options def make_objspace(cmdlineopt): - if cmdlineopt.objspace == 'std': - from pypy.objspace.std import Space - elif cmdlineopt.objspace == 'thunk': - from pypy.objspace.thunk import Space - elif cmdlineopt.objspace == 'logic': - from pypy.objspace.logic import Space - else: - raise ValueError("cannot instantiate %r space" %(cmdlineopt.objspace,)) + mod = __import__('pypy.objspace.%s' % cmdlineopt.objspace, + None, None, ['Space']) + Space = mod.Space space = Space(usemodules = cmdlineopt.usemodules, nofaking = cmdlineopt.nofaking, Modified: pypy/dist/pypy/conftest.py ============================================================================== --- pypy/dist/pypy/conftest.py (original) +++ pypy/dist/pypy/conftest.py Sun Apr 16 18:13:35 2006 @@ -58,7 +58,7 @@ try: return _SPACECACHE[key] except KeyError: - assert name in ('std', 'thunk', 'logic'), name + #assert name in ('std', 'thunk', 'logic', etc.), name mod = __import__('pypy.objspace.%s' % name, None, None, ['Space']) Space = mod.Space try: Added: pypy/dist/pypy/objspace/test/test_trivialobjspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/test/test_trivialobjspace.py Sun Apr 16 18:13:35 2006 @@ -0,0 +1,13 @@ +from pypy.conftest import gettestobjspace + +class AppTest_Trivial: + + def setup_class(cls): + cls.space = gettestobjspace('trivial') + + def test_pystone(self): + from test import pystone + try: + pystone.main(1000) + except ZeroDivisionError: + pass # null measured time Added: pypy/dist/pypy/objspace/trivial.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/trivial.py Sun Apr 16 18:13:35 2006 @@ -0,0 +1,236 @@ +""" +A simple object space wrapping real CPython objects to PyPy, +and wrapping back PyPy W_Xxx objects to CPython. +""" +# XXX proof of concept, not fully working. + +import sys, __builtin__ +from pypy.interpreter import baseobjspace +from pypy.interpreter import typedef # force the 'typedef' attributes +from pypy.interpreter.baseobjspace import W_Root, Wrappable +from pypy.interpreter.error import OperationError +from pypy.interpreter.function import Function +from pypy.interpreter.argument import Arguments +from pypy.tool.sourcetools import func_with_new_name +from pypy.rpython.rarithmetic import r_uint + +class W_Box(W_Root): + + def __init__(self, obj): + self.obj = obj + + def __repr__(self): + return 'W_Box(%r)' % (self.obj,) + + def getclass(self, space): + return space.wrap(type(self.obj)) + +# ____________________________________________________________ + +class CPyFacade(object): + __slots__ = ('obj', '__weakref__') + +facade_obj = CPyFacade.obj +facade_new = CPyFacade.__new__ +del CPyFacade.obj + +# ____________________________________________________________ + +class TrivialObjSpace(baseobjspace.ObjSpace): + + def initialize(self): + self.facadetypes = {} + + for key, value in __builtin__.__dict__.items(): + name = 'w_' + key + if not hasattr(self, name): + setattr(self, name, W_Box(value)) + self.make_builtins() + + from pypy.interpreter.module import Module + import exceptions + mod = Module(self, self.wrap('exceptions'), + self.wrap(exceptions.__dict__)) + self.setitem(self.sys.get('modules'), self.wrap('exceptions'), + self.wrap(mod)) + + self.setup_builtin_modules() + + def setoptions(self, **ignore): + pass + + def getfacadetype(self, typedef): + try: + return self.facadetypes[typedef] + except KeyError: + + basedef = typedef.base + if basedef is None: + base = CPyFacade + else: + base = self.getfacadetype(basedef) + + d = {'__slots__': ()} + fixup_classes = [] + for key, value in typedef.rawdict.items(): + if key == '__repr__': + continue # for easier debugging + w_obj = self.wrap(value) + if isinstance(w_obj, Wrappable): + x = facade_new(CPyFacade) + facade_obj.__set__(x, w_obj) + fixup_classes.append((x, w_obj.typedef)) + else: + x = w_obj + d[key] = x + + if typedef is Function.typedef: + # a special case to avoid infinite recursion + def function_call(func, *args, **kwds): + w_func = self.wrap(func) + args_w = [self.wrap(x) for x in args] + kwds_w = dict([(key, self.wrap(value)) + for key, value in kwds.items()]) + a = Arguments(self, args_w, kwds_w) + res = w_func.call_args(a) + return self.unwrap(res) + d['__call__'] = function_call + + T = type(typedef.name, (base,), d) + + if '__slots__' not in typedef.rawdict: + del T.__slots__ + + self.facadetypes[typedef] = T + + for x, typedef in fixup_classes: + x.__class__ = self.getfacadetype(typedef) + + return T + + def wrap(self, x): + if isinstance(x, Wrappable): + return x.__spacebind__(self) + elif isinstance(x, CPyFacade): + return facade_obj.__get__(x) + else: + return W_Box(x) + + newint = wrap + + def unwrap(self, w_obj): + if isinstance(w_obj, Wrappable): + try: + return w_obj.__trivialfacade + except AttributeError: + T = self.getfacadetype(w_obj.typedef) + f = facade_new(T) + facade_obj.__set__(f, w_obj) + w_obj.__trivialfacade = f + return f + else: + return w_obj.obj + + def type(self, w_obj): + return w_obj.getclass(self) + + def is_true(self, w_obj): + return bool(self.unwrap(w_obj)) + + def check_type(self, x, classes): + if not isinstance(x, classes): + msg = '%s expected, got %r object instead' % (classes, + type(x).__name__) + raise OperationError(self.w_TypeError, self.wrap(msg)) + + def str_w(self, w_obj): + x = self.unwrap(w_obj) + self.check_type(x, (str, unicode)) + return str(x) + + def int_w(self, w_obj): + x = self.unwrap(w_obj) + self.check_type(x, (int, long)) + return int(x) + + def uint_w(self, w_obj): + x = self.unwrap(w_obj) + self.check_type(x, (r_uint, int, long)) + return r_uint(x) + + def float_w(self, w_obj): + x = self.unwrap(w_obj) + self.check_type(x, (float, int, long)) + return float(x) + + def is_w(self, w_x, w_y): + return self.unwrap(w_x) is self.unwrap(w_y) + + def newtuple(self, items_w): + items = [self.unwrap(w_item) for w_item in items_w] + return W_Box(tuple(items)) + + def newlist(self, items_w): + items = [self.unwrap(w_item) for w_item in items_w] + return W_Box(items) + + def newdict(self, items_w): + items = [(self.unwrap(w_key), self.unwrap(w_value)) + for w_key, w_value in items_w] + return W_Box(dict(items)) + + def newstring(self, chars): + s = ''.join([chr(n) for n in chars]) + return W_Box(s) + + def newunicode(self, chars): + u = u''.join([unichr(n) for n in chars]) + return W_Box(u) + + def newslice(self, w_start, w_stop, w_step): + sl = slice(self.unwrap(w_start), + self.unwrap(w_stop), + self.unwrap(w_step)) + return W_Box(sl) + + def call_args(self, w_callable, args): + callable = self.unwrap(w_callable) + args_w, kwds_w = args.unpack() + args = [self.unwrap(w_value) for w_value in args_w] + kwds = dict([(key, self.unwrap(w_value)) for key, w_value in kwds_w]) + try: + res = callable(*args, **kwds) + except: + self.wrap_exception() + return self.wrap(res) + + def wrap_exception(self): + # real-to-wrapped exceptions + exc, value, tb = sys.exc_info() + if exc is OperationError: + raise exc, value, tb # just re-raise it + w_exc = self.wrap(exc) + w_value = self.wrap(value) + raise OperationError, OperationError(w_exc, w_value), tb + + +def setup(): + from pypy.objspace.flow.operation import FunctionByName + for name, symbol, arity, specialnames in baseobjspace.ObjSpace.MethodTable: + if hasattr(TrivialObjSpace, name): + continue + def make_function(): + operator = FunctionByName[name] + def do(self, *args_w): + args = [self.unwrap(a) for a in args_w] + try: + res = operator(*args) + except: + self.wrap_exception() + return self.wrap(res) + return func_with_new_name(do, name) + setattr(TrivialObjSpace, name, make_function()) +setup() +del setup + +Space = TrivialObjSpace From cfbolz at codespeak.net Sun Apr 16 18:19:14 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 16 Apr 2006 18:19:14 +0200 (CEST) Subject: [pypy-svn] r25872 - pypy/dist/pypy/translator/backendopt Message-ID: <20060416161914.B8E4110095@code0.codespeak.net> Author: cfbolz Date: Sun Apr 16 18:19:13 2006 New Revision: 25872 Modified: pypy/dist/pypy/translator/backendopt/all.py Log: hum, enabling backendopt broke translation :-(. Will track it down, sorry. Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Sun Apr 16 18:19:13 2006 @@ -18,7 +18,7 @@ mallocs=True, ssa_form=True, merge_if_blocks_to_switch=True, - propagate=True, + propagate=False, heap2stack=False, clever_malloc_removal=False): From cfbolz at codespeak.net Sun Apr 16 19:03:09 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 16 Apr 2006 19:03:09 +0200 (CEST) Subject: [pypy-svn] r25873 - pypy/dist/pypy/translator/backendopt/test Message-ID: <20060416170309.5508E10095@code0.codespeak.net> Author: cfbolz Date: Sun Apr 16 19:03:07 2006 New Revision: 25873 Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: for some reason there is a surviving same_as Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Sun Apr 16 19:03:07 2006 @@ -185,7 +185,7 @@ appendgraph = graph.startblock.operations[3].args[0].value._obj.graph appendarg0 = appendgraph.startblock.inputargs[0] appendstate = adi.getstate(appendarg0) - resizegraph = appendgraph.startblock.operations[2].args[0].value._obj.graph + resizegraph = [op for op in appendgraph.startblock.operations if op.opname != "same_as"][2].args[0].value._obj.graph resizearg0 = resizegraph.startblock.inputargs[0] resizestate = adi.getstate(resizearg0) reallygraph = resizegraph.startblock.exits[0].target.operations[0].args[0].value._obj.graph From cfbolz at codespeak.net Sun Apr 16 19:07:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 16 Apr 2006 19:07:49 +0200 (CEST) Subject: [pypy-svn] r25874 - pypy/dist/pypy/translator/c/test Message-ID: <20060416170749.C333810095@code0.codespeak.net> Author: cfbolz Date: Sun Apr 16 19:07:48 2006 New Revision: 25874 Modified: pypy/dist/pypy/translator/c/test/test_newgc.py Log: make test a bit less brittle Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Sun Apr 16 19:07:48 2006 @@ -209,7 +209,9 @@ fn = self.getcompiled(f) res = fn() assert res == 2 - assert len(self.t.graphs[0].startblock.exits[False].target.operations) == 10 + operations = self.t.graphs[0].startblock.exits[False].target.operations + assert len([op for op in operations if op.opname == "gc_reload_possibly_moved"]) == 0 + def test_framework_safe_pushpop(self): class A(object): pass From arigo at codespeak.net Sun Apr 16 19:37:30 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 16 Apr 2006 19:37:30 +0200 (CEST) Subject: [pypy-svn] r25875 - pypy/dist/pypy/objspace Message-ID: <20060416173730.625D010095@code0.codespeak.net> Author: arigo Date: Sun Apr 16 19:37:29 2006 New Revision: 25875 Modified: pypy/dist/pypy/objspace/trivial.py Log: Rename unwrap() to expose() to avoid confusion. Modified: pypy/dist/pypy/objspace/trivial.py ============================================================================== --- pypy/dist/pypy/objspace/trivial.py (original) +++ pypy/dist/pypy/objspace/trivial.py Sun Apr 16 19:37:29 2006 @@ -93,7 +93,7 @@ for key, value in kwds.items()]) a = Arguments(self, args_w, kwds_w) res = w_func.call_args(a) - return self.unwrap(res) + return self.expose(res) d['__call__'] = function_call T = type(typedef.name, (base,), d) @@ -120,6 +120,15 @@ def unwrap(self, w_obj): if isinstance(w_obj, Wrappable): + return w_obj + else: + return w_obj.obj + + def expose(self, w_obj): + """Turn a W_Root instance into an interp-level object that behaves + on top of CPython in the same way as w_obj behaves on top of PyPy. + """ + if isinstance(w_obj, Wrappable): try: return w_obj.__trivialfacade except AttributeError: @@ -135,7 +144,7 @@ return w_obj.getclass(self) def is_true(self, w_obj): - return bool(self.unwrap(w_obj)) + return bool(self.expose(w_obj)) def check_type(self, x, classes): if not isinstance(x, classes): @@ -144,38 +153,38 @@ raise OperationError(self.w_TypeError, self.wrap(msg)) def str_w(self, w_obj): - x = self.unwrap(w_obj) + x = self.expose(w_obj) self.check_type(x, (str, unicode)) return str(x) def int_w(self, w_obj): - x = self.unwrap(w_obj) + x = self.expose(w_obj) self.check_type(x, (int, long)) return int(x) def uint_w(self, w_obj): - x = self.unwrap(w_obj) + x = self.expose(w_obj) self.check_type(x, (r_uint, int, long)) return r_uint(x) def float_w(self, w_obj): - x = self.unwrap(w_obj) + x = self.expose(w_obj) self.check_type(x, (float, int, long)) return float(x) def is_w(self, w_x, w_y): - return self.unwrap(w_x) is self.unwrap(w_y) + return self.expose(w_x) is self.expose(w_y) def newtuple(self, items_w): - items = [self.unwrap(w_item) for w_item in items_w] + items = [self.expose(w_item) for w_item in items_w] return W_Box(tuple(items)) def newlist(self, items_w): - items = [self.unwrap(w_item) for w_item in items_w] + items = [self.expose(w_item) for w_item in items_w] return W_Box(items) def newdict(self, items_w): - items = [(self.unwrap(w_key), self.unwrap(w_value)) + items = [(self.expose(w_key), self.expose(w_value)) for w_key, w_value in items_w] return W_Box(dict(items)) @@ -188,16 +197,16 @@ return W_Box(u) def newslice(self, w_start, w_stop, w_step): - sl = slice(self.unwrap(w_start), - self.unwrap(w_stop), - self.unwrap(w_step)) + sl = slice(self.expose(w_start), + self.expose(w_stop), + self.expose(w_step)) return W_Box(sl) def call_args(self, w_callable, args): - callable = self.unwrap(w_callable) + callable = self.expose(w_callable) args_w, kwds_w = args.unpack() - args = [self.unwrap(w_value) for w_value in args_w] - kwds = dict([(key, self.unwrap(w_value)) for key, w_value in kwds_w]) + args = [self.expose(w_value) for w_value in args_w] + kwds = dict([(key, self.expose(w_value)) for key, w_value in kwds_w]) try: res = callable(*args, **kwds) except: @@ -222,7 +231,7 @@ def make_function(): operator = FunctionByName[name] def do(self, *args_w): - args = [self.unwrap(a) for a in args_w] + args = [self.expose(a) for a in args_w] try: res = operator(*args) except: From arigo at codespeak.net Mon Apr 17 12:44:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 12:44:52 +0200 (CEST) Subject: [pypy-svn] r25878 - in pypy/dist/pypy/rpython: . rctypes rctypes/test Message-ID: <20060417104452.390121009C@code0.codespeak.net> Author: arigo Date: Mon Apr 17 12:44:45 2006 New Revision: 25878 Modified: pypy/dist/pypy/rpython/rctypes/aarray.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rptr.py Log: 'value' attribute of char arrays. Modified: pypy/dist/pypy/rpython/rctypes/aarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/aarray.py Mon Apr 17 12:44:45 2006 @@ -1,5 +1,5 @@ -from ctypes import ARRAY, c_int -from pypy.annotation.model import SomeCTypesObject, SomeBuiltin +from ctypes import ARRAY, c_int, c_char +from pypy.annotation.model import SomeCTypesObject, SomeBuiltin, SomeString from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype @@ -28,6 +28,12 @@ from pypy.rpython.rctypes.rarray import ArrayRepr return ArrayRepr(rtyper, s_array) -extregistry.register_metatype(ArrayType, +entry = extregistry.register_metatype(ArrayType, compute_annotation=array_instance_compute_annotation, get_repr=arraytype_get_repr) +def char_array_get_field_annotation(s_array, fieldname): + assert fieldname == 'value' + if s_array.knowntype._type_ != c_char: + raise Exception("only arrays of chars have a .value attribute") + return SomeString() # can_be_None = False +entry.get_field_annotation = char_array_get_field_annotation Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Mon Apr 17 12:44:45 2006 @@ -1,5 +1,6 @@ from ctypes import ARRAY, c_int from pypy.rpython.rbuiltin import gen_cast_subarray_pointer +from pypy.rpython.rstr import string_repr from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype @@ -38,6 +39,14 @@ # ByValue case p.c_data[i] = llitem.c_data[0] + def rtype_getattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + assert self.r_item.ll_type == lltype.Char # .value: char arrays only + v_box = hop.inputarg(self, 0) + return hop.gendirectcall(ll_chararrayvalue, v_box) + def get_c_data_of_item(self, llops, v_array, v_index): v_c_array = self.get_c_data(llops, v_array) if isinstance(self.r_item, CTypesRefRepr): @@ -89,3 +98,14 @@ # ByValue case (optimization; the above also works in this case) v_newvalue = r_array.r_item.getvalue(hop.llops, v_item) r_array.set_item_value(hop.llops, v_array, v_index, v_newvalue) + + +def ll_chararrayvalue(box): + from pypy.rpython.rctypes import rchar_p + p = box.c_data + p1 = lltype.cast_subarray_pointer(rchar_p.CCHARP, p, 0) + length = rchar_p.ll_strnlen(p1, len(p)) + newstr = lltype.malloc(string_repr.lowleveltype.TO, length) + for i in range(length): + newstr.chars[i] = p[i] + return newstr Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 17 12:44:45 2006 @@ -89,3 +89,30 @@ p1 = cast(pointer(x), c_void_p) p2 = cast(p1, POINTER(c_int)) assert p2.contents.value == 12 + +def test_char_array(): + a = (c_char * 3)() + a[0] = 'x' + a[1] = 'y' + assert a.value == 'xy' + a[2] = 'z' + assert a.value == 'xyz' + + b = create_string_buffer(3) + assert type(b) is type(a) + + b.value = "nxw" + assert b[0] == 'n' + assert b[1] == 'x' + assert b[2] == 'w' + + b.value = "?" + assert b[0] == '?' + assert b[1] == '\x00' + assert b[2] == 'w' + + class S(Structure): + _fields_ = [('p', POINTER(c_char))] + + s = S() + s.p = b Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Mon Apr 17 12:44:45 2006 @@ -4,6 +4,7 @@ import py.test import pypy.rpython.rctypes.implementation +from pypy.annotation import model as annmodel from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy import conftest @@ -16,7 +17,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int, c_short, ARRAY, POINTER, pointer, c_char_p +from ctypes import c_int, c_short, ARRAY, POINTER, pointer, c_char_p, c_char c_int_10 = ARRAY(c_int,10) @@ -146,6 +147,20 @@ a.translator.view() assert s.knowntype == int + def test_annotate_char_array_value(self): + A = c_char * 3 + def func(): + a = A() + a[0] = 'x' + a[1] = 'y' + return a.value + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(func, []) + if conftest.option.view: + a.translator.view() + assert s == annmodel.SomeString() + class Test_specialization: def test_specialize_array(self): def create_array(): @@ -190,6 +205,16 @@ res = interpret(func, []) assert res == expected + def test_specialize_char_array_value(self): + A = c_char * 3 + def func(): + a = A() + a[0] = 'x' + a[1] = 'y' + return a.value + res = interpret(func, []) + assert ''.join(res.chars) == "xy" + class Test_compilation: def test_compile_array_access(self): def access_array(): @@ -221,3 +246,13 @@ func, expected = maketest() fn = compile(func, []) assert fn() == expected + + def test_compile_char_array_value(self): + A = c_char * 3 + def func(): + a = A() + a[0] = 'x' + a[1] = 'y' + return a.value + fn = compile(func, []) + assert fn() == "xy" Modified: pypy/dist/pypy/rpython/rptr.py ============================================================================== --- pypy/dist/pypy/rpython/rptr.py (original) +++ pypy/dist/pypy/rpython/rptr.py Mon Apr 17 12:44:45 2006 @@ -2,7 +2,7 @@ from pypy.annotation import model as annmodel from pypy.objspace.flow import model as flowmodel from pypy.rpython.lltypesystem.lltype import \ - Ptr, ContainerType, Void, Signed, Bool, FuncType, typeOf + Ptr, ContainerType, Void, Signed, Bool, FuncType, typeOf, FixedSizeArray from pypy.rpython.error import TyperError from pypy.rpython.rmodel import Repr, IntegerRepr @@ -47,9 +47,13 @@ hop.genop('setfield', vlist) def rtype_len(self, hop): - vlist = hop.inputargs(self) - return hop.genop('getarraysize', vlist, - resulttype = hop.r_result.lowleveltype) + ARRAY = hop.args_r[0].lowleveltype.TO + if isinstance(ARRAY, FixedSizeArray): + return hop.inputconst(Signed, ARRAY.length) + else: + vlist = hop.inputargs(self) + return hop.genop('getarraysize', vlist, + resulttype = hop.r_result.lowleveltype) def rtype_is_true(self, hop): vlist = hop.inputargs(self) From arigo at codespeak.net Mon Apr 17 12:59:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 12:59:48 +0200 (CEST) Subject: [pypy-svn] r25879 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060417105948.D6E1410093@code0.codespeak.net> Author: arigo Date: Mon Apr 17 12:59:47 2006 New Revision: 25879 Added: pypy/dist/pypy/rpython/rctypes/astringbuf.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py Log: create_string_buffer() needs special support to be able to return arrays of characters of variable size. Basic annotation done. Added: pypy/dist/pypy/rpython/rctypes/astringbuf.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/astringbuf.py Mon Apr 17 12:59:47 2006 @@ -0,0 +1,24 @@ +from pypy.rpython import extregistry +from pypy.annotation import model as annmodel + +from ctypes import create_string_buffer, c_char + + +class StringBufferType(object): + """Placeholder for the result type of create_string_buffer(), + which cannot be represented as a regular ctypes type because + the length is not an annotation-time constant. + """ + _type_ = c_char + #_length_ = unspecified + + +def stringbuf_compute_result_annotation(s_length): + if s_length.knowntype != int: + raise Exception("rctypes only supports create_string_buffer(length)") + return annmodel.SomeCTypesObject(StringBufferType, + annmodel.SomeCTypesObject.OWNSMEMORY) + +extregistry.register_value(create_string_buffer, + compute_result_annotation=stringbuf_compute_result_annotation, + ) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Mon Apr 17 12:59:47 2006 @@ -6,6 +6,7 @@ import pypy.rpython.rctypes.achar_p import pypy.rpython.rctypes.astruct import pypy.rpython.rctypes.avoid_p +import pypy.rpython.rctypes.astringbuf # Register the correspondance between SomeCTypesObject and the get_repr() Added: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Mon Apr 17 12:59:47 2006 @@ -0,0 +1,48 @@ +""" +Test the rctypes implementation. +""" + +import py.test +import pypy.rpython.rctypes.implementation +from pypy.annotation import model as annmodel +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy import conftest +from pypy.translator.c.test.test_genc import compile +import sys +from pypy.rpython.test.test_llinterp import interpret + +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + +from ctypes import create_string_buffer +from pypy.rpython.rctypes.astringbuf import StringBufferType + + +class Test_annotation: + def test_annotate_create(self): + def func(n): + return create_string_buffer(n) + + a = RPythonAnnotator() + s = a.build_types(func, [int]) + assert s.knowntype == StringBufferType + + if conftest.option.view: + a.translator.view() + + def test_annotate_access(self): + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + return buf[0] + + a = RPythonAnnotator() + s = a.build_types(func, [int]) + assert s == annmodel.SomeChar() + + if conftest.option.view: + a.translator.view() From arigo at codespeak.net Mon Apr 17 13:40:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 13:40:20 +0200 (CEST) Subject: [pypy-svn] r25880 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060417114020.0790F10093@code0.codespeak.net> Author: arigo Date: Mon Apr 17 13:40:19 2006 New Revision: 25880 Added: pypy/dist/pypy/rpython/rctypes/rstringbuf.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Log: RTyping string buffers. Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/astringbuf.py Mon Apr 17 13:40:19 2006 @@ -19,6 +19,24 @@ return annmodel.SomeCTypesObject(StringBufferType, annmodel.SomeCTypesObject.OWNSMEMORY) +def stringbuf_specialize_call(hop): + from pypy.rpython.lltypesystem import lltype + [v_length] = hop.inputargs(lltype.Signed) + r_stringbuf = hop.r_result + return hop.genop("malloc_varsize", [ + hop.inputconst(lltype.Void, r_stringbuf.lowleveltype.TO), + v_length, + ], resulttype=r_stringbuf.lowleveltype, + ) + extregistry.register_value(create_string_buffer, compute_result_annotation=stringbuf_compute_result_annotation, + specialize_call=stringbuf_specialize_call, ) + +def stringbuf_get_repr(rtyper, s_stringbuf): + from pypy.rpython.rctypes import rstringbuf + return rstringbuf.StringBufRepr(rtyper, s_stringbuf, rstringbuf.STRBUFTYPE) + +extregistry.register_type(StringBufferType, + get_repr=stringbuf_get_repr) Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Mon Apr 17 13:40:19 2006 @@ -51,21 +51,20 @@ if self.ownsmemory: self.r_memoryowner = self - self.lowleveltype = lltype.Ptr( - lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data", self.c_data_type ), - *content_keepalives - ) - ) + fields = content_keepalives + [ + ( "c_data", self.c_data_type ), + ] else: s_memoryowner = SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) self.r_memoryowner = rtyper.getrepr(s_memoryowner) - self.lowleveltype = lltype.Ptr( + fields = content_keepalives + [ + ( "c_data_owner_keepalive", self.r_memoryowner.lowleveltype ), + ( "c_data", lltype.Ptr(self.c_data_type) ), + ] + self.lowleveltype = lltype.Ptr( lltype.GcStruct( "CtypesBox_%s" % (ctype.__name__,), - ( "c_data", lltype.Ptr(self.c_data_type) ), - ( "c_data_owner_keepalive", self.r_memoryowner.lowleveltype ), - *content_keepalives + *fields ) ) self.const_cache = {} # store generated const values+original value Added: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Mon Apr 17 13:40:19 2006 @@ -0,0 +1,9 @@ +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.rctypes.rmodel import CTypesRefRepr + + +class StringBufRepr(CTypesRefRepr): + pass + + +STRBUFTYPE = lltype.Array(lltype.Char) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Mon Apr 17 13:40:19 2006 @@ -46,3 +46,15 @@ if conftest.option.view: a.translator.view() + +class Test_specialization: + def test_specialize_create(self): + def func(n): + return create_string_buffer(n) + + res = interpret(func, [17]) + c_data = res.c_data + assert c_data[0] == '\x00' + assert c_data[16] == '\x00' + assert len(c_data) == 17 + py.test.raises(IndexError, "c_data[17]") From arigo at codespeak.net Mon Apr 17 13:52:27 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 13:52:27 +0200 (CEST) Subject: [pypy-svn] r25881 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060417115227.2AD901008F@code0.codespeak.net> Author: arigo Date: Mon Apr 17 13:52:25 2006 New Revision: 25881 Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py pypy/dist/pypy/rpython/rctypes/rstringbuf.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Log: Operations on string buffers. Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/astringbuf.py Mon Apr 17 13:52:25 2006 @@ -38,5 +38,9 @@ from pypy.rpython.rctypes import rstringbuf return rstringbuf.StringBufRepr(rtyper, s_stringbuf, rstringbuf.STRBUFTYPE) -extregistry.register_type(StringBufferType, +entry = extregistry.register_type(StringBufferType, get_repr=stringbuf_get_repr) +def stringbuf_get_field_annotation(s_array, fieldname): + assert fieldname == 'value' + return annmodel.SomeString() # can_be_None = False +entry.get_field_annotation = stringbuf_get_field_annotation Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Mon Apr 17 13:52:25 2006 @@ -1,9 +1,39 @@ from pypy.rpython.lltypesystem import lltype +from pypy.annotation.pairtype import pairtype +from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.rctypes.rmodel import CTypesRefRepr class StringBufRepr(CTypesRefRepr): - pass + + def rtype_len(self, hop): + [v_stringbuf] = hop.inputargs(self) + v_array = self.get_c_data(hop.llops, v_stringbuf) + return hop.genop('getarraysize', [v_array], + resulttype = lltype.Signed) + + def rtype_getattr(self, hop): + from pypy.rpython.rctypes.rarray import ll_chararrayvalue + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + v_box = hop.inputarg(self, 0) + return hop.gendirectcall(ll_chararrayvalue, v_box) + + +class __extend__(pairtype(StringBufRepr, IntegerRepr)): + def rtype_getitem((r_stringbuf, r_int), hop): + v_stringbuf, v_index = hop.inputargs(r_stringbuf, lltype.Signed) + v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) + return hop.genop('getarrayitem', [v_array, v_index], + resulttype = lltype.Char) + + def rtype_setitem((r_stringbuf, r_int), hop): + v_stringbuf, v_index, v_item = hop.inputargs(r_stringbuf, + lltype.Signed, + lltype.Char) + v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) + hop.genop('setarrayitem', [v_array, v_index, v_item]) STRBUFTYPE = lltype.Array(lltype.Char) Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 17 13:52:25 2006 @@ -100,6 +100,7 @@ b = create_string_buffer(3) assert type(b) is type(a) + assert len(b) == 3 b.value = "nxw" assert b[0] == 'n' @@ -116,3 +117,5 @@ s = S() s.p = b + + assert len(create_string_buffer(0)) == 0 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Mon Apr 17 13:52:25 2006 @@ -47,6 +47,18 @@ if conftest.option.view: a.translator.view() + def test_annotate_len(self): + def func(n): + buf = create_string_buffer(n) + return len(buf) + + a = RPythonAnnotator() + s = a.build_types(func, [int]) + assert s.knowntype == int + + if conftest.option.view: + a.translator.view() + class Test_specialization: def test_specialize_create(self): def func(n): @@ -58,3 +70,33 @@ assert c_data[16] == '\x00' assert len(c_data) == 17 py.test.raises(IndexError, "c_data[17]") + + def test_specialize_access(self): + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + return buf[0] + + res = interpret(func, [17]) + assert res == 'x' + + def test_specialize_len(self): + def func(n): + buf = create_string_buffer(n) + return len(buf) + + res = interpret(func, [17]) + assert res == 17 + res = interpret(func, [0]) + assert res == 0 + + def test_annotate_value(self): + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + return buf.value + + res = interpret(func, [12]) + assert ''.join(res.chars) == "xy" From arigo at codespeak.net Mon Apr 17 14:09:56 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 14:09:56 +0200 (CEST) Subject: [pypy-svn] r25885 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060417120956.18908100AD@code0.codespeak.net> Author: arigo Date: Mon Apr 17 14:09:54 2006 New Revision: 25885 Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rstringbuf.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: Automatic cast from a string buffer to c_char_p function arguments. Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Mon Apr 17 14:09:54 2006 @@ -3,6 +3,7 @@ from pypy.rpython.rstr import StringRepr, string_repr from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO from pypy.rpython.rctypes.rarray import ArrayRepr +from pypy.rpython.rctypes.rstringbuf import StringBufRepr from pypy.annotation.pairtype import pairtype from ctypes import c_char, c_char_p @@ -75,6 +76,17 @@ r_temp.setvalue(llops, v_owned_box, v_c_array) return llops.convertvar(v_owned_box, r_temp, r_to) +class __extend__(pairtype(StringBufRepr, CCharPRepr)): + def convert_from_to((r_from, r_to), v, llops): + # warning: no keepalives, only for short-lived conversions like + # in argument passing + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + v_c_array = r_from.get_c_data_of_item(llops, v, C_ZERO) + r_temp.setvalue(llops, v_owned_box, v_c_array) + return llops.convertvar(v_owned_box, r_temp, r_to) + # XXX some code duplication above + CCHARP = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Mon Apr 17 14:09:54 2006 @@ -1,5 +1,6 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype +from pypy.rpython.rbuiltin import gen_cast_subarray_pointer from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.rctypes.rmodel import CTypesRefRepr @@ -20,6 +21,13 @@ v_box = hop.inputarg(self, 0) return hop.gendirectcall(ll_chararrayvalue, v_box) + def get_c_data_of_item(self, llops, v_stringbuf, v_index): + v_array = self.get_c_data(llops, v_stringbuf) + return gen_cast_subarray_pointer(llops, ONE_CHAR_PTR, + v_array, v_index) + +ONE_CHAR_PTR = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) + class __extend__(pairtype(StringBufRepr, IntegerRepr)): def rtype_getitem((r_stringbuf, r_int), hop): Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 17 14:09:54 2006 @@ -117,5 +117,7 @@ s = S() s.p = b + s.p.contents.value = '!' + assert b.value == '!' assert len(create_string_buffer(0)) == 0 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Mon Apr 17 14:09:54 2006 @@ -38,6 +38,10 @@ return res return func, 1234 * ord('e') +def test_base(): + func, expected = maketest() + assert func() == expected + class Test_annotation: def test_annotate_array(self): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Mon Apr 17 14:09:54 2006 @@ -9,7 +9,7 @@ from pypy.rpython.lltypesystem import lltype from ctypes import cdll -from ctypes import c_int, c_long, c_char_p, c_char +from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer # __________ the standard C library __________ @@ -122,6 +122,17 @@ res = [interpret(fn, [i]) for i in range(4)] assert res == [0, 42, 0, 18238] + def test_specialize_atoi_stringbuf(self): + def fn(n): + buf = create_string_buffer(n) + buf[0] = '4' + buf[1] = '2' + return atoi(buf) + + assert fn(11) == 42 + res = interpret(fn, [11]) + assert res == 42 + class Test_compile: def test_compile_labs(self): fn = compile(test_labs, [int]) From antocuni at codespeak.net Mon Apr 17 14:14:17 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 17 Apr 2006 14:14:17 +0200 (CEST) Subject: [pypy-svn] r25886 - in pypy/dist/pypy: rpython rpython/lltypesystem rpython/ootypesystem rpython/ootypesystem/test rpython/test translator/cli/src Message-ID: <20060417121417.D7E8E100AD@code0.codespeak.net> Author: antocuni Date: Mon Apr 17 14:13:43 2006 New Revision: 25886 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/translator/cli/src/pypylib.cs Log: ootypesystem's lists and lltypesystem's lists have been unified as much as possible. Now they share almost all ll_* helper functions that have been placed in rpython/rlist.py. Two ADT methods have been added both to lltypesystem's ListRepr and ootypesystem's ListRepr: _ll_resize_ge and _ll_resize_le. The CLI backend has been modified to reflect the changes to the interface. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Mon Apr 17 14:13:43 2006 @@ -5,12 +5,13 @@ from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.rmodel import externalvsinternal from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ - AbstractFixedSizeListRepr, AbstractListIteratorRepr, rtype_newlist + AbstractFixedSizeListRepr, AbstractListIteratorRepr, rtype_newlist, \ + ll_setitem_nonneg from pypy.rpython.rlist import dum_nocheck, dum_checkidx from pypy.rpython.lltypesystem.rslice import SliceRepr from pypy.rpython.lltypesystem.rslice import startstop_slice_repr, startonly_slice_repr from pypy.rpython.lltypesystem.rslice import minusone_slice_repr -from pypy.rpython.lltypesystem. lltype import \ +from pypy.rpython.lltypesystem.lltype import \ GcForwardReference, Ptr, GcArray, GcStruct, \ Void, Signed, malloc, typeOf, Primitive, \ Bool, nullptr, typeMethod @@ -49,15 +50,15 @@ self.listitem = listitem self.list_cache = {} # setup() needs to be called to finish this initialization - self.ll_concat = ll_concat - self.ll_extend = ll_extend - self.ll_listslice_startonly = ll_listslice_startonly - self.ll_listslice = ll_listslice - self.ll_listslice_minusone = ll_listslice_minusone - self.ll_listsetslice = ll_listsetslice - self.ll_listdelslice_startonly = ll_listdelslice_startonly - self.ll_listdelslice = ll_listdelslice - self.ll_listindex = ll_listindex +## self.ll_concat = ll_concat +## self.ll_extend = ll_extend +## self.ll_listslice_startonly = ll_listslice_startonly +## self.ll_listslice = ll_listslice +## self.ll_listslice_minusone = ll_listslice_minusone +## self.ll_listsetslice = ll_listsetslice +## self.ll_listdelslice_startonly = ll_listdelslice_startonly +## self.ll_listdelslice = ll_listdelslice +## self.ll_listindex = ll_listindex self.list_builder = ListBuilder() def _setup_repr_final(self): @@ -92,24 +93,6 @@ def get_eqfunc(self): return inputconst(Void, self.item_repr.get_ll_eq_function()) - def rtype_bltn_list(self, hop): - v_lst = hop.inputarg(self, 0) - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - return hop.gendirectcall(ll_copy, cRESLIST, v_lst) - - def rtype_len(self, hop): - v_lst, = hop.inputargs(self) - return hop.gendirectcall(ll_len, v_lst) - - def rtype_is_true(self, hop): - v_lst, = hop.inputargs(self) - return hop.gendirectcall(ll_list_is_true, v_lst) - - def rtype_method_reverse(self, hop): - v_lst, = hop.inputargs(self) - hop.exception_cannot_occur() - hop.gendirectcall(ll_reverse,v_lst) - def make_iterator_repr(self): return ListIteratorRepr(self) @@ -206,6 +189,8 @@ "ITEM": ITEM, "ll_getitem_fast": ll_getitem_fast, "ll_setitem_fast": ll_setitem_fast, + "_ll_resize_ge": _ll_list_resize_ge, + "_ll_resize_le": _ll_list_resize_le, }) ) @@ -218,53 +203,6 @@ result.items = malloc(self.LIST.items.TO, n) return result - def rtype_method_append(self, hop): - v_lst, v_value = hop.inputargs(self, self.item_repr) - hop.exception_cannot_occur() - hop.gendirectcall(ll_append, v_lst, v_value) - - def rtype_method_insert(self, hop): - v_lst, v_index, v_value = hop.inputargs(self, Signed, self.item_repr) - arg1 = hop.args_s[1] - args = v_lst, v_index, v_value - if arg1.is_constant() and arg1.const == 0: - llfn = ll_prepend - args = v_lst, v_value - elif arg1.nonneg: - llfn = ll_insert_nonneg - else: - raise TyperError("insert() index must be proven non-negative") - hop.exception_cannot_occur() - hop.gendirectcall(llfn, *args) - - def rtype_method_extend(self, hop): - v_lst1, v_lst2 = hop.inputargs(*hop.args_r) - hop.exception_cannot_occur() - hop.gendirectcall(ll_extend, v_lst1, v_lst2) - - def rtype_method_pop(self, hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - if hop.nb_args == 2: - args = hop.inputargs(self, Signed) - assert hasattr(args[1], 'concretetype') - arg1 = hop.args_s[1] - if arg1.is_constant() and arg1.const == 0: - llfn = ll_pop_zero - args = args[:1] - elif hop.args_s[1].nonneg: - llfn = ll_pop_nonneg - else: - llfn = ll_pop - else: - args = hop.inputargs(self) - llfn = ll_pop_default - hop.exception_is_here() - v_res = hop.gendirectcall(llfn, v_func, *args) - return self.recast(hop.llops, v_res) class FixedSizeListRepr(AbstractFixedSizeListRepr, BaseListRepr): @@ -281,7 +219,9 @@ "list_builder": self.list_builder, "ITEM": ITEM, "ll_getitem_fast": ll_fixed_getitem_fast, - "ll_setitem_fast": ll_fixed_setitem_fast + "ll_setitem_fast": ll_fixed_setitem_fast, + "_ll_resize_ge": _ll_list_resize_ge, + "_ll_resize_le": _ll_list_resize_le, }) self.LIST.become(ITEMARRAY) @@ -293,99 +233,6 @@ result = malloc(self.LIST, n, immortal=True) return result -class __extend__(pairtype(BaseListRepr, Repr)): - - def rtype_contains((r_lst, _), hop): - v_lst, v_any = hop.inputargs(r_lst, r_lst.item_repr) - return hop.gendirectcall(ll_listcontains, v_lst, v_any, r_lst.get_eqfunc()) - - -class __extend__(pairtype(BaseListRepr, IntegerRepr)): - - def rtype_getitem((r_lst, r_int), hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index = hop.inputargs(r_lst, Signed) - if hop.args_s[1].nonneg: - llfn = ll_getitem_nonneg - else: - llfn = ll_getitem - hop.exception_is_here() - v_res = hop.gendirectcall(llfn, v_func, v_lst, v_index) - return r_lst.recast(hop.llops, v_res) - - def rtype_setitem((r_lst, r_int), hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index, v_item = hop.inputargs(r_lst, Signed, r_lst.item_repr) - if hop.args_s[1].nonneg: - llfn = ll_setitem_nonneg - else: - llfn = ll_setitem - hop.exception_is_here() - return hop.gendirectcall(llfn, v_func, v_lst, v_index, v_item) - - def rtype_mul((r_lst, r_int), hop): - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - v_lst, v_factor = hop.inputargs(r_lst, Signed) - return hop.gendirectcall(ll_mul, cRESLIST, v_lst, v_factor) - -class __extend__(pairtype(ListRepr, IntegerRepr)): - - def rtype_delitem((r_lst, r_int), hop): - if hop.has_implicit_exception(IndexError): - spec = dum_checkidx - else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index = hop.inputargs(r_lst, Signed) - if hop.args_s[1].nonneg: - llfn = ll_delitem_nonneg - else: - llfn = ll_delitem - hop.exception_is_here() - return hop.gendirectcall(llfn, v_func, v_lst, v_index) - - def rtype_inplace_mul((r_lst, r_int), hop): - v_lst, v_factor = hop.inputargs(r_lst, Signed) - return hop.gendirectcall(ll_inplace_mul, v_lst, v_factor) - - -class __extend__(pairtype(BaseListRepr, BaseListRepr)): - def convert_from_to((r_lst1, r_lst2), v, llops): - if r_lst1.listitem is None or r_lst2.listitem is None: - return NotImplemented - if r_lst1.listitem is not r_lst2.listitem: - return NotImplemented - return v - - def rtype_is_((r_lst1, r_lst2), hop): - if r_lst1.lowleveltype != r_lst2.lowleveltype: - # obscure logic, the is can be true only if both are None - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - return hop.gendirectcall(ll_both_none, v_lst1, v_lst2) - - return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) - - def rtype_eq((r_lst1, r_lst2), hop): - assert r_lst1.item_repr == r_lst2.item_repr - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - return hop.gendirectcall(ll_listeq, v_lst1, v_lst2, r_lst1.get_eqfunc()) - - def rtype_ne((r_lst1, r_lst2), hop): - assert r_lst1.item_repr == r_lst2.item_repr - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - flag = hop.gendirectcall(ll_listeq, v_lst1, v_lst2, r_lst1.get_eqfunc()) - return hop.genop('bool_not', [flag], resulttype=Bool) - -def ll_both_none(lst1, lst2): - return not lst1 and not lst2 # ____________________________________________________________ # @@ -468,370 +315,9 @@ _ll_list_resize_really(l, newsize) -def ll_copy(RESLIST, l): - length = l.ll_length() - new_lst = RESLIST.ll_newlist(length) - i = 0 - while i < length: - new_lst.ll_setitem_fast(i, l.ll_getitem_fast(i)) - i += 1 - return new_lst -ll_copy.oopspec = 'list.copy(l)' - -def ll_len(l): - return l.ll_length() -ll_len.oopspec = 'list.len(l)' - -def ll_list_is_true(l): - # check if a list is True, allowing for None - return bool(l) and l.ll_length() != 0 -ll_list_is_true.oopspec = 'list.nonzero(l)' - -def ll_append(l, newitem): - length = l.ll_length() - _ll_list_resize_ge(l, length+1) - l.ll_setitem_fast(length, newitem) -ll_append.oopspec = 'list.append(l, newitem)' - -# this one is for the special case of insert(0, x) -def ll_prepend(l, newitem): - length = l.ll_length() - _ll_list_resize_ge(l, length+1) - dst = length - while dst > 0: - src = dst - 1 - l.ll_setitem_fast(dst, l.ll_getitem_fast(src)) - dst = src - l.ll_setitem_fast(0, newitem) -ll_prepend.oopspec = 'list.insert(l, 0, newitem)' - -def ll_insert_nonneg(l, index, newitem): - length = l.ll_length() - _ll_list_resize_ge(l, length+1) - dst = length - while dst > index: - src = dst - 1 - l.ll_setitem_fast(dst, l.ll_getitem_fast(src)) - dst = src - l.ll_setitem_fast(index, newitem) -ll_insert_nonneg.oopspec = 'list.insert(l, index, newitem)' - -def ll_pop_nonneg(func, l, index): - if func is dum_checkidx and (index >= l.ll_length()): - raise IndexError - res = l.ll_getitem_fast(index) - ll_delitem_nonneg(dum_nocheck, l, index) - return res -ll_pop_nonneg.oopspec = 'list.pop(l, index)' - -def ll_pop_default(func, l): - length = l.ll_length() - if func is dum_checkidx and (length == 0): - raise IndexError - index = length - 1 - newlength = index - res = l.ll_getitem_fast(index) - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - l.ll_setitem_fast(index, nullptr(ITEM.TO)) - _ll_list_resize_le(l, newlength) - return res -ll_pop_default.oopspec = 'list.pop(l)' - -def ll_pop_zero(func, l): - length = l.ll_length() - if func is dum_checkidx and (length == 0): - raise IndexError - newlength = length - 1 - res = l.ll_getitem_fast(0) - j = 0 - j1 = j+1 - while j < newlength: - l.ll_setitem_fast(j, l.ll_getitem_fast(j1)) - j = j1 - j1 += 1 - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - l.ll_setitem_fast(newlength, nullptr(ITEM.TO)) - _ll_list_resize_le(l, newlength) - return res -ll_pop_zero.oopspec = 'list.pop(l, 0)' - -def ll_pop(func, l, index): - length = l.ll_length() - if index < 0: - index += length - if func is dum_checkidx and (index < 0 or index >= length): - raise IndexError - res = l.ll_getitem_fast(index) - ll_delitem_nonneg(dum_nocheck, l, index) - return res -ll_pop.oopspec = 'list.pop(l, index)' - -def ll_reverse(l): - length = l.ll_length() - i = 0 - length_1_i = length-1-i - while i < length_1_i: - tmp = l.ll_getitem_fast(i) - l.ll_setitem_fast(i, l.ll_getitem_fast(length_1_i)) - l.ll_setitem_fast(length_1_i, tmp) - i += 1 - length_1_i -= 1 -ll_reverse.oopspec = 'list.reverse(l)' - -def ll_getitem_nonneg(func, l, index): - if func is dum_checkidx and (index >= l.ll_length()): - raise IndexError - return l.ll_getitem_fast(index) -ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' - -def ll_getitem(func, l, index): - length = l.ll_length() - if index < 0: - index += length - if func is dum_checkidx and (index < 0 or index >= length): - raise IndexError - return l.ll_getitem_fast(index) -ll_getitem.oopspec = 'list.getitem(l, index)' - -def ll_setitem_nonneg(func, l, index, newitem): - if func is dum_checkidx and (index >= l.ll_length()): - raise IndexError - l.ll_setitem_fast(index, newitem) -ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' - -def ll_setitem(func, l, index, newitem): - length = l.ll_length() - if index < 0: - index += length - if func is dum_checkidx and (index < 0 or index >= length): - raise IndexError - l.ll_setitem_fast(index, newitem) -ll_setitem.oopspec = 'list.setitem(l, index, newitem)' - -def ll_delitem_nonneg(func, l, index): - length = l.ll_length() - if func is dum_checkidx and (index >= length): - raise IndexError - newlength = length - 1 - j = index - j1 = j+1 - while j < newlength: - l.ll_setitem_fast(j, l.ll_getitem_fast(j1)) - j = j1 - j1 += 1 - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - l.ll_setitem_fast(newlength, nullptr(ITEM.TO)) - _ll_list_resize_le(l, newlength) -ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' - -def ll_delitem(func, l, i): - length = l.ll_length() - if i < 0: - i += length - if func is dum_checkidx and (i < 0 or i >= length): - raise IndexError - ll_delitem_nonneg(dum_nocheck, l, i) -ll_delitem.oopspec = 'list.delitem(l, i)' - -def ll_concat(RESLIST, l1, l2): - len1 = l1.ll_length() - len2 = l2.ll_length() - newlength = len1 + len2 - l = RESLIST.ll_newlist(newlength) - j = 0 - while j < len1: - l.ll_setitem_fast(j, l1.ll_getitem_fast(j)) - j += 1 - i = 0 - while i < len2: - l.ll_setitem_fast(j, l2.ll_getitem_fast(i)) - i += 1 - j += 1 - return l -ll_concat.oopspec = 'list.concat(l1, l2)' - -def ll_extend(l1, l2): - len1 = l1.ll_length() - len2 = l2.ll_length() - newlength = len1 + len2 - _ll_list_resize_ge(l1, newlength) - i = 0 - j = len1 - while i < len2: - l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) - i += 1 - j += 1 - -def ll_listslice_startonly(RESLIST, l1, start): - len1 = l1.ll_length() - newlength = len1 - start - l = RESLIST.ll_newlist(newlength) - j = 0 - i = start - while i < len1: - l.ll_setitem_fast(j, l1.ll_getitem_fast(i)) - i += 1 - j += 1 - return l - -def ll_listslice(RESLIST, l1, slice): - start = slice.start - stop = slice.stop - length = l1.ll_length() - if stop > length: - stop = length - newlength = stop - start - l = RESLIST.ll_newlist(newlength) - j = 0 - i = start - while i < stop: - l.ll_setitem_fast(j, l1.ll_getitem_fast(i)) - i += 1 - j += 1 - return l - -def ll_listslice_minusone(RESLIST, l1): - newlength = l1.ll_length() - 1 - assert newlength >= 0 - l = RESLIST.ll_newlist(newlength) - j = 0 - while j < newlength: - l.ll_setitem_fast(j, l1.ll_getitem_fast(j)) - j += 1 - return l - -def ll_listdelslice_startonly(l, start): - newlength = start - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - j = l.ll_length() - 1 - while j >= newlength: - l.ll_setitem_fast(j, nullptr(ITEM.TO)) - j -= 1 - _ll_list_resize_le(l, newlength) - -def ll_listdelslice(l, slice): - start = slice.start - stop = slice.stop - length = l.ll_length() - if stop > length: - stop = length - newlength = length - (stop-start) - j = start - i = stop - while j < newlength: - l.ll_setitem_fast(j, l.ll_getitem_fast(i)) - i += 1 - j += 1 - ITEM = typeOf(l).TO.ITEM - if isinstance(ITEM, Ptr): - j = length - 1 - while j >= newlength: - l.ll_setitem_fast(j, nullptr(ITEM.TO)) - j -= 1 - _ll_list_resize_le(l, newlength) - -def ll_listsetslice(l1, slice, l2): - count = l2.ll_length() - assert count == slice.stop - slice.start, ( - "setslice cannot resize lists in RPython") - # XXX but it should be easy enough to support, soon - start = slice.start - j = start - i = 0 - while i < count: - l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) - i += 1 - j += 1 - -# ____________________________________________________________ -# -# Comparison. - -def ll_listeq(l1, l2, eqfn): - if not l1 and not l2: - return True - if not l1 or not l2: - return False - len1 = l1.ll_length() - len2 = l2.ll_length() - if len1 != len2: - return False - j = 0 - while j < len1: - if eqfn is None: - if l1.ll_getitem_fast(j) != l2.ll_getitem_fast(j): - return False - else: - if not eqfn(l1.ll_getitem_fast(j), l2.ll_getitem_fast(j)): - return False - j += 1 - return True - -def ll_listcontains(lst, obj, eqfn): - lng = lst.ll_length() - j = 0 - while j < lng: - if eqfn is None: - if lst.ll_getitem_fast(j) == obj: - return True - else: - if eqfn(lst.ll_getitem_fast(j), obj): - return True - j += 1 - return False - -def ll_listindex(lst, obj, eqfn): - lng = lst.ll_length() - j = 0 - while j < lng: - if eqfn is None: - if lst.ll_getitem_fast(j) == obj: - return j - else: - if eqfn(lst.ll_getitem_fast(j), obj): - return j - j += 1 - raise ValueError # can't say 'list.index(x): x not in list' TEMP = GcArray(Ptr(rstr.STR)) -def ll_inplace_mul(l, factor): - length = l.ll_length() - if factor < 0: - factor = 0 - resultlen = length * factor - res = l - _ll_list_resize(res, resultlen) - j = length - while j < resultlen: - i = 0 - while i < length: - p = j + i - res.ll_setitem_fast(p, l.ll_getitem_fast(i)) - i += 1 - j += length - return res - - -def ll_mul(RESLIST, l, factor): - length = l.ll_length() - if factor < 0: - factor = 0 - resultlen = length * factor - res = RESLIST.ll_newlist(resultlen) - j = 0 - while j < resultlen: - i = 0 - while i < length: - p = j + i - res.ll_setitem_fast(p, l.ll_getitem_fast(i)) - i += 1 - j += length - return res # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 17 14:13:43 2006 @@ -206,6 +206,8 @@ "ll_length": Meth([], Signed), "ll_getitem_fast": Meth([Signed], self.ITEMTYPE_T), "ll_setitem_fast": Meth([Signed, self.ITEMTYPE_T], Void), + "_ll_resize_ge": Meth([Signed], Void), + "_ll_resize_le": Meth([Signed], Void), "append": Meth([self.ITEMTYPE_T], Void), "extend": Meth([self.SELFTYPE_T], Void), "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) @@ -215,8 +217,10 @@ # this is the equivalent of the lltypesystem ll_newlist that is # marked as typeMethod. - def ll_newlist(self): - return new(self) + def ll_newlist(self, length): + lst = new(self) + lst._ll_resize_ge(length) + return lst def _setup_methods(self, generic_types): methods = {} @@ -522,7 +526,7 @@ class _list(object): def __init__(self, LIST): - self._TYPE = LIST + self._TYPE = LIST self._list = [] def __getattribute__(self, name): @@ -541,6 +545,19 @@ # NOT_RPYTHON return len(self._list) + def _ll_resize_ge(self, length): + # NOT_RPYTHON + if len(self._list) < length: + diff = length - len(self._list) + self._list += [self._TYPE._ITEMTYPE._defl()] * diff + assert len(self._list) >= length + + def _ll_resize_le(self, length): + # NOT_RPYTHON + if length < len(self._list): + del self._list[length:] + assert len(self._list) <= length + def append(self, item): # NOT_RPYTHON assert typeOf(item) == self._TYPE._ITEMTYPE Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Mon Apr 17 14:13:43 2006 @@ -25,15 +25,15 @@ self.lowleveltype = self.LIST self.listitem = listitem self.list_cache = {} - self.ll_concat = ll_concat - self.ll_extend = ll_extend - self.ll_listslice_startonly = ll_listslice_startonly - self.ll_listslice = ll_listslice - self.ll_listslice_minusone = ll_listslice_minusone - self.ll_listsetslice = ll_listsetslice - self.ll_listdelslice_startonly = ll_listdelslice_startonly - self.ll_listdelslice = ll_listdelslice - self.ll_listindex = ll_listindex +## self.ll_concat = ll_concat +## self.ll_extend = ll_extend +## self.ll_listslice_startonly = ll_listslice_startonly +## self.ll_listslice = ll_listslice +## self.ll_listslice_minusone = ll_listslice_minusone +## self.ll_listsetslice = ll_listsetslice +## self.ll_listdelslice_startonly = ll_listdelslice_startonly +## self.ll_listdelslice = ll_listdelslice +## self.ll_listindex = ll_listindex # setup() needs to be called to finish this initialization def _setup_repr(self): @@ -55,24 +55,24 @@ def get_eqfunc(self): return inputconst(Void, self.item_repr.get_ll_eq_function()) - def rtype_len(self, hop): - return self.send_message(hop, "ll_length") +## def rtype_len(self, hop): +## return self.send_message(hop, "ll_length") - def rtype_is_true(self, hop): - v_lst, = hop.inputargs(self) - return hop.gendirectcall(ll_list_is_true, v_lst) - - def rtype_bltn_list(self, hop): - v_lst = hop.inputarg(self, 0) - c_start = hop.inputconst(Signed, 0) - cRESLIST = hop.inputconst(Void, hop.r_result.LIST) - return hop.gendirectcall(self.ll_listslice_startonly, cRESLIST, v_lst, c_start) +## def rtype_is_true(self, hop): +## v_lst, = hop.inputargs(self) +## return hop.gendirectcall(ll_list_is_true, v_lst) + +## def rtype_bltn_list(self, hop): +## v_lst = hop.inputarg(self, 0) +## c_start = hop.inputconst(Signed, 0) +## cRESLIST = hop.inputconst(Void, hop.r_result.LIST) +## return hop.gendirectcall(self.ll_listslice_startonly, cRESLIST, v_lst, c_start) - def rtype_method_append(self, hop): - return self.send_message(hop, "append") +## def rtype_method_append(self, hop): +## return self.send_message(hop, "append") - def rtype_method_extend(self, hop): - return self.send_message(hop, "extend") +## def rtype_method_extend(self, hop): +## return self.send_message(hop, "extend") def make_iterator_repr(self): return ListIteratorRepr(self) @@ -83,37 +83,37 @@ FixedSizeListRepr = ListRepr -class __extend__(pairtype(BaseListRepr, IntegerRepr)): +##class __extend__(pairtype(BaseListRepr, IntegerRepr)): - def rtype_getitem((r_list, r_int), hop): - if hop.args_s[1].nonneg: - return r_list.send_message(hop, "ll_getitem_fast", can_raise=True) - else: - v_list, v_index = hop.inputargs(r_list, Signed) - hop.exception_is_here() - v_res = hop.gendirectcall(ll_getitem, v_list, v_index) - return r_list.recast(hop.llops, v_res) - - def rtype_setitem((r_list, r_int), hop): - if hop.args_s[1].nonneg: - return r_list.send_message(hop, "ll_setitem_fast", can_raise=True) - else: - v_list, v_index, v_item = hop.inputargs(r_list, Signed, r_list.item_repr) - hop.exception_is_here() - return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) - - -class __extend__(pairtype(ListRepr, IntegerRepr)): - - def rtype_delitem((r_list, r_int), hop): - v_list, v_index = hop.inputargs(r_list, Signed) - if hop.args_s[1].nonneg: - c_count = hop.inputconst(Signed, 1) - return r_list.send_message(hop, "remove_range",can_raise=True, - v_args=[v_list, v_index, c_count]) - else: - hop.exception_is_here() - return hop.gendirectcall(ll_delitem, v_list, v_index) +## def rtype_getitem((r_list, r_int), hop): +## if hop.args_s[1].nonneg: +## return r_list.send_message(hop, "ll_getitem_fast", can_raise=True) +## else: +## v_list, v_index = hop.inputargs(r_list, Signed) +## hop.exception_is_here() +## v_res = hop.gendirectcall(ll_getitem, v_list, v_index) +## return r_list.recast(hop.llops, v_res) + +## def rtype_setitem((r_list, r_int), hop): +## if hop.args_s[1].nonneg: +## return r_list.send_message(hop, "ll_setitem_fast", can_raise=True) +## else: +## v_list, v_index, v_item = hop.inputargs(r_list, Signed, r_list.item_repr) +## hop.exception_is_here() +## return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) + + +##class __extend__(pairtype(ListRepr, IntegerRepr)): + +## def rtype_delitem((r_list, r_int), hop): +## v_list, v_index = hop.inputargs(r_list, Signed) +## if hop.args_s[1].nonneg: +## c_count = hop.inputconst(Signed, 1) +## return r_list.send_message(hop, "remove_range",can_raise=True, +## v_args=[v_list, v_index, c_count]) +## else: +## hop.exception_is_here() +## return hop.gendirectcall(ll_delitem, v_list, v_index) def newlist(llops, r_list, items_v): @@ -128,127 +128,113 @@ # These helpers are sometimes trivial but help encapsulation -def ll_newlist(LIST): - return ootype.new(LIST) +##def ll_newlist(LIST): +## return ootype.new(LIST) -def ll_getitem(lst, index): - if index < 0: - index += lst.ll_length() - return lst.ll_getitem_fast(index) - -def ll_setitem(lst, index, item): - if index < 0: - index += lst.ll_length() - return lst.ll_setitem_fast(index, item) - -def ll_delitem(lst, index): - if index < 0: - index += lst.ll_length() - return lst.remove_range(index, 1) - -def ll_list_is_true(lst): - return bool(lst) and lst.ll_length() != 0 - -def ll_append(lst, item): - lst.append(item) - -def ll_extend(l1, l2): - # This is a bit inefficient, could also add extend to the list interface - len2 = l2.ll_length() - i = 0 - while i < len2: - l1.append(l2.ll_getitem_fast(i)) - i += 1 - -def ll_concat(RESLIST, l1, l2): - len1 = l1.ll_length() - len2 = l2.ll_length() - #l = ootype.new(RESLIST) - l = RESLIST.ll_newlist() - i = 0 - while i < len1: - l.append(l1.ll_getitem_fast(i)) - i += 1 - i = 0 - while i < len2: - l.append(l2.ll_getitem_fast(i)) - i += 1 - return l - -def ll_listslice_startonly(RESLIST, lst, start): - len1 = lst.ll_length() - #newlength = len1 - start - res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements - i = start - while i < len1: - res.append(lst.ll_getitem_fast(i)) - i += 1 - return res - -def ll_listslice(RESLIST, lst, slice): - start = slice.start - stop = slice.stop - length = lst.ll_length() - if stop > length: - stop = length - #newlength = stop - start - res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements - i = start - while i < stop: - res.append(lst.ll_getitem_fast(i)) - i += 1 - return res - -def ll_listslice_minusone(RESLIST, lst): - newlength = lst.ll_length() - 1 - #assert newlength >= 0 # TODO: asserts seems to have problems with ootypesystem - res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements - i = 0 - while i < newlength: - res.append(lst.ll_getitem_fast(i)) - i += 1 - return res - -def ll_listsetslice(l1, slice, l2): - count = l2.ll_length() -## assert count == slice.stop - slice.start, ( # TODO: see above -## "setslice cannot resize lists in RPython") - # XXX but it should be easy enough to support, soon - start = slice.start - j = start - i = 0 - while i < count: - l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) - i += 1 - j += 1 - -def ll_listdelslice_startonly(lst, start): - count = lst.ll_length() - start - if count > 0: - lst.remove_range(start, count) - -def ll_listdelslice(lst, slice): - start = slice.start - stop = slice.stop - length = lst.ll_length() - if stop > length: - stop = length - count = stop - start - if count > 0: - lst.remove_range(start, count) - -def ll_listindex(lst, obj, eqfn): - lng = lst.ll_length() - j = 0 - while j < lng: - if eqfn is None: - if lst.ll_getitem_fast(j) == obj: - return j - else: - if eqfn(lst.ll_getitem_fast(j), obj): - return j - j += 1 - raise ValueError # can't say 'list.index(x): x not in list' +##def ll_getitem(lst, index): +## if index < 0: +## index += lst.ll_length() +## return lst.ll_getitem_fast(index) + +##def ll_setitem(lst, index, item): +## if index < 0: +## index += lst.ll_length() +## return lst.ll_setitem_fast(index, item) + +##def ll_delitem(lst, index): +## if index < 0: +## index += lst.ll_length() +## return lst.remove_range(index, 1) + +##def ll_list_is_true(lst): +## return bool(lst) and lst.ll_length() != 0 + +##def ll_append(lst, item): +## lst.append(item) + +##def ll_extend(l1, l2): +## # This is a bit inefficient, could also add extend to the list interface +## len2 = l2.ll_length() +## i = 0 +## while i < len2: +## l1.append(l2.ll_getitem_fast(i)) +## i += 1 + + +##def ll_listslice_startonly(RESLIST, lst, start): +## len1 = lst.ll_length() +## #newlength = len1 - start +## res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements +## i = start +## while i < len1: +## res.append(lst.ll_getitem_fast(i)) +## i += 1 +## return res + +##def ll_listslice(RESLIST, lst, slice): +## start = slice.start +## stop = slice.stop +## length = lst.ll_length() +## if stop > length: +## stop = length +## #newlength = stop - start +## res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements +## i = start +## while i < stop: +## res.append(lst.ll_getitem_fast(i)) +## i += 1 +## return res + +##def ll_listslice_minusone(RESLIST, lst): +## newlength = lst.ll_length() - 1 +## #assert newlength >= 0 # TODO: asserts seems to have problems with ootypesystem +## res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements +## i = 0 +## while i < newlength: +## res.append(lst.ll_getitem_fast(i)) +## i += 1 +## return res + +##def ll_listsetslice(l1, slice, l2): +## count = l2.ll_length() +#### assert count == slice.stop - slice.start, ( # TODO: see above +#### "setslice cannot resize lists in RPython") +## # XXX but it should be easy enough to support, soon +## start = slice.start +## j = start +## i = 0 +## while i < count: +## l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) +## i += 1 +## j += 1 + +##def ll_listdelslice_startonly(lst, start): +## count = lst.ll_length() - start +## if count > 0: +## lst.remove_range(start, count) + +##def ll_listdelslice(lst, slice): +## start = slice.start +## stop = slice.stop +## length = lst.ll_length() +## if stop > length: +## stop = length +## count = stop - start +## if count > 0: +## lst.remove_range(start, count) + +##def ll_listindex(lst, obj, eqfn): +## lng = lst.ll_length() +## j = 0 +## while j < lng: +## if eqfn is None: +## if lst.ll_getitem_fast(j) == obj: +## return j +## else: +## if eqfn(lst.ll_getitem_fast(j), obj): +## return j +## j += 1 +## raise ValueError # can't say 'list.index(x): x not in list' Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Mon Apr 17 14:13:43 2006 @@ -7,6 +7,12 @@ l = new(LT) assert typeOf(l) == LT +def test_ll_newlist(): + LT = List(Signed) + l = LT.ll_newlist(10) + assert typeOf(l) == LT + assert l.ll_length() == 10 + def test_len(): LT = List(Signed) l = new(LT) Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Mon Apr 17 14:13:43 2006 @@ -1,10 +1,13 @@ -from pypy.annotation.pairtype import pairtype +from pypy.annotation.pairtype import pairtype, pair from pypy.annotation import model as annmodel -from pypy.rpython.rmodel import Repr, IteratorRepr, inputconst +from pypy.rpython.rmodel import Repr, IteratorRepr, IntegerRepr, inputconst from pypy.rpython.rslice import AbstractSliceRepr -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem.lltype import typeOf, Ptr, Void, Signed, Bool, nullptr from pypy.rpython import robject +def dum_checkidx(): pass +def dum_nocheck(): pass + class __extend__(annmodel.SomeList): def rtyper_makerepr(self, rtyper): @@ -35,20 +38,178 @@ def recast(self, llops, v): return llops.convertvar(v, self.item_repr, self.external_item_repr) + def rtype_bltn_list(self, hop): + v_lst = hop.inputarg(self, 0) + cRESLIST = hop.inputconst(Void, hop.r_result.LIST) + return hop.gendirectcall(ll_copy, cRESLIST, v_lst) + + def rtype_len(self, hop): + v_lst, = hop.inputargs(self) + return hop.gendirectcall(ll_len, v_lst) + + def rtype_is_true(self, hop): + v_lst, = hop.inputargs(self) + return hop.gendirectcall(ll_list_is_true, v_lst) + + def rtype_method_reverse(self, hop): + v_lst, = hop.inputargs(self) + hop.exception_cannot_occur() + hop.gendirectcall(ll_reverse,v_lst) + def rtype_method_index(self, hop): v_lst, v_value = hop.inputargs(self, self.item_repr) hop.has_implicit_exception(ValueError) # record that we know about it hop.exception_is_here() - return hop.gendirectcall(self.ll_listindex, v_lst, v_value, self.get_eqfunc()) + return hop.gendirectcall(ll_listindex, v_lst, v_value, self.get_eqfunc()) class AbstractListRepr(AbstractBaseListRepr): - pass + + def rtype_method_append(self, hop): + v_lst, v_value = hop.inputargs(self, self.item_repr) + hop.exception_cannot_occur() + hop.gendirectcall(ll_append, v_lst, v_value) + + def rtype_method_insert(self, hop): + v_lst, v_index, v_value = hop.inputargs(self, Signed, self.item_repr) + arg1 = hop.args_s[1] + args = v_lst, v_index, v_value + if arg1.is_constant() and arg1.const == 0: + llfn = ll_prepend + args = v_lst, v_value + elif arg1.nonneg: + llfn = ll_insert_nonneg + else: + raise TyperError("insert() index must be proven non-negative") + hop.exception_cannot_occur() + hop.gendirectcall(llfn, *args) + + def rtype_method_extend(self, hop): + v_lst1, v_lst2 = hop.inputargs(*hop.args_r) + hop.exception_cannot_occur() + hop.gendirectcall(ll_extend, v_lst1, v_lst2) + + def rtype_method_pop(self, hop): + if hop.has_implicit_exception(IndexError): + spec = dum_checkidx + else: + spec = dum_nocheck + v_func = hop.inputconst(Void, spec) + if hop.nb_args == 2: + args = hop.inputargs(self, Signed) + assert hasattr(args[1], 'concretetype') + arg1 = hop.args_s[1] + if arg1.is_constant() and arg1.const == 0: + llfn = ll_pop_zero + args = args[:1] + elif hop.args_s[1].nonneg: + llfn = ll_pop_nonneg + else: + llfn = ll_pop + else: + args = hop.inputargs(self) + llfn = ll_pop_default + hop.exception_is_here() + v_res = hop.gendirectcall(llfn, v_func, *args) + return self.recast(hop.llops, v_res) + class AbstractFixedSizeListRepr(AbstractBaseListRepr): pass +class __extend__(pairtype(AbstractBaseListRepr, Repr)): + + def rtype_contains((r_lst, _), hop): + v_lst, v_any = hop.inputargs(r_lst, r_lst.item_repr) + return hop.gendirectcall(ll_listcontains, v_lst, v_any, r_lst.get_eqfunc()) + +class __extend__(pairtype(AbstractBaseListRepr, IntegerRepr)): + + def rtype_getitem((r_lst, r_int), hop): + if hop.has_implicit_exception(IndexError): + spec = dum_checkidx + else: + spec = dum_nocheck + v_func = hop.inputconst(Void, spec) + v_lst, v_index = hop.inputargs(r_lst, Signed) + if hop.args_s[1].nonneg: + llfn = ll_getitem_nonneg + else: + llfn = ll_getitem + hop.exception_is_here() + v_res = hop.gendirectcall(llfn, v_func, v_lst, v_index) + return r_lst.recast(hop.llops, v_res) + + def rtype_setitem((r_lst, r_int), hop): + if hop.has_implicit_exception(IndexError): + spec = dum_checkidx + else: + spec = dum_nocheck + v_func = hop.inputconst(Void, spec) + v_lst, v_index, v_item = hop.inputargs(r_lst, Signed, r_lst.item_repr) + if hop.args_s[1].nonneg: + llfn = ll_setitem_nonneg + else: + llfn = ll_setitem + hop.exception_is_here() + return hop.gendirectcall(llfn, v_func, v_lst, v_index, v_item) + + def rtype_mul((r_lst, r_int), hop): + cRESLIST = hop.inputconst(Void, hop.r_result.LIST) + v_lst, v_factor = hop.inputargs(r_lst, Signed) + return hop.gendirectcall(ll_mul, cRESLIST, v_lst, v_factor) + + +class __extend__(pairtype(AbstractListRepr, IntegerRepr)): + + def rtype_delitem((r_lst, r_int), hop): + if hop.has_implicit_exception(IndexError): + spec = dum_checkidx + else: + spec = dum_nocheck + v_func = hop.inputconst(Void, spec) + v_lst, v_index = hop.inputargs(r_lst, Signed) + if hop.args_s[1].nonneg: + llfn = ll_delitem_nonneg + else: + llfn = ll_delitem + hop.exception_is_here() + return hop.gendirectcall(llfn, v_func, v_lst, v_index) + + def rtype_inplace_mul((r_lst, r_int), hop): + v_lst, v_factor = hop.inputargs(r_lst, Signed) + return hop.gendirectcall(ll_inplace_mul, v_lst, v_factor) + + +class __extend__(pairtype(AbstractBaseListRepr, AbstractBaseListRepr)): + def convert_from_to((r_lst1, r_lst2), v, llops): + if r_lst1.listitem is None or r_lst2.listitem is None: + return NotImplemented + if r_lst1.listitem is not r_lst2.listitem: + return NotImplemented + return v + + def rtype_is_((r_lst1, r_lst2), hop): + if r_lst1.lowleveltype != r_lst2.lowleveltype: + # obscure logic, the is can be true only if both are None + v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) + return hop.gendirectcall(ll_both_none, v_lst1, v_lst2) + + return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) + + def rtype_eq((r_lst1, r_lst2), hop): + assert r_lst1.item_repr == r_lst2.item_repr + v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) + return hop.gendirectcall(ll_listeq, v_lst1, v_lst2, r_lst1.get_eqfunc()) + + def rtype_ne((r_lst1, r_lst2), hop): + assert r_lst1.item_repr == r_lst2.item_repr + v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) + flag = hop.gendirectcall(ll_listeq, v_lst1, v_lst2, r_lst1.get_eqfunc()) + return hop.genop('bool_not', [flag], resulttype=Bool) + + def rtype_newlist(hop): nb_args = hop.nb_args r_list = hop.r_result @@ -66,22 +227,18 @@ return hop.rtyper.type_system.rlist.newlist(hop.llops, r_list, items_v) -def dum_checkidx(): pass -def dum_nocheck(): pass - - class __extend__(pairtype(AbstractBaseListRepr, AbstractBaseListRepr)): def rtype_add((r_lst1, r_lst2), hop): v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - cRESLIST = hop.inputconst(lltype.Void, hop.r_result.LIST) - return hop.gendirectcall(hop.r_result.ll_concat, cRESLIST, v_lst1, v_lst2) + cRESLIST = hop.inputconst(Void, hop.r_result.LIST) + return hop.gendirectcall(ll_concat, cRESLIST, v_lst1, v_lst2) class __extend__(pairtype(AbstractListRepr, AbstractBaseListRepr)): def rtype_inplace_add((r_lst1, r_lst2), hop): v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - hop.gendirectcall(r_lst1.ll_extend, v_lst1, v_lst2) + hop.gendirectcall(ll_extend, v_lst1, v_lst2) return v_lst1 @@ -89,16 +246,16 @@ def rtype_getitem((r_lst, r_slic), hop): rs = r_lst.rtyper.type_system.rslice - cRESLIST = hop.inputconst(lltype.Void, hop.r_result.LIST) + cRESLIST = hop.inputconst(Void, hop.r_result.LIST) if r_slic == rs.startonly_slice_repr: v_lst, v_start = hop.inputargs(r_lst, rs.startonly_slice_repr) - return hop.gendirectcall(r_lst.ll_listslice_startonly, cRESLIST, v_lst, v_start) + return hop.gendirectcall(ll_listslice_startonly, cRESLIST, v_lst, v_start) if r_slic == rs.startstop_slice_repr: v_lst, v_slice = hop.inputargs(r_lst, rs.startstop_slice_repr) - return hop.gendirectcall(r_lst.ll_listslice, cRESLIST, v_lst, v_slice) + return hop.gendirectcall(ll_listslice, cRESLIST, v_lst, v_slice) if r_slic == rs.minusone_slice_repr: v_lst, v_ignored = hop.inputargs(r_lst, rs.minusone_slice_repr) - return hop.gendirectcall(r_lst.ll_listslice_minusone, cRESLIST, v_lst) + return hop.gendirectcall(ll_listslice_minusone, cRESLIST, v_lst) raise TyperError('getitem does not support slices with %r' % (r_slic,)) def rtype_setitem((r_lst, r_slic), hop): @@ -108,7 +265,7 @@ if r_slic == rs.startstop_slice_repr: v_lst, v_slice, v_lst2 = hop.inputargs(r_lst, rs.startstop_slice_repr, hop.args_r[2]) - hop.gendirectcall(r_lst.ll_listsetslice, v_lst, v_slice, v_lst2) + hop.gendirectcall(ll_listsetslice, v_lst, v_slice, v_lst2) return raise TyperError('setitem does not support slices with %r' % (r_slic,)) @@ -119,11 +276,11 @@ rs = r_lst.rtyper.type_system.rslice if r_slic == rs.startonly_slice_repr: v_lst, v_start = hop.inputargs(r_lst, rs.startonly_slice_repr) - hop.gendirectcall(r_lst.ll_listdelslice_startonly, v_lst, v_start) + hop.gendirectcall(ll_listdelslice_startonly, v_lst, v_start) return if r_slic == rs.startstop_slice_repr: v_lst, v_slice = hop.inputargs(r_lst, rs.startstop_slice_repr) - hop.gendirectcall(r_lst.ll_listdelslice, v_lst, v_slice) + hop.gendirectcall(ll_listdelslice, v_lst, v_slice) return raise TyperError('delitem does not support slices with %r' % (r_slic,)) @@ -136,7 +293,7 @@ def newiter(self, hop): v_lst, = hop.inputargs(self.r_list) - citerptr = hop.inputconst(lltype.Void, self.lowleveltype) + citerptr = hop.inputconst(Void, self.lowleveltype) return hop.gendirectcall(self.ll_listiter, citerptr, v_lst) def rtype_next(self, hop): @@ -146,3 +303,391 @@ v_res = hop.gendirectcall(self.ll_listnext, v_iter) return self.r_list.recast(hop.llops, v_res) + + +# ____________________________________________________________ +# +# Low-level methods. These can be run for testing, but are meant to +# be direct_call'ed from rtyped flow graphs, which means that they will +# get flowed and annotated, mostly with SomePtr. + +def ll_both_none(lst1, lst2): + return not lst1 and not lst2 + +# return a nullptr() if lst is a list of pointers it, else None. Note +# that if we are using ootypesystem there are not pointers, so we +# always return None. +def ll_null_item(lst): + LIST = typeOf(lst) + if isinstance(LIST, Ptr): + ITEM = LIST.TO.ITEM + if isinstance(ITEM, Ptr): + return nullptr(ITEM.TO) + return None + + +def ll_copy(RESLIST, l): + length = l.ll_length() + new_lst = RESLIST.ll_newlist(length) + i = 0 + while i < length: + new_lst.ll_setitem_fast(i, l.ll_getitem_fast(i)) + i += 1 + return new_lst +ll_copy.oopspec = 'list.copy(l)' + +def ll_len(l): + return l.ll_length() +ll_len.oopspec = 'list.len(l)' + +def ll_list_is_true(l): + # check if a list is True, allowing for None + return bool(l) and l.ll_length() != 0 +ll_list_is_true.oopspec = 'list.nonzero(l)' + +def ll_append(l, newitem): + length = l.ll_length() + l._ll_resize_ge(length+1) + l.ll_setitem_fast(length, newitem) +ll_append.oopspec = 'list.append(l, newitem)' + +# this one is for the special case of insert(0, x) +def ll_prepend(l, newitem): + length = l.ll_length() + l._ll_resize_ge(length+1) + dst = length + while dst > 0: + src = dst - 1 + l.ll_setitem_fast(dst, l.ll_getitem_fast(src)) + dst = src + l.ll_setitem_fast(0, newitem) +ll_prepend.oopspec = 'list.insert(l, 0, newitem)' + +def ll_concat(RESLIST, l1, l2): + len1 = l1.ll_length() + len2 = l2.ll_length() + newlength = len1 + len2 + l = RESLIST.ll_newlist(newlength) + j = 0 + while j < len1: + l.ll_setitem_fast(j, l1.ll_getitem_fast(j)) + j += 1 + i = 0 + while i < len2: + l.ll_setitem_fast(j, l2.ll_getitem_fast(i)) + i += 1 + j += 1 + return l +ll_concat.oopspec = 'list.concat(l1, l2)' + +def ll_insert_nonneg(l, index, newitem): + length = l.ll_length() + l._ll_resize_ge(length+1) + dst = length + while dst > index: + src = dst - 1 + l.ll_setitem_fast(dst, l.ll_getitem_fast(src)) + dst = src + l.ll_setitem_fast(index, newitem) +ll_insert_nonneg.oopspec = 'list.insert(l, index, newitem)' + +def ll_pop_nonneg(func, l, index): + if func is dum_checkidx and (index >= l.ll_length()): + raise IndexError + res = l.ll_getitem_fast(index) + ll_delitem_nonneg(dum_nocheck, l, index) + return res +ll_pop_nonneg.oopspec = 'list.pop(l, index)' + +def ll_pop_default(func, l): + length = l.ll_length() + if func is dum_checkidx and (length == 0): + raise IndexError + index = length - 1 + newlength = index + res = l.ll_getitem_fast(index) + null = ll_null_item(l) + if null is not None: + l.ll_setitem_fast(index, null) + l._ll_resize_le(newlength) + return res +ll_pop_default.oopspec = 'list.pop(l)' + +def ll_pop_zero(func, l): + length = l.ll_length() + if func is dum_checkidx and (length == 0): + raise IndexError + newlength = length - 1 + res = l.ll_getitem_fast(0) + j = 0 + j1 = j+1 + while j < newlength: + l.ll_setitem_fast(j, l.ll_getitem_fast(j1)) + j = j1 + j1 += 1 + null = ll_null_item(l) + if null is not None: + l.ll_setitem_fast(newlength, null) + l._ll_resize_le(newlength) + return res +ll_pop_zero.oopspec = 'list.pop(l, 0)' + +def ll_pop(func, l, index): + length = l.ll_length() + if index < 0: + index += length + if func is dum_checkidx and (index < 0 or index >= length): + raise IndexError + res = l.ll_getitem_fast(index) + ll_delitem_nonneg(dum_nocheck, l, index) + return res +ll_pop.oopspec = 'list.pop(l, index)' + +def ll_reverse(l): + length = l.ll_length() + i = 0 + length_1_i = length-1-i + while i < length_1_i: + tmp = l.ll_getitem_fast(i) + l.ll_setitem_fast(i, l.ll_getitem_fast(length_1_i)) + l.ll_setitem_fast(length_1_i, tmp) + i += 1 + length_1_i -= 1 +ll_reverse.oopspec = 'list.reverse(l)' + +def ll_getitem_nonneg(func, l, index): + if func is dum_checkidx and (index >= l.ll_length()): + raise IndexError + return l.ll_getitem_fast(index) +ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' + +def ll_getitem(func, l, index): + length = l.ll_length() + if index < 0: + index += length + if func is dum_checkidx and (index < 0 or index >= length): + raise IndexError + return l.ll_getitem_fast(index) +ll_getitem.oopspec = 'list.getitem(l, index)' + +def ll_setitem_nonneg(func, l, index, newitem): + if func is dum_checkidx and (index >= l.ll_length()): + raise IndexError + l.ll_setitem_fast(index, newitem) +ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' + +def ll_setitem(func, l, index, newitem): + length = l.ll_length() + if index < 0: + index += length + if func is dum_checkidx and (index < 0 or index >= length): + raise IndexError + l.ll_setitem_fast(index, newitem) +ll_setitem.oopspec = 'list.setitem(l, index, newitem)' + +def ll_delitem_nonneg(func, l, index): + length = l.ll_length() + if func is dum_checkidx and (index >= length): + raise IndexError + newlength = length - 1 + j = index + j1 = j+1 + while j < newlength: + l.ll_setitem_fast(j, l.ll_getitem_fast(j1)) + j = j1 + j1 += 1 + + null = ll_null_item(l) + if null is not None: + l.ll_setitem_fast(newlength, null) + l._ll_resize_le(newlength) +ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' + +def ll_delitem(func, l, i): + length = l.ll_length() + if i < 0: + i += length + if func is dum_checkidx and (i < 0 or i >= length): + raise IndexError + ll_delitem_nonneg(dum_nocheck, l, i) +ll_delitem.oopspec = 'list.delitem(l, i)' + + +def ll_extend(l1, l2): + len1 = l1.ll_length() + len2 = l2.ll_length() + newlength = len1 + len2 + l1._ll_resize_ge(newlength) + i = 0 + j = len1 + while i < len2: + l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) + i += 1 + j += 1 + +def ll_listslice_startonly(RESLIST, l1, start): + len1 = l1.ll_length() + newlength = len1 - start + l = RESLIST.ll_newlist(newlength) + j = 0 + i = start + while i < len1: + l.ll_setitem_fast(j, l1.ll_getitem_fast(i)) + i += 1 + j += 1 + return l + +def ll_listslice(RESLIST, l1, slice): + start = slice.start + stop = slice.stop + length = l1.ll_length() + if stop > length: + stop = length + newlength = stop - start + l = RESLIST.ll_newlist(newlength) + j = 0 + i = start + while i < stop: + l.ll_setitem_fast(j, l1.ll_getitem_fast(i)) + i += 1 + j += 1 + return l + +def ll_listslice_minusone(RESLIST, l1): + newlength = l1.ll_length() - 1 + #assert newlength >= 0 # XXX assert doesn't work in ootypesystem + l = RESLIST.ll_newlist(newlength) + j = 0 + while j < newlength: + l.ll_setitem_fast(j, l1.ll_getitem_fast(j)) + j += 1 + return l + +def ll_listdelslice_startonly(l, start): + newlength = start + null = ll_null_item(l) + if null is not None: + j = l.ll_length() - 1 + while j >= newlength: + l.ll_setitem_fast(j, null) + j -= 1 + l._ll_resize_le(newlength) + +def ll_listdelslice(l, slice): + start = slice.start + stop = slice.stop + length = l.ll_length() + if stop > length: + stop = length + newlength = length - (stop-start) + j = start + i = stop + while j < newlength: + l.ll_setitem_fast(j, l.ll_getitem_fast(i)) + i += 1 + j += 1 + null = ll_null_item(l) + if null is not None: + j = length - 1 + while j >= newlength: + l.ll_setitem_fast(j, null) + j -= 1 + l._ll_resize_le(newlength) + +def ll_listsetslice(l1, slice, l2): + count = l2.ll_length() + # XXX: assert doesn't work in ootypesystem + #assert count == slice.stop - slice.start, "setslice cannot resize lists in RPython") + # XXX but it should be easy enough to support, soon + start = slice.start + j = start + i = 0 + while i < count: + l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) + i += 1 + j += 1 + +# ____________________________________________________________ +# +# Comparison. + +def ll_listeq(l1, l2, eqfn): + if not l1 and not l2: + return True + if not l1 or not l2: + return False + len1 = l1.ll_length() + len2 = l2.ll_length() + if len1 != len2: + return False + j = 0 + while j < len1: + if eqfn is None: + if l1.ll_getitem_fast(j) != l2.ll_getitem_fast(j): + return False + else: + if not eqfn(l1.ll_getitem_fast(j), l2.ll_getitem_fast(j)): + return False + j += 1 + return True + +def ll_listcontains(lst, obj, eqfn): + lng = lst.ll_length() + j = 0 + while j < lng: + if eqfn is None: + if lst.ll_getitem_fast(j) == obj: + return True + else: + if eqfn(lst.ll_getitem_fast(j), obj): + return True + j += 1 + return False + +def ll_listindex(lst, obj, eqfn): + lng = lst.ll_length() + j = 0 + while j < lng: + if eqfn is None: + if lst.ll_getitem_fast(j) == obj: + return j + else: + if eqfn(lst.ll_getitem_fast(j), obj): + return j + j += 1 + raise ValueError # can't say 'list.index(x): x not in list' + +def ll_inplace_mul(l, factor): + length = l.ll_length() + if factor < 0: + factor = 0 + resultlen = length * factor + res = l + #_ll_list_resize(res, resultlen) + res._ll_resize_ge(resultlen) + j = length + while j < resultlen: + i = 0 + while i < length: + p = j + i + res.ll_setitem_fast(p, l.ll_getitem_fast(i)) + i += 1 + j += length + return res + + +def ll_mul(RESLIST, l, factor): + length = l.ll_length() + if factor < 0: + factor = 0 + resultlen = length * factor + res = RESLIST.ll_newlist(resultlen) + j = 0 + while j < resultlen: + i = 0 + while i < length: + p = j + i + res.ll_setitem_fast(p, l.ll_getitem_fast(i)) + i += 1 + j += length + return res Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Mon Apr 17 14:13:43 2006 @@ -1,7 +1,8 @@ import sys from pypy.translator.translator import TranslationContext from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.lltypesystem.rlist import * +from pypy.rpython.rlist import * +from pypy.rpython.lltypesystem.rlist import ListRepr, FixedSizeListRepr, ll_newlist, ll_fixed_newlist from pypy.rpython.lltypesystem.rslice import ll_newslice from pypy.rpython.rint import signed_repr from pypy.rpython.test.test_llinterp import interpret, interpret_raises Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Mon Apr 17 14:13:43 2006 @@ -29,6 +29,27 @@ this[index] = item; } + public void _ll_resize_ge(int length) + { + if (this.Count < length) + { + // TODO: this is inefficient because it can triggers + // many array resize + int diff = length - this.Count; + for(int i=0; i Author: arigo Date: Mon Apr 17 14:15:49 2006 New Revision: 25887 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: Automatic cast from array to pointer. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Mon Apr 17 14:15:49 2006 @@ -5,8 +5,9 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr -from pypy.rpython.rctypes.rmodel import genreccopy, reccopy +from pypy.rpython.rctypes.rmodel import genreccopy, reccopy, C_ZERO from pypy.rpython.rctypes.rprimitive import PrimitiveRepr +from pypy.rpython.rctypes.rpointer import PointerRepr from pypy.annotation.model import SomeCTypesObject ArrayType = type(ARRAY(c_int, 10)) @@ -100,6 +101,16 @@ r_array.set_item_value(hop.llops, v_array, v_index, v_newvalue) +class __extend__(pairtype(ArrayRepr, PointerRepr)): + def convert_from_to((r_from, r_to), v, llops): + # XXX keepalives + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + v_c_array = r_from.get_c_data_of_item(llops, v, C_ZERO) + r_temp.setvalue(llops, v_owned_box, v_c_array) + return llops.convertvar(v_owned_box, r_temp, r_to) + + def ll_chararrayvalue(box): from pypy.rpython.rctypes import rchar_p p = box.c_data Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Mon Apr 17 14:15:49 2006 @@ -121,3 +121,14 @@ assert b.value == '!' assert len(create_string_buffer(0)) == 0 + +def test_array(): + a = (c_int * 10)() + + class S(Structure): + _fields_ = [('p', POINTER(c_int))] + + s = S() + s.p = a + s.p.contents.value = 42 + assert a[0] == 42 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Mon Apr 17 14:15:49 2006 @@ -17,7 +17,8 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int, c_short, ARRAY, POINTER, pointer, c_char_p, c_char +from ctypes import c_int, c_short, c_char_p, c_char, pointer +from ctypes import ARRAY, POINTER, Structure c_int_10 = ARRAY(c_int,10) @@ -219,6 +220,19 @@ res = interpret(func, []) assert ''.join(res.chars) == "xy" + def test_automatic_cast_array_to_pointer(self): + A = c_int * 10 + class S(Structure): + _fields_ = [('p', POINTER(c_int))] + def func(): + a = A() + s = S() + s.p = a + s.p.contents.value = 42 + return a[0] + res = interpret(func, []) + assert res == 42 + class Test_compilation: def test_compile_array_access(self): def access_array(): From arigo at codespeak.net Mon Apr 17 14:31:53 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 14:31:53 +0200 (CEST) Subject: [pypy-svn] r25888 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060417123153.7B5CA100B0@code0.codespeak.net> Author: arigo Date: Mon Apr 17 14:31:52 2006 New Revision: 25888 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: Test and bug fix. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Mon Apr 17 14:31:52 2006 @@ -77,14 +77,17 @@ class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) - if isinstance(r_array.r_item, CTypesRefRepr): + if isinstance(r_array.r_item, PrimitiveRepr): + # primitive case (optimization; the below also works in this case) + # NB. this optimization is invalid for PointerReprs! See for + # example: a[0].contents = ... to change the first pointer of + # an array of pointers. + v_value = r_array.get_item_value(hop.llops, v_array, v_index) + return r_array.r_item.return_value(hop.llops, v_value) + else: # ByRef case v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) return r_array.r_item.return_c_data(hop.llops, v_c_data) - else: - # ByValue case (optimization; the above also works in this case) - v_value = r_array.get_item_value(hop.llops, v_array, v_index) - return r_array.r_item.return_value(hop.llops, v_value) def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Mon Apr 17 14:31:52 2006 @@ -233,6 +233,20 @@ res = interpret(func, []) assert res == 42 + def test_array_of_pointers(self): + class S(Structure): + _fields_ = [('x', c_int)] + A = POINTER(S) * 10 + def func(): + a = A() + s = S() + s.x = 11 + a[2].contents = s + a[3] = pointer(s) + return a[2].contents.x * a[3].contents.x + res = interpret(func, []) + assert res == 121 + class Test_compilation: def test_compile_array_access(self): def access_array(): From arigo at codespeak.net Mon Apr 17 14:39:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 14:39:59 +0200 (CEST) Subject: [pypy-svn] r25889 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060417123959.62AAB100B2@code0.codespeak.net> Author: arigo Date: Mon Apr 17 14:39:58 2006 New Revision: 25889 Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Same bug as in r25888 for structs. Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Mon Apr 17 14:39:58 2006 @@ -75,14 +75,17 @@ name = s_attr.const r_field = self.r_fields[name] v_struct, v_attr = hop.inputargs(self, lltype.Void) - if isinstance(r_field, CTypesRefRepr): + if isinstance(r_field, PrimitiveRepr): + # primitive case (optimization; the below also works in this case) + # NB. this optimization is invalid for PointerReprs! See for + # example: s.p.contents = ... to change the pointer field 'p' + # of 's'. + v_value = self.get_field_value(hop.llops, v_struct, name) + return r_field.return_value(hop.llops, v_value) + else: # ByRef case v_c_data = self.get_c_data_of_field(hop.llops, v_struct, name) return r_field.return_c_data(hop.llops, v_c_data) - else: - # ByValue case (optimization; the above also works in this case) - v_value = self.get_field_value(hop.llops, v_struct, name) - return r_field.return_value(hop.llops, v_value) def rtype_setattr(self, hop): s_attr = hop.args_s[1] Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Mon Apr 17 14:39:58 2006 @@ -138,6 +138,22 @@ res = interpret(func, []) assert res == expected + def test_struct_of_pointers(self): + class S(Structure): + _fields_ = [('x', c_int)] + class T(Structure): + _fields_ = [('p', POINTER(S))] + def func(): + t1 = T() + t2 = T() + s = S() + s.x = 11 + t1.p = pointer(s) + t2.p.contents = s + return t1.p.contents.x * t2.p.contents.x + res = interpret(func, []) + assert res == 121 + class Test_compilation: def test_compile_struct_access(self): def access_struct(n): From arigo at codespeak.net Mon Apr 17 15:01:57 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 15:01:57 +0200 (CEST) Subject: [pypy-svn] r25892 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060417130157.5E715100AF@code0.codespeak.net> Author: arigo Date: Mon Apr 17 15:01:56 2006 New Revision: 25892 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: Reminder: keepalives not implemented for arrays and structs. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Mon Apr 17 15:01:56 2006 @@ -2,7 +2,7 @@ Test the rctypes implementation. """ -import py.test +import py import pypy.rpython.rctypes.implementation from pypy.annotation import model as annmodel from pypy.annotation.annrpython import RPythonAnnotator @@ -247,6 +247,22 @@ res = interpret(func, []) assert res == 121 + def test_specialize_keepalive(self): + py.test.skip("in-progress") + class S(Structure): + _fields_ = [('x', c_int)] + A = POINTER(S) * 10 + def func(): + a = A() + for i in range(10): + s = S() + s.x = i*i + a[i].contents = s + for i in range(10): + assert a[i].contents.x == i*i + func() + interpret(func, []) + class Test_compilation: def test_compile_array_access(self): def access_array(): From arigo at codespeak.net Mon Apr 17 15:06:43 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 17 Apr 2006 15:06:43 +0200 (CEST) Subject: [pypy-svn] r25894 - in pypy/dist/pypy/rpython: . lltypesystem rctypes rctypes/test Message-ID: <20060417130643.90BFD100B0@code0.codespeak.net> Author: arigo Date: Mon Apr 17 15:06:42 2006 New Revision: 25894 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: Some support for pointer[non-zero-index]. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Mon Apr 17 15:06:42 2006 @@ -908,7 +908,16 @@ def _cast_to_adr(self): from pypy.rpython.lltypesystem import llmemory - return llmemory.fakeaddress(self) + if isinstance(self._obj, _subarray): + # return an address built as an offset in the whole array + parent, parentindex = parentlink(self._obj) + T = typeOf(parent) + addr = llmemory.fakeaddress(_ptr(Ptr(T), parent)) + addr += llmemory.itemoffsetof(T, parentindex) + return addr + else: + # normal case + return llmemory.fakeaddress(self) assert not '__dict__' in dir(_ptr) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Mon Apr 17 15:06:42 2006 @@ -335,6 +335,19 @@ return llops.genop('cast_adr_to_ptr', [v_base_adr], resulttype = ARRAYPTRTYPE) +def gen_add_itemoffset_to_pointer(llops, ITEMTYPE, v_ptr, v_index): + "Generates address manipulations equivalent to the C expression ptr+index." + from pypy.rpython.lltypesystem import llmemory + v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], + resulttype = llmemory.Address) + c_ofs = inputconst(lltype.Signed, llmemory.ItemOffset(ITEMTYPE)) + v_ofs = llops.genop('int_mul', [c_ofs, v_index], + resulttype = lltype.Signed) + v_newadr = llops.genop('adr_add', [v_adr, v_ofs], + resulttype = llmemory.Address) + return llops.genop('cast_adr_to_ptr', [v_newadr], + resulttype = v_ptr.concretetype) + def rtype_cast_structfield_pointer(hop): assert hop.args_s[0].is_constant() assert hop.args_s[2].is_constant() Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Mon Apr 17 15:06:42 2006 @@ -1,3 +1,4 @@ +from pypy.rpython.rbuiltin import gen_add_itemoffset_to_pointer from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.error import TyperError from pypy.rpython.lltypesystem import lltype @@ -64,11 +65,14 @@ def rtype_getitem((r_ptr, _), hop): self = r_ptr v_ptr, v_index = hop.inputargs(self, lltype.Signed) + v_c_ptr = self.getvalue(hop.llops, v_ptr) if hop.args_s[1].is_constant() and hop.args_s[1].const == 0: - v_c_ptr = self.getvalue(hop. llops, v_ptr) - return self.r_contents.return_c_data(hop.llops, v_c_ptr) + pass # skip adding the offset else: - raise NotImplementedError("XXX: pointer[non-zero-index]") + v_c_ptr = gen_add_itemoffset_to_pointer(hop.llops, + r_ptr.r_contents.ll_type, + v_c_ptr, v_index) + return self.r_contents.return_c_data(hop.llops, v_c_ptr) def rtype_setitem((r_ptr, _), hop): # p[0] = x is not the same as p.contents.value = x Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Mon Apr 17 15:06:42 2006 @@ -2,13 +2,14 @@ Test the rctypes pointer implementation """ -import py.test +import py import pypy.rpython.rctypes.implementation from pypy.translator.translator import TranslationContext from pypy import conftest from pypy.rpython.test.test_llinterp import interpret +from pypy.translator.c.test.test_genc import compile -from ctypes import c_int, c_float, POINTER, pointer +from ctypes import c_int, c_float, POINTER, pointer, Structure class Test_annotation: def test_simple(self): @@ -251,3 +252,32 @@ res = interpret(fn, []) float_c_data = res.c_data[0] assert float_c_data[0] == 6.25 + + def test_specialize_getitem_nonzero_index(self): + A = c_int * 10 + class S(Structure): + _fields_ = [('x', POINTER(c_int))] + def fn(): + a = A() + a[3] = 5 + s = S() + s.x = a + return s.x[3] + assert fn() == 5 + res = interpret(fn, []) + assert res == 5 + +class Test_compilation: + def test_compile_getitem_nonzero_index(self): + A = c_int * 10 + class S(Structure): + _fields_ = [('x', POINTER(c_int))] + def func(): + a = A() + a[3] = 5 + s = S() + s.x = a + return s.x[3] + fn = compile(func, []) + res = fn() + assert res == 5 From sanxiyn at codespeak.net Mon Apr 17 15:32:53 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 17 Apr 2006 15:32:53 +0200 (CEST) Subject: [pypy-svn] r25900 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060417133253.C3073100B6@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 17 15:32:38 2006 New Revision: 25900 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: Tokyo sprint Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Mon Apr 17 15:32:38 2006 @@ -18,6 +18,7 @@ Anders Chrigstr?m 22/4 - 30/4 Akihabare Washington Hotel Samuele Pedroni 22/4 - 30/4 Akihabare Washington Hotel Valentino Volonghi 22/4 - 1/5 Hotel Edoya (Ueno) +Seo Sanghyeon 22/4 - 29/4 ==================== ============== ===================== People on the following list were present at previous sprints: From antocuni at codespeak.net Mon Apr 17 16:45:17 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 17 Apr 2006 16:45:17 +0200 (CEST) Subject: [pypy-svn] r25903 - in pypy/dist/pypy: rpython rpython/lltypesystem rpython/ootypesystem rpython/test translator/cli/test Message-ID: <20060417144517.92A14100B2@code0.codespeak.net> Author: antocuni Date: Mon Apr 17 16:44:58 2006 New Revision: 25903 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/rootype.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/translator/cli/test/compile.py Log: - Added support for is_, eq and ne between two OOInstanceRepr. - Added convert_constant to ootypeststem/rlist.py - Added more tests for ootypesystem's list. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Mon Apr 17 16:44:58 2006 @@ -1,6 +1,5 @@ from pypy.annotation.pairtype import pairtype, pair from pypy.annotation import model as annmodel -from pypy.objspace.flow.model import Constant from pypy.rpython.error import TyperError from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst from pypy.rpython.rmodel import externalvsinternal @@ -64,31 +63,8 @@ def _setup_repr_final(self): self.list_builder.setup(self) - def convert_const(self, listobj): - # get object from bound list method - #listobj = getattr(listobj, '__self__', listobj) - if listobj is None: - return nullptr(self.LIST) - if not isinstance(listobj, list): - raise TyperError("expected a list: %r" % (listobj,)) - try: - key = Constant(listobj) - return self.list_cache[key] - except KeyError: - self.setup() - n = len(listobj) - result = self.prepare_const(n) - self.list_cache[key] = result - r_item = self.item_repr - if r_item.lowleveltype is not Void: - items = result.ll_items() - for i in range(n): - x = listobj[i] - items[i] = r_item.convert_const(x) - return result - - def prepare_const(self, nitems): - raise NotImplementedError + def null_const(self): + return nullptr(self.LIST) def get_eqfunc(self): return inputconst(Void, self.item_repr.get_ll_eq_function()) @@ -170,6 +146,16 @@ return 1 # bad but not used alone +class __extend__(pairtype(BaseListRepr, BaseListRepr)): + def rtype_is_((r_lst1, r_lst2), hop): + if r_lst1.lowleveltype != r_lst2.lowleveltype: + # obscure logic, the is can be true only if both are None + v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) + return hop.gendirectcall(ll_both_none, v_lst1, v_lst2) + + return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) + + class ListRepr(AbstractListRepr, BaseListRepr): def _setup_repr(self): @@ -318,6 +304,8 @@ TEMP = GcArray(Ptr(rstr.STR)) +def ll_both_none(lst1, lst2): + return not lst1 and not lst2 # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Mon Apr 17 16:44:58 2006 @@ -43,6 +43,14 @@ if isinstance(self.lowleveltype, ootype.ForwardReference): self.lowleveltype.become(ootype.List(self.item_repr.lowleveltype)) + def null_const(self): + return self.LIST._null + + def prepare_const(self, n): + result = self.LIST.ll_newlist(n) + return result + + def send_message(self, hop, message, can_raise=False, v_args=None): if v_args is None: v_args = hop.inputargs(self, *hop.args_r[1:]) @@ -55,25 +63,6 @@ def get_eqfunc(self): return inputconst(Void, self.item_repr.get_ll_eq_function()) -## def rtype_len(self, hop): -## return self.send_message(hop, "ll_length") - -## def rtype_is_true(self, hop): -## v_lst, = hop.inputargs(self) -## return hop.gendirectcall(ll_list_is_true, v_lst) - -## def rtype_bltn_list(self, hop): -## v_lst = hop.inputarg(self, 0) -## c_start = hop.inputconst(Signed, 0) -## cRESLIST = hop.inputconst(Void, hop.r_result.LIST) -## return hop.gendirectcall(self.ll_listslice_startonly, cRESLIST, v_lst, c_start) - -## def rtype_method_append(self, hop): -## return self.send_message(hop, "append") - -## def rtype_method_extend(self, hop): -## return self.send_message(hop, "extend") - def make_iterator_repr(self): return ListIteratorRepr(self) @@ -83,37 +72,13 @@ FixedSizeListRepr = ListRepr -##class __extend__(pairtype(BaseListRepr, IntegerRepr)): +class __extend__(pairtype(BaseListRepr, BaseListRepr)): + + def rtype_is_((r_lst1, r_lst2), hop): + # NB. this version performs no cast to the common base class + vlist = hop.inputargs(r_lst1, r_lst2) + return hop.genop('oois', vlist, resulttype=ootype.Bool) -## def rtype_getitem((r_list, r_int), hop): -## if hop.args_s[1].nonneg: -## return r_list.send_message(hop, "ll_getitem_fast", can_raise=True) -## else: -## v_list, v_index = hop.inputargs(r_list, Signed) -## hop.exception_is_here() -## v_res = hop.gendirectcall(ll_getitem, v_list, v_index) -## return r_list.recast(hop.llops, v_res) - -## def rtype_setitem((r_list, r_int), hop): -## if hop.args_s[1].nonneg: -## return r_list.send_message(hop, "ll_setitem_fast", can_raise=True) -## else: -## v_list, v_index, v_item = hop.inputargs(r_list, Signed, r_list.item_repr) -## hop.exception_is_here() -## return hop.gendirectcall(ll_setitem, v_list, v_index, v_item) - - -##class __extend__(pairtype(ListRepr, IntegerRepr)): - -## def rtype_delitem((r_list, r_int), hop): -## v_list, v_index = hop.inputargs(r_list, Signed) -## if hop.args_s[1].nonneg: -## c_count = hop.inputconst(Signed, 1) -## return r_list.send_message(hop, "remove_range",can_raise=True, -## v_args=[v_list, v_index, c_count]) -## else: -## hop.exception_is_here() -## return hop.gendirectcall(ll_delitem, v_list, v_index) def newlist(llops, r_list, items_v): @@ -126,116 +91,6 @@ resulttype=ootype.Void) return v_result -# These helpers are sometimes trivial but help encapsulation - -##def ll_newlist(LIST): -## return ootype.new(LIST) - -##def ll_getitem(lst, index): -## if index < 0: -## index += lst.ll_length() -## return lst.ll_getitem_fast(index) - -##def ll_setitem(lst, index, item): -## if index < 0: -## index += lst.ll_length() -## return lst.ll_setitem_fast(index, item) - -##def ll_delitem(lst, index): -## if index < 0: -## index += lst.ll_length() -## return lst.remove_range(index, 1) - -##def ll_list_is_true(lst): -## return bool(lst) and lst.ll_length() != 0 - -##def ll_append(lst, item): -## lst.append(item) - -##def ll_extend(l1, l2): -## # This is a bit inefficient, could also add extend to the list interface -## len2 = l2.ll_length() -## i = 0 -## while i < len2: -## l1.append(l2.ll_getitem_fast(i)) -## i += 1 - - -##def ll_listslice_startonly(RESLIST, lst, start): -## len1 = lst.ll_length() -## #newlength = len1 - start -## res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements -## i = start -## while i < len1: -## res.append(lst.ll_getitem_fast(i)) -## i += 1 -## return res - -##def ll_listslice(RESLIST, lst, slice): -## start = slice.start -## stop = slice.stop -## length = lst.ll_length() -## if stop > length: -## stop = length -## #newlength = stop - start -## res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements -## i = start -## while i < stop: -## res.append(lst.ll_getitem_fast(i)) -## i += 1 -## return res - -##def ll_listslice_minusone(RESLIST, lst): -## newlength = lst.ll_length() - 1 -## #assert newlength >= 0 # TODO: asserts seems to have problems with ootypesystem -## res = ootype.new(RESLIST) # TODO: pre-allocate newlength elements -## i = 0 -## while i < newlength: -## res.append(lst.ll_getitem_fast(i)) -## i += 1 -## return res - -##def ll_listsetslice(l1, slice, l2): -## count = l2.ll_length() -#### assert count == slice.stop - slice.start, ( # TODO: see above -#### "setslice cannot resize lists in RPython") -## # XXX but it should be easy enough to support, soon -## start = slice.start -## j = start -## i = 0 -## while i < count: -## l1.ll_setitem_fast(j, l2.ll_getitem_fast(i)) -## i += 1 -## j += 1 - -##def ll_listdelslice_startonly(lst, start): -## count = lst.ll_length() - start -## if count > 0: -## lst.remove_range(start, count) - -##def ll_listdelslice(lst, slice): -## start = slice.start -## stop = slice.stop -## length = lst.ll_length() -## if stop > length: -## stop = length -## count = stop - start -## if count > 0: -## lst.remove_range(start, count) - -##def ll_listindex(lst, obj, eqfn): -## lng = lst.ll_length() -## j = 0 -## while j < lng: -## if eqfn is None: -## if lst.ll_getitem_fast(j) == obj: -## return j -## else: -## if eqfn(lst.ll_getitem_fast(j), obj): -## return j -## j += 1 -## raise ValueError # can't say 'list.index(x): x not in list' - # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/ootypesystem/rootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rootype.py Mon Apr 17 16:44:58 2006 @@ -59,6 +59,12 @@ vlist = hop.inputargs(r_ins1, r_ins2) return hop.genop('oois', vlist, resulttype=ootype.Bool) + rtype_eq = rtype_is_ + + def rtype_ne(rpair, hop): + v = rpair.rtype_eq(hop) + return hop.genop("bool_not", [v], resulttype=ootype.Bool) + class OOBoundMethRepr(Repr): def __init__(self, ootype, name): Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Mon Apr 17 16:44:58 2006 @@ -1,4 +1,5 @@ from pypy.annotation.pairtype import pairtype, pair +from pypy.objspace.flow.model import Constant from pypy.annotation import model as annmodel from pypy.rpython.rmodel import Repr, IteratorRepr, IntegerRepr, inputconst from pypy.rpython.rslice import AbstractSliceRepr @@ -38,6 +39,34 @@ def recast(self, llops, v): return llops.convertvar(v, self.item_repr, self.external_item_repr) + def convert_const(self, listobj): + # get object from bound list method + #listobj = getattr(listobj, '__self__', listobj) + if listobj is None: + return self.null_const() + if not isinstance(listobj, list): + raise TyperError("expected a list: %r" % (listobj,)) + try: + key = Constant(listobj) + return self.list_cache[key] + except KeyError: + self.setup() + n = len(listobj) + result = self.prepare_const(n) + self.list_cache[key] = result + r_item = self.item_repr + if r_item.lowleveltype is not Void: + for i in range(n): + x = listobj[i] + result.ll_setitem_fast(i, r_item.convert_const(x)) + return result + + def null_const(self): + raise NotImplementedError + + def prepare_const(self, nitems): + raise NotImplementedError + def rtype_bltn_list(self, hop): v_lst = hop.inputarg(self, 0) cRESLIST = hop.inputconst(Void, hop.r_result.LIST) @@ -190,14 +219,15 @@ return NotImplemented return v - def rtype_is_((r_lst1, r_lst2), hop): - if r_lst1.lowleveltype != r_lst2.lowleveltype: - # obscure logic, the is can be true only if both are None - v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) - return hop.gendirectcall(ll_both_none, v_lst1, v_lst2) - - return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) +## # TODO: move it to lltypesystem +## def rtype_is_((r_lst1, r_lst2), hop): +## if r_lst1.lowleveltype != r_lst2.lowleveltype: +## # obscure logic, the is can be true only if both are None +## v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) +## return hop.gendirectcall(ll_both_none, v_lst1, v_lst2) +## return pairtype(Repr, Repr).rtype_is_(pair(r_lst1, r_lst2), hop) + def rtype_eq((r_lst1, r_lst2), hop): assert r_lst1.item_repr == r_lst2.item_repr v_lst1, v_lst2 = hop.inputargs(r_lst1, r_lst2) @@ -311,8 +341,6 @@ # be direct_call'ed from rtyped flow graphs, which means that they will # get flowed and annotated, mostly with SomePtr. -def ll_both_none(lst1, lst2): - return not lst1 and not lst2 # return a nullptr() if lst is a list of pointers it, else None. Note # that if we are using ootypesystem there are not pointers, so we Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Mon Apr 17 16:44:58 2006 @@ -1,11 +1,13 @@ import sys from pypy.translator.translator import TranslationContext from pypy.rpython.lltypesystem.lltype import * +from pypy.rpython.ootypesystem import ootype from pypy.rpython.rlist import * from pypy.rpython.lltypesystem.rlist import ListRepr, FixedSizeListRepr, ll_newlist, ll_fixed_newlist from pypy.rpython.lltypesystem.rslice import ll_newslice from pypy.rpython.rint import signed_repr -from pypy.rpython.test.test_llinterp import interpret, interpret_raises +from pypy.rpython.test.test_llinterp import interpret #as interpret2 +from pypy.rpython.test.test_llinterp import interpret_raises # as interpret_raises2 from pypy.translator.translator import TranslationContext from pypy.objspace.flow.model import Constant, Variable @@ -161,13 +163,27 @@ # ____________________________________________________________ +class Foo: pass + +class Bar(Foo): pass + class BaseTestListRtyping: + def interpret(self, fn, args): + return interpret(fn, args, type_system=self.ts) + + def interpret_raises(self, exc, fn, args): + return interpret_raises(exc, fn, args, type_system=self.ts) + + def _skip_tuples(self): + if self.ts == 'ootype': + py.test.skip("ootypesystem doesn't support returning tuples of lists, yet") + def test_simple(self): def dummyfn(): l = [10, 20, 30] return l[2] - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res == 30 def test_append(self): @@ -179,7 +195,7 @@ l.append(80) l.append(90) return len(l), l[0], l[-1] - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res.item0 == 5 assert res.item1 == 50 assert res.item2 == 90 @@ -188,14 +204,14 @@ def dummyfn(): l = [5, 10] return len(l) - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res == 2 def dummyfn(): l = [5] l.append(6) return len(l) - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res == 2 def test_iterate(self): @@ -204,7 +220,7 @@ for x in [1, 3, 5, 7, 9]: total += x return total - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res == 25 def dummyfn(): total = 0 @@ -213,7 +229,7 @@ for x in l: total += x return total - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res == 25 def test_recursive(self): @@ -223,7 +239,7 @@ l = [l] N -= 1 return len(l) - res = interpret(dummyfn, [5], type_system=self.ts) + res = self.interpret(dummyfn, [5]) assert res == 1 def dummyfn(N): @@ -232,7 +248,7 @@ l.append(l) N -= 1 return len(l) - res = interpret(dummyfn, [5]) + res = self.interpret(dummyfn, [5]) assert res == 5 def test_add(self): @@ -240,7 +256,7 @@ l = [5] l += [6,7] return l + [8] - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert self.ll_to_list(res) == [5, 6, 7, 8] def dummyfn(): @@ -249,16 +265,15 @@ l2 = l + [8] l2.append(9) return l2 - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert self.ll_to_list(res) == [5, 6, 7, 8, 9] def test_slice(self): - if self.ts == 'ootype': - py.test.skip("ootypesystem doesn't support returning tuples of lists, yet") + self._skip_tuples() def dummyfn(): l = [5, 6, 7, 8, 9] return l[:2], l[1:4], l[3:] - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert self.ll_to_list(res.item0) == [5, 6] assert self.ll_to_list(res.item1) == [6, 7, 8] assert self.ll_to_list(res.item2) == [8, 9] @@ -267,7 +282,7 @@ l = [5, 6, 7, 8] l.append(9) return l[:2], l[1:4], l[3:] - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert self.ll_to_list(res.item0) == [5, 6] assert self.ll_to_list(res.item1) == [6, 7, 8] assert self.ll_to_list(res.item2) == [8, 9] @@ -278,7 +293,7 @@ l[1] = 55 l[-1] = 66 return l - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert self.ll_to_list(res) == [5, 55, 66] def dummyfn(): @@ -289,7 +304,7 @@ l[1] = 55 l[-1] = 66 return l - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert self.ll_to_list(res) == [5, 55, 66] def dummyfn(): @@ -300,17 +315,16 @@ del l[-1] del l[:] return len(l) - res = interpret(dummyfn, [], type_system=self.ts) + res = self.interpret(dummyfn, []) assert res == 0 - + def test_setslice(self): - if self.ts == 'ootype': - py.test.skip("ootypesystem doesn't support returning tuples of lists, yet") + self._skip_tuples() def dummyfn(): l = [10, 9, 8, 7] l[:2] = [6, 5] return l[0], l[1], l[2], l[3] - res = interpret(dummyfn, (), type_system=self.ts) + res = self.interpret(dummyfn, ()) assert res.item0 == 6 assert res.item1 == 5 assert res.item2 == 8 @@ -321,7 +335,7 @@ l.append(7) l[:2] = [6, 5] return l[0], l[1], l[2], l[3] - res = interpret(dummyfn, (), type_system=self.ts) + res = self.interpret(dummyfn, ()) assert res.item0 == 6 assert res.item1 == 5 assert res.item2 == 8 @@ -333,7 +347,7 @@ l2 = list(l1) l2[0] = 0 return l1[0] - res = interpret(dummyfn, (), type_system=self.ts) + res = self.interpret(dummyfn, ()) assert res == 42 def test_is_true(self): @@ -348,266 +362,264 @@ return is_true([]) def dummyfn3(): return is_true([0]) - assert interpret(dummyfn1, (), type_system=self.ts) == False - assert interpret(dummyfn2, (), type_system=self.ts) == False - assert interpret(dummyfn3, (), type_system=self.ts) == True + assert self.interpret(dummyfn1, ()) == False + assert self.interpret(dummyfn2, ()) == False + assert self.interpret(dummyfn3, ()) == True def test_list_index_simple(self): def dummyfn(i): l = [5,6,7,8] return l.index(i) - res = interpret(dummyfn, (6,), type_system=self.ts) + res = self.interpret(dummyfn, (6,)) assert res == 1 - interpret_raises(ValueError, dummyfn, [42], type_system=self.ts) + interpret_raises(ValueError, dummyfn, [42]) -def test_insert_pop(): - def dummyfn(): - l = [6, 7, 8] - l.insert(0, 5) - l.insert(1, 42) - l.pop(2) - l.pop(0) - l.pop(-1) - l.pop() - return l[-1] - res = interpret(dummyfn, ())#, view=True) - assert res == 42 + def test_insert_pop(self): + def dummyfn(): + l = [6, 7, 8] + l.insert(0, 5) + l.insert(1, 42) + l.pop(2) + l.pop(0) + l.pop(-1) + l.pop() + return l[-1] + res = self.interpret(dummyfn, ())#, view=True) + assert res == 42 -def test_insert_bug(): - def dummyfn(n): - l = [1] - l = l[:] - l.pop(0) - if n < 0: - l.insert(0, 42) - else: - l.insert(n, 42) - return l - res = interpret(dummyfn, [0]) - assert res.ll_length() == 1 - assert res.ll_items()[0] == 42 - res = interpret(dummyfn, [-1]) - assert res.ll_length() == 1 - assert res.ll_items()[0] == 42 + def test_insert_bug(self): + def dummyfn(n): + l = [1] + l = l[:] + l.pop(0) + if n < 0: + l.insert(0, 42) + else: + l.insert(n, 42) + return l + res = self.interpret(dummyfn, [0]) + assert res.ll_length() == 1 + assert res.ll_getitem_fast(0) == 42 + res = self.interpret(dummyfn, [-1]) + assert res.ll_length() == 1 + assert res.ll_getitem_fast(0) == 42 -def test_inst_pop(): - class A: - pass - l = [A(), A()] - def f(idx): - try: - return l.pop(idx) - except IndexError: - return None - res = interpret(f, [1]) - assert ''.join(res.super.typeptr.name) == 'A\00' + def test_inst_pop(self): + class A: + pass + l = [A(), A()] + def f(idx): + try: + return l.pop(idx) + except IndexError: + return None + res = self.interpret(f, [1]) + assert self.class_name(res) == 'A' + #''.join(res.super.typeptr.name) == 'A\00' -def test_reverse(): - def dummyfn(): - l = [5, 3, 2] - l.reverse() - return l[0]*100 + l[1]*10 + l[2] - res = interpret(dummyfn, ()) - assert res == 235 + def test_reverse(self): + def dummyfn(): + l = [5, 3, 2] + l.reverse() + return l[0]*100 + l[1]*10 + l[2] + res = self.interpret(dummyfn, ()) + assert res == 235 - def dummyfn(): - l = [5] - l.append(3) - l.append(2) - l.reverse() - return l[0]*100 + l[1]*10 + l[2] - res = interpret(dummyfn, ()) - assert res == 235 - -def test_prebuilt_list(): - klist = ['a', 'd', 'z', 'k'] - def dummyfn(n): - return klist[n] - res = interpret(dummyfn, [0]) - assert res == 'a' - res = interpret(dummyfn, [3]) - assert res == 'k' - res = interpret(dummyfn, [-2]) - assert res == 'z' + def dummyfn(): + l = [5] + l.append(3) + l.append(2) + l.reverse() + return l[0]*100 + l[1]*10 + l[2] + res = self.interpret(dummyfn, ()) + assert res == 235 - klist = ['a', 'd', 'z'] - def mkdummyfn(): + def test_prebuilt_list(self): + klist = ['a', 'd', 'z', 'k'] def dummyfn(n): - klist.append('k') return klist[n] - return dummyfn - res = interpret(mkdummyfn(), [0]) - assert res == 'a' - res = interpret(mkdummyfn(), [3]) - assert res == 'k' - res = interpret(mkdummyfn(), [-2]) - assert res == 'z' - -def test_bound_list_method(): - klist = [1, 2, 3] - # for testing constant methods without actually mutating the constant - def dummyfn(n): - klist.extend([]) - interpret(dummyfn, [7]) - -def test_list_is(): - def dummyfn(): - l1 = [] - return l1 is l1 - res = interpret(dummyfn, []) - assert res is True - def dummyfn(): - l2 = [1, 2] - return l2 is l2 - res = interpret(dummyfn, []) - assert res is True - def dummyfn(): - l1 = [2] - l2 = [1, 2] - return l1 is l2 - res = interpret(dummyfn, []) - assert res is False - def dummyfn(): - l1 = [1, 2] - l2 = [1] - l2.append(2) - return l1 is l2 - res = interpret(dummyfn, []) - assert res is False - - def dummyfn(): - l1 = None - l2 = [1, 2] - return l1 is l2 - res = interpret(dummyfn, []) - assert res is False - - def dummyfn(): - l1 = None - l2 = [1] - l2.append(2) - return l1 is l2 - res = interpret(dummyfn, []) - assert res is False - -def test_list_compare(): - def fn(i, j, neg=False): - s1 = [[1, 2, 3], [4, 5, 1], None] - s2 = [[1, 2, 3], [4, 5, 1], [1], [1, 2], [4, 5, 1, 6], - [7, 1, 1, 8, 9, 10], None] - if neg: return s1[i] != s2[i] - return s1[i] == s2[j] - for i in range(3): - for j in range(7): - for case in False, True: - res = interpret(fn, [i,j,case]) - assert res is fn(i, j, case) - - def fn(i, j, neg=False): - s1 = [[1, 2, 3], [4, 5, 1], None] - l = [] - l.extend([1,2,3]) - s2 = [l, [4, 5, 1], [1], [1, 2], [4, 5, 1, 6], - [7, 1, 1, 8, 9, 10], None] - if neg: return s1[i] != s2[i] - return s1[i] == s2[j] - for i in range(3): - for j in range(7): - for case in False, True: - res = interpret(fn, [i,j,case]) - assert res is fn(i, j, case) - + res = self.interpret(dummyfn, [0]) + assert res == 'a' + res = self.interpret(dummyfn, [3]) + assert res == 'k' + res = self.interpret(dummyfn, [-2]) + assert res == 'z' + + klist = ['a', 'd', 'z'] + def mkdummyfn(): + def dummyfn(n): + klist.append('k') + return klist[n] + return dummyfn + res = self.interpret(mkdummyfn(), [0]) + assert res == 'a' + res = self.interpret(mkdummyfn(), [3]) + assert res == 'k' + res = self.interpret(mkdummyfn(), [-2]) + assert res == 'z' + + def test_bound_list_method(self): + klist = [1, 2, 3] + # for testing constant methods without actually mutating the constant + def dummyfn(n): + klist.extend([]) + self.interpret(dummyfn, [7]) -def test_list_comparestr(): - def fn(i, j, neg=False): - s1 = [["hell"], ["hello", "world"]] - s1[0][0] += "o" # ensure no interning - s2 = [["hello"], ["world"]] - if neg: return s1[i] != s2[i] - return s1[i] == s2[j] - for i in range(2): - for j in range(2): - for case in False, True: - res = interpret(fn, [i,j,case]) - assert res is fn(i, j, case) + def test_list_is(self): + def dummyfn(): + l1 = [] + return l1 is l1 + res = self.interpret(dummyfn, []) + assert res is True + def dummyfn(): + l2 = [1, 2] + return l2 is l2 + res = self.interpret(dummyfn, []) + assert res is True + def dummyfn(): + l1 = [2] + l2 = [1, 2] + return l1 is l2 + res = self.interpret(dummyfn, []) + assert res is False + def dummyfn(): + l1 = [1, 2] + l2 = [1] + l2.append(2) + return l1 is l2 + res = self.interpret(dummyfn, []) + assert res is False -class Foo: pass + def dummyfn(): + l1 = None + l2 = [1, 2] + return l1 is l2 + res = self.interpret(dummyfn, []) + assert res is False -class Bar(Foo): pass + def dummyfn(): + l1 = None + l2 = [1] + l2.append(2) + return l1 is l2 + res = self.interpret(dummyfn, []) + assert res is False + + def test_list_compare(self): + def fn(i, j, neg=False): + s1 = [[1, 2, 3], [4, 5, 1], None] + s2 = [[1, 2, 3], [4, 5, 1], [1], [1, 2], [4, 5, 1, 6], + [7, 1, 1, 8, 9, 10], None] + if neg: return s1[i] != s2[i] + return s1[i] == s2[j] + for i in range(3): + for j in range(7): + for case in False, True: + res = self.interpret(fn, [i,j,case]) + assert res is fn(i, j, case) -def test_list_compareinst(): - def fn(i, j, neg=False): - foo1 = Foo() - foo2 = Foo() - bar1 = Bar() - s1 = [[foo1], [foo2], [bar1]] - s2 = s1[:] - if neg: return s1[i] != s2[i] - return s1[i] == s2[j] - for i in range(3): - for j in range(3): + def fn(i, j, neg=False): + s1 = [[1, 2, 3], [4, 5, 1], None] + l = [] + l.extend([1,2,3]) + s2 = [l, [4, 5, 1], [1], [1, 2], [4, 5, 1, 6], + [7, 1, 1, 8, 9, 10], None] + if neg: return s1[i] != s2[i] + return s1[i] == s2[j] + for i in range(3): + for j in range(7): + for case in False, True: + res = self.interpret(fn, [i,j,case]) + assert res is fn(i, j, case) + + + def test_list_comparestr(self): + def fn(i, j, neg=False): + s1 = [["hell"], ["hello", "world"]] + s1[0][0] += "o" # ensure no interning + s2 = [["hello"], ["world"]] + if neg: return s1[i] != s2[i] + return s1[i] == s2[j] + for i in range(2): + for j in range(2): + for case in False, True: + res = self.interpret(fn, [i,j,case]) + assert res is fn(i, j, case) + + + def test_list_compareinst(self): + def fn(i, j, neg=False): + foo1 = Foo() + foo2 = Foo() + bar1 = Bar() + s1 = [[foo1], [foo2], [bar1]] + s2 = s1[:] + if neg: return s1[i] != s2[i] + return s1[i] == s2[j] + for i in range(3): + for j in range(3): + for case in False, True: + res = self.interpret(fn, [i, j, case]) + assert res is fn(i, j, case) + + def fn(i, j, neg=False): + foo1 = Foo() + foo2 = Foo() + bar1 = Bar() + s1 = [[foo1], [foo2], [bar1]] + s2 = s1[:] + + s2[0].extend([]) + + if neg: return s1[i] != s2[i] + return s1[i] == s2[j] + for i in range(3): + for j in range(3): + for case in False, True: + res = self.interpret(fn, [i, j, case]) + assert res is fn(i, j, case) + + + def test_list_contains(self): + def fn(i, neg=False): + foo1 = Foo() + foo2 = Foo() + bar1 = Bar() + bar2 = Bar() + lis = [foo1, foo2, bar1] + args = lis + [bar2] + if neg : return args[i] not in lis + return args[i] in lis + for i in range(4): for case in False, True: - res = interpret(fn, [i, j, case]) - assert res is fn(i, j, case) - - def fn(i, j, neg=False): - foo1 = Foo() - foo2 = Foo() - bar1 = Bar() - s1 = [[foo1], [foo2], [bar1]] - s2 = s1[:] + res = self.interpret(fn, [i, case]) + assert res is fn(i, case) - s2[0].extend([]) - - if neg: return s1[i] != s2[i] - return s1[i] == s2[j] - for i in range(3): - for j in range(3): + def fn(i, neg=False): + foo1 = Foo() + foo2 = Foo() + bar1 = Bar() + bar2 = Bar() + lis = [foo1, foo2, bar1] + lis.append(lis.pop()) + args = lis + [bar2] + if neg : return args[i] not in lis + return args[i] in lis + for i in range(4): for case in False, True: - res = interpret(fn, [i, j, case]) - assert res is fn(i, j, case) - - -def test_list_contains(): - def fn(i, neg=False): - foo1 = Foo() - foo2 = Foo() - bar1 = Bar() - bar2 = Bar() - lis = [foo1, foo2, bar1] - args = lis + [bar2] - if neg : return args[i] not in lis - return args[i] in lis - for i in range(4): - for case in False, True: - res = interpret(fn, [i, case]) - assert res is fn(i, case) - - def fn(i, neg=False): - foo1 = Foo() - foo2 = Foo() - bar1 = Bar() - bar2 = Bar() - lis = [foo1, foo2, bar1] - lis.append(lis.pop()) - args = lis + [bar2] - if neg : return args[i] not in lis - return args[i] in lis - for i in range(4): - for case in False, True: - res = interpret(fn, [i, case]) - assert res is fn(i, case) + res = self.interpret(fn, [i, case]) + assert res is fn(i, case) -def test_not_a_char_list_after_all(): - def fn(): - l = ['h', 'e', 'l', 'l', 'o'] - return 'world' in l - res = interpret(fn, []) - assert res is False + def test_not_a_char_list_after_all(self): + def fn(): + l = ['h', 'e', 'l', 'l', 'o'] + return 'world' in l + res = self.interpret(fn, []) + assert res is False def test_list_index(): def fn(i): @@ -1093,9 +1105,16 @@ def ll_to_list(self, l): return map(None, l.ll_items())[:l.ll_length()] + def class_name(self, value): + return "".join(value.super.typeptr.name)[:-1] + + class TestOotypeRtyping(BaseTestListRtyping): ts = "ootype" def ll_to_list(self, l): return l._list[:] + + def class_name(self, value): + return ootype.dynamicType(value)._name.split(".")[-1] Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Mon Apr 17 16:44:58 2006 @@ -32,10 +32,13 @@ i += 1 return total -def bar(x, y): - lst = [1,2,3,x,y] + [1,2] - #return sum_(list(lst)) - return lst[4] +class A: + pass + +##def bar(x, y): +## lst = [A(), A()] +## return lst.pop(1) + f = compile_function(bar, [int, int]) From cfbolz at codespeak.net Mon Apr 17 23:25:10 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 17 Apr 2006 23:25:10 +0200 (CEST) Subject: [pypy-svn] r25913 - pypy/dist/pypy/translator/goal Message-ID: <20060417212510.6C76610094@code0.codespeak.net> Author: cfbolz Date: Mon Apr 17 23:25:08 2006 New Revision: 25913 Added: pypy/dist/pypy/translator/goal/targetpushpop.py Log: small target that shows a push and a pop for checking the assembly that is generated for this Added: pypy/dist/pypy/translator/goal/targetpushpop.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetpushpop.py Mon Apr 17 23:25:08 2006 @@ -0,0 +1,36 @@ +""" +A simple standalone target. + +The target below specifies None as the argument types list. +This is a case treated specially in driver.py . If the list +of input types is empty, it is meant to be a list of strings, +actually implementing argv of the executable. +""" + +import os, sys + +# __________ Entry point __________ + +class A(object): + pass + +class B(object): + pass + +def f(x): + b = B() + b.x = x + return b + +global_a = A() + +def entry_point(argv): + # push argv + global_a.b = f(len(argv)) + # pop argv + return global_a.b.x + +# _____ Define and setup target ___ + +def target(*args): + return entry_point, None From cfbolz at codespeak.net Mon Apr 17 23:38:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 17 Apr 2006 23:38:17 +0200 (CEST) Subject: [pypy-svn] r25914 - pypy/dist/pypy/translator/goal Message-ID: <20060417213817.A1C8F10094@code0.codespeak.net> Author: cfbolz Date: Mon Apr 17 23:38:04 2006 New Revision: 25914 Modified: pypy/dist/pypy/translator/goal/targetpushpop.py Log: more pushs and pops in a row Modified: pypy/dist/pypy/translator/goal/targetpushpop.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpushpop.py (original) +++ pypy/dist/pypy/translator/goal/targetpushpop.py Mon Apr 17 23:38:04 2006 @@ -25,10 +25,24 @@ global_a = A() def entry_point(argv): - # push argv - global_a.b = f(len(argv)) - # pop argv - return global_a.b.x + a1 = A() + a2 = A() + a3 = A() + a4 = A() + global_a.next = a1 + a1.x = 1 + a2.x = 2 + a3.x = 3 + a4.x = 4 + a1.next = a2 + a2.next = a3 + a3.next = a4 + a4.next = None + # push stuff + global_a.b = B() + global_a.b.x = len(argv) + # pop stuff + return a1.x + a2.x + a3.x + a4.x + global_a.b.x # _____ Define and setup target ___ From cfbolz at codespeak.net Mon Apr 17 23:59:55 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 17 Apr 2006 23:59:55 +0200 (CEST) Subject: [pypy-svn] r25915 - pypy/dist/pypy/translator/goal Message-ID: <20060417215955.3C89110094@code0.codespeak.net> Author: cfbolz Date: Mon Apr 17 23:59:53 2006 New Revision: 25915 Modified: pypy/dist/pypy/translator/goal/targetpushpop.py Log: don't inline the call we are interested in, to make it a it easier to decipher the assembler Modified: pypy/dist/pypy/translator/goal/targetpushpop.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpushpop.py (original) +++ pypy/dist/pypy/translator/goal/targetpushpop.py Mon Apr 17 23:59:53 2006 @@ -18,6 +18,8 @@ pass def f(x): + if x == 0: + return f(x - 1) b = B() b.x = x return b @@ -39,7 +41,7 @@ a3.next = a4 a4.next = None # push stuff - global_a.b = B() + global_a.b = f(len(argv)) global_a.b.x = len(argv) # pop stuff return a1.x + a2.x + a3.x + a4.x + global_a.b.x From antocuni at codespeak.net Tue Apr 18 00:02:22 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 00:02:22 +0200 (CEST) Subject: [pypy-svn] r25916 - in pypy/dist/pypy/rpython: . lltypesystem Message-ID: <20060417220222.ED9EA10092@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 00:02:16 2006 New Revision: 25916 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/rlist.py Log: rtype_alloc_and_set and ll_alloc_and_set have been moved to rpython/rlist.py and made typesystem indipendent, in order to use them in ootypesystem, too. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Tue Apr 18 00:02:16 2006 @@ -5,7 +5,7 @@ from pypy.rpython.rmodel import externalvsinternal from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ AbstractFixedSizeListRepr, AbstractListIteratorRepr, rtype_newlist, \ - ll_setitem_nonneg + rtype_alloc_and_set, ll_setitem_nonneg from pypy.rpython.rlist import dum_nocheck, dum_checkidx from pypy.rpython.lltypesystem.rslice import SliceRepr from pypy.rpython.lltypesystem.rslice import startstop_slice_repr, startonly_slice_repr @@ -362,24 +362,6 @@ llops.gendirectcall(ll_setitem_nonneg, v_func, v_result, ci, v_item) return v_result -def ll_alloc_and_set(LIST, count, item): - if count < 0: - count = 0 - l = LIST.ll_newlist(count) - if item: # as long as malloc it is known to zero the allocated memory avoid zeroing twice - i = 0 - items = l.ll_items() - while i < count: - items[i] = item - i += 1 - return l -ll_alloc_and_set.oopspec = 'newlist(count, item)' - -def rtype_alloc_and_set(hop): - r_list = hop.r_result - v_count, v_item = hop.inputargs(Signed, r_list.item_repr) - cLIST = hop.inputconst(Void, r_list.LIST) - return hop.gendirectcall(ll_alloc_and_set, cLIST, v_count, v_item) # ____________________________________________________________ # Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Tue Apr 18 00:02:16 2006 @@ -256,6 +256,12 @@ items_v = [hop.inputarg(r_listitem, arg=i) for i in range(nb_args)] return hop.rtyper.type_system.rlist.newlist(hop.llops, r_list, items_v) +def rtype_alloc_and_set(hop): + r_list = hop.r_result + v_count, v_item = hop.inputargs(Signed, r_list.item_repr) + cLIST = hop.inputconst(Void, r_list.LIST) + return hop.gendirectcall(ll_alloc_and_set, cLIST, v_count, v_item) + class __extend__(pairtype(AbstractBaseListRepr, AbstractBaseListRepr)): @@ -341,6 +347,18 @@ # be direct_call'ed from rtyped flow graphs, which means that they will # get flowed and annotated, mostly with SomePtr. +def ll_alloc_and_set(LIST, count, item): + if count < 0: + count = 0 + l = LIST.ll_newlist(count) + if item: # as long as malloc it is known to zero the allocated memory avoid zeroing twice + i = 0 + while i < count: + l.ll_setitem_fast(i, item) + i += 1 + return l +ll_alloc_and_set.oopspec = 'newlist(count, item)' + # return a nullptr() if lst is a list of pointers it, else None. Note # that if we are using ootypesystem there are not pointers, so we From antocuni at codespeak.net Tue Apr 18 00:20:41 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 00:20:41 +0200 (CEST) Subject: [pypy-svn] r25917 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060417222041.B3A951008F@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 00:20:37 2006 New Revision: 25917 Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py Log: Added support for rtype_alloc_and_set in ootypesystem's list. Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Tue Apr 18 00:20:37 2006 @@ -1,6 +1,6 @@ from pypy.annotation.pairtype import pairtype from pypy.rpython.rlist import AbstractBaseListRepr, AbstractListRepr, \ - AbstractListIteratorRepr, rtype_newlist + AbstractListIteratorRepr, rtype_newlist, rtype_alloc_and_set from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.rmodel import inputconst, externalvsinternal from pypy.rpython.lltypesystem.lltype import Signed, Void From antocuni at codespeak.net Tue Apr 18 00:22:46 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 00:22:46 +0200 (CEST) Subject: [pypy-svn] r25918 - in pypy/dist/pypy/rpython: . lltypesystem ootypesystem Message-ID: <20060417222246.DF4851008F@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 00:22:40 2006 New Revision: 25918 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/rlist.py Log: Added _ll_resize to the ADT/_GENERIC_METHODS list interface. Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Tue Apr 18 00:22:40 2006 @@ -177,6 +177,7 @@ "ll_setitem_fast": ll_setitem_fast, "_ll_resize_ge": _ll_list_resize_ge, "_ll_resize_le": _ll_list_resize_le, + "_ll_resize": _ll_list_resize, }) ) @@ -206,8 +207,6 @@ "ITEM": ITEM, "ll_getitem_fast": ll_fixed_getitem_fast, "ll_setitem_fast": ll_fixed_setitem_fast, - "_ll_resize_ge": _ll_list_resize_ge, - "_ll_resize_le": _ll_list_resize_le, }) self.LIST.become(ITEMARRAY) Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 18 00:22:40 2006 @@ -208,6 +208,7 @@ "ll_setitem_fast": Meth([Signed, self.ITEMTYPE_T], Void), "_ll_resize_ge": Meth([Signed], Void), "_ll_resize_le": Meth([Signed], Void), + "_ll_resize": Meth([Signed], Void), "append": Meth([self.ITEMTYPE_T], Void), "extend": Meth([self.SELFTYPE_T], Void), "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) @@ -558,6 +559,14 @@ del self._list[length:] assert len(self._list) <= length + def _ll_resize(self, length): + # NOT_RPYTHON + if length > len(self._list): + self._ll_resize_ge(length) + elif length < len(self._list): + self._ll_resize_le(length) + assert len(self._list) == length + def append(self, item): # NOT_RPYTHON assert typeOf(item) == self._TYPE._ITEMTYPE @@ -669,3 +678,4 @@ ROOT = Instance('Root', None, _is_root=True) + Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Tue Apr 18 00:22:40 2006 @@ -709,8 +709,8 @@ factor = 0 resultlen = length * factor res = l - #_ll_list_resize(res, resultlen) - res._ll_resize_ge(resultlen) + res._ll_resize(resultlen) + #res._ll_resize_ge(resultlen) j = length while j < resultlen: i = 0 From tismer at codespeak.net Tue Apr 18 05:45:22 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 18 Apr 2006 05:45:22 +0200 (CEST) Subject: [pypy-svn] r25921 - pypy/dist/pypy/annotation Message-ID: <20060418034522.D971C1008F@code0.codespeak.net> Author: tismer Date: Tue Apr 18 05:45:20 2006 New Revision: 25921 Modified: pypy/dist/pypy/annotation/model.py Log: clean up old comments which are not useful Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Tue Apr 18 05:45:20 2006 @@ -40,25 +40,6 @@ DEBUG = True # set to False to disable recording of debugging information TLS = tlsobject() -""" -Some history (Chris): - -As a first approach to break this thing down, I slottified -all of these objects. The result was not overwhelming: -A savingof 5MB, but four percent of slowdown, since object -comparison got much more expensive, by lacking a __dict__. - -So I trashed 8 hours of work, without a check-in. (Just -writing this here to leave *some* trace of work). - -Then I tried to make all instances unique and wrote a lot -of attribute tracking code here, locked write access -outside of __init__, and patched many modules and several -hundred lines of code. -Finally I came up with a better concept, which is postponed. - -""" - class SomeObject: """The set of all objects. Each instance stands for an arbitrary object about which nothing is known.""" From tismer at codespeak.net Tue Apr 18 06:02:35 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 18 Apr 2006 06:02:35 +0200 (CEST) Subject: [pypy-svn] r25922 - pypy/dist/pypy/annotation Message-ID: <20060418040235.9D4D61008F@code0.codespeak.net> Author: tismer Date: Tue Apr 18 06:02:24 2006 New Revision: 25922 Modified: pypy/dist/pypy/annotation/model.py Log: being more consistent about == and "is" Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Tue Apr 18 06:02:24 2006 @@ -546,7 +546,7 @@ raise ValueError("%sshould return a low-level type,\ngot instead %r" % ( info, s_val)) -ll_to_annotation_map = dict([(ll, ann) for ann,ll in annotation_to_ll_map]) +ll_to_annotation_map = dict([(ll, ann) for ann, ll in annotation_to_ll_map]) def lltype_to_annotation(T): try: @@ -657,7 +657,7 @@ else: flattened = args for arg in flattened: - if arg.__class__ == SomeObject and arg.knowntype != type: + if arg.__class__ is SomeObject and arg.knowntype is not type: return SomeObject() bookkeeper = pypy.annotation.bookkeeper.getbookkeeper() bookkeeper.warning("no precise annotation supplied for %s%r" % (name, args)) From tismer at codespeak.net Tue Apr 18 06:41:56 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 18 Apr 2006 06:41:56 +0200 (CEST) Subject: [pypy-svn] r25923 - pypy/dist/pypy/translator/backendopt Message-ID: <20060418044156.27CF51008F@code0.codespeak.net> Author: tismer Date: Tue Apr 18 06:41:53 2006 New Revision: 25923 Modified: pypy/dist/pypy/translator/backendopt/removenoops.py Log: after removal of traverse also remove the import Modified: pypy/dist/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/removenoops.py Tue Apr 18 06:41:53 2006 @@ -1,5 +1,4 @@ from pypy.objspace.flow.model import Block, Variable, Constant -from pypy.objspace.flow.model import traverse from pypy.rpython.lltypesystem.lltype import Void from pypy.translator.backendopt.support import log from pypy.translator import simplify From tismer at codespeak.net Tue Apr 18 07:33:43 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 18 Apr 2006 07:33:43 +0200 (CEST) Subject: [pypy-svn] r25924 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060418053343.5D18B1008F@code0.codespeak.net> Author: tismer Date: Tue Apr 18 07:33:40 2006 New Revision: 25924 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: sorry I can't make it. Don't nag me, I'm exhausted. Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Tue Apr 18 07:33:40 2006 @@ -1,4 +1,4 @@ -People coming to the Tokyo sprint 2006 +?People coming to the Tokyo sprint 2006 ================================================== People who have a ``?`` in their arrive/depart or accomodation @@ -9,7 +9,6 @@ Name Arrive/Depart Accomodation ==================== ============== ===================== Beatrice D?ring 21/4 - 25/4 -Christian Tismer 22/4 - 30/4 Akihabare Washington Hotel Eric van Riet Paap 18/4 - 1/5 Akihabare Washington Hotel Jacob Hall?n Laura Creighton @@ -27,6 +26,7 @@ Name Arrive/Depart Accomodation ==================== ============== ===================== Armin Rigo not coming +Christian Tismer not coming Ludovic Aubry Adrien Di Mascio Laura Creighton From lac at codespeak.net Tue Apr 18 08:05:11 2006 From: lac at codespeak.net (lac at codespeak.net) Date: Tue, 18 Apr 2006 08:05:11 +0200 (CEST) Subject: [pypy-svn] r25925 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060418060511.370031008A@code0.codespeak.net> Author: lac Date: Tue Apr 18 08:05:10 2006 New Revision: 25925 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: Confirming what most of you already know. Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Tue Apr 18 08:05:10 2006 @@ -10,8 +10,6 @@ ==================== ============== ===================== Beatrice D??ring 21/4 - 25/4 Eric van Riet Paap 18/4 - 1/5 Akihabare Washington Hotel -Jacob Hall??n -Laura Creighton Anders Lehmann 23/4 - 1/5 Akihabare Washington Hotel Niklaus Haldimann 20/4 - 1/5 Akihabare Washington Hotel Anders Chrigstr??m 22/4 - 30/4 Akihabare Washington Hotel @@ -27,6 +25,8 @@ ==================== ============== ===================== Armin Rigo not coming Christian Tismer not coming +Jacob Hall?n not coming +Laura Creighton not coming Ludovic Aubry Adrien Di Mascio Laura Creighton From hpk at codespeak.net Tue Apr 18 09:16:20 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 18 Apr 2006 09:16:20 +0200 (CEST) Subject: [pypy-svn] r25927 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060418071620.CB3BF1008F@code0.codespeak.net> Author: hpk Date: Tue Apr 18 09:16:18 2006 New Revision: 25927 Added: pypy/extradoc/sprintinfo/iceland/ pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Log: add a first draft for iceland announcement - to be send out today Added: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Tue Apr 18 09:16:18 2006 @@ -0,0 +1,22 @@ +Hi folks, + +from 21st (arrival) till 28th (departure) May some of the PyPy +group are going to meet in Iceland and sprint on the upcoming +0.9 release of PyPy. This sprint is kindly hosted and sponsored +by EWT and CCPgames and thus we should be able to arrange +funding for travels and accomodation. There are also non-pypy +activities going on and people such as Tim Peters and other +python-dev core developers plan to attend. If you'd like to +help and join PyPy hacking around some vulcanos (actually +we are meeting in Reykjavik, but well :), please mail Armin +or Holger until end this week (friday 21st). + +We intend to give a PyPy tutorial but will otherwise focus on +release goals, so we may select participants by their +interest and ability to help there. The release targets +a variety of features (extension module compiler + extension coding, +stackless transformations, documentation & testing, many other +little topics like work on the CLI backend?) so don't feel +easily discouraged :) + +... From hpk at codespeak.net Tue Apr 18 10:16:38 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 18 Apr 2006 10:16:38 +0200 (CEST) Subject: [pypy-svn] r25928 - pypy/extradoc/minute Message-ID: <20060418081638.D17061008F@code0.codespeak.net> Author: hpk Date: Tue Apr 18 10:16:32 2006 New Revision: 25928 Added: pypy/extradoc/minute/invite-pypy-sync-04-20-2006.txt Log: invitation draft to pypy-sync this week Added: pypy/extradoc/minute/invite-pypy-sync-04-20-2006.txt ============================================================================== --- (empty file) +++ pypy/extradoc/minute/invite-pypy-sync-04-20-2006.txt Tue Apr 18 10:16:32 2006 @@ -0,0 +1,52 @@ +Hi there, + +the next pypy-sync meeting of active developers is + + THURSDAY, 20th April, 5:30 PM GMT+2 + +with the following topics (plus the ones that get mailed ahead +of the meeting ): + +- activity reports + blockers + +- summer of X, how do we go about it? + there are three possibilities: + + - aim for Summer of PyPy through the EU (becomes unlikely) + - aim for becoming an "Summer of Code" organisation + - provide mentors and participate at Googles Summer-of-Code + through the PSF and try to get a bit of external + funding for having participants attend sprints? + + decision time :) + +- what needs to be done until Iceland (21st May) for 0.9? + Samuele and Holger went through the issue tracker + and identified 0.9 release issues in Leysin. But we are + missing assignments ... could everyone have a look and take + responsibility for resolving particular issues? + + https://codespeak.net/issue/pypy-dev/ + + (and hit "Show 0.9") + +- tackling and assigning the major 0.9 tasks to be worked + on until Iceland: + + - integrating/refining the new stackless CFG transformations + instead of the genc-support + - adding missing support (if any) for app level stackless module + (e.g. yield_current_frame_to_caller ?) + - implementing tasklet pickling + - advancing rctypes to support interfacing with CPython runtime + through a CPyObjSpace + - tackling the actual extension module compiler tool + (including testing improvements and writing a tutorial) + - overhauling and updating translation documentation + (also in preparation for EU reports pending in June) + +best, + + holger + + From arigo at codespeak.net Tue Apr 18 10:23:38 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 10:23:38 +0200 (CEST) Subject: [pypy-svn] r25929 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060418082338.5F59710084@code0.codespeak.net> Author: arigo Date: Tue Apr 18 10:23:37 2006 New Revision: 25929 Modified: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Log: Updated. I tried to follow the results of the IRC discussion. Modified: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt (original) +++ pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Tue Apr 18 10:23:37 2006 @@ -1,22 +1,41 @@ Hi folks, -from 21st (arrival) till 28th (departure) May some of the PyPy -group are going to meet in Iceland and sprint on the upcoming -0.9 release of PyPy. This sprint is kindly hosted and sponsored -by EWT and CCPgames and thus we should be able to arrange -funding for travels and accomodation. There are also non-pypy -activities going on and people such as Tim Peters and other -python-dev core developers plan to attend. If you'd like to -help and join PyPy hacking around some vulcanos (actually -we are meeting in Reykjavik, but well :), please mail Armin -or Holger until end this week (friday 21st). - -We intend to give a PyPy tutorial but will otherwise focus on -release goals, so we may select participants by their -interest and ability to help there. The release targets -a variety of features (extension module compiler + extension coding, -stackless transformations, documentation & testing, many other -little topics like work on the CLI backend?) so don't feel -easily discouraged :) +We're pleased to announce the next PyPy sprint in Iceland from 21st +(arrival) to 28th (depature) of May 2006. This sprint is kindly hosted +and sponsored by EWT and CCPgames and thus we should be able to arrange +funding for travels and accomodation for interested people. There are +also non-PyPy activities going on -- see the full `EWT announcement`_ +and people such as Tim Peters and other python-dev core developers plan +to attend. If you'd like to help and join PyPy hacking around some +geysers (actually we are meeting in Reykjavik, but well :), please mail +us, preferrably on the `pypy-sprint mailing list`_. Please at least +mention that you are interested *until end this week* (friday 21st). -... +The sprint goals for PyPy are, as usual, kept open to the interest of +the participants, especially more so that there will be many non-PyPy +people at the sprint to talk to. However, it is also likely that we +will have to keep some focus on the goals of the upcoming June 0.9 +release: + +* The extension module compiler. The goal is to be able to use a single + RPython source code as an extension module for both PyPy and CPython. + The means to get there is -- most likely -- by compiling ctypes-based + modules into either pypy-c or a CPython dll/so. + +* Write some more modules in this style with ctypes. Sockets come to + mind :-) + +* Stackless: the big missing feature is pickling running tasklets. + There is also some smaller work that needs to be done, like exposing + all the existing RPython-level interfaces to app-level + (e.g. greenlets). + +* Write more documentation. + +* Misc topics, depending on interests: back-ends (CLI, Squeak), testing + framework, modified semantics (security/sandboxing...), etc. + + + +.. _`EWT announcement`: http://mail.python.org/pipermail/python-announce-list/2006-April/004849.html +.. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint From hpk at codespeak.net Tue Apr 18 10:35:38 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 18 Apr 2006 10:35:38 +0200 (CEST) Subject: [pypy-svn] r25930 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060418083538.568071008F@code0.codespeak.net> Author: hpk Date: Tue Apr 18 10:35:36 2006 New Revision: 25930 Modified: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Log: minor adjustment and footer - can go out from my side Modified: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt (original) +++ pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Tue Apr 18 10:35:36 2006 @@ -8,8 +8,9 @@ and people such as Tim Peters and other python-dev core developers plan to attend. If you'd like to help and join PyPy hacking around some geysers (actually we are meeting in Reykjavik, but well :), please mail -us, preferrably on the `pypy-sprint mailing list`_. Please at least -mention that you are interested *until end this week* (friday 21st). +us, preferrably on the `pypy-sprint mailing list`_. Please state your +interest *until end this week* (friday 21st) as planes and hotels +need to be booked. The sprint goals for PyPy are, as usual, kept open to the interest of the participants, especially more so that there will be many non-PyPy @@ -36,6 +37,10 @@ framework, modified semantics (security/sandboxing...), etc. +a bientot, + + Armin + Holger + .. _`EWT announcement`: http://mail.python.org/pipermail/python-announce-list/2006-April/004849.html .. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint From arigo at codespeak.net Tue Apr 18 10:36:39 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 10:36:39 +0200 (CEST) Subject: [pypy-svn] r25931 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060418083639.DF8B41008A@code0.codespeak.net> Author: arigo Date: Tue Apr 18 10:36:37 2006 New Revision: 25931 Modified: pypy/dist/pypy/translator/backendopt/malloc.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: A test that shows a situation where malloc removal needs an extra 'keepalive' to be valid. Fix. Modified: pypy/dist/pypy/translator/backendopt/malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/malloc.py Tue Apr 18 10:36:37 2006 @@ -153,11 +153,12 @@ for block, vars in variables_by_block.items(): - def flowin(var, newvarsmap): + def flowin(var, newvarsmap, insert_keepalive=False): # in this 'block', follow where the 'var' goes to and replace # it by a flattened-out family of variables. This family is given # by newvarsmap, whose keys are the 'flatnames'. vars = {var: True} + last_removed_access = None def list_newvars(): return [newvarsmap[key] for key in flatnames] @@ -175,11 +176,13 @@ [newvarsmap[S, fldname]], op.result) newops.append(newop) + last_removed_access = len(newops) elif op.opname == "setfield": S = op.args[0].concretetype.TO fldname = op.args[1].value assert (S, fldname) in newvarsmap newvarsmap[S, fldname] = op.args[2] + last_removed_access = len(newops) elif op.opname in ("same_as", "cast_pointer"): assert op.result not in vars vars[op.result] = True @@ -188,7 +191,7 @@ # flattened list of variables for both, as a "setfield" # via one pointer must be reflected in the other. elif op.opname == 'keepalive': - pass + last_removed_access = len(newops) else: raise AssertionError, op.opname elif op.result in vars: @@ -198,7 +201,7 @@ # drop the "malloc" operation else: newops.append(op) - block.operations[:] = newops + assert block.exitswitch not in vars for link in block.exits: @@ -206,10 +209,24 @@ for arg in link.args: if arg in vars: newargs += list_newvars() + insert_keepalive = False # kept alive by the link else: newargs.append(arg) link.args[:] = newargs + if insert_keepalive and last_removed_access is not None: + keepalives = [] + for v in list_newvars(): + T = v.concretetype + if isinstance(T, lltype.Ptr) and T._needsgc(): + v0 = Variable() + v0.concretetype = lltype.Void + newop = SpaceOperation('keepalive', [v], v0) + keepalives.append(newop) + newops[last_removed_access:last_removed_access] = keepalives + + block.operations[:] = newops + # look for variables arriving from outside the block for var in vars: if var in block.inputargs: @@ -224,7 +241,7 @@ newinputargs += block.inputargs[i+1:] block.inputargs[:] = newinputargs assert var not in block.inputargs - flowin(var, newvarsmap) + flowin(var, newvarsmap, insert_keepalive=True) # look for variables created inside the block by a malloc vars_created_here = [] Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Tue Apr 18 10:36:37 2006 @@ -3,6 +3,7 @@ from pypy.translator.backendopt.inline import inline_function from pypy.translator.backendopt.all import backend_optimizations from pypy.translator.translator import TranslationContext, graphof +from pypy.translator import simplify from pypy.objspace.flow.model import checkgraph, flatten, Block from pypy.rpython.llinterp import LLInterpreter from pypy.conftest import option @@ -28,6 +29,8 @@ if option.view: t.view() remove_simple_mallocs(graph) + # to detect missing keepalives: + simplify.transform_dead_op_vars_in_blocks(list(graph.iterblocks())) if option.view: t.view() if must_be_removed: @@ -146,3 +149,33 @@ op = graph.startblock.exits[0].target.exits[1].target.operations[0] assert op.opname == "malloc" +def test_add_keepalives(): + from pypy.rpython.lltypesystem import lltype + class A: + pass + SMALL = lltype.Struct('SMALL', ('x', lltype.Signed)) + BIG = lltype.GcStruct('BIG', ('z', lltype.Signed), ('s', SMALL)) + def fn7(i): + big = lltype.malloc(BIG) + a = A() + a.big = big + a.small = big.s + while i > 0: + a.small.x += i + i -= 1 + return a.small.x + check(fn7, [int], [10], 55, must_be_removed=False) + +def test_getsubstruct(): + py.test.skip("in-progress") + from pypy.rpython.lltypesystem import lltype + SMALL = lltype.Struct('SMALL', ('x', lltype.Signed)) + BIG = lltype.GcStruct('BIG', ('z', lltype.Signed), ('s', SMALL)) + + def fn(n1, n2): + b = lltype.malloc(BIG) + b.z = n1 + b.s.x = n2 + return b.z - b.s.x + + check(fn, [int, int], [100, 58], 42) From arigo at codespeak.net Tue Apr 18 10:38:29 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 10:38:29 +0200 (CEST) Subject: [pypy-svn] r25932 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060418083829.2366B1008F@code0.codespeak.net> Author: arigo Date: Tue Apr 18 10:38:23 2006 New Revision: 25932 Modified: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Log: More tweaks. Modified: pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt (original) +++ pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt Tue Apr 18 10:38:23 2006 @@ -9,8 +9,8 @@ to attend. If you'd like to help and join PyPy hacking around some geysers (actually we are meeting in Reykjavik, but well :), please mail us, preferrably on the `pypy-sprint mailing list`_. Please state your -interest *until end this week* (friday 21st) as planes and hotels -need to be booked. +interest *until end this week* (friday 21st) to help the organizers +plan for the sprint. The sprint goals for PyPy are, as usual, kept open to the interest of the participants, especially more so that there will be many non-PyPy @@ -37,7 +37,7 @@ framework, modified semantics (security/sandboxing...), etc. -a bientot, +A bientot, Armin + Holger From arigo at codespeak.net Tue Apr 18 11:27:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 11:27:01 +0200 (CEST) Subject: [pypy-svn] r25933 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060418092701.B40511008F@code0.codespeak.net> Author: arigo Date: Tue Apr 18 11:27:00 2006 New Revision: 25933 Modified: pypy/dist/pypy/translator/backendopt/malloc.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: Malloc removal in the presence of random 'getsubstruct' operations. This is needed to remove the overhead of rctypes, but it's probably not buying us anything for the rest of PyPy. Modified: pypy/dist/pypy/translator/backendopt/malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/malloc.py Tue Apr 18 11:27:00 2006 @@ -19,6 +19,27 @@ self.usepoints.update(other.usepoints) +def equivalent_substruct(S, fieldname): + # we consider a pointer to a GcStruct S as equivalent to a + # pointer to a substructure 'S.fieldname' if it's the first + # inlined sub-GcStruct. As an extension we also allow a pointer + # to a GcStruct containing just one Struct to be equivalent to + # a pointer to that Struct only (although a mere cast_pointer + # would not allow casting). This is needed to malloc-remove + # the 'wrapper' GcStructs introduced by previous passes of + # malloc removal. + if not isinstance(S, lltype.GcStruct): + return False + if fieldname != S._names[0]: + return False + FIELDTYPE = S._flds[fieldname] + if isinstance(FIELDTYPE, lltype.GcStruct): + return True + if len(S._names) == 1 and isinstance(FIELDTYPE, lltype.Struct): + return 'wrapper' + return False + + def compute_lifetimes(graph): """Compute the static data flow of the graph: returns a list of LifeTime instances, each of which corresponds to a set of Variables from the graph. @@ -57,11 +78,17 @@ # special-case these operations to identify their input # and output variables union(node, op.args[0], node, op.result) - else: - for i in range(len(op.args)): - if isinstance(op.args[i], Variable): - set_use_point(node, op.args[i], "op", node, op, i) - set_creation_point(node, op.result, "op", node, op) + continue + if op.opname == 'getsubstruct': + S = op.args[0].concretetype.TO + if equivalent_substruct(S, op.args[1].value): + # assumed to be similar to a cast_pointer + union(node, op.args[0], node, op.result) + continue + for i in range(len(op.args)): + if isinstance(op.args[i], Variable): + set_use_point(node, op.args[i], "op", node, op, i) + set_creation_point(node, op.result, "op", node, op) if isinstance(node.exitswitch, Variable): set_use_point(node, node.exitswitch, "exitswitch", node) @@ -108,14 +135,18 @@ STRUCT = lltypes.keys()[0].TO assert isinstance(STRUCT, lltype.GcStruct) - # must be only ever accessed via getfield/setfield or touched by keepalive + # must be only ever accessed via getfield/setfield/getsubstruct + # or touched by keepalive. Note that same_as and cast_pointer + # are not recorded in usepoints. + VALID = dict.fromkeys([("getfield", 0), + ("setfield", 0), + ("getsubstruct", 0), + ("keepalive", 0)]) for up in info.usepoints: if up[0] != "op": return False kind, node, op, index = up - if (op.opname, index) in [("getfield", 0), - ("setfield", 0), - ("keepalive", 0)]: + if (op.opname, index) in VALID: continue # ok return False @@ -130,18 +161,42 @@ # success: replace each variable with a family of variables (one per field) example = STRUCT._container_example() + + # 'flatnames' is a list of (STRUCTTYPE, fieldname_in_that_struct) that + # describes the list of variables that should replace the single + # malloc'ed pointer variable that we are about to remove. For primitive + # or pointer fields, the new corresponding variable just stores the + # actual value. For substructures, if pointers to them are "equivalent" + # to pointers to the parent structure (see equivalent_substruct()) then + # they are just merged, and flatnames will also list the fields within + # that substructure. Other substructures are replaced by a single new + # variable which is a pointer to a GcStruct-wrapper; each is malloc'ed + # individually, in an exploded way. (The next malloc removal pass will + # get rid of them again, in the typical case.) flatnames = [] flatconstants = {} + needsubmallocs = [] + newvarstype = {} # map {item-of-flatnames: concretetype} + def flatten(S, example): start = 0 - if S._names and isinstance(S._flds[S._names[0]], lltype.GcStruct): + if S._names and equivalent_substruct(S, S._names[0]): flatten(S._flds[S._names[0]], getattr(example, S._names[0])) start = 1 for name in S._names[start:]: - flatnames.append((S, name)) - constant = Constant(getattr(example, name)) - constant.concretetype = lltype.typeOf(constant.value) - flatconstants[S, name] = constant + key = S, name + flatnames.append(key) + FIELDTYPE = S._flds[name] + if isinstance(FIELDTYPE, lltype.ContainerType): + needsubmallocs.append(key) + newvarstype[key] = lltype.Ptr(lltype.GcStruct('wrapper', + ('data', FIELDTYPE))) + else: + constant = Constant(getattr(example, name)) + constant.concretetype = FIELDTYPE + flatconstants[key] = constant + newvarstype[key] = FIELDTYPE + flatten(STRUCT, example) variables_by_block = {} @@ -192,6 +247,29 @@ # via one pointer must be reflected in the other. elif op.opname == 'keepalive': last_removed_access = len(newops) + elif op.opname == "getsubstruct": + S = op.args[0].concretetype.TO + fldname = op.args[1].value + equiv = equivalent_substruct(S, fldname) + if equiv == "wrapper": + # reading the only Struct field of a GcStruct: + # do it with a getsubstruct on the wrapper ptr var + v = newvarsmap[S, fldname] + newop = SpaceOperation("getsubstruct", + [v, op.args[1]], + op.result) + newops.append(newop) + elif equiv: + # exactly like a cast_pointer + assert op.result not in vars + vars[op.result] = True + else: + # (S, fldname) in flatnames and in needsubmallocs + newop = SpaceOperation("same_as", + [newvarsmap[S, fldname]], + op.result) + newops.append(newop) + last_removed_access = len(newops) else: raise AssertionError, op.opname elif op.result in vars: @@ -199,6 +277,17 @@ assert vars == {var: True} count[0] += 1 # drop the "malloc" operation + newvarsmap = flatconstants.copy() # zero initial values + # if there are substructures, they are now individually + # malloc'ed in an exploded way. (They will typically be + # removed again by the next malloc removal pass.) + for key in needsubmallocs: + v = Variable() + v.concretetype = newvarstype[key] + c = Constant(v.concretetype.TO, lltype.Void) + newop = SpaceOperation("malloc", [c], v) + newops.append(newop) + newvarsmap[key] = v else: newops.append(op) @@ -235,7 +324,7 @@ newvarsmap = {} for key in flatnames: newvar = Variable() - newvar.concretetype = flatconstants[key].concretetype + newvar.concretetype = newvarstype[key] newvarsmap[key] = newvar newinputargs.append(newvar) newinputargs += block.inputargs[i+1:] @@ -249,8 +338,7 @@ if op.opname == "malloc" and op.result in vars: vars_created_here.append(op.result) for var in vars_created_here: - newvarsmap = flatconstants.copy() # dummy initial values - flowin(var, newvarsmap) + flowin(var, newvarsmap=None) assert count[0] return count[0] @@ -261,7 +349,7 @@ lifetimes = compute_lifetimes(graph) progress = 0 for info in lifetimes: - progress += _try_inline_malloc(info) + progress += _try_inline_malloc(info) return progress def remove_simple_mallocs(graph): Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Tue Apr 18 11:27:00 2006 @@ -167,7 +167,6 @@ check(fn7, [int], [10], 55, must_be_removed=False) def test_getsubstruct(): - py.test.skip("in-progress") from pypy.rpython.lltypesystem import lltype SMALL = lltype.Struct('SMALL', ('x', lltype.Signed)) BIG = lltype.GcStruct('BIG', ('z', lltype.Signed), ('s', SMALL)) From arigo at codespeak.net Tue Apr 18 11:55:19 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 11:55:19 +0200 (CEST) Subject: [pypy-svn] r25934 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060418095519.5ADD91008F@code0.codespeak.net> Author: arigo Date: Tue Apr 18 11:55:18 2006 New Revision: 25934 Modified: pypy/dist/pypy/translator/backendopt/malloc.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: Fix rather broken code, as better testing shows. Modified: pypy/dist/pypy/translator/backendopt/malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/malloc.py Tue Apr 18 11:55:18 2006 @@ -36,7 +36,7 @@ if isinstance(FIELDTYPE, lltype.GcStruct): return True if len(S._names) == 1 and isinstance(FIELDTYPE, lltype.Struct): - return 'wrapper' + return True return False @@ -251,25 +251,19 @@ S = op.args[0].concretetype.TO fldname = op.args[1].value equiv = equivalent_substruct(S, fldname) - if equiv == "wrapper": - # reading the only Struct field of a GcStruct: - # do it with a getsubstruct on the wrapper ptr var - v = newvarsmap[S, fldname] - newop = SpaceOperation("getsubstruct", - [v, op.args[1]], - op.result) - newops.append(newop) - elif equiv: + if equiv: # exactly like a cast_pointer assert op.result not in vars vars[op.result] = True else: - # (S, fldname) in flatnames and in needsubmallocs - newop = SpaceOperation("same_as", - [newvarsmap[S, fldname]], + # do it with a getsubstruct on the independently + # malloc'ed GcStruct + v = newvarsmap[S, fldname] + cname = Constant('data', lltype.Void) + newop = SpaceOperation("getsubstruct", + [v, cname], op.result) newops.append(newop) - last_removed_access = len(newops) else: raise AssertionError, op.opname elif op.result in vars: Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Tue Apr 18 11:55:18 2006 @@ -1,5 +1,5 @@ import py -from pypy.translator.backendopt.malloc import remove_simple_mallocs +from pypy.translator.backendopt.malloc import remove_mallocs_once from pypy.translator.backendopt.inline import inline_function from pypy.translator.backendopt.all import backend_optimizations from pypy.translator.translator import TranslationContext, graphof @@ -28,16 +28,20 @@ graph = graphof(t, fn) if option.view: t.view() - remove_simple_mallocs(graph) - # to detect missing keepalives: - simplify.transform_dead_op_vars_in_blocks(list(graph.iterblocks())) - if option.view: - t.view() + # to detect missing keepalives and broken intermediate graphs, + # we do the loop ourselves instead of calling remove_simple_mallocs() + while True: + progress = remove_mallocs_once(graph) + simplify.transform_dead_op_vars_in_blocks(list(graph.iterblocks())) + if progress and option.view: + t.view() + interp = LLInterpreter(t.rtyper) + res = interp.eval_graph(graph, args) + assert res == expected_result + if not progress: + break if must_be_removed: check_malloc_removed(graph) - interp = LLInterpreter(t.rtyper) - res = interp.eval_graph(graph, args) - assert res == expected_result def test_fn1(): From arigo at codespeak.net Tue Apr 18 11:58:39 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 11:58:39 +0200 (CEST) Subject: [pypy-svn] r25935 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060418095839.6EDC61008F@code0.codespeak.net> Author: arigo Date: Tue Apr 18 11:58:38 2006 New Revision: 25935 Modified: pypy/dist/pypy/translator/backendopt/malloc.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: Minimal support for FixedSizeArray in malloc removal. Modified: pypy/dist/pypy/translator/backendopt/malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/malloc.py Tue Apr 18 11:58:38 2006 @@ -142,12 +142,18 @@ ("setfield", 0), ("getsubstruct", 0), ("keepalive", 0)]) + MAYBE_VALID = dict.fromkeys([("getarrayitem", 0), + ("setarrayitem", 0), + ("getarraysubstruct", 0)]) for up in info.usepoints: if up[0] != "op": return False kind, node, op, index = up if (op.opname, index) in VALID: continue # ok + if (op.opname, index) in MAYBE_VALID: + if isinstance(op.args[1], Constant): + continue # ok if the index is constant return False # must not remove mallocs of structures that have a RTTI with a destructor @@ -224,17 +230,21 @@ for arg in op.args[1:]: # should be the first arg only assert arg not in vars if op.args and op.args[0] in vars: - if op.opname == "getfield": + if op.opname in ("getfield", "getarrayitem"): S = op.args[0].concretetype.TO fldname = op.args[1].value + if op.opname == "getarrayitem": + fldname = 'item%d' % fldname newop = SpaceOperation("same_as", [newvarsmap[S, fldname]], op.result) newops.append(newop) last_removed_access = len(newops) - elif op.opname == "setfield": + elif op.opname in ("setfield", "setarrayitem"): S = op.args[0].concretetype.TO fldname = op.args[1].value + if op.opname == "setarrayitem": + fldname = 'item%d' % fldname assert (S, fldname) in newvarsmap newvarsmap[S, fldname] = op.args[2] last_removed_access = len(newops) @@ -247,9 +257,11 @@ # via one pointer must be reflected in the other. elif op.opname == 'keepalive': last_removed_access = len(newops) - elif op.opname == "getsubstruct": + elif op.opname in ("getsubstruct", "getarraysubstruct"): S = op.args[0].concretetype.TO fldname = op.args[1].value + if op.opname == "getarraysubstruct": + fldname = 'item%d' % fldname equiv = equivalent_substruct(S, fldname) if equiv: # exactly like a cast_pointer Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Tue Apr 18 11:58:38 2006 @@ -182,3 +182,17 @@ return b.z - b.s.x check(fn, [int, int], [100, 58], 42) + +def test_fixedsizearray(): + from pypy.rpython.lltypesystem import lltype + A = lltype.FixedSizeArray(lltype.Signed, 3) + S = lltype.GcStruct('S', ('a', A)) + + def fn(n1, n2): + s = lltype.malloc(S) + a = s.a + a[0] = n1 + a[2] = n2 + return a[0]-a[2] + + check(fn, [int, int], [100, 42], 58) From arigo at codespeak.net Tue Apr 18 11:59:34 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 11:59:34 +0200 (CEST) Subject: [pypy-svn] r25936 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060418095934.7109C1008F@code0.codespeak.net> Author: arigo Date: Tue Apr 18 11:59:33 2006 New Revision: 25936 Added: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py (contents, props changed) Log: Yay! malloc removal can now remove all this rctypes cruft, at least in this simple case. Added: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_overhead.py Tue Apr 18 11:59:33 2006 @@ -0,0 +1,39 @@ +""" +Check that backendopt is able to remove most of the overhead introduced +by the rctypes rtyping. +""" + +import pypy.rpython.rctypes.implementation +from pypy import conftest +from pypy.rpython.test.test_llinterp import gengraph +from pypy.translator.backendopt.all import backend_optimizations + +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + +from ctypes import c_int + + +def find_mallocs(func, argtypes): + t, typer, graph = gengraph(func, argtypes) + backend_optimizations(t) + if conftest.option.view: + t.view() + + result = [] + for block in t.graphs[0].iterblocks(): + for op in block.operations: + if op.opname.startswith('malloc'): + result.append(op.result.concretetype) + return result + + +def test_simple(): + def func(n): + x = c_int(n) + return x.value + + mallocs = find_mallocs(func, [int]) + assert not mallocs From arigo at codespeak.net Tue Apr 18 12:30:40 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 12:30:40 +0200 (CEST) Subject: [pypy-svn] r25938 - in pypy/dist/pypy/rpython: . rctypes rctypes/test Message-ID: <20060418103040.25C4F10088@code0.codespeak.net> Author: arigo Date: Tue Apr 18 12:30:38 2006 New Revision: 25938 Modified: pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_overhead.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Log: More passing minimal-overhead tests. Avoid generating dummy operations to add a zero offset. Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Tue Apr 18 12:30:38 2006 @@ -327,16 +327,21 @@ c_ofs1 = inputconst(lltype.Signed, llmemory.ArrayItemsOffset(SRCTYPE)) v_array_items_adr = llops.genop('adr_add', [v_array_adr, c_ofs1], resulttype = llmemory.Address) - c_ofs2 = inputconst(lltype.Signed, llmemory.ItemOffset(SRCTYPE.OF)) - v_ofs3 = llops.genop('int_mul', [c_ofs2, v_baseoffset], - resulttype = lltype.Signed) - v_base_adr = llops.genop('adr_add', [v_array_items_adr, v_ofs3], - resulttype = llmemory.Address) + if isinstance(v_baseoffset, Constant) and v_baseoffset.value == 0: + v_base_adr = v_array_items_adr + else: + c_ofs2 = inputconst(lltype.Signed, llmemory.ItemOffset(SRCTYPE.OF)) + v_ofs3 = llops.genop('int_mul', [c_ofs2, v_baseoffset], + resulttype = lltype.Signed) + v_base_adr = llops.genop('adr_add', [v_array_items_adr, v_ofs3], + resulttype = llmemory.Address) return llops.genop('cast_adr_to_ptr', [v_base_adr], resulttype = ARRAYPTRTYPE) def gen_add_itemoffset_to_pointer(llops, ITEMTYPE, v_ptr, v_index): "Generates address manipulations equivalent to the C expression ptr+index." + if isinstance(v_index, Constant) and v_index.value == 0: + return v_ptr from pypy.rpython.lltypesystem import llmemory v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], resulttype = llmemory.Address) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 18 12:30:38 2006 @@ -66,12 +66,9 @@ self = r_ptr v_ptr, v_index = hop.inputargs(self, lltype.Signed) v_c_ptr = self.getvalue(hop.llops, v_ptr) - if hop.args_s[1].is_constant() and hop.args_s[1].const == 0: - pass # skip adding the offset - else: - v_c_ptr = gen_add_itemoffset_to_pointer(hop.llops, - r_ptr.r_contents.ll_type, - v_c_ptr, v_index) + v_c_ptr = gen_add_itemoffset_to_pointer(hop.llops, + r_ptr.r_contents.ll_type, + v_c_ptr, v_index) return self.r_contents.return_c_data(hop.llops, v_c_ptr) def rtype_setitem((r_ptr, _), hop): Modified: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_overhead.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_overhead.py Tue Apr 18 12:30:38 2006 @@ -37,3 +37,19 @@ mallocs = find_mallocs(func, [int]) assert not mallocs + +def test_labs(): + from pypy.rpython.rctypes.test import test_rfunc + def func(n): + return test_rfunc.labs(n) + + mallocs = find_mallocs(func, [int]) + assert not mallocs + +def test_atoi(): + from pypy.rpython.rctypes.test import test_rfunc + def func(s): + return test_rfunc.atoi(s) + + mallocs = find_mallocs(func, [str]) + assert not mallocs Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Tue Apr 18 12:30:38 2006 @@ -9,6 +9,11 @@ from pypy.rpython.test.test_llinterp import interpret from pypy.translator.c.test.test_genc import compile +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + from ctypes import c_int, c_float, POINTER, pointer, Structure class Test_annotation: From arigo at codespeak.net Tue Apr 18 12:37:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 12:37:05 +0200 (CEST) Subject: [pypy-svn] r25939 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060418103705.B533210088@code0.codespeak.net> Author: arigo Date: Tue Apr 18 12:37:04 2006 New Revision: 25939 Modified: pypy/dist/pypy/rpython/rctypes/aprimitive.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Log: c_wchar support. No c_wchar_p so far. Modified: pypy/dist/pypy/rpython/rctypes/aprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/aprimitive.py Tue Apr 18 12:37:04 2006 @@ -1,12 +1,13 @@ from ctypes import c_char, c_byte, c_ubyte, c_short, c_ushort, c_int, c_uint from ctypes import c_long, c_ulong, c_longlong, c_ulonglong, c_float -from ctypes import c_double, c_char_p +from ctypes import c_double, c_wchar, c_char_p from pypy.annotation import model as annmodel from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype ctypes_annotation_list = { c_char: lltype.Char, + c_wchar: lltype.UniChar, c_byte: lltype.Signed, c_ubyte: lltype.Unsigned, c_short: lltype.Signed, Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Tue Apr 18 12:37:04 2006 @@ -1,7 +1,7 @@ from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype -from pypy.rpython.rmodel import IntegerRepr, FloatRepr, CharRepr +from pypy.rpython.rmodel import IntegerRepr, FloatRepr, CharRepr, UniCharRepr from pypy.rpython.error import TyperError from pypy.rpython.rctypes.rmodel import CTypesValueRepr @@ -37,7 +37,8 @@ class __extend__(pairtype(IntegerRepr, PrimitiveRepr), pairtype(FloatRepr, PrimitiveRepr), - pairtype(CharRepr, PrimitiveRepr)): + pairtype(CharRepr, PrimitiveRepr), + pairtype(UniCharRepr, PrimitiveRepr)): def convert_from_to((r_from, r_to), v, llops): # first convert 'v' to the precise expected low-level type r_input = r_to.rtyper.primitive_to_repr[r_to.ll_type] Modified: pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Tue Apr 18 12:37:04 2006 @@ -20,7 +20,7 @@ from ctypes import c_char, c_byte, c_ubyte, c_short, c_ushort, c_int, c_uint from ctypes import c_long, c_ulong, c_longlong, c_ulonglong, c_float -from ctypes import c_double, c_char_p, pointer +from ctypes import c_double, c_wchar, c_char_p, pointer class Test_annotation: def test_simple(self): @@ -224,6 +224,9 @@ assert x.value == 2.75 x.value -= 1 assert x.value == 1.75 + x = c_wchar(u'A') + x.value = unichr(ord(x.value) + 1) + assert x.value == u'B' interpret(func, []) def test_convert_from_llvalue(self): @@ -250,6 +253,9 @@ assert x.value == 2.75 pointer(x)[0] -= 1 assert x.value == 1.75 + x = c_wchar(u'A') + pointer(x)[0] = unichr(ord(pointer(x)[0]) + 1) + assert x.value == u'B' interpret(func, []) class Test_compilation: From arigo at codespeak.net Tue Apr 18 13:21:30 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 13:21:30 +0200 (CEST) Subject: [pypy-svn] r25940 - pypy/dist/pypy/jit/timeshifter Message-ID: <20060418112130.A33461008A@code0.codespeak.net> Author: arigo Date: Tue Apr 18 13:21:29 2006 New Revision: 25940 Modified: pypy/dist/pypy/jit/timeshifter/timeshift.py Log: Use Antonio's new rlist interface here. Modified: pypy/dist/pypy/jit/timeshifter/timeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/timeshift.py (original) +++ pypy/dist/pypy/jit/timeshifter/timeshift.py Tue Apr 18 13:21:29 2006 @@ -223,10 +223,8 @@ def read_out_box(self, llops, v_boxes, i): - c_dum_nocheck = rmodel.inputconst(lltype.Void, rlist.dum_nocheck) c_i = rmodel.inputconst(lltype.Signed, i) - s_dum_nocheck = self.rtyper.annotator.bookkeeper.immutablevalue(rlist.dum_nocheck) - v_box = llops.gendirectcall(rlist.ll_getitem_nonneg, c_dum_nocheck, v_boxes, c_i) + v_box = llops.gendirectcall(rlist.ll_getitem_fast, v_boxes, c_i) v_box = llops.convertvar(v_box, self.r_box_list.item_repr, self.r_RedBox) return v_box From auc at codespeak.net Tue Apr 18 13:30:43 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 18 Apr 2006 13:30:43 +0200 (CEST) Subject: [pypy-svn] r25942 - pypy/dist/pypy/lib/logic/computation_space Message-ID: <20060418113043.6F4311008A@code0.codespeak.net> Author: auc Date: Tue Apr 18 13:30:41 2006 New Revision: 25942 Modified: pypy/dist/pypy/lib/logic/computation_space/computationspace.py pypy/dist/pypy/lib/logic/computation_space/test_variable.py Log: disable test for now Modified: pypy/dist/pypy/lib/logic/computation_space/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/computationspace.py Tue Apr 18 13:30:41 2006 @@ -360,6 +360,6 @@ dom.reset_flags() if entailed: # we should also remove the constraint from - # the set of satifiable constraints + # the set of satifiable constraints of the space if const in affected_constraints: affected_constraints.remove(const) Modified: pypy/dist/pypy/lib/logic/computation_space/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/test_variable.py Tue Apr 18 13:30:41 2006 @@ -255,21 +255,6 @@ class EOL: pass -def generate(thread, Xs, n, limit): - """(eager generation of a stream 0|1|2|...) - declare - fun {Generate N Limit} - if N Author: antocuni Date: Tue Apr 18 13:42:18 2006 New Revision: 25943 Modified: pypy/dist/pypy/rpython/test/test_rlist.py Log: Added more tests for ootypesystem's list. Some tests skip because by now ootypesystem lack support for: - returning tuples of lists - strings - PyObject* - assert statement - _freeze_ The _skip_oo calls have to be removed as soon as ootypesystem support the feature above. Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Tue Apr 18 13:42:18 2006 @@ -4,10 +4,12 @@ from pypy.rpython.ootypesystem import ootype from pypy.rpython.rlist import * from pypy.rpython.lltypesystem.rlist import ListRepr, FixedSizeListRepr, ll_newlist, ll_fixed_newlist +from pypy.rpython.lltypesystem import rlist as ll_rlist +from pypy.rpython.ootypesystem import rlist as oo_rlist from pypy.rpython.lltypesystem.rslice import ll_newslice from pypy.rpython.rint import signed_repr -from pypy.rpython.test.test_llinterp import interpret #as interpret2 -from pypy.rpython.test.test_llinterp import interpret_raises # as interpret_raises2 +from pypy.rpython.test.test_llinterp import interpret +from pypy.rpython.test.test_llinterp import interpret_raises from pypy.translator.translator import TranslationContext from pypy.objspace.flow.model import Constant, Variable @@ -74,6 +76,16 @@ ll_listsetslice(l1, s, l2) self.check_list(l1, expected) + +# helper used by some tests below +def list_is_clear(lis, idx): + items = lis._obj.items._obj.items + for i in range(idx, len(items)): + if items[i]._obj is not None: + return False + return True + + class TestListImpl(BaseTestListImpl): @@ -161,11 +173,86 @@ self.check_list(lvar1, [42, 43, 44, 45] * 3) + # ____________________________________________________________ -class Foo: pass +# these classes are used in the tests below +class Foo: + pass + +class Bar(Foo): + pass + +class Freezing: + def _freeze_(self): + return True + +def test_list_builder(): + def fixed_size_case(): + return [42] + def variable_size_case(): + lst = [] + lst.append(42) + return lst + + from pypy.rpython import rgenop + from pypy.rpython.module import support + + class DummyBlockBuilder: + + def __init__(self): + self.newblock = rgenop.newblock() + self.bareblock = support.from_opaque_object(self.newblock.obj) + + def genop(self, opname, args, RESULT_TYPE): + return rgenop.genop(self.newblock, opname, args, + rgenop.constTYPE(RESULT_TYPE)) + + def genconst(self, llvalue): + return rgenop.genconst(llvalue) + + # inspection + def __getitem__(self, index): + return self.bareblock.operations[index] + + def __len__(self): + return len(self.bareblock.operations) + + + for fn in [fixed_size_case, variable_size_case]: + t = TranslationContext() + t.buildannotator().build_types(fn, []) + t.buildrtyper().specialize() + LIST = t.graphs[0].getreturnvar().concretetype.TO + llop = DummyBlockBuilder() + v0 = Constant(42) + v0.concretetype = Signed + opq_v0 = support.to_opaque_object(v0) + v1 = Variable() + v1.concretetype = Signed + opq_v1 = support.to_opaque_object(v1) + vr = LIST.list_builder(llop, [opq_v0, opq_v1]) + vr = rgenop.reveal(vr) + assert len(llop) == 3 + assert llop[0].opname == 'direct_call' + assert len(llop[0].args) == 3 + assert llop[0].args[1].concretetype == Void + assert llop[0].args[1].value == LIST + assert llop[0].args[2].concretetype == Signed + assert llop[0].args[2].value == 2 + assert llop[0].result is vr + for op, i, vi in [(llop[1], 0, v0), (llop[2], 1, v1)]: + assert op.opname == 'direct_call' + assert len(op.args) == 5 + assert op.args[1].value is dum_nocheck + assert op.args[2] is vr + assert op.args[3].concretetype == Signed + assert op.args[3].value == i + assert op.args[4] is vi + assert op.result.concretetype is Void + + -class Bar(Foo): pass class BaseTestListRtyping: @@ -175,9 +262,9 @@ def interpret_raises(self, exc, fn, args): return interpret_raises(exc, fn, args, type_system=self.ts) - def _skip_tuples(self): + def _skip_oo(self, reason): if self.ts == 'ootype': - py.test.skip("ootypesystem doesn't support returning tuples of lists, yet") + py.test.skip("ootypesystem doesn't support %s, yet" % reason) def test_simple(self): def dummyfn(): @@ -269,7 +356,7 @@ assert self.ll_to_list(res) == [5, 6, 7, 8, 9] def test_slice(self): - self._skip_tuples() + self._skip_oo('tuples of lists') def dummyfn(): l = [5, 6, 7, 8, 9] return l[:2], l[1:4], l[3:] @@ -319,7 +406,7 @@ assert res == 0 def test_setslice(self): - self._skip_tuples() + self._skip_oo('tuples of lists') def dummyfn(): l = [10, 9, 8, 7] l[:2] = [6, 5] @@ -621,497 +708,433 @@ res = self.interpret(fn, []) assert res is False -def test_list_index(): - def fn(i): - foo1 = Foo() - foo2 = Foo() - bar1 = Bar() - bar2 = Bar() - lis = [foo1, foo2, bar1] - args = lis + [bar2] - return lis.index(args[i]) - for i in range(4): - for varsize in False, True: - try: - res2 = fn(i) - res1 = interpret(fn, [i]) - assert res1 == res2 - except Exception, e: - interpret_raises(e.__class__, fn, [i]) - - def fn(i): - foo1 = Foo() - foo2 = Foo() - bar1 = Bar() - bar2 = Bar() - lis = [foo1, foo2, bar1] - args = lis + [bar2] - lis.append(lis.pop()) - return lis.index(args[i]) - for i in range(4): - for varsize in False, True: - try: - res2 = fn(i) - res1 = interpret(fn, [i]) - assert res1 == res2 - except Exception, e: - interpret_raises(e.__class__, fn, [i]) + def test_list_index(self): + def fn(i): + foo1 = Foo() + foo2 = Foo() + bar1 = Bar() + bar2 = Bar() + lis = [foo1, foo2, bar1] + args = lis + [bar2] + return lis.index(args[i]) + for i in range(4): + for varsize in False, True: + try: + res2 = fn(i) + res1 = self.interpret(fn, [i]) + assert res1 == res2 + except Exception, e: + self.interpret_raises(e.__class__, fn, [i]) + def fn(i): + foo1 = Foo() + foo2 = Foo() + bar1 = Bar() + bar2 = Bar() + lis = [foo1, foo2, bar1] + args = lis + [bar2] + lis.append(lis.pop()) + return lis.index(args[i]) + for i in range(4): + for varsize in False, True: + try: + res2 = fn(i) + res1 = self.interpret(fn, [i]) + assert res1 == res2 + except Exception, e: + self.interpret_raises(e.__class__, fn, [i]) -def test_list_str(): - def fn(): - return str([1,2,3]) - - res = interpret(fn, []) - assert ''.join(res.chars) == fn() - def fn(): - return str([[1,2,3]]) - - res = interpret(fn, []) - assert ''.join(res.chars) == fn() + def test_list_str(self): + self._skip_oo('strings') + def fn(): + return str([1,2,3]) - def fn(): - l = [1,2] - l.append(3) - return str(l) - - res = interpret(fn, []) - assert ''.join(res.chars) == fn() + res = self.interpret(fn, []) + assert ''.join(res.chars) == fn() - def fn(): - l = [1,2] - l.append(3) - return str([l]) - - res = interpret(fn, []) - assert ''.join(res.chars) == fn() + def fn(): + return str([[1,2,3]]) -def test_list_or_None(): - empty_list = [] - nonempty_list = [1, 2] - def fn(i): - test = [None, empty_list, nonempty_list][i] - if test: - return 1 - else: - return 0 - - res = interpret(fn, [0]) - assert res == 0 - res = interpret(fn, [1]) - assert res == 0 - res = interpret(fn, [2]) - assert res == 1 - - - nonempty_list = [1, 2] - def fn(i): - empty_list = [1] - empty_list.pop() - nonempty_list = [] - nonempty_list.extend([1,2]) - test = [None, empty_list, nonempty_list][i] - if test: - return 1 - else: - return 0 - - res = interpret(fn, [0]) - assert res == 0 - res = interpret(fn, [1]) - assert res == 0 - res = interpret(fn, [2]) - assert res == 1 - - -def test_inst_list(): - def fn(): - l = [None] - l[0] = Foo() - l.append(Bar()) - l2 = [l[1], l[0], l[0]] - l.extend(l2) - for x in l2: - l.append(x) - x = l.pop() - x = l.pop() - x = l.pop() - x = l2.pop() - return str(x)+";"+str(l) - res = interpret(fn, []) - assert ''.join(res.chars) == ';[, , , , ]' - - def fn(): - l = [None] * 2 - l[0] = Foo() - l[1] = Bar() - l2 = [l[1], l[0], l[0]] - l = l + [None] * 3 - i = 2 - for x in l2: - l[i] = x - i += 1 - return str(l) - res = interpret(fn, []) - assert ''.join(res.chars) == '[, , , , ]' - -def test_list_slice_minusone(): - def fn(i): - lst = [i, i+1, i+2] - lst2 = lst[:-1] - return lst[-1] * lst2[-1] - res = interpret(fn, [5]) - assert res == 42 - - def fn(i): - lst = [i, i+1, i+2, 7] - lst.pop() - lst2 = lst[:-1] - return lst[-1] * lst2[-1] - res = interpret(fn, [5]) - assert res == 42 - -def test_list_multiply(): - def fn(i): - lst = [i] * i - ret = len(lst) - if ret: - ret *= lst[-1] - return ret - for arg in (1, 9, 0, -1, -27): - res = interpret(fn, [arg]) - assert res == fn(arg) - def fn(i): - lst = [i, i + 1] * i - ret = len(lst) - if ret: - ret *= lst[-1] - return ret - for arg in (1, 9, 0, -1, -27): - res = interpret(fn, [arg]) - assert res == fn(arg) - -def test_list_inplace_multiply(): - def fn(i): - lst = [i] - lst *= i - ret = len(lst) - if ret: - ret *= lst[-1] - return ret - for arg in (1, 9, 0, -1, -27): - res = interpret(fn, [arg]) - assert res == fn(arg) - def fn(i): - lst = [i, i + 1] - lst *= i - ret = len(lst) - if ret: - ret *= lst[-1] - return ret - for arg in (1, 9, 0, -1, -27): - res = interpret(fn, [arg]) - assert res == fn(arg) - -def test_indexerror(): - def fn(i): - l = [5, 8, 3] - try: - l[i] = 99 - except IndexError: - pass - try: - del l[i] - except IndexError: - pass - try: - return l[2] # implicit int->PyObj conversion here - except IndexError: - return "oups" - res = interpret(fn, [6]) - assert res._obj.value == 3 - res = interpret(fn, [-2]) - assert res._obj.value == "oups" - - def fn(i): - l = [5, 8] - l.append(3) - try: - l[i] = 99 - except IndexError: - pass - try: - del l[i] - except IndexError: - pass - try: - return l[2] # implicit int->PyObj conversion here - except IndexError: - return "oups" - res = interpret(fn, [6]) - assert res._obj.value == 3 - res = interpret(fn, [-2]) - assert res._obj.value == "oups" + res = self.interpret(fn, []) + assert ''.join(res.chars) == fn() -def list_is_clear(lis, idx): - items = lis._obj.items._obj.items - for i in range(idx, len(items)): - if items[i]._obj is not None: - return False - return True + def fn(): + l = [1,2] + l.append(3) + return str(l) -def test_no_unneeded_refs(): - def fndel(p, q): - lis = ["5", "3", "99"] - assert q >= 0 - assert p >= 0 - del lis[p:q] - return lis - def fnpop(n): - lis = ["5", "3", "99"] - while n: - lis.pop() - n -=1 - return lis - for i in range(2, 3+1): - lis = interpret(fndel, [0, i]) - assert list_is_clear(lis, 3-i) - for i in range(3): - lis = interpret(fnpop, [i]) - assert list_is_clear(lis, 3-i) - -def test_list_basic_ops(): - def list_basic_ops(i=int, j=int): - l = [1,2,3] - l.insert(0, 42) - del l[1] - l.append(i) - listlen = len(l) - l.extend(l) - del l[listlen:] - l += [5,6] - l[1] = i - return l[j] - for i in range(6): - for j in range(6): - res = interpret(list_basic_ops, [i, j]) - assert res == list_basic_ops(i, j) - -def test_valueerror(): - def fn(i): - l = [4, 7, 3] - try: - j = l.index(i) - except ValueError: - j = 100 - return j - res = interpret(fn, [4]) - assert res == 0 - res = interpret(fn, [7]) - assert res == 1 - res = interpret(fn, [3]) - assert res == 2 - res = interpret(fn, [6]) - assert res == 100 - - def fn(i): - l = [5, 8] - l.append(3) - try: - l[i] = 99 - except IndexError: - pass - try: - del l[i] - except IndexError: - pass - try: - return l[2] # implicit int->PyObj conversion here - except IndexError: - return "oups" - res = interpret(fn, [6]) - assert res._obj.value == 3 - res = interpret(fn, [-2]) - assert res._obj.value == "oups" - -def test_memoryerror(): - def fn(i): - lst = [0] * i - lst[i-1] = 5 - return lst[0] - res = interpret(fn, [1]) - assert res == 5 - res = interpret(fn, [2]) - assert res == 0 - interpret_raises(MemoryError, fn, [sys.maxint]) + res = self.interpret(fn, []) + assert ''.join(res.chars) == fn() -def test_list_builder(): - def fixed_size_case(): - return [42] - def variable_size_case(): - lst = [] - lst.append(42) - return lst + def fn(): + l = [1,2] + l.append(3) + return str([l]) - from pypy.rpython import rgenop - from pypy.rpython.module import support + res = self.interpret(fn, []) + assert ''.join(res.chars) == fn() - class DummyBlockBuilder: + def test_list_or_None(self): + empty_list = [] + nonempty_list = [1, 2] + def fn(i): + test = [None, empty_list, nonempty_list][i] + if test: + return 1 + else: + return 0 - def __init__(self): - self.newblock = rgenop.newblock() - self.bareblock = support.from_opaque_object(self.newblock.obj) + res = self.interpret(fn, [0]) + assert res == 0 + res = self.interpret(fn, [1]) + assert res == 0 + res = self.interpret(fn, [2]) + assert res == 1 - def genop(self, opname, args, RESULT_TYPE): - return rgenop.genop(self.newblock, opname, args, - rgenop.constTYPE(RESULT_TYPE)) - def genconst(self, llvalue): - return rgenop.genconst(llvalue) + nonempty_list = [1, 2] + def fn(i): + empty_list = [1] + empty_list.pop() + nonempty_list = [] + nonempty_list.extend([1,2]) + test = [None, empty_list, nonempty_list][i] + if test: + return 1 + else: + return 0 - # inspection - def __getitem__(self, index): - return self.bareblock.operations[index] + res = self.interpret(fn, [0]) + assert res == 0 + res = self.interpret(fn, [1]) + assert res == 0 + res = self.interpret(fn, [2]) + assert res == 1 - def __len__(self): - return len(self.bareblock.operations) + def test_inst_list(self): + self._skip_oo('strings') + def fn(): + l = [None] + l[0] = Foo() + l.append(Bar()) + l2 = [l[1], l[0], l[0]] + l.extend(l2) + for x in l2: + l.append(x) + x = l.pop() + x = l.pop() + x = l.pop() + x = l2.pop() + return str(x)+";"+str(l) + res = self.interpret(fn, []) + assert ''.join(res.chars) == ';[, , , , ]' - for fn in [fixed_size_case, variable_size_case]: - t = TranslationContext() - t.buildannotator().build_types(fn, []) - t.buildrtyper().specialize() - LIST = t.graphs[0].getreturnvar().concretetype.TO - llop = DummyBlockBuilder() - v0 = Constant(42) - v0.concretetype = Signed - opq_v0 = support.to_opaque_object(v0) - v1 = Variable() - v1.concretetype = Signed - opq_v1 = support.to_opaque_object(v1) - vr = LIST.list_builder(llop, [opq_v0, opq_v1]) - vr = rgenop.reveal(vr) - assert len(llop) == 3 - assert llop[0].opname == 'direct_call' - assert len(llop[0].args) == 3 - assert llop[0].args[1].concretetype == Void - assert llop[0].args[1].value == LIST - assert llop[0].args[2].concretetype == Signed - assert llop[0].args[2].value == 2 - assert llop[0].result is vr - for op, i, vi in [(llop[1], 0, v0), (llop[2], 1, v1)]: - assert op.opname == 'direct_call' - assert len(op.args) == 5 - assert op.args[1].value is dum_nocheck - assert op.args[2] is vr - assert op.args[3].concretetype == Signed - assert op.args[3].value == i - assert op.args[4] is vi - assert op.result.concretetype is Void + def fn(): + l = [None] * 2 + l[0] = Foo() + l[1] = Bar() + l2 = [l[1], l[0], l[0]] + l = l + [None] * 3 + i = 2 + for x in l2: + l[i] = x + i += 1 + return str(l) + res = self.interpret(fn, []) + assert ''.join(res.chars) == '[, , , , ]' -class Freezing: - def _freeze_(self): - return True + def test_list_slice_minusone(self): + def fn(i): + lst = [i, i+1, i+2] + lst2 = lst[:-1] + return lst[-1] * lst2[-1] + res = self.interpret(fn, [5]) + assert res == 42 -def test_voidlist_prebuilt(): - frlist = [Freezing()] * 17 - def mylength(l): - return len(l) - def f(): - return mylength(frlist) - res = interpret(f, []) - assert res == 17 - -def test_voidlist_fixed(): - fr = Freezing() - def f(): - return len([fr, fr]) - res = interpret(f, []) - assert res == 2 - -def test_voidlist_nonfixed(): - class Freezing: - def _freeze_(self): - return True - fr = Freezing() - def f(): - lst = [fr, fr] - lst.append(fr) - del lst[1] - assert lst[0] is fr - return len(lst) - res = interpret(f, []) - assert res == 2 - - -def test_type_erase_fixed_size(): - class A(object): - pass - class B(object): - pass - - def f(): - return [A()], [B()] - - t = TranslationContext() - s = t.buildannotator().build_types(f, []) - rtyper = t.buildrtyper() - rtyper.specialize() + def fn(i): + lst = [i, i+1, i+2, 7] + lst.pop() + lst2 = lst[:-1] + return lst[-1] * lst2[-1] + res = self.interpret(fn, [5]) + assert res == 42 - s_A_list = s.items[0] - s_B_list = s.items[1] - - r_A_list = rtyper.getrepr(s_A_list) - assert isinstance(r_A_list, FixedSizeListRepr) - r_B_list = rtyper.getrepr(s_B_list) - assert isinstance(r_B_list, FixedSizeListRepr) - - assert r_A_list.lowleveltype == r_B_list.lowleveltype - -def test_type_erase_var_size(): - class A(object): - pass - class B(object): - pass - - def f(): - la = [A()] - lb = [B()] - la.append(None) - lb.append(None) - return la, lb - - t = TranslationContext() - s = t.buildannotator().build_types(f, []) - rtyper = t.buildrtyper() - rtyper.specialize() + def test_list_multiply(self): + def fn(i): + lst = [i] * i + ret = len(lst) + if ret: + ret *= lst[-1] + return ret + for arg in (1, 9, 0, -1, -27): + res = self.interpret(fn, [arg]) + assert res == fn(arg) + def fn(i): + lst = [i, i + 1] * i + ret = len(lst) + if ret: + ret *= lst[-1] + return ret + for arg in (1, 9, 0, -1, -27): + res = self.interpret(fn, [arg]) + assert res == fn(arg) + + def test_list_inplace_multiply(self): + def fn(i): + lst = [i] + lst *= i + ret = len(lst) + if ret: + ret *= lst[-1] + return ret + for arg in (1, 9, 0, -1, -27): + res = self.interpret(fn, [arg]) + assert res == fn(arg) + def fn(i): + lst = [i, i + 1] + lst *= i + ret = len(lst) + if ret: + ret *= lst[-1] + return ret + for arg in (1, 9, 0, -1, -27): + res = self.interpret(fn, [arg]) + assert res == fn(arg) + + def test_indexerror(self): + self._skip_oo('PyObject*') + def fn(i): + l = [5, 8, 3] + try: + l[i] = 99 + except IndexError: + pass + try: + del l[i] + except IndexError: + pass + try: + return l[2] # implicit int->PyObj conversion here + except IndexError: + return "oups" + res = self.interpret(fn, [6]) + assert res._obj.value == 3 + res = self.interpret(fn, [-2]) + assert res._obj.value == "oups" - s_A_list = s.items[0] - s_B_list = s.items[1] - - r_A_list = rtyper.getrepr(s_A_list) - assert isinstance(r_A_list, ListRepr) - r_B_list = rtyper.getrepr(s_B_list) - assert isinstance(r_B_list, ListRepr) + def fn(i): + l = [5, 8] + l.append(3) + try: + l[i] = 99 + except IndexError: + pass + try: + del l[i] + except IndexError: + pass + try: + return l[2] # implicit int->PyObj conversion here + except IndexError: + return "oups" + res = self.interpret(fn, [6]) + assert res._obj.value == 3 + res = self.interpret(fn, [-2]) + assert res._obj.value == "oups" + + def test_no_unneeded_refs(self): + self._skip_oo('assert') + def fndel(p, q): + lis = ["5", "3", "99"] + assert q >= 0 + assert p >= 0 + del lis[p:q] + return lis + def fnpop(n): + lis = ["5", "3", "99"] + while n: + lis.pop() + n -=1 + return lis + for i in range(2, 3+1): + lis = self.interpret(fndel, [0, i]) + assert list_is_clear(lis, 3-i) + for i in range(3): + lis = self.interpret(fnpop, [i]) + assert list_is_clear(lis, 3-i) + + def test_list_basic_ops(self): + def list_basic_ops(i=int, j=int): + l = [1,2,3] + l.insert(0, 42) + del l[1] + l.append(i) + listlen = len(l) + l.extend(l) + del l[listlen:] + l += [5,6] + l[1] = i + return l[j] + for i in range(6): + for j in range(6): + res = self.interpret(list_basic_ops, [i, j]) + assert res == list_basic_ops(i, j) + + def test_valueerror(self): + def fn(i): + l = [4, 7, 3] + try: + j = l.index(i) + except ValueError: + j = 100 + return j + res = self.interpret(fn, [4]) + assert res == 0 + res = self.interpret(fn, [7]) + assert res == 1 + res = self.interpret(fn, [3]) + assert res == 2 + res = self.interpret(fn, [6]) + assert res == 100 + + self._skip_oo('PyObject*') + def fn(i): + l = [5, 8] + l.append(3) + try: + l[i] = 99 + except IndexError: + pass + try: + del l[i] + except IndexError: + pass + try: + return l[2] # implicit int->PyObj conversion here + except IndexError: + return "oups" + res = self.interpret(fn, [6]) + assert res._obj.value == 3 + res = self.interpret(fn, [-2]) + assert res._obj.value == "oups" + + def test_voidlist_prebuilt(self): + frlist = [Freezing()] * 17 + def mylength(l): + return len(l) + def f(): + return mylength(frlist) + res = self.interpret(f, []) + assert res == 17 + + def test_voidlist_fixed(self): + self._skip_oo('_freeze_') + fr = Freezing() + def f(): + return len([fr, fr]) + res = self.interpret(f, []) + assert res == 2 + + def test_voidlist_nonfixed(self): + self._skip_oo('_freeze_') + class Freezing: + def _freeze_(self): + return True + fr = Freezing() + def f(): + lst = [fr, fr] + lst.append(fr) + del lst[1] + assert lst[0] is fr + return len(lst) + res = self.interpret(f, []) + assert res == 2 + + def test_type_erase_fixed_size(self): + self._skip_oo('tuples of lists') + class A(object): + pass + class B(object): + pass + + def f(): + return [A()], [B()] + + t = TranslationContext() + s = t.buildannotator().build_types(f, []) + rtyper = t.buildrtyper(type_system=self.ts) + rtyper.specialize() + + s_A_list = s.items[0] + s_B_list = s.items[1] + + r_A_list = rtyper.getrepr(s_A_list) + assert isinstance(r_A_list, self.rlist.FixedSizeListRepr) + r_B_list = rtyper.getrepr(s_B_list) + assert isinstance(r_B_list, self.rlist.FixedSizeListRepr) + + assert r_A_list.lowleveltype == r_B_list.lowleveltype + + def test_type_erase_var_size(self): + self._skip_oo('tuples of lists') + class A(object): + pass + class B(object): + pass + + def f(): + la = [A()] + lb = [B()] + la.append(None) + lb.append(None) + return la, lb + + t = TranslationContext() + s = t.buildannotator().build_types(f, []) + rtyper = t.buildrtyper(type_system=self.ts) + rtyper.specialize() + + s_A_list = s.items[0] + s_B_list = s.items[1] + + r_A_list = rtyper.getrepr(s_A_list) + assert isinstance(r_A_list, self.rlist.ListRepr) + r_B_list = rtyper.getrepr(s_B_list) + assert isinstance(r_B_list, self.rlist.ListRepr) + + assert r_A_list.lowleveltype == r_B_list.lowleveltype - assert r_A_list.lowleveltype == r_B_list.lowleveltype class TestLltypeRtyping(BaseTestListRtyping): ts = "lltype" + rlist = ll_rlist def ll_to_list(self, l): return map(None, l.ll_items())[:l.ll_length()] def class_name(self, value): return "".join(value.super.typeptr.name)[:-1] + + def test_memoryerror(self): + def fn(i): + lst = [0] * i + lst[i-1] = 5 + return lst[0] + res = self.interpret(fn, [1]) + assert res == 5 + res = self.interpret(fn, [2]) + assert res == 0 + self.interpret_raises(MemoryError, fn, [sys.maxint]) class TestOotypeRtyping(BaseTestListRtyping): ts = "ootype" + rlist = oo_rlist def ll_to_list(self, l): return l._list[:] From antocuni at codespeak.net Tue Apr 18 15:24:35 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 15:24:35 +0200 (CEST) Subject: [pypy-svn] r25945 - pypy/dist/pypy/rpython/test Message-ID: <20060418132435.57C631008E@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 15:24:30 2006 New Revision: 25945 Modified: pypy/dist/pypy/rpython/test/test_rlist.py Log: test_indexerror and test_valueerror has been refactored in order not to rely on automatic convertion to PyObject*, so they can be used by ootypesystem, too. Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Tue Apr 18 15:24:30 2006 @@ -904,7 +904,6 @@ assert res == fn(arg) def test_indexerror(self): - self._skip_oo('PyObject*') def fn(i): l = [5, 8, 3] try: @@ -916,13 +915,13 @@ except IndexError: pass try: - return l[2] # implicit int->PyObj conversion here + return l[2] except IndexError: - return "oups" + return -1 res = self.interpret(fn, [6]) - assert res._obj.value == 3 + assert res == 3 res = self.interpret(fn, [-2]) - assert res._obj.value == "oups" + assert res == -1 def fn(i): l = [5, 8] @@ -936,13 +935,13 @@ except IndexError: pass try: - return l[2] # implicit int->PyObj conversion here + return l[2] except IndexError: - return "oups" + return -1 res = self.interpret(fn, [6]) - assert res._obj.value == 3 + assert res == 3 res = self.interpret(fn, [-2]) - assert res._obj.value == "oups" + assert res == -1 def test_no_unneeded_refs(self): self._skip_oo('assert') @@ -999,7 +998,6 @@ res = self.interpret(fn, [6]) assert res == 100 - self._skip_oo('PyObject*') def fn(i): l = [5, 8] l.append(3) @@ -1012,13 +1010,13 @@ except IndexError: pass try: - return l[2] # implicit int->PyObj conversion here + return l[2] except IndexError: - return "oups" + return -1 res = self.interpret(fn, [6]) - assert res._obj.value == 3 + assert res == 3 res = self.interpret(fn, [-2]) - assert res._obj.value == "oups" + assert res == -1 def test_voidlist_prebuilt(self): frlist = [Freezing()] * 17 From arigo at codespeak.net Tue Apr 18 15:42:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 15:42:05 +0200 (CEST) Subject: [pypy-svn] r25947 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060418134205.096781008E@code0.codespeak.net> Author: arigo Date: Tue Apr 18 15:42:03 2006 New Revision: 25947 Modified: pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_overhead.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: Keepalives for rctypes arrays. Light refactorings, tests. All kept under control by test_overhead. Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Tue Apr 18 15:42:03 2006 @@ -5,7 +5,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr -from pypy.rpython.rctypes.rmodel import genreccopy, reccopy, C_ZERO +from pypy.rpython.rctypes.rmodel import genreccopy_arrayitem, reccopy, C_ZERO from pypy.rpython.rctypes.rprimitive import PrimitiveRepr from pypy.rpython.rctypes.rpointer import PointerRepr from pypy.annotation.model import SomeCTypesObject @@ -30,6 +30,14 @@ super(ArrayRepr, self).__init__(rtyper, s_array, c_data_type) + def get_content_keepalive_type(self): + "An extra array of keepalives, one per item." + item_keepalive_type = self.r_item.get_content_keepalive_type() + if not item_keepalive_type: + return None + else: + return lltype.FixedSizeArray(item_keepalive_type, self.length) + def initialize_const(self, p, value): for i in range(self.length): llitem = self.r_item.convert_const(value[i]) @@ -67,11 +75,11 @@ return llops.genop('getarrayitem', [v_c_array, v_index], resulttype = self.r_item.ll_type) - def set_item_value(self, llops, v_array, v_index, v_newvalue): - # ByValue case only - assert isinstance(self.r_item, CTypesValueRepr) - v_c_array = self.get_c_data(llops, v_array) - llops.genop('setarrayitem', [v_c_array, v_index, v_newvalue]) +## def set_item_value(self, llops, v_array, v_index, v_newvalue): +## # ByValue case only +## assert isinstance(self.r_item, CTypesValueRepr) +## v_c_array = self.get_c_data(llops, v_array) +## llops.genop('setarrayitem', [v_c_array, v_index, v_newvalue]) class __extend__(pairtype(ArrayRepr, IntegerRepr)): @@ -92,16 +100,16 @@ def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item) - if isinstance(r_array.r_item, CTypesRefRepr): - # ByRef case - v_item_c_data = r_array.r_item.get_c_data(hop.llops, v_item) - v_c_data = r_array.get_c_data_of_item(hop.llops, v_array, v_index) - # copy the whole structure's content over - genreccopy(hop.llops, v_item_c_data, v_c_data) - else: - # ByValue case (optimization; the above also works in this case) - v_newvalue = r_array.r_item.getvalue(hop.llops, v_item) - r_array.set_item_value(hop.llops, v_array, v_index, v_newvalue) + v_newvalue = r_array.r_item.get_c_data_or_value(hop.llops, v_item) + # copy the new value (which might be a whole structure) + v_c_array = r_array.get_c_data(hop.llops, v_array) + genreccopy_arrayitem(hop.llops, v_newvalue, v_c_array, v_index) + # copy the keepalive information too + v_keepalive_array = r_array.getkeepalive(hop.llops, v_array) + if v_keepalive_array is not None: + v_newkeepalive = r_array.r_item.getkeepalive(hop.llops, v_item) + genreccopy_arrayitem(hop.llops, v_newkeepalive, + v_keepalive_array, v_index) class __extend__(pairtype(ArrayRepr, PointerRepr)): Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Tue Apr 18 15:42:03 2006 @@ -23,9 +23,9 @@ # field instead of the c_data pointer return llops.gendirectcall(ll_charp2str, v_value) - def get_content_keepalives(self): - "Return an extra keepalive field used for the RPython string." - return [('keepalive_str', string_repr.lowleveltype)] + def get_content_keepalive_type(self): + "An extra keepalive used for the RPython string." + return string_repr.lowleveltype def getstring(self, llops, v_box): return llops.gendirectcall(ll_getstring, v_box) @@ -115,12 +115,12 @@ def ll_getstring(box): p = box.c_data[0] if p: - if box.keepalive_str and ll_str2charp(box.keepalive_str) == p: - maxlen = len(box.keepalive_str.chars) + if box.keepalive and ll_str2charp(box.keepalive) == p: + maxlen = len(box.keepalive.chars) length = ll_strnlen(p, maxlen) if length == maxlen: # no embedded zero in the string - return box.keepalive_str + return box.keepalive else: length = ll_strlen(p) newstr = lltype.malloc(string_repr.lowleveltype.TO, length) @@ -135,4 +135,4 @@ box.c_data[0] = ll_str2charp(string) else: box.c_data[0] = lltype.nullptr(CCHARP.TO) - box.keepalive_str = string + box.keepalive = string Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Tue Apr 18 15:42:03 2006 @@ -47,18 +47,20 @@ raise TyperError("unsupported ctypes memorystate %r" % memorystate) self.c_data_type = self.get_c_data_type(ll_type) - content_keepalives = self.get_content_keepalives() + + fields = [] + content_keepalive_type = self.get_content_keepalive_type() + if content_keepalive_type: + fields.append(( "keepalive", content_keepalive_type )) if self.ownsmemory: self.r_memoryowner = self - fields = content_keepalives + [ - ( "c_data", self.c_data_type ), - ] + fields.append(( "c_data", self.c_data_type )) else: s_memoryowner = SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) self.r_memoryowner = rtyper.getrepr(s_memoryowner) - fields = content_keepalives + [ + fields += [ ( "c_data_owner_keepalive", self.r_memoryowner.lowleveltype ), ( "c_data", lltype.Ptr(self.c_data_type) ), ] @@ -69,9 +71,10 @@ ) self.const_cache = {} # store generated const values+original value - def get_content_keepalives(self): - "Return extra keepalive fields used for the content of this object." - return [] + def get_content_keepalive_type(self): + """Return the type of the extra keepalive field used for the content + of this object.""" + return None def convert_const(self, value): if isinstance(value, self.ctype): @@ -148,6 +151,21 @@ # XXX add v_c_data_owner return self.allocate_instance_ref(llops, v_c_data) + def getkeepalive(self, llops, v_box): + try: + TYPE = self.lowleveltype.TO.keepalive + except AttributeError: + return None + else: + if isinstance(TYPE, lltype.ContainerType): + TYPE = lltype.Ptr(TYPE) + opname = 'getsubstruct' + else: + opname = 'getfield' + c_name = inputconst(lltype.Void, 'keepalive') + return llops.genop(opname, [v_box, c_name], + resulttype = TYPE) + class __extend__(pairtype(CTypesRepr, CTypesRepr)): @@ -157,7 +175,13 @@ if (r_from.ctype == r_to.ctype and r_from.ownsmemory and not r_to.ownsmemory): v_c_data = r_from.get_c_data(llops, v) - return r_to.allocate_instance_ref(llops, v_c_data, v) + v_result = r_to.allocate_instance_ref(llops, v_c_data, v) + # copy of the 'keepalive' field over + v_keepalive = r_from.getkeepalive(llops, v) + if v_keepalive is not None: + genreccopy_structfield(llops, v_keepalive, + v_result, 'keepalive') + return v_result else: return NotImplemented @@ -170,6 +194,9 @@ assert isinstance(ll_type, lltype.ContainerType) return ll_type + def get_c_data_or_value(self, llops, v_box): + return self.get_c_data(llops, v_box) + class CTypesValueRepr(CTypesRepr): """Base class for ctypes repr that have some kind of by-value @@ -195,6 +222,8 @@ v_c_data = self.get_c_data(llops, v_box) self.setvalue_inside_c_data(llops, v_c_data, v_value) + get_c_data_or_value = getvalue + def initialize_const(self, p, value): if isinstance(value, self.ctype): value = value.value @@ -295,9 +324,19 @@ def genreccopy_arrayitem(llops, v_source, v_destarray, v_destindex): ITEMTYPE = v_destarray.concretetype.TO.OF - if isinstance(ITEMTYPE, lltype.Primitive): - llops.genop('setarrayitem', [v_destarray, v_destindex, v_source]) - else: + if isinstance(ITEMTYPE, lltype.ContainerType): v_dest = llops.genop('getarraysubstruct', [v_destarray, v_destindex], resulttype = lltype.Ptr(ITEMTYPE)) genreccopy(llops, v_source, v_dest) + else: + llops.genop('setarrayitem', [v_destarray, v_destindex, v_source]) + +def genreccopy_structfield(llops, v_source, v_deststruct, fieldname): + c_name = inputconst(lltype.Void, fieldname) + FIELDTYPE = getattr(v_deststruct.concretetype.TO, fieldname) + if isinstance(FIELDTYPE, lltype.ContainerType): + v_dest = llops.genop('getsubstruct', [v_deststruct, c_name], + resulttype = lltype.Ptr(FIELDTYPE)) + genreccopy(llops, v_source, v_dest) + else: + llops.genop('setfield', [v_deststruct, c_name, v_source]) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 18 15:42:03 2006 @@ -20,13 +20,12 @@ super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) - def get_content_keepalives(self): - "Return an extra keepalive field used for the pointer's contents." - return [('keepalive_contents', - self.r_contents.r_memoryowner.lowleveltype)] + def get_content_keepalive_type(self): + "Keepalive for the box that holds the data that 'self' points to." + return self.r_contents.r_memoryowner.lowleveltype def setkeepalive(self, llops, v_box, v_owner): - inputargs = [v_box, inputconst(lltype.Void, 'keepalive_contents'), + inputargs = [v_box, inputconst(lltype.Void, 'keepalive'), v_owner] llops.genop('setfield', inputargs) @@ -35,7 +34,7 @@ p.c_data[0] = llcontents.c_data # the following line is probably pointless, as 'llcontents' will be # an immortal global constant just like 'p', but better safe than sorry - p.keepalive_contents = llcontents.c_data_owner_keepalive + p.keepalive = llcontents.c_data_owner_keepalive def setcontents(self, llops, v_ptr, v_contentsbox): v_c_data = self.r_contents.get_c_data(llops, v_contentsbox) Modified: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_overhead.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_overhead.py Tue Apr 18 15:42:03 2006 @@ -13,7 +13,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") -from ctypes import c_int +from ctypes import c_int, Structure, pointer, POINTER def find_mallocs(func, argtypes): @@ -53,3 +53,28 @@ mallocs = find_mallocs(func, [str]) assert not mallocs + +def test_array_getitem(): + A = c_int * 10 + def func(n): + a = A() + a[n] = n*2 + return a[n] + + mallocs = find_mallocs(func, [int]) + assert len(mallocs) <= 1 # for A() only + +def test_array_setitem(): + class S(Structure): + _fields_ = [('x', c_int)] + A = POINTER(S) * 10 + + def func(a, s, i): + while i > 0: + s.x = i*i + a[i] = pointer(s) + i -= 1 + return a[i].contents.x + + mallocs = find_mallocs(func, [A, S, int]) + assert not mallocs Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Tue Apr 18 15:42:03 2006 @@ -248,7 +248,6 @@ assert res == 121 def test_specialize_keepalive(self): - py.test.skip("in-progress") class S(Structure): _fields_ = [('x', c_int)] A = POINTER(S) * 10 @@ -257,7 +256,7 @@ for i in range(10): s = S() s.x = i*i - a[i].contents = s + a[i] = pointer(s) for i in range(10): assert a[i].contents.x == i*i func() From antocuni at codespeak.net Tue Apr 18 15:43:14 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 15:43:14 +0200 (CEST) Subject: [pypy-svn] r25948 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060418134314.37BD710088@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 15:43:09 2006 New Revision: 25948 Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py Log: Modified newlist in order to use _ll_resize and ll_setitem_fast instead of append. Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Tue Apr 18 15:43:09 2006 @@ -82,13 +82,22 @@ def newlist(llops, r_list, items_v): - c_1ist = inputconst(ootype.Void, r_list.lowleveltype) - v_result = llops.genop("new", [c_1ist], resulttype=r_list.lowleveltype) - c_append = inputconst(ootype.Void, "append") - # This is very inefficient for a large amount of initial items ... - for v_item in items_v: - llops.genop("oosend", [c_append, v_result, v_item], - resulttype=ootype.Void) + c_list = inputconst(ootype.Void, r_list.lowleveltype) + v_result = llops.genop("new", [c_list], resulttype=r_list.lowleveltype) +## c_append = inputconst(ootype.Void, "append") +## # This is very inefficient for a large amount of initial items ... +## for v_item in items_v: +## llops.genop("oosend", [c_append, v_result, v_item], +## resulttype=ootype.Void) + + c_resize = inputconst(ootype.Void, "_ll_resize") + c_length = inputconst(ootype.Signed, len(items_v)) + llops.genop("oosend", [c_resize, v_result, c_length], resulttype=ootype.Void) + + c_setitem = inputconst(ootype.Void, "ll_setitem_fast") + for i, v_item in enumerate(items_v): + ci = inputconst(Signed, i) + llops.genop("oosend", [c_setitem, v_result, ci, v_item], resulttype=ootype.Void) return v_result From antocuni at codespeak.net Tue Apr 18 16:03:48 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 16:03:48 +0200 (CEST) Subject: [pypy-svn] r25949 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060418140348.4D55C10087@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 16:03:41 2006 New Revision: 25949 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: 'append', 'extend' and 'remove_range' has been removed from ootype.List._GENERIC_METHODS because they are no longer needed. Some tests have been modified because the used 'append'. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 18 16:03:41 2006 @@ -209,9 +209,9 @@ "_ll_resize_ge": Meth([Signed], Void), "_ll_resize_le": Meth([Signed], Void), "_ll_resize": Meth([Signed], Void), - "append": Meth([self.ITEMTYPE_T], Void), - "extend": Meth([self.SELFTYPE_T], Void), - "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) +## "append": Meth([self.ITEMTYPE_T], Void), +## "extend": Meth([self.SELFTYPE_T], Void), +## "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) }) self._setup_methods(generic_types) @@ -567,11 +567,6 @@ self._ll_resize_le(length) assert len(self._list) == length - def append(self, item): - # NOT_RPYTHON - assert typeOf(item) == self._TYPE._ITEMTYPE - self._list.append(item) - def ll_getitem_fast(self, index): # NOT_RPYTHON assert typeOf(index) == Signed @@ -585,14 +580,19 @@ assert index >= 0 self._list[index] = item - def extend(self, other): - # NOT_RPYTHON - assert typeOf(other) == typeOf(self) - self._list.extend(other._list) - - def remove_range(self, start, count): - # NOT_RPYTHON - del self._list[start:start+count] +## def append(self, item): +## # NOT_RPYTHON +## assert typeOf(item) == self._TYPE._ITEMTYPE +## self._list.append(item) + +## def extend(self, other): +## # NOT_RPYTHON +## assert typeOf(other) == typeOf(self) +## self._list.extend(other._list) + +## def remove_range(self, start, count): +## # NOT_RPYTHON +## del self._list[start:start+count] class _null_list(_null_mixin(_list), _list): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 18 16:03:41 2006 @@ -1,6 +1,7 @@ import py from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.ootypesystem.ootype import * +from pypy.rpython.rlist import ll_append def test_new(): LT = List(Signed) @@ -18,25 +19,37 @@ l = new(LT) assert l.ll_length() == 0 -def test_append(): +def test_resize(): LT = List(Signed) - l = new(LT) - l.append(1) - assert l.ll_length() == 1 - -def test_extend(): - LT = List(Signed) - l1 = new(LT) - l2 = new(LT) - l1.append(1) - l2.append(2) - l1.extend(l2) - assert l1.ll_length() == 2 + lst = new(LT) + lst._ll_resize(10) + assert lst.ll_length() == 10 + lst._ll_resize_ge(9) + assert lst.ll_length() == 10 + lst._ll_resize_ge(20) + assert lst.ll_length() >= 20 + lst._ll_resize_le(10) + assert lst.ll_length() <= 10 + +##def test_append(): +## LT = List(Signed) +## l = new(LT) +## l.append(1) +## assert l.ll_length() == 1 + +##def test_extend(): +## LT = List(Signed) +## l1 = new(LT) +## l2 = new(LT) +## l1.append(1) +## l2.append(2) +## l1.extend(l2) +## assert l1.ll_length() == 2 def test_setitem_getitem(): LT = List(Signed) l = new(LT) - l.append(2) + ll_append(l, 2) assert l.ll_getitem_fast(0) == 2 l.ll_setitem_fast(0, 3) assert l.ll_getitem_fast(0) == 3 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Tue Apr 18 16:03:41 2006 @@ -110,24 +110,24 @@ rettype = g.getreturnvar().concretetype assert rettype == Signed -def test_list_append(): - LT = List(Signed) - - def oof(): - l = new(LT) - l.append(1) - return l.ll_length() +##def test_list_append(): +## LT = List(Signed) - g = gengraph(oof, []) - rettype = g.getreturnvar().concretetype - assert rettype == Signed +## def oof(): +## l = new(LT) +## l.append(1) +## return l.ll_length() + +## g = gengraph(oof, []) +## rettype = g.getreturnvar().concretetype +## assert rettype == Signed def test_list_getitem_setitem(): LT = List(Signed) def oof(): l = new(LT) - l.append(1) + l._ll_resize(1) l.ll_setitem_fast(0, 2) return l.ll_getitem_fast(0) @@ -165,6 +165,6 @@ return lst.ll_length() lst = new(LIST) - lst.append(1) + lst._ll_resize(1) res = interpret(oof, [lst], type_system='ootype') assert res == 1 From antocuni at codespeak.net Tue Apr 18 16:08:51 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 18 Apr 2006 16:08:51 +0200 (CEST) Subject: [pypy-svn] r25950 - pypy/dist/pypy/translator/cli/src Message-ID: <20060418140851.8154C1008F@code0.codespeak.net> Author: antocuni Date: Tue Apr 18 16:08:41 2006 New Revision: 25950 Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs Log: Added the _ll_resize method. Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Tue Apr 18 16:08:41 2006 @@ -29,6 +29,14 @@ this[index] = item; } + public void _ll_resize(int length) + { + if (length > this.Count) + this._ll_resize_ge(length); + else if (length < this.Count) + this._ll_resize_le(length); + } + public void _ll_resize_ge(int length) { if (this.Count < length) @@ -50,6 +58,7 @@ } } + /* public void append(T item) { this.Add(item); @@ -69,5 +78,6 @@ { return this.IndexOf(item); } + */ } } From arigo at codespeak.net Tue Apr 18 16:18:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 16:18:52 +0200 (CEST) Subject: [pypy-svn] r25951 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060418141852.42CEB10091@code0.codespeak.net> Author: arigo Date: Tue Apr 18 16:18:50 2006 New Revision: 25951 Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_overhead.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Keepalives for rctypes structs. Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Tue Apr 18 16:18:50 2006 @@ -2,7 +2,7 @@ from pypy.rpython.rbuiltin import gen_cast_structfield_pointer from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr -from pypy.rpython.rctypes.rmodel import genreccopy, reccopy +from pypy.rpython.rctypes.rmodel import genreccopy_structfield, reccopy from pypy.rpython.rctypes.rprimitive import PrimitiveRepr from pypy.annotation.model import SomeCTypesObject @@ -28,6 +28,19 @@ super(StructRepr, self).__init__(rtyper, s_struct, c_data_type) + def get_content_keepalive_type(self): + "An extra struct of keepalives, one per field." + keepalives = [] + for name in self.c_data_type._names: + r_field = self.r_fields[name] + field_keepalive_type = r_field.get_content_keepalive_type() + if field_keepalive_type: + keepalives.append((name, field_keepalive_type)) + if not keepalives: + return None + else: + return lltype.Struct('keepalives', *keepalives) + def initialize_const(self, p, value): for name, r_field in self.r_fields.items(): llitem = r_field.convert_const(getattr(value, name)) @@ -61,13 +74,13 @@ return llops.genop('getfield', [v_c_struct, c_fieldname], resulttype = r_field.ll_type) - def set_field_value(self, llops, v_struct, fieldname, v_newvalue): - # ByValue case only - r_field = self.r_fields[fieldname] - assert isinstance(r_field, CTypesValueRepr) - v_c_struct = self.get_c_data(llops, v_struct) - c_fieldname = inputconst(lltype.Void, fieldname) - llops.genop('setfield', [v_c_struct, c_fieldname, v_newvalue]) +## def set_field_value(self, llops, v_struct, fieldname, v_newvalue): +## # ByValue case only +## r_field = self.r_fields[fieldname] +## assert isinstance(r_field, CTypesValueRepr) +## v_c_struct = self.get_c_data(llops, v_struct) +## c_fieldname = inputconst(lltype.Void, fieldname) +## llops.genop('setfield', [v_c_struct, c_fieldname, v_newvalue]) def rtype_getattr(self, hop): s_attr = hop.args_s[1] @@ -93,13 +106,13 @@ name = s_attr.const r_field = self.r_fields[name] v_struct, v_attr, v_item = hop.inputargs(self, lltype.Void, r_field) - if isinstance(r_field, CTypesRefRepr): - # ByRef case - v_new_c_data = r_field.get_c_data(hop.llops, v_item) - v_c_data = self.get_c_data_of_field(hop.llops, v_struct, name) - # copy the whole structure's content over - genreccopy(hop.llops, v_new_c_data, v_c_data) - else: - # ByValue case (optimization; the above also works in this case) - v_newvalue = r_field.getvalue(hop.llops, v_item) - self.set_field_value(hop.llops, v_struct, name, v_newvalue) + v_newvalue = r_field.get_c_data_or_value(hop.llops, v_item) + # copy the new value (which might be a whole substructure) + v_c_struct = self.get_c_data(hop.llops, v_struct) + genreccopy_structfield(hop.llops, v_newvalue, v_c_struct, name) + # copy the keepalive information too + v_newkeepalive = r_field.getkeepalive(hop.llops, v_item) + if v_newkeepalive is not None: + v_keepalive_struct = self.getkeepalive(hop.llops, v_struct) + genreccopy_structfield(hop.llops, v_newkeepalive, + v_keepalive_struct, name) Modified: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_overhead.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_overhead.py Tue Apr 18 16:18:50 2006 @@ -3,6 +3,7 @@ by the rctypes rtyping. """ +import py import pypy.rpython.rctypes.implementation from pypy import conftest from pypy.rpython.test.test_llinterp import gengraph @@ -18,7 +19,7 @@ def find_mallocs(func, argtypes): t, typer, graph = gengraph(func, argtypes) - backend_optimizations(t) + backend_optimizations(t, inline_threshold=10) if conftest.option.view: t.view() @@ -78,3 +79,22 @@ mallocs = find_mallocs(func, [A, S, int]) assert not mallocs + +def test_struct_setitem(): + py.test.skip("not working yet") + class S(Structure): + _fields_ = [('x', c_int)] + class T(Structure): + _fields_ = [('s', POINTER(S))] + def make_t(i): + t = T() + s = S() + s.x = i*i + t.s = pointer(s) + return t + def func(): + t = make_t(17) + return t.s.contents.x + + mallocs = find_mallocs(func, []) + assert not mallocs # depends on inlining Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Tue Apr 18 16:18:50 2006 @@ -154,6 +154,24 @@ res = interpret(func, []) assert res == 121 + def test_specialize_keepalive(self): + class S(Structure): + _fields_ = [('x', c_int)] + class T(Structure): + _fields_ = [('s', POINTER(S))] + def make_t(i): + t = T() + s = S() + s.x = i*i + t.s = pointer(s) + return t + def func(): + t = make_t(17) + return t.s.contents.x + + res = interpret(func, []) + assert res == 289 + class Test_compilation: def test_compile_struct_access(self): def access_struct(n): From cfbolz at codespeak.net Tue Apr 18 16:49:46 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 18 Apr 2006 16:49:46 +0200 (CEST) Subject: [pypy-svn] r25953 - pypy/dist/pypy/rpython/test Message-ID: <20060418144946.0D60210088@code0.codespeak.net> Author: cfbolz Date: Tue Apr 18 16:49:46 2006 New Revision: 25953 Modified: pypy/dist/pypy/rpython/test/test_objectmodel.py Log: test that shows the exception problem with r_dicts Modified: pypy/dist/pypy/rpython/test/test_objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_objectmodel.py (original) +++ pypy/dist/pypy/rpython/test/test_objectmodel.py Tue Apr 18 16:49:46 2006 @@ -147,6 +147,22 @@ res = interpret(fn, [2]) assert res == 2 +def test_rtype_r_dict_exceptions(): + def raising_hash(obj): + if id(obj) % 2 == 0: + raise TypeError + return 1 + def eq(obj1, obj2): + return obj1 is obj2 + def f(): + d1 = r_dict(eq, raising_hash) + try: + x = d1["blabla"] + except Exception: + return 1 + return x + res = interpret(f, []) + def test_rtype_keepalive(): from pypy.rpython import objectmodel def f(): From arigo at codespeak.net Tue Apr 18 17:18:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 17:18:05 +0200 (CEST) Subject: [pypy-svn] r25955 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem rpython/lltypesystem/test rpython/rctypes rpython/rctypes/test translator/c translator/c/test Message-ID: <20060418151805.007CC1008E@code0.codespeak.net> Author: arigo Date: Tue Apr 18 17:18:01 2006 New Revision: 25955 Modified: pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rstringbuf.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/test/test_lltyped.py Log: Major cleanup to get rid of the obscure address manipulation routines cast_subarray_pointer() & co. Instead, we have three new ll operations that each have a very straightforward C equivalent: * direct_fieldptr(struct, name) -> get a ptr to the field * direct_arrayitems(array) -> get a ptr to the first item * direct_ptradd(ptr, n) -> shift a ptr by n items They manipulate a kind of "pointer to primitives" represented as a Ptr(FixedSizeArray(T, 1)). See test_lltype.py for examples. (This is all for rctypes.) Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Tue Apr 18 17:18:01 2006 @@ -421,23 +421,23 @@ cast_p = lltype.cast_pointer(PtrT.const, s_p.ll_ptrtype._defl()) return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) -def cast_subarray_pointer(PtrT, s_p, s_offset): - assert isinstance(s_p, SomePtr), "casting of non-pointer: %r" % s_p - assert PtrT.is_constant() - cast_p = lltype.cast_subarray_pointer(PtrT.const, - s_p.ll_ptrtype._example(), - 0) +def direct_fieldptr(s_p, s_fieldname): + assert isinstance(s_p, SomePtr), "direct_* of non-pointer: %r" % s_p + assert s_fieldname.is_constant() + cast_p = lltype.direct_fieldptr(s_p.ll_ptrtype._example(), + s_fieldname.const) return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) -def cast_structfield_pointer(PtrT, s_p, s_fieldname): - assert isinstance(s_p, SomePtr), "casting of non-pointer: %r" % s_p - assert PtrT.is_constant() - assert s_fieldname.is_constant() - cast_p = lltype.cast_structfield_pointer(PtrT.const, - s_p.ll_ptrtype._example(), - s_fieldname.const) +def direct_arrayitems(s_p): + assert isinstance(s_p, SomePtr), "direct_* of non-pointer: %r" % s_p + cast_p = lltype.direct_arrayitems(s_p.ll_ptrtype._example()) return SomePtr(ll_ptrtype=lltype.typeOf(cast_p)) +def direct_ptradd(s_p, s_n): + assert isinstance(s_p, SomePtr), "direct_* of non-pointer: %r" % s_p + # don't bother with an example here: the resulting pointer is the same + return s_p + def cast_ptr_to_int(s_ptr): # xxx return SomeInteger() @@ -454,8 +454,9 @@ BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr BUILTIN_ANALYZERS[lltype.cast_pointer] = cast_pointer -BUILTIN_ANALYZERS[lltype.cast_subarray_pointer] = cast_subarray_pointer -BUILTIN_ANALYZERS[lltype.cast_structfield_pointer] = cast_structfield_pointer +BUILTIN_ANALYZERS[lltype.direct_fieldptr] = direct_fieldptr +BUILTIN_ANALYZERS[lltype.direct_arrayitems] = direct_arrayitems +BUILTIN_ANALYZERS[lltype.direct_ptradd] = direct_ptradd BUILTIN_ANALYZERS[lltype.cast_ptr_to_int] = cast_ptr_to_int BUILTIN_ANALYZERS[lltype.getRuntimeTypeInfo] = getRuntimeTypeInfo BUILTIN_ANALYZERS[lltype.runtime_type_info] = runtime_type_info Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Tue Apr 18 17:18:01 2006 @@ -536,6 +536,20 @@ assert checkptr(ptr1) return not bool(ptr1) + def op_direct_fieldptr(self, obj, field): + assert checkptr(obj) + assert isinstance(field, str) + return lltype.direct_fieldptr(obj, field) + + def op_direct_arrayitems(self, obj): + assert checkptr(obj) + return lltype.direct_arrayitems(obj) + + def op_direct_ptradd(self, obj, index): + assert checkptr(obj) + assert isinstance(index, int) + return lltype.direct_ptradd(obj, index) + def op_cast_ptr_to_int(self, ptr1): assert checkptr(ptr1) assert isinstance(lltype.typeOf(ptr1).TO, (lltype.Array, lltype.Struct)) Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Tue Apr 18 17:18:01 2006 @@ -219,17 +219,15 @@ if (isinstance(ref, _arrayitemref) and isinstance(EXPECTED_TYPE.TO, lltype.FixedSizeArray) and ref.type() == EXPECTED_TYPE.TO.OF): - # special case that requires cast_subarray_pointer - return lltype.cast_subarray_pointer(EXPECTED_TYPE, - ref.array, - ref.index) + # special case that requires direct_arrayitems + p_items = lltype.direct_arrayitems(ref.array) + return lltype.direct_ptradd(p_items, ref.index) elif (isinstance(ref, _structfieldref) and isinstance(EXPECTED_TYPE.TO, lltype.FixedSizeArray) and ref.type() == EXPECTED_TYPE.TO.OF): - # special case that requires cast_structfield_pointer - return lltype.cast_structfield_pointer(EXPECTED_TYPE, - ref.struct, - ref.fieldname) + # special case that requires direct_fieldptr + return lltype.direct_fieldptr(ref.struct, + ref.fieldname) else: # regular case return lltype.cast_pointer(EXPECTED_TYPE, ref.get()) Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Tue Apr 18 17:18:01 2006 @@ -265,6 +265,9 @@ 'ptr_nonzero': LLOp(canfold=True), 'ptr_iszero': LLOp(canfold=True), 'cast_ptr_to_int': LLOp(sideeffects=False), + 'direct_fieldptr': LLOp(canfold=True), + 'direct_arrayitems': LLOp(canfold=True), + 'direct_ptradd': LLOp(canfold=True), # _________ XXX l3interp hacks ___________ Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Tue Apr 18 17:18:01 2006 @@ -614,53 +614,44 @@ raise TypeError, "can only cast pointers to other pointers" return ptr._cast_to(PTRTYPE) -def cast_subarray_pointer(ARRAYPTRTYPE, arrayptr, baseoffset): - CURPTRTYPE = typeOf(arrayptr) - if not isinstance(CURPTRTYPE, Ptr) or not isinstance(ARRAYPTRTYPE, Ptr): - raise TypeError, "can only cast pointers to other pointers" - ARRAYTYPE = ARRAYPTRTYPE.TO - if (not isinstance(CURPTRTYPE.TO, (Array, FixedSizeArray)) or - not isinstance(ARRAYTYPE, FixedSizeArray)): - raise TypeError, "can only cast from arrays to FixedSizeArray" - if CURPTRTYPE.TO.OF != ARRAYTYPE.OF: - raise TypeError, "mismatching array item types" +def direct_fieldptr(structptr, fieldname): + """Get a pointer to a field in the struct. The resulting + pointer is actually of type Ptr(FixedSizeArray(FIELD, 1)). + It can be used in a regular getarrayitem(0) or setarrayitem(0) + to read or write to the field. + """ + CURTYPE = typeOf(structptr).TO + if not isinstance(CURTYPE, Struct): + raise TypeError, "direct_fieldptr: not a struct" + if fieldname not in CURTYPE._flds: + raise TypeError, "%s has no field %r" % (CURTYPE, fieldname) + if not structptr: + raise RuntimeError("direct_fieldptr: NULL argument") + return _subarray._makeptr(structptr._obj, fieldname) + +def direct_arrayitems(arrayptr): + """Get a pointer to the first item of the array. The resulting + pointer is actually of type Ptr(FixedSizeArray(ITEM, 1)) but can + be used in a regular getarrayitem(n) or direct_ptradd(n) to access + further elements. + """ + CURTYPE = typeOf(arrayptr).TO + if not isinstance(CURTYPE, (Array, FixedSizeArray)): + raise TypeError, "direct_arrayitems: not an array" if not arrayptr: - raise RuntimeError("cast_subarray_pointer: NULL argument") - try: - cache = _subarray._cache[arrayptr._obj] - except KeyError: - cache = _subarray._cache[arrayptr._obj] = {} - key = (ARRAYTYPE, baseoffset) - try: - subarray = cache[key] - except KeyError: - subarray = cache[key] = _subarray(ARRAYTYPE, arrayptr._obj, baseoffset) - return _ptr(ARRAYPTRTYPE, subarray) + raise RuntimeError("direct_arrayitems: NULL argument") + return _subarray._makeptr(arrayptr._obj, 0) -def cast_structfield_pointer(ARRAYPTRTYPE, structptr, fieldname): - CURPTRTYPE = typeOf(structptr) - if not isinstance(CURPTRTYPE, Ptr) or not isinstance(ARRAYPTRTYPE, Ptr): - raise TypeError, "can only cast pointers to other pointers" - ARRAYTYPE = ARRAYPTRTYPE.TO - if (not isinstance(CURPTRTYPE.TO, Struct) or - not isinstance(ARRAYTYPE, FixedSizeArray)): - raise TypeError, "can only cast from (Gc)Struct to FixedSizeArray" - if ARRAYTYPE.length != 1: - raise TypeError, "can only cast to FixedSizeArray of length 1" - if getattr(CURPTRTYPE.TO, fieldname) != ARRAYTYPE.OF: - raise TypeError, "mismatching array item type" - if not structptr: - raise RuntimeError("cast_structfield_pointer: NULL argument") - try: - cache = _subarray._cache[structptr._obj] - except KeyError: - cache = _subarray._cache[structptr._obj] = {} - try: - subarray = cache[fieldname] - except KeyError: - subarray = cache[fieldname] = _subarray(ARRAYTYPE, structptr._obj, - fieldname) - return _ptr(ARRAYPTRTYPE, subarray) +def direct_ptradd(ptr, n): + """Shift a pointer forward or backward by n items. The pointer must + have been built by direct_arrayitems(). + """ + if not ptr: + raise RuntimeError("direct_ptradd: NULL argument") + if not isinstance(ptr._obj, _subarray): + raise TypeError("direct_ptradd: only for direct_arrayitems() ptrs") + parent, base = parentlink(ptr._obj) + return _subarray._makeptr(parent, base + n) def _expose(val, solid=False): """XXX A nice docstring here""" @@ -1138,6 +1129,24 @@ else: self._parentstructure().setitem(baseoffset + index, value) + def _makeptr(parent, baseoffset_or_fieldname): + cache = _subarray._cache.setdefault(parent, {}) + try: + subarray = cache[baseoffset_or_fieldname] + except KeyError: + PARENTTYPE = typeOf(parent) + if isinstance(baseoffset_or_fieldname, str): + # for direct_fieldptr + ITEMTYPE = getattr(PARENTTYPE, baseoffset_or_fieldname) + else: + # for direct_arrayitems + ITEMTYPE = PARENTTYPE.OF + ARRAYTYPE = FixedSizeArray(ITEMTYPE, 1) + subarray = _subarray(ARRAYTYPE, parent, baseoffset_or_fieldname) + cache[baseoffset_or_fieldname] = subarray + return _ptr(Ptr(subarray._TYPE), subarray) + _makeptr = staticmethod(_makeptr) + class _func(object): def __init__(self, TYPE, **attrs): Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_lltype.py Tue Apr 18 17:18:01 2006 @@ -578,7 +578,7 @@ assert len(s.a) == 5 py.test.raises(TypeError, "s.a = a") -def test_cast_subarray_pointer(): +def test_direct_arrayitems(): for a in [malloc(GcArray(Signed), 5), malloc(FixedSizeArray(Signed, 5), immortal=True)]: a[0] = 0 @@ -586,36 +586,34 @@ a[2] = 20 a[3] = 30 a[4] = 40 - BOX = Ptr(FixedSizeArray(Signed, 2)) - b01 = cast_subarray_pointer(BOX, a, 0) - b12 = cast_subarray_pointer(BOX, a, 1) - b23 = cast_subarray_pointer(BOX, a, 2) - b34 = cast_subarray_pointer(BOX, a, 3) - assert b01[0] == 0 - assert b01[1] == 10 - assert b12[0] == 10 - assert b12[1] == 20 - assert b23[0] == 20 - assert b23[1] == 30 - assert b34[0] == 30 - assert b34[1] == 40 - b23[0] = 23 + b0 = direct_arrayitems(a) + b1 = direct_ptradd(b0, 1) + b2 = direct_ptradd(b1, 1) + b3 = direct_ptradd(b0, 3) + assert b0[0] == 0 + assert b0[1] == 10 + assert b0[4] == 40 + assert b1[0] == 10 + assert b1[1] == 20 + assert b2[0] == 20 + assert b2[1] == 30 + assert b3[-2] == 10 + assert b3[0] == 30 + assert b3[1] == 40 + assert b2[-2] == 0 + assert b1[3] == 40 + b2[0] = 23 assert a[2] == 23 - b12[1] += 1 + b1[1] += 1 assert a[2] == 24 - # out-of-bound access is allowed, if it's within the parent's bounds - assert len(b23) == 2 - assert b23[-1] == 10 - assert b12[3] == 40 - py.test.raises(IndexError, "b01[-1]") - py.test.raises(IndexError, "b34[2]") - py.test.raises(IndexError, "b12[4]") + py.test.raises(IndexError, "b0[-1]") + py.test.raises(IndexError, "b3[2]") + py.test.raises(IndexError, "b1[4]") -def test_cast_structfield_pointer(): +def test_direct_fieldptr(): S = GcStruct('S', ('x', Signed), ('y', Signed)) - A = FixedSizeArray(Signed, 1) s = malloc(S) - a = cast_structfield_pointer(Ptr(A), s, 'y') + a = direct_fieldptr(s, 'y') a[0] = 34 assert s.y == 34 py.test.raises(IndexError, "a[1]") Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Tue Apr 18 17:18:01 2006 @@ -307,77 +307,27 @@ return hop.genop('cast_pointer', [v_input], # v_type implicit in r_result resulttype = hop.r_result.lowleveltype) -def rtype_cast_subarray_pointer(hop): - assert hop.args_s[0].is_constant() - assert isinstance(hop.args_r[1], rptr.PtrRepr) - v_type, v_input, v_baseoffset = hop.inputargs(lltype.Void, hop.args_r[1], - lltype.Signed) +def rtype_direct_fieldptr(hop): + assert isinstance(hop.args_r[0], rptr.PtrRepr) + assert hop.args_s[1].is_constant() + vlist = hop.inputargs(hop.args_r[0], lltype.Void) hop.exception_cannot_occur() - # instead of adding a 'cast_subarray_pointer' ll operation, - # we do it with address manipulations - return gen_cast_subarray_pointer(hop.llops, hop.r_result.lowleveltype, - v_input, v_baseoffset) - -def gen_cast_subarray_pointer(llops, ARRAYPTRTYPE, v_array, v_baseoffset): - "Helper to generate the equivalent of a 'cast_subarray_pointer' operation." - from pypy.rpython.lltypesystem import llmemory - SRCTYPE = v_array.concretetype.TO - v_array_adr = llops.genop('cast_ptr_to_adr', [v_array], - resulttype = llmemory.Address) - c_ofs1 = inputconst(lltype.Signed, llmemory.ArrayItemsOffset(SRCTYPE)) - v_array_items_adr = llops.genop('adr_add', [v_array_adr, c_ofs1], - resulttype = llmemory.Address) - if isinstance(v_baseoffset, Constant) and v_baseoffset.value == 0: - v_base_adr = v_array_items_adr - else: - c_ofs2 = inputconst(lltype.Signed, llmemory.ItemOffset(SRCTYPE.OF)) - v_ofs3 = llops.genop('int_mul', [c_ofs2, v_baseoffset], - resulttype = lltype.Signed) - v_base_adr = llops.genop('adr_add', [v_array_items_adr, v_ofs3], - resulttype = llmemory.Address) - return llops.genop('cast_adr_to_ptr', [v_base_adr], - resulttype = ARRAYPTRTYPE) - -def gen_add_itemoffset_to_pointer(llops, ITEMTYPE, v_ptr, v_index): - "Generates address manipulations equivalent to the C expression ptr+index." - if isinstance(v_index, Constant) and v_index.value == 0: - return v_ptr - from pypy.rpython.lltypesystem import llmemory - v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], - resulttype = llmemory.Address) - c_ofs = inputconst(lltype.Signed, llmemory.ItemOffset(ITEMTYPE)) - v_ofs = llops.genop('int_mul', [c_ofs, v_index], - resulttype = lltype.Signed) - v_newadr = llops.genop('adr_add', [v_adr, v_ofs], - resulttype = llmemory.Address) - return llops.genop('cast_adr_to_ptr', [v_newadr], - resulttype = v_ptr.concretetype) + return hop.genop('direct_fieldptr', vlist, + resulttype=hop.r_result.lowleveltype) -def rtype_cast_structfield_pointer(hop): - assert hop.args_s[0].is_constant() - assert hop.args_s[2].is_constant() - fieldname = hop.args_s[2].const - assert isinstance(hop.args_r[1], rptr.PtrRepr) - v_type, v_input, v_fieldname = hop.inputargs(lltype.Void, hop.args_r[1], - lltype.Void) +def rtype_direct_arrayitems(hop): + assert isinstance(hop.args_r[0], rptr.PtrRepr) + vlist = hop.inputargs(hop.args_r[0]) + hop.exception_cannot_occur() + return hop.genop('direct_arrayitems', vlist, + resulttype=hop.r_result.lowleveltype) + +def rtype_direct_ptradd(hop): + assert isinstance(hop.args_r[0], rptr.PtrRepr) + vlist = hop.inputargs(hop.args_r[0], lltype.Signed) hop.exception_cannot_occur() - # instead of adding a 'cast_structfield_pointer' ll operation, - # we do it with address manipulations - return gen_cast_structfield_pointer(hop.llops, hop.r_result.lowleveltype, - v_input, fieldname) - -def gen_cast_structfield_pointer(llops, ARRAYPTRTYPE, v_struct, fieldname): - "Helper to generate the equivalent of a 'cast_structfield_pointer' op." - from pypy.rpython.lltypesystem import llmemory - SRCTYPE = v_struct.concretetype.TO - v_struct_adr = llops.genop('cast_ptr_to_adr', [v_struct], - resulttype = llmemory.Address) - c_ofs1 = inputconst(lltype.Signed, - llmemory.FieldOffset(SRCTYPE, fieldname)) - v_base_adr = llops.genop('adr_add', [v_struct_adr, c_ofs1], - resulttype = llmemory.Address) - return llops.genop('cast_adr_to_ptr', [v_base_adr], - resulttype = ARRAYPTRTYPE) + return hop.genop('direct_ptradd', vlist, + resulttype=hop.r_result.lowleveltype) def rtype_cast_primitive(hop): assert hop.args_s[0].is_constant() @@ -413,8 +363,9 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer -BUILTIN_TYPER[lltype.cast_subarray_pointer] = rtype_cast_subarray_pointer -BUILTIN_TYPER[lltype.cast_structfield_pointer] = rtype_cast_structfield_pointer +BUILTIN_TYPER[lltype.direct_fieldptr] = rtype_direct_fieldptr +BUILTIN_TYPER[lltype.direct_arrayitems] = rtype_direct_arrayitems +BUILTIN_TYPER[lltype.direct_ptradd] = rtype_direct_ptradd BUILTIN_TYPER[lltype.cast_ptr_to_int] = rtype_cast_ptr_to_int BUILTIN_TYPER[lltype.typeOf] = rtype_const_result BUILTIN_TYPER[lltype.nullptr] = rtype_const_result Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Tue Apr 18 17:18:01 2006 @@ -1,5 +1,4 @@ from ctypes import ARRAY, c_int -from pypy.rpython.rbuiltin import gen_cast_subarray_pointer from pypy.rpython.rstr import string_repr from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype @@ -9,6 +8,7 @@ from pypy.rpython.rctypes.rprimitive import PrimitiveRepr from pypy.rpython.rctypes.rpointer import PointerRepr from pypy.annotation.model import SomeCTypesObject +from pypy.objspace.flow.model import Constant ArrayType = type(ARRAY(c_int, 10)) @@ -64,9 +64,15 @@ lltype.Ptr(self.r_item.c_data_type)) else: # ByValue case - A = lltype.FixedSizeArray(self.r_item.ll_type, 1) - return gen_cast_subarray_pointer(llops, lltype.Ptr(A), - v_c_array, v_index) + P = lltype.Ptr(lltype.FixedSizeArray(self.r_item.ll_type, 1)) + v_items = llops.genop('direct_arrayitems', [v_c_array], + resulttype = P) + if isinstance(v_index, Constant) and v_index.value == 0: + pass # skip direct_ptradd + else: + v_items = llops.genop('direct_ptradd', [v_items, v_index], + resulttype = P) + return v_items def get_item_value(self, llops, v_array, v_index): # ByValue case only @@ -125,8 +131,7 @@ def ll_chararrayvalue(box): from pypy.rpython.rctypes import rchar_p p = box.c_data - p1 = lltype.cast_subarray_pointer(rchar_p.CCHARP, p, 0) - length = rchar_p.ll_strnlen(p1, len(p)) + length = rchar_p.ll_strnlen(lltype.direct_arrayitems(p), len(p)) newstr = lltype.malloc(string_repr.lowleveltype.TO, length) for i in range(length): newstr.chars[i] = p[i] Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Tue Apr 18 17:18:01 2006 @@ -103,7 +103,7 @@ return i def ll_str2charp(s): - return lltype.cast_subarray_pointer(CCHARP, s.chars, 0) + return lltype.direct_arrayitems(s.chars) def ll_charp2str(p): length = ll_strlen(p) Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Tue Apr 18 17:18:01 2006 @@ -1,10 +1,10 @@ -from pypy.rpython.rbuiltin import gen_add_itemoffset_to_pointer from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.error import TyperError from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rmodel import CTypesValueRepr, genreccopy from pypy.annotation.model import SomeCTypesObject +from pypy.objspace.flow.model import Constant class PointerRepr(CTypesValueRepr): @@ -65,9 +65,11 @@ self = r_ptr v_ptr, v_index = hop.inputargs(self, lltype.Signed) v_c_ptr = self.getvalue(hop.llops, v_ptr) - v_c_ptr = gen_add_itemoffset_to_pointer(hop.llops, - r_ptr.r_contents.ll_type, - v_c_ptr, v_index) + if isinstance(v_index, Constant) and v_index.value == 0: + pass # skip direct_ptradd + else: + v_c_ptr = hop.genop('direct_ptradd', [v_c_ptr, v_index], + resulttype = r_ptr.ll_type) return self.r_contents.return_c_data(hop.llops, v_c_ptr) def rtype_setitem((r_ptr, _), hop): Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Tue Apr 18 17:18:01 2006 @@ -1,8 +1,8 @@ from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype -from pypy.rpython.rbuiltin import gen_cast_subarray_pointer from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.rctypes.rmodel import CTypesRefRepr +from pypy.objspace.flow.model import Constant class StringBufRepr(CTypesRefRepr): @@ -23,8 +23,14 @@ def get_c_data_of_item(self, llops, v_stringbuf, v_index): v_array = self.get_c_data(llops, v_stringbuf) - return gen_cast_subarray_pointer(llops, ONE_CHAR_PTR, - v_array, v_index) + v_char_p = llops.genop('direct_arrayitems', [v_array], + resulttype = ONE_CHAR_PTR) + if isinstance(v_index, Constant) and v_index.value == 0: + pass # skip direct_ptradd + else: + v_char_p = llops.genop('direct_ptradd', [v_char_p, v_index], + resulttype = ONE_CHAR_PTR) + return v_char_p ONE_CHAR_PTR = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Tue Apr 18 17:18:01 2006 @@ -1,5 +1,4 @@ from pypy.rpython.rmodel import inputconst -from pypy.rpython.rbuiltin import gen_cast_structfield_pointer from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesRefRepr, CTypesValueRepr from pypy.rpython.rctypes.rmodel import genreccopy_structfield, reccopy @@ -54,16 +53,16 @@ def get_c_data_of_field(self, llops, v_struct, fieldname): v_c_struct = self.get_c_data(llops, v_struct) r_field = self.r_fields[fieldname] + c_fieldname = inputconst(lltype.Void, fieldname) if isinstance(r_field, CTypesRefRepr): # ByRef case - c_fieldname = inputconst(lltype.Void, fieldname) return llops.genop('getsubstruct', [v_c_struct, c_fieldname], lltype.Ptr(r_field.c_data_type)) else: # ByValue case - A = lltype.FixedSizeArray(r_field.ll_type, 1) - return gen_cast_structfield_pointer(llops, lltype.Ptr(A), - v_c_struct, fieldname) + P = lltype.Ptr(lltype.FixedSizeArray(r_field.ll_type, 1)) + return llops.genop('direct_fieldptr', [v_c_struct, c_fieldname], + resulttype = P) def get_field_value(self, llops, v_struct, fieldname): # ByValue case only Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Tue Apr 18 17:18:01 2006 @@ -72,8 +72,7 @@ def str2subarray(string): llstring = string_repr.convert_const(string) keepalive.append(llstring) - A = lltype.FixedSizeArray(lltype.Char, 1) - return lltype.cast_subarray_pointer(lltype.Ptr(A), llstring.chars, 0) + return lltype.direct_arrayitems(llstring.chars) assert ll_atoi(str2subarray("")) == 0 assert ll_atoi(str2subarray("42z7")) == 42 assert ll_atoi(str2subarray("blah")) == 0 Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Tue Apr 18 17:18:01 2006 @@ -564,6 +564,21 @@ else: raise NotImplementedError + def OP_DIRECT_FIELDPTR(self, op): + return self.OP_GETFIELD(op, ampersand='&') + + def OP_DIRECT_ARRAYITEMS(self, op): + ARRAY = self.lltypemap(op.args[0]).TO + items = self.expr(op.args[0]) + if not isinstance(ARRAY, FixedSizeArray): + items += '->items' + return '%s = %s;' % (self.expr(op.result), items) + + def OP_DIRECT_PTRADD(self, op): + return '%s = %s + %s;' % (self.expr(op.result), + self.expr(op.args[0]), + self.expr(op.args[1])) + def OP_CAST_POINTER(self, op): TYPE = self.lltypemap(op.result) typename = self.db.gettype(TYPE) Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Tue Apr 18 17:18:01 2006 @@ -129,7 +129,7 @@ res = fn(4) assert res == -1 - def test_cast_subarray_pointer(self): + def test_direct_arrayitems(self): for a in [malloc(GcArray(Signed), 5), malloc(FixedSizeArray(Signed, 5), immortal=True)]: a[0] = 0 @@ -137,16 +137,16 @@ a[2] = 20 a[3] = 30 a[4] = 40 - BOX = Ptr(FixedSizeArray(Signed, 2)) - b01 = cast_subarray_pointer(BOX, a, 0) - b12 = cast_subarray_pointer(BOX, a, 1) - b23 = cast_subarray_pointer(BOX, a, 2) - b34 = cast_subarray_pointer(BOX, a, 3) + b0 = direct_arrayitems(a) + b1 = direct_ptradd(b0, 1) + b2 = direct_ptradd(b1, 1) def llf(n=int): + b0 = direct_arrayitems(a) + b3 = direct_ptradd(direct_ptradd(b0, 5), -2) saved = a[n] a[n] = 1000 try: - return b01[0] + b12[0] + b23[1] + b34[1] + return b0[0] + b3[-2] + b2[1] + b1[3] finally: a[n] = saved fn = self.getcompiled(llf) @@ -161,13 +161,11 @@ res = fn(4) assert res == 0 + 10 + 30 + 1000 - def test_cast_structfield_pointer(self): + def test_direct_fieldptr(self): S = GcStruct('S', ('x', Signed), ('y', Signed)) - SUBARRAY = FixedSizeArray(Signed, 1) - P = Ptr(SUBARRAY) def llf(n=int): s = malloc(S) - a = cast_structfield_pointer(P, s, 'y') + a = direct_fieldptr(s, 'y') a[0] = n return s.y @@ -183,11 +181,10 @@ a2[1] = 600 s.x = 50 s.y = 4 - P = Ptr(FixedSizeArray(Signed, 1)) - p1 = cast_subarray_pointer(P, a1, 3) - p2 = cast_subarray_pointer(P, a2, 1) - p3 = cast_structfield_pointer(P, s, 'x') - p4 = cast_structfield_pointer(P, s, 'y') + p1 = direct_ptradd(direct_arrayitems(a1), 3) + p2 = direct_ptradd(direct_arrayitems(a2), 1) + p3 = direct_fieldptr(s, 'x') + p4 = direct_fieldptr(s, 'y') def llf(): a1[3] += 1000 a2[1] += 100 From arigo at codespeak.net Tue Apr 18 17:51:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 18 Apr 2006 17:51:10 +0200 (CEST) Subject: [pypy-svn] r25958 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060418155110.C27AB10091@code0.codespeak.net> Author: arigo Date: Tue Apr 18 17:51:09 2006 New Revision: 25958 Modified: pypy/dist/pypy/translator/backendopt/malloc.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: A case where malloc removal would hang, trying repeatedly to "remove" the same GcStruct without progress. Modified: pypy/dist/pypy/translator/backendopt/malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/malloc.py Tue Apr 18 17:51:09 2006 @@ -164,6 +164,14 @@ return False except (ValueError, AttributeError), e: pass + + # to avoid an infinite loop of "removals" without actual progress, + # be careful when trying to remove something that already looks like + # a GcStruct wrapper + if (len(STRUCT._names) == 1 and + isinstance(STRUCT._flds[STRUCT._names[0]], lltype.ContainerType) and + not equivalent_substruct(STRUCT, STRUCT._names[0])): + return False # success: replace each variable with a family of variables (one per field) example = STRUCT._container_example() Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Tue Apr 18 17:51:09 2006 @@ -196,3 +196,16 @@ return a[0]-a[2] check(fn, [int, int], [100, 42], 58) + +def test_wrapper_cannot_be_removed(): + from pypy.rpython.lltypesystem import lltype + SMALL = lltype.OpaqueType('SMALL') + BIG = lltype.GcStruct('BIG', ('z', lltype.Signed), ('s', SMALL)) + + def g(small): + return -1 + def fn(): + b = lltype.malloc(BIG) + g(b.s) + + check(fn, [], [], None, must_be_removed=False) From pedronis at codespeak.net Tue Apr 18 20:24:45 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 18 Apr 2006 20:24:45 +0200 (CEST) Subject: [pypy-svn] r25961 - pypy/dist/pypy/objspace/constraint Message-ID: <20060418182445.D808610085@code0.codespeak.net> Author: pedronis Date: Tue Apr 18 20:24:45 2006 New Revision: 25961 Modified: pypy/dist/pypy/objspace/constraint/ (props changed) Log: set svn:ignore From pedronis at codespeak.net Tue Apr 18 20:25:42 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 18 Apr 2006 20:25:42 +0200 (CEST) Subject: [pypy-svn] r25962 - pypy/dist/pypy/objspace/constraint/test Message-ID: <20060418182542.7367610085@code0.codespeak.net> Author: pedronis Date: Tue Apr 18 20:25:41 2006 New Revision: 25962 Modified: pypy/dist/pypy/objspace/constraint/test/ (props changed) Log: set svn:ignore From pedronis at codespeak.net Tue Apr 18 20:26:49 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 18 Apr 2006 20:26:49 +0200 (CEST) Subject: [pypy-svn] r25963 - in pypy/dist/pypy/translator/llvm/pyllvm: . test Message-ID: <20060418182649.8DCF710085@code0.codespeak.net> Author: pedronis Date: Tue Apr 18 20:26:48 2006 New Revision: 25963 Modified: pypy/dist/pypy/translator/llvm/pyllvm/ (props changed) pypy/dist/pypy/translator/llvm/pyllvm/test/ (props changed) Log: set svn:ignore From tismer at codespeak.net Tue Apr 18 22:59:29 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 18 Apr 2006 22:59:29 +0200 (CEST) Subject: [pypy-svn] r25964 - pypy/dist/pypy/translator/c/test Message-ID: <20060418205929.79FE410086@code0.codespeak.net> Author: tismer Date: Tue Apr 18 22:59:27 2006 New Revision: 25964 Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py Log: baby-steps towards some completion Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Tue Apr 18 22:59:27 2006 @@ -216,16 +216,16 @@ def setup_new_module(mod, modname): # note the name clash with py.test on setup_module - from types import module - m = module(modname) + import types + m = types.ModuleType(modname) allobjs = mod.__dict__.values() funcs = eval('[]') # or import list from __builtin__ from twisted.internet import reactor print dir(reactor) #whow this works + isinstance = eval('isinstance') for obj in allobjs: - print obj, dir(obj) - if hasattr(42, 'func_name'): + if isinstance(obj, types.FunctionType): funcs.append( (obj.func_name, obj) ) print 'funcs=', funcs funcs.sort() From pedronis at codespeak.net Wed Apr 19 01:13:32 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 19 Apr 2006 01:13:32 +0200 (CEST) Subject: [pypy-svn] r25965 - in pypy/dist/pypy: annotation objspace/flow objspace/flow/test rpython rpython/test translator Message-ID: <20060418231332.62A8610088@code0.codespeak.net> Author: pedronis Date: Wed Apr 19 01:13:26 2006 New Revision: 25965 Modified: pypy/dist/pypy/annotation/annrpython.py pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/objspace/flow/objspace.py pypy/dist/pypy/objspace/flow/test/test_objspace.py pypy/dist/pypy/rpython/rdict.py pypy/dist/pypy/rpython/rtuple.py pypy/dist/pypy/rpython/test/test_objectmodel.py pypy/dist/pypy/rpython/test/test_rdict.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/rpython/test/test_rtuple.py pypy/dist/pypy/translator/simplify.py Log: fix propagation of exception out r_dict operations. Notice that this required having the flow space assume that getitem/setitem/delitem can raise any exceptions. Changes with some tests at various levels to cope with this. Notice that the pruning of excepts links when a getitem appears in an unrelated try/except is less strong as a consequence. Now {}[[]] fails with a TypeError as expected in a translated PyPy. Modified: pypy/dist/pypy/annotation/annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/annrpython.py (original) +++ pypy/dist/pypy/annotation/annrpython.py Wed Apr 19 01:13:26 2006 @@ -516,11 +516,11 @@ arg1 = self.binding(op.args[0]) arg2 = self.binding(op.args[1]) binop = getattr(pair(arg1, arg2), op.opname, None) - can_only_throw = getattr(binop, "can_only_throw", None) + can_only_throw = read_can_only_throw(binop, arg1, arg2) elif op.opname in annmodel.UNARY_OPERATIONS: arg1 = self.binding(op.args[0]) unop = getattr(arg1, op.opname, None) - can_only_throw = getattr(unop, "can_only_throw", None) + can_only_throw = read_can_only_throw(unop, arg1) else: can_only_throw = None @@ -722,3 +722,9 @@ return "" %(break_at, self.op) __str__ = __repr__ + +def read_can_only_throw(opimpl, *args): + can_only_throw = getattr(opimpl, "can_only_throw", None) + if can_only_throw is None or isinstance(can_only_throw, list): + return can_only_throw + return can_only_throw(*args) Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Wed Apr 19 01:13:26 2006 @@ -400,22 +400,27 @@ class __extend__(pairtype(SomeDict, SomeObject)): + def _can_only_throw(dic1, *ignore): + if dic1.dictdef.dictkey.custom_eq_hash: + return None + return [KeyError] + def getitem((dic1, obj2)): getbookkeeper().count("dict_getitem", dic1) dic1.dictdef.generalize_key(obj2) return dic1.dictdef.read_value() - getitem.can_only_throw = [KeyError] + getitem.can_only_throw = _can_only_throw def setitem((dic1, obj2), s_value): getbookkeeper().count("dict_setitem", dic1) dic1.dictdef.generalize_key(obj2) dic1.dictdef.generalize_value(s_value) - setitem.can_only_throw = [KeyError] + setitem.can_only_throw = _can_only_throw def delitem((dic1, obj2)): getbookkeeper().count("dict_delitem", dic1) dic1.dictdef.generalize_key(obj2) - delitem.can_only_throw = [KeyError] + delitem.can_only_throw = _can_only_throw class __extend__(pairtype(SomeSlice, SomeSlice)): Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Wed Apr 19 01:13:26 2006 @@ -460,6 +460,12 @@ int: [ValueError], # built-ins that can always raise exceptions chr: [ValueError], unichr: [ValueError], + # specifying IndexError, and KeyError beyond Exception, + # allows the annotator to be more precise, see test_reraiseAnything/KeyError in + # the annotator tests + 'getitem': [IndexError, KeyError, Exception], + 'setitem': [IndexError, KeyError, Exception], + 'delitem': [IndexError, KeyError, Exception], } def _add_exceptions(names, exc): @@ -477,13 +483,13 @@ lis.append(OverflowError) implicit_exceptions[name+"_ovf"] = lis -for _err in IndexError, KeyError: - _add_exceptions("""getitem setitem delitem""", _err) +#for _err in IndexError, KeyError: +# _add_exceptions("""getitem setitem delitem""", _err) for _name in 'getattr', 'delattr': _add_exceptions(_name, AttributeError) for _name in 'iter', 'coerce': _add_exceptions(_name, TypeError) -del _name, _err +del _name#, _err _add_exceptions("""div mod divmod truediv floordiv pow inplace_div inplace_mod inplace_divmod inplace_truediv Modified: pypy/dist/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_objspace.py Wed Apr 19 01:13:26 2006 @@ -248,15 +248,32 @@ self.show(x) #__________________________________________________________ - def implicitIndexError(lst): + def implicitException(lst): try: x = lst[5] - except IndexError: + except Exception: return 'catch' return lst[3] # not caught - def test_implicitIndexError(self): - x = self.codetest(self.implicitIndexError) + def test_implicitException(self): + x = self.codetest(self.implicitException) + simplify_graph(x) + self.show(x) + def cannot_reach_exceptblock(link): + if isinstance(link, Link): + assert link.target is not x.exceptblock + traverse(cannot_reach_exceptblock, x) + + + def implicitAttributeError(x): + try: + x = getattr(x, "y") + except AttributeError: + return 'catch' + return getattr(x, "z") # not caught + + def test_implicitAttributeError(self): + x = self.codetest(self.implicitAttributeError) simplify_graph(x) self.show(x) def cannot_reach_exceptblock(link): @@ -296,32 +313,72 @@ assert d == {None: True, OSError: True, Exception: True} #__________________________________________________________ - def reraiseKeyError(dic): + def reraiseAttributeError(v): + try: + x = getattr(v, "y") + except AttributeError: + raise + + def test_reraiseAttributeError(self): + x = self.codetest(self.reraiseAttributeError) + simplify_graph(x) + self.show(x) + found_AttributeError = [] + def only_raise_AttributeError(link): + if isinstance(link, Link): + if link.target is x.exceptblock: + assert link.args[0] == Constant(AttributeError) + found_AttributeError.append(link) + traverse(only_raise_AttributeError, x) + assert found_AttributeError + + def reraiseTypeError(dic): try: x = dic[5] - except KeyError: + except TypeError: raise - def test_reraiseKeyError(self): - x = self.codetest(self.reraiseKeyError) + def test_reraiseTypeError(self): + x = self.codetest(self.reraiseTypeError) simplify_graph(x) self.show(x) - found_KeyError = [] - def only_raise_KeyError(link): + found = [] + def can_reach_exceptblock(link): if isinstance(link, Link): if link.target is x.exceptblock: - assert link.args[0] == Constant(KeyError) - found_KeyError.append(link) - traverse(only_raise_KeyError, x) - assert found_KeyError + found.append(link) + traverse(can_reach_exceptblock, x) + assert found + #__________________________________________________________ - def reraiseAnything(dic): + def reraiseAnythingDicCase(dic): try: dic[5] except: raise + def test_reraiseAnythingDicCase(self): + x = self.codetest(self.reraiseAnythingDicCase) + simplify_graph(x) + self.show(x) + found = {} + def find_exceptions(link): + if isinstance(link, Link): + if link.target is x.exceptblock: + if isinstance(link.args[0], Constant): + found[link.args[0].value] = True + else: + found[link.exitcase] = None + traverse(find_exceptions, x) + assert found == {IndexError: True, KeyError: True, Exception: None} + + def reraiseAnything(x): + try: + pow(x, 5) + except: + raise + def test_reraiseAnything(self): x = self.codetest(self.reraiseAnything) simplify_graph(x) @@ -333,7 +390,7 @@ assert isinstance(link.args[0], Constant) found[link.args[0].value] = True traverse(find_exceptions, x) - assert found == {KeyError: True, IndexError: True} + assert found == {ValueError: True, ZeroDivisionError: True, OverflowError: True} #__________________________________________________________ def freevar(self, x): Modified: pypy/dist/pypy/rpython/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/rdict.py (original) +++ pypy/dist/pypy/rpython/rdict.py Wed Apr 19 01:13:26 2006 @@ -329,19 +329,25 @@ def rtype_getitem((r_dict, r_key), hop): v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) - hop.has_implicit_exception(KeyError) # record that we know about it + if not r_dict.custom_eq_hash: + hop.has_implicit_exception(KeyError) # record that we know about it hop.exception_is_here() v_res = hop.gendirectcall(ll_dict_getitem, v_dict, v_key) return r_dict.recast_value(hop.llops, v_res) def rtype_delitem((r_dict, r_key), hop): v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) - hop.has_implicit_exception(KeyError) # record that we know about it + if not r_dict.custom_eq_hash: + hop.has_implicit_exception(KeyError) # record that we know about it hop.exception_is_here() return hop.gendirectcall(ll_dict_delitem, v_dict, v_key) def rtype_setitem((r_dict, r_key), hop): v_dict, v_key, v_value = hop.inputargs(r_dict, r_dict.key_repr, r_dict.value_repr) + if r_dict.custom_eq_hash: + hop.exception_is_here() + else: + hop.exception_cannot_occur() hop.gendirectcall(ll_dict_setitem, v_dict, v_key, v_value) def rtype_contains((r_dict, r_key), hop): Modified: pypy/dist/pypy/rpython/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/rtuple.py (original) +++ pypy/dist/pypy/rpython/rtuple.py Wed Apr 19 01:13:26 2006 @@ -139,6 +139,8 @@ v_tuple, v_index = hop.inputargs(r_tup, Signed) if not isinstance(v_index, Constant): raise TyperError("non-constant tuple index") + if hop.has_implicit_exception(IndexError): + hop.exception_cannot_occur() index = v_index.value v = r_tup.getitem(hop.llops, v_tuple, index) return hop.llops.convertvar(v, r_tup.items_r[index], r_tup.external_items_r[index]) Modified: pypy/dist/pypy/rpython/test/test_objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_objectmodel.py (original) +++ pypy/dist/pypy/rpython/test/test_objectmodel.py Wed Apr 19 01:13:26 2006 @@ -149,19 +149,44 @@ def test_rtype_r_dict_exceptions(): def raising_hash(obj): - if id(obj) % 2 == 0: + if obj.startswith("bla"): raise TypeError return 1 def eq(obj1, obj2): return obj1 is obj2 def f(): d1 = r_dict(eq, raising_hash) + d1['xxx'] = 1 try: x = d1["blabla"] except Exception: - return 1 + return 42 return x res = interpret(f, []) + assert res == 42 + + def f(): + d1 = r_dict(eq, raising_hash) + d1['xxx'] = 1 + try: + x = d1["blabla"] + except TypeError: + return 42 + return x + res = interpret(f, []) + assert res == 42 + + def f(): + d1 = r_dict(eq, raising_hash) + d1['xxx'] = 1 + try: + d1["blabla"] = 2 + except TypeError: + return 42 + return 0 + res = interpret(f, []) + assert res == 42 + def test_rtype_keepalive(): from pypy.rpython import objectmodel @@ -183,3 +208,36 @@ assert res == 5 +def test_access_in_try(): + h = lambda x: 1 + eq = lambda x,y: x==y + def f(d): + try: + return d[2] + except ZeroDivisionError: + return 42 + return -1 + def g(n): + d = r_dict(eq, h) + d[1] = n + d[2] = 2*n + return f(d) + res = interpret(g, [3]) + assert res == 6 + +def test_access_in_try_set(): + h = lambda x: 1 + eq = lambda x,y: x==y + def f(d): + try: + d[2] = 77 + except ZeroDivisionError: + return 42 + return -1 + def g(n): + d = r_dict(eq, h) + d[1] = n + f(d) + return d[2] + res = interpret(g, [3]) + assert res == 77 Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Wed Apr 19 01:13:26 2006 @@ -680,3 +680,30 @@ assert res == 2 res = interpret(f, [6]) assert res == 0 + +def test_access_in_try(): + def f(d): + try: + return d[2] + except ZeroDivisionError: + return 42 + return -1 + def g(n): + d = {1: n, 2: 2*n} + return f(d) + res = interpret(g, [3]) + assert res == 6 + +def test_access_in_try_set(): + def f(d): + try: + d[2] = 77 + except ZeroDivisionError: + return 42 + return -1 + def g(n): + d = {1: n} + f(d) + return d[2] + res = interpret(g, [3]) + assert res == 77 Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Wed Apr 19 01:13:26 2006 @@ -1104,6 +1104,32 @@ assert r_A_list.lowleveltype == r_B_list.lowleveltype + def test_access_in_try(self): + def f(sq): + try: + return sq[2] + except ZeroDivisionError: + return 42 + return -1 + def g(n): + l = [1] * n + return f(l) + res = self.interpret(g, [3]) + assert res == 1 + + def test_access_in_try_set(self): + def f(sq): + try: + sq[2] = 77 + except ZeroDivisionError: + return 42 + return -1 + def g(n): + l = [1] * n + f(l) + return l[2] + res = self.interpret(g, [3]) + assert res == 77 class TestLltypeRtyping(BaseTestListRtyping): Modified: pypy/dist/pypy/rpython/test/test_rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rtuple.py (original) +++ pypy/dist/pypy/rpython/test/test_rtuple.py Wed Apr 19 01:13:26 2006 @@ -224,6 +224,18 @@ res = self.interpret(f, [0]) assert self.class_name(res) == "B" + def test_access_in_try(self): + def f(sq): + try: + return sq[2] + except ZeroDivisionError: + return 42 + return -1 + def g(n): + t = (1,2,n) + return f(t) + res = self.interpret(g, [3]) + assert res == 3 class TestLLTuple(AbstractTestRTuple): Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Wed Apr 19 01:13:26 2006 @@ -214,15 +214,17 @@ def visit(block): if not (isinstance(block, Block) - and block.exitswitch == clastexc and len(block.exits) == 2 - and block.exits[1].exitcase is Exception): + and block.exitswitch == clastexc + and block.exits[-1].exitcase is Exception): return + covered = [link.exitcase for link in block.exits[1:-1]] seen = [] - norm, exc = block.exits + preserve = list(block.exits[:-1]) + exc = block.exits[-1] last_exception = exc.last_exception last_exc_value = exc.last_exc_value query = exc.target - switches = [ (None, norm) ] + switches = [] # collect the targets while len(query.exits) == 2: newrenaming = {} @@ -238,7 +240,13 @@ lno, lyes = query.exits assert lno.exitcase == False and lyes.exitcase == True if case not in seen: - switches.append( (case, lyes) ) + is_covered = False + for cov in covered: + if issubclass(case, cov): + is_covered = True + break + if not is_covered: + switches.append( (case, lyes) ) seen.append(case) exc = lno query = exc.target @@ -248,20 +256,20 @@ exits = [] for case, oldlink in switches: link = oldlink.copy(rename) - if case is not None: - link.last_exception = last_exception - link.last_exc_value = last_exc_value - # make the above two variables unique - renaming2 = {} - def rename2(v): - return renaming2.get(v, v) - for v in link.getextravars(): - renaming2[v] = Variable(v) - link = link.copy(rename2) + assert case is not None + link.last_exception = last_exception + link.last_exc_value = last_exc_value + # make the above two variables unique + renaming2 = {} + def rename2(v): + return renaming2.get(v, v) + for v in link.getextravars(): + renaming2[v] = Variable(v) + link = link.copy(rename2) link.exitcase = case link.prevblock = block exits.append(link) - block.exits = tuple(exits) + block.exits = tuple(preserve + exits) traverse(visit, graph) From pedronis at codespeak.net Wed Apr 19 01:16:26 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 19 Apr 2006 01:16:26 +0200 (CEST) Subject: [pypy-svn] r25966 - pypy/dist/pypy/translator/c/test Message-ID: <20060418231626.170A910088@code0.codespeak.net> Author: pedronis Date: Wed Apr 19 01:16:25 2006 New Revision: 25966 Modified: pypy/dist/pypy/translator/c/test/test_typed.py Log: forgot to check in these genc level tests in the last checkin, about exception propagation out of r_dict ops. Modified: pypy/dist/pypy/translator/c/test/test_typed.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_typed.py (original) +++ pypy/dist/pypy/translator/c/test/test_typed.py Wed Apr 19 01:16:25 2006 @@ -722,3 +722,48 @@ fn = self.getcompiled(g) assert fn(0) == 1 assert fn(1) == 42 + + def test_r_dict_exceptions(self): + from pypy.rpython.objectmodel import r_dict + + def raising_hash(obj): + if obj.startswith("bla"): + raise TypeError + return 1 + def eq(obj1, obj2): + return obj1 is obj2 + def f(): + d1 = r_dict(eq, raising_hash) + d1['xxx'] = 1 + try: + x = d1["blabla"] + except Exception: + return 42 + return x + fn = self.getcompiled(f) + res = fn() + assert res == 42 + + def f(): + d1 = r_dict(eq, raising_hash) + d1['xxx'] = 1 + try: + x = d1["blabla"] + except TypeError: + return 42 + return x + fn = self.getcompiled(f) + res = fn() + assert res == 42 + + def f(): + d1 = r_dict(eq, raising_hash) + d1['xxx'] = 1 + try: + d1["blabla"] = 2 + except TypeError: + return 42 + return 0 + fn = self.getcompiled(f) + res = fn() + assert res == 42 From tismer at codespeak.net Wed Apr 19 02:33:35 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 19 Apr 2006 02:33:35 +0200 (CEST) Subject: [pypy-svn] r25967 - pypy/dist/pypy/objspace/flow Message-ID: <20060419003335.178451008E@code0.codespeak.net> Author: tismer Date: Wed Apr 19 02:33:32 2006 New Revision: 25967 Modified: pypy/dist/pypy/objspace/flow/objspace.py Log: removing some left-overs. Will revive part of it in some 'experiments' folder. Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Wed Apr 19 02:33:32 2006 @@ -599,52 +599,17 @@ make_op(*line) """ -Strategy for a new import logic -------------------------------- - -It is an old problem to decide whether to use do_imports_immediately. -In general, it would be nicer not to use this flag for RPython, in order -to make it easy to support imports at run-time for extensions. - -On the other hand, there are situations where this is absolutely needed: -Some of the ll helper functions need to import something late, to -avoid circular imports. Not doing the import immediately would cause -a crash, because the imported object would become SomeObject. - -We would like to have control over imports even on a per-import policy. - -As a general solution, I came up with the following trick, or maybe it's -not a trick but a good concept: - -By declaring the imported subject as a global, you trigger the immediate -import. This is consistent with the RPython concept that globals -should never change, just with the addition that objects may be added. -In addition, we consider global modules to be immutable, making attribute -access a constant operation. - -As a generalisation, we can enforce that getattr/setattr on any -object that is unwrappable for computation is evaluated -immediately. This gives us early detection of programming errors. -XXX this step isn't done, yet, need to discuss this. - -Implementation --------------- - -It is not completely trivial, since we have to intercept the process -of flowing, to keep trak of which variable might become a constant. -Finally I ended up with a rather simple solution: -Flowcontext monitors every link creation, by no longer using -Link() directly, but redirecting this to a function make_link, -which can be patched to record the creation of links. - -The actual tracking and constant resolving is implemented in the -ConstTracker class below. - +This is just a placeholder for some code I'm checking in elsewhere. +It is provenly possible to determine constantness of certain expressions +a little later. I introduced this a bit too early, together with tieing +this to something being global, which was a bad idea. +The concept is still valid, and it can be used to force something to +be evaluated immediately because it is supposed to be a constant. +One good possible use of this is loop unrolling. +This will be found in an 'experimental' folder with some use cases. """ def override(): - from __builtin__ import getattr as _getattr # uhmm - def getattr(self, w_obj, w_name): # handling special things like sys # unfortunately this will never vanish with a unique import logic :-( From tismer at codespeak.net Wed Apr 19 04:33:45 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 19 Apr 2006 04:33:45 +0200 (CEST) Subject: [pypy-svn] r25968 - pypy/dist/pypy/rpython Message-ID: <20060419023345.9EE981008E@code0.codespeak.net> Author: tismer Date: Wed Apr 19 04:33:40 2006 New Revision: 25968 Modified: pypy/dist/pypy/rpython/rslice.py (props changed) Log: still some files with missing eol-style From tismer at codespeak.net Wed Apr 19 07:24:19 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 19 Apr 2006 07:24:19 +0200 (CEST) Subject: [pypy-svn] r25969 - in pypy/dist/pypy: rpython/lltypesystem translator/c/test Message-ID: <20060419052419.D23D61009C@code0.codespeak.net> Author: tismer Date: Wed Apr 19 07:24:16 2006 New Revision: 25969 Modified: pypy/dist/pypy/rpython/lltypesystem/rslice.py pypy/dist/pypy/translator/c/test/test_wrapping.py Log: some builtin support and some slicing. a bit tedious, should avoid spending too much time Modified: pypy/dist/pypy/rpython/lltypesystem/rslice.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rslice.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rslice.py Wed Apr 19 07:24:16 2006 @@ -1,6 +1,9 @@ from pypy.rpython.rslice import AbstractSliceRepr from pypy.rpython.lltypesystem.lltype import \ - GcStruct, Signed, Ptr, Void, malloc + GcStruct, Signed, Ptr, Void, malloc, PyObject, nullptr +from pypy.annotation.pairtype import pairtype +from pypy.rpython.robject import PyObjRepr, pyobj_repr +from pypy.rpython.rmodel import inputconst, PyObjPtr, IntegerRepr # ____________________________________________________________ # @@ -36,3 +39,27 @@ s.stop = stop return s +# ____________________________________________________________ +# +# limited support for casting into PyObject + +# stuff like this should go into one file maybe + +class __extend__(pairtype(SliceRepr, PyObjRepr)): + def convert_from_to((r_from, r_to), v, llops): + null = inputconst(Ptr(PyObject), nullptr(PyObject)) + def pyint(v): + return llops.gencapicall('PyInt_FromLong', [v], resulttype=r_to) + v_step = v_start = v_stop = null + if r_from.lowleveltype is Signed: + v_start = pyint(v) + elif r_from.lowleveltype is Void: + v_stop = inputconst(r_to, -1) + else: + v_start = pyint(llops.genop('getfield', [v, inputconst(Void, 'start')], + resulttype=Signed)) + v_stop = pyint(llops.genop('getfield', [v, inputconst(Void, 'stop')], + resulttype=Signed)) + return llops.gencapicall('PySlice_New', + [v_start, v_stop, v_step], + resulttype = pyobj_repr) Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Wed Apr 19 07:24:16 2006 @@ -5,6 +5,7 @@ from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem import lltype from pypy.rpython import robject, rclass +from pypy.translator.tool.cbuild import enable_fast_compilation import sys @@ -214,20 +215,53 @@ # _______________________________________________ # creating our own setup function for the module +# this class *can* be used for faster access. +# the compiler anyway chews quite a bit on it... +class BuiltinHelper(object): + # the following would be much easier if we had + # loop unrolling right inside the flowing process + src = [] + src.append('def __init__(self):') + src.append(' import __builtin__ as b') + import __builtin__ + for name in dir(__builtin__): + obj = getattr(__builtin__, name) + if callable(obj) and hasattr(obj, '__name__'): + src.append(' self.%s = b.%s' % (name, obj.__name__)) + src = '\n'.join(src) + #print src + exec src + del __builtin__, name, obj, src + def setup_new_module(mod, modname): # note the name clash with py.test on setup_module import types m = types.ModuleType(modname) allobjs = mod.__dict__.values() funcs = eval('[]') # or import list from __builtin__ - from twisted.internet import reactor - print dir(reactor) + # one alternative + #bltn = BuiltinHelper() + # less code: + import __builtin__ as bltn + print bltn.list('hallo') + #from twisted.internet import reactor + #print dir(reactor) #whow this works isinstance = eval('isinstance') + # above is possible, this is probably a better compromise: + isinstance = bltn.isinstance for obj in allobjs: if isinstance(obj, types.FunctionType): funcs.append( (obj.func_name, obj) ) print 'funcs=', funcs + print funcs[3:] + #funcs += [2, 3, 5] + # not yet + stuff = bltn.range(10) + print stuff[3:] + print stuff[:3] + print stuff[3:7] + print stuff[:-1] funcs.sort() return m From hpk at codespeak.net Wed Apr 19 10:30:55 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 19 Apr 2006 10:30:55 +0200 (CEST) Subject: [pypy-svn] r25971 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419083055.D011A100B9@code0.codespeak.net> Author: hpk Date: Wed Apr 19 10:30:53 2006 New Revision: 25971 Added: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: added a first preliminary draft for gathering info about surely or maybe-coming people to Iceland we need much info: please fill in and decide before friday about your participance readyness! take yourself out if you know you can't make it. Added: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 10:30:53 2006 @@ -0,0 +1,109 @@ +Travel planning for Iceland PyPy sprinters +================================================== + +This sprint's travel organisation is to be handlded +by EWT and CCP games, the sponsoring organisations. +Therefore we need to provide more information +(which is needed for bookings and asking back) +Sprint core time is 21st till 28th (both including ASFAIK) + + +Armin Rigo +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: arigo at tunes.org +birthdate: +address: +organisation: +comments: + +Holger Krekel +------------------------------ + +preferred arrival date: 20th acceptable arrival dates: 19th, 21st +preferred departure date: 28th acceptable departure dates: 27th,29th +start city / airport: Hildesheim / Frankfurt (FRA) +mail: holger at merlinux.de + +birthdate: 14th nov. 1969, phone number: +49 171 464 8622 +address: Steinbergstr. 42, 31139 Hildesheim, Germany +organisation: merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany +comments: not before 11 AM as i need a 3 hour train travel to FRA airport + +Samuele Pedroni +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: pedronis at strakt.com +birthdate: +address: +organisation: AB Strakt, +comments: + +Anders Chrigstroem +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: pedronis at strakt.com +birthdate: +address: +organisation: AB Strakt, +comments: + +Jan Balster +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: jan at merlinux.de + +birthdate: +address: +organisation: merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany +comments: + + +Michael Hudson +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: arigo at tunes.org +birthdate: +address: +organisation: +comments: + +Antonio Cuni +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: +birthdate: +address: +organisation: +comments: + + +(Wanja Saatkamp) +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: wanja at merlinux.de +birthdate: +address: +organisation: +comments: From hpk at codespeak.net Wed Apr 19 10:35:20 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Wed, 19 Apr 2006 10:35:20 +0200 (CEST) Subject: [pypy-svn] r25972 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419083520.5B7CE100B6@code0.codespeak.net> Author: hpk Date: Wed Apr 19 10:35:19 2006 New Revision: 25972 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: added Bea (although i don't really know if she comes or for how long etc.) Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 10:35:19 2006 @@ -20,6 +20,18 @@ organisation: comments: +Beatrice During +------------------------------ + +preferred arrival date: acceptable arrival dates: +preferred departure date: acceptable departure dates: +start city / airport: +mail: bea at changemaker.nu +birthdate: +address: +organisation: +comments: + Holger Krekel ------------------------------ From jan at codespeak.net Wed Apr 19 12:20:21 2006 From: jan at codespeak.net (jan at codespeak.net) Date: Wed, 19 Apr 2006 12:20:21 +0200 (CEST) Subject: [pypy-svn] r25976 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419102021.E9F30100B6@code0.codespeak.net> Author: jan Date: Wed Apr 19 12:20:20 2006 New Revision: 25976 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: Added info for Jan Balster Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 12:20:20 2006 @@ -72,13 +72,13 @@ Jan Balster ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: +preferred arrival date: 20th acceptable arrival dates: 19th, 21st +preferred departure date: 28th acceptable departure dates: 27th, 29th +start city / airport: L?beck / Hamburg mail: jan at merlinux.de -birthdate: -address: +birthdate: 11th jan. 1978, +address: Buergerweide 1, 23562 L?beck, Germany organisation: merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany comments: From arigo at codespeak.net Wed Apr 19 13:17:41 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 13:17:41 +0200 (CEST) Subject: [pypy-svn] r25977 - in pypy/dist/pypy: rpython/rctypes/test translator/backendopt translator/backendopt/test Message-ID: <20060419111741.D2D4A100B5@code0.codespeak.net> Author: arigo Date: Wed Apr 19 13:17:40 2006 New Revision: 25977 Modified: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py pypy/dist/pypy/translator/backendopt/malloc.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: direct_fieldptr support in malloc removal, for rctypes. The code is getting rather involved, but still has some kind of convoluted elegance to it. Modified: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_overhead.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_overhead.py Wed Apr 19 13:17:40 2006 @@ -81,7 +81,6 @@ assert not mallocs def test_struct_setitem(): - py.test.skip("not working yet") class S(Structure): _fields_ = [('x', c_int)] class T(Structure): Modified: pypy/dist/pypy/translator/backendopt/malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/malloc.py Wed Apr 19 13:17:40 2006 @@ -23,8 +23,8 @@ # we consider a pointer to a GcStruct S as equivalent to a # pointer to a substructure 'S.fieldname' if it's the first # inlined sub-GcStruct. As an extension we also allow a pointer - # to a GcStruct containing just one Struct to be equivalent to - # a pointer to that Struct only (although a mere cast_pointer + # to a GcStruct containing just one field to be equivalent to + # a pointer to that field only (although a mere cast_pointer # would not allow casting). This is needed to malloc-remove # the 'wrapper' GcStructs introduced by previous passes of # malloc removal. @@ -35,7 +35,7 @@ FIELDTYPE = S._flds[fieldname] if isinstance(FIELDTYPE, lltype.GcStruct): return True - if len(S._names) == 1 and isinstance(FIELDTYPE, lltype.Struct): + if len(S._names) == 1: return True return False @@ -79,7 +79,7 @@ # and output variables union(node, op.args[0], node, op.result) continue - if op.opname == 'getsubstruct': + if op.opname in ('getsubstruct', 'direct_fieldptr'): S = op.args[0].concretetype.TO if equivalent_substruct(S, op.args[1].value): # assumed to be similar to a cast_pointer @@ -135,26 +135,42 @@ STRUCT = lltypes.keys()[0].TO assert isinstance(STRUCT, lltype.GcStruct) - # must be only ever accessed via getfield/setfield/getsubstruct - # or touched by keepalive. Note that same_as and cast_pointer - # are not recorded in usepoints. - VALID = dict.fromkeys([("getfield", 0), - ("setfield", 0), - ("getsubstruct", 0), - ("keepalive", 0)]) - MAYBE_VALID = dict.fromkeys([("getarrayitem", 0), - ("setarrayitem", 0), - ("getarraysubstruct", 0)]) + # must be only ever accessed via getfield/setfield/getsubstruct/ + # direct_fieldptr, or touched by keepalive. Note that same_as and + # cast_pointer are not recorded in usepoints. + FIELD_ACCESS = dict.fromkeys(["getfield", + "setfield", + "keepalive", + "getarrayitem", + "setarrayitem"]) + SUBSTRUCT_ACCESS = dict.fromkeys(["getsubstruct", + "direct_fieldptr", + "getarraysubstruct"]) + + CHECK_ARRAY_INDEX = dict.fromkeys(["getarrayitem", + "setarrayitem", + "getarraysubstruct"]) + accessed_substructs = {} + for up in info.usepoints: if up[0] != "op": return False kind, node, op, index = up - if (op.opname, index) in VALID: - continue # ok - if (op.opname, index) in MAYBE_VALID: - if isinstance(op.args[1], Constant): - continue # ok if the index is constant - return False + if index != 0: + return False + if op.opname in CHECK_ARRAY_INDEX: + if not isinstance(op.args[1], Constant): + return False # non-constant array index + if op.opname in FIELD_ACCESS: + pass # ok + elif op.opname in SUBSTRUCT_ACCESS: + S = op.args[0].concretetype.TO + name = op.args[1].value + if not isinstance(name, str): # access by index + name = 'item%d' % (name,) + accessed_substructs[S, name] = True + else: + return False # must not remove mallocs of structures that have a RTTI with a destructor @@ -165,16 +181,7 @@ except (ValueError, AttributeError), e: pass - # to avoid an infinite loop of "removals" without actual progress, - # be careful when trying to remove something that already looks like - # a GcStruct wrapper - if (len(STRUCT._names) == 1 and - isinstance(STRUCT._flds[STRUCT._names[0]], lltype.ContainerType) and - not equivalent_substruct(STRUCT, STRUCT._names[0])): - return False - # success: replace each variable with a family of variables (one per field) - example = STRUCT._container_example() # 'flatnames' is a list of (STRUCTTYPE, fieldname_in_that_struct) that # describes the list of variables that should replace the single @@ -191,27 +198,47 @@ flatconstants = {} needsubmallocs = [] newvarstype = {} # map {item-of-flatnames: concretetype} + direct_fieldptr_key = {} - def flatten(S, example): + def flatten(S): start = 0 if S._names and equivalent_substruct(S, S._names[0]): - flatten(S._flds[S._names[0]], getattr(example, S._names[0])) - start = 1 + SUBTYPE = S._flds[S._names[0]] + if isinstance(SUBTYPE, lltype.Struct): + flatten(SUBTYPE) + start = 1 + else: + ARRAY = lltype.FixedSizeArray(SUBTYPE, 1) + direct_fieldptr_key[ARRAY, 'item0'] = S, S._names[0] for name in S._names[start:]: key = S, name flatnames.append(key) FIELDTYPE = S._flds[name] - if isinstance(FIELDTYPE, lltype.ContainerType): + if key in accessed_substructs: needsubmallocs.append(key) newvarstype[key] = lltype.Ptr(lltype.GcStruct('wrapper', ('data', FIELDTYPE))) - else: - constant = Constant(getattr(example, name)) + elif not isinstance(FIELDTYPE, lltype.ContainerType): + example = FIELDTYPE._defl() + constant = Constant(example) constant.concretetype = FIELDTYPE flatconstants[key] = constant newvarstype[key] = FIELDTYPE + #else: + # the inlined substructure is never accessed, drop it + + flatten(STRUCT) + assert len(direct_fieldptr_key) <= 1 - flatten(STRUCT, example) + def key_for_field_access(S, fldname): + if isinstance(S, lltype.FixedSizeArray): + if not isinstance(fldname, str): # access by index + fldname = 'item%d' % (fldname,) + try: + return direct_fieldptr_key[S, fldname] + except KeyError: + pass + return S, fldname variables_by_block = {} for block, var in info.variables: @@ -241,20 +268,31 @@ if op.opname in ("getfield", "getarrayitem"): S = op.args[0].concretetype.TO fldname = op.args[1].value - if op.opname == "getarrayitem": - fldname = 'item%d' % fldname - newop = SpaceOperation("same_as", - [newvarsmap[S, fldname]], - op.result) + key = key_for_field_access(S, fldname) + if key in accessed_substructs: + c_name = Constant('data', lltype.Void) + newop = SpaceOperation("getfield", + [newvarsmap[key], c_name], + op.result) + else: + newop = SpaceOperation("same_as", + [newvarsmap[key]], + op.result) newops.append(newop) last_removed_access = len(newops) elif op.opname in ("setfield", "setarrayitem"): S = op.args[0].concretetype.TO fldname = op.args[1].value - if op.opname == "setarrayitem": - fldname = 'item%d' % fldname - assert (S, fldname) in newvarsmap - newvarsmap[S, fldname] = op.args[2] + key = key_for_field_access(S, fldname) + assert key in newvarsmap + if key in accessed_substructs: + c_name = Constant('data', lltype.Void) + newop = SpaceOperation("setfield", + [newvarsmap[key], c_name, op.args[2]], + op.result) + newops.append(newop) + else: + newvarsmap[key] = op.args[2] last_removed_access = len(newops) elif op.opname in ("same_as", "cast_pointer"): assert op.result not in vars @@ -265,7 +303,8 @@ # via one pointer must be reflected in the other. elif op.opname == 'keepalive': last_removed_access = len(newops) - elif op.opname in ("getsubstruct", "getarraysubstruct"): + elif op.opname in ("getsubstruct", "getarraysubstruct", + "direct_fieldptr"): S = op.args[0].concretetype.TO fldname = op.args[1].value if op.opname == "getarraysubstruct": @@ -278,9 +317,13 @@ else: # do it with a getsubstruct on the independently # malloc'ed GcStruct + if op.opname == "direct_fieldptr": + opname = "direct_fieldptr" + else: + opname = "getsubstruct" v = newvarsmap[S, fldname] cname = Constant('data', lltype.Void) - newop = SpaceOperation("getsubstruct", + newop = SpaceOperation(opname, [v, cname], op.result) newops.append(newop) @@ -289,7 +332,7 @@ elif op.result in vars: assert op.opname == "malloc" assert vars == {var: True} - count[0] += 1 + progress = True # drop the "malloc" operation newvarsmap = flatconstants.copy() # zero initial values # if there are substructures, they are now individually @@ -299,9 +342,13 @@ v = Variable() v.concretetype = newvarstype[key] c = Constant(v.concretetype.TO, lltype.Void) + if c.value == op.args[0].value: + progress = False # replacing a malloc with + # the same malloc! newop = SpaceOperation("malloc", [c], v) newops.append(newop) newvarsmap[key] = v + count[0] += progress else: newops.append(op) @@ -354,7 +401,6 @@ for var in vars_created_here: flowin(var, newvarsmap=None) - assert count[0] return count[0] def remove_mallocs_once(graph): Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Wed Apr 19 13:17:40 2006 @@ -209,3 +209,46 @@ g(b.s) check(fn, [], [], None, must_be_removed=False) + +def test_direct_fieldptr(): + from pypy.rpython.lltypesystem import lltype + S = lltype.GcStruct('S', ('x', lltype.Signed)) + + def fn(): + s = lltype.malloc(S) + s.x = 11 + p = lltype.direct_fieldptr(s, 'x') + return p[0] + + check(fn, [], [], 11) + +def test_direct_fieldptr_2(): + from pypy.rpython.lltypesystem import lltype + T = lltype.GcStruct('T', ('z', lltype.Signed)) + S = lltype.GcStruct('S', ('t', T), + ('x', lltype.Signed), + ('y', lltype.Signed)) + def fn(): + s = lltype.malloc(S) + s.x = 10 + s.t.z = 1 + px = lltype.direct_fieldptr(s, 'x') + py = lltype.direct_fieldptr(s, 'y') + pz = lltype.direct_fieldptr(s.t, 'z') + py[0] = 31 + return px[0] + s.y + pz[0] + + check(fn, [], [], 42) + +def test_getarraysubstruct(): + from pypy.rpython.lltypesystem import lltype + U = lltype.Struct('U', ('n', lltype.Signed)) + for length in [1, 2]: + S = lltype.GcStruct('S', ('a', lltype.FixedSizeArray(U, length))) + for index in range(length): + + def fn(): + s = lltype.malloc(S) + s.a[index].n = 12 + return s.a[index].n + check(fn, [], [], 12) From arigo at codespeak.net Wed Apr 19 13:18:44 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 13:18:44 +0200 (CEST) Subject: [pypy-svn] r25978 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060419111844.C68B4100B5@code0.codespeak.net> Author: arigo Date: Wed Apr 19 13:18:42 2006 New Revision: 25978 Added: pypy/dist/pypy/rpython/rctypes/test/conftest.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/test/test_import.py pypy/dist/pypy/rpython/rctypes/test/test_overhead.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Log: Use a single conftest.py to detect Pythons where ctypes is not installed. Added: pypy/dist/pypy/rpython/rctypes/test/conftest.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/conftest.py Wed Apr 19 13:18:42 2006 @@ -0,0 +1,10 @@ +import py + +class Directory(py.test.collect.Directory): + + def run(self): + try: + import ctypes + except ImportError: + py.test.skip("these tests need ctypes installed") + return super(Directory, self).run() Modified: pypy/dist/pypy/rpython/rctypes/test/test_import.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_import.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_import.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Checks that importing and registering the annotations for rctypes +Check that importing and registering the annotations for rctypes doesn't bring in the whole rtyper. """ Modified: pypy/dist/pypy/rpython/rctypes/test/test_overhead.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_overhead.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_overhead.py Wed Apr 19 13:18:42 2006 @@ -9,11 +9,6 @@ from pypy.rpython.test.test_llinterp import gengraph from pypy.translator.backendopt.all import backend_optimizations -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_int, Structure, pointer, POINTER Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Test the rctypes implementation. +Test the array implementation. """ import py @@ -12,11 +12,6 @@ import sys from pypy.rpython.test.test_llinterp import interpret -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_int, c_short, c_char_p, c_char, pointer from ctypes import ARRAY, POINTER, Structure Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Test the rctypes implementation. +Test the c_char_p implementation. """ import py.test @@ -12,11 +12,6 @@ import sys from pypy.rpython.test.test_llinterp import interpret -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_char_p, pointer, Structure class Test_annotation: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Apr 19 13:18:42 2006 @@ -15,12 +15,6 @@ thisdir = py.path.local(__file__).dirpath() -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - - from ctypes import cdll from ctypes import POINTER, Structure, c_int, byref, pointer, c_void_p Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Wed Apr 19 13:18:42 2006 @@ -1,3 +1,7 @@ +""" +Test external function calls. +""" + import py import sys import pypy.rpython.rctypes.implementation Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Wed Apr 19 13:18:42 2006 @@ -9,11 +9,6 @@ from pypy.rpython.test.test_llinterp import interpret from pypy.translator.c.test.test_genc import compile -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_int, c_float, POINTER, pointer, Structure class Test_annotation: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Test the rctypes implementation. +Test the primitive c_* implementation. """ import py.test @@ -13,11 +13,6 @@ from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.rarithmetic import r_longlong, r_ulonglong -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_char, c_byte, c_ubyte, c_short, c_ushort, c_int, c_uint from ctypes import c_long, c_ulong, c_longlong, c_ulonglong, c_float from ctypes import c_double, c_wchar, c_char_p, pointer Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Test the rctypes implementation. +Test the create_string_buffer() implementation. """ import py.test @@ -12,11 +12,6 @@ import sys from pypy.rpython.test.test_llinterp import interpret -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import create_string_buffer from pypy.rpython.rctypes.astringbuf import StringBufferType Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Test the rctypes implementation. +Test the Structure implementation. """ import py.test @@ -11,11 +11,6 @@ import sys from pypy.rpython.test.test_llinterp import interpret -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_int, c_short, Structure, POINTER, pointer, c_char_p class tagpoint(Structure): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Wed Apr 19 13:18:42 2006 @@ -1,5 +1,5 @@ """ -Test the rctypes implementation. +Test the c_void_p implementation. """ import py.test @@ -11,11 +11,6 @@ from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.test.test_llinterp import interpret -try: - import ctypes -except ImportError: - py.test.skip("this test needs ctypes installed") - from ctypes import c_void_p, c_int, cast, pointer, POINTER class Test_annotation: From arigo at codespeak.net Wed Apr 19 13:22:24 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 13:22:24 +0200 (CEST) Subject: [pypy-svn] r25979 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060419112224.91BB2100AD@code0.codespeak.net> Author: arigo Date: Wed Apr 19 13:22:22 2006 New Revision: 25979 Modified: pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/rctypes/avoid_p.py pypy/dist/pypy/rpython/rctypes/rmodel.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Log: Truth value of various ctypes objects. Using None in calls to functions expecting pointers. Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Wed Apr 19 13:22:22 2006 @@ -645,6 +645,9 @@ else: return SomeObject() + def is_true(cto): + return SomeBool() + #_________________________________________ # memory addresses Modified: pypy/dist/pypy/rpython/rctypes/avoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/avoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/avoid_p.py Wed Apr 19 13:22:22 2006 @@ -7,11 +7,18 @@ # c_void_p() as a function -def c_void_p_compute_result_annotation(s_arg=None): - raise NotImplementedError("XXX calling c_void_p()") +def c_void_p_compute_result_annotation(): + return annmodel.SomeCTypesObject(c_void_p, + annmodel.SomeCTypesObject.OWNSMEMORY) + +def c_void_specialize_call(hop): + r_void_p = hop.r_result + v_result = r_void_p.allocate_instance(hop.llops) + return v_result extregistry.register_value(c_void_p, compute_result_annotation=c_void_p_compute_result_annotation, + specialize_call=c_void_specialize_call, ) # c_void_p instances Modified: pypy/dist/pypy/rpython/rctypes/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rmodel.py (original) +++ pypy/dist/pypy/rpython/rctypes/rmodel.py Wed Apr 19 13:22:22 2006 @@ -1,6 +1,6 @@ from pypy.rpython.rmodel import Repr, inputconst from pypy.rpython.error import TyperError -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.annotation.model import SomeCTypesObject from pypy.annotation.pairtype import pairtype @@ -237,8 +237,28 @@ r_temp.setvalue(llops, v_owned_box, v_value) return llops.convertvar(v_owned_box, r_temp, self) + def rtype_is_true(self, hop): + [v_box] = hop.inputargs(self) + v_value = self.getvalue(hop.llops, v_box) + if v_value.concretetype in (lltype.Char, lltype.UniChar): + llfn = ll_c_char_is_true + elif v_value.concretetype == llmemory.Address: + llfn = ll_address_is_true + else: + llfn = ll_is_true + return hop.gendirectcall(llfn, v_value) + # ____________________________________________________________ +def ll_is_true(x): + return bool(x) + +def ll_c_char_is_true(x): + return bool(ord(x)) + +def ll_address_is_true(x): + return x != llmemory.NULL + C_ZERO = inputconst(lltype.Signed, 0) def reccopy(source, dest): Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Wed Apr 19 13:22:22 2006 @@ -30,6 +30,8 @@ llops.genop('setfield', inputargs) def initialize_const(self, p, ptr): + if ptr is None: # passed as argument to functions expecting pointers + return llcontents = self.r_contents.convert_const(ptr.contents) p.c_data[0] = llcontents.c_data # the following line is probably pointless, as 'llcontents' will be Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Wed Apr 19 13:22:22 2006 @@ -132,3 +132,26 @@ s.p = a s.p.contents.value = 42 assert a[0] == 42 + +def test_truth_value(): + p = POINTER(c_int)() + assert not p + p.contents = c_int(12) + assert p + # I can't figure out how to reset p to NULL... + + assert c_int(12) + assert not c_int(0) # a bit strange, if you ask me + assert c_int(-1) + assert not c_byte(0) + assert not c_char('\x00') # hum + assert not c_float(0.0) + assert not c_double(0.0) + assert not c_ulonglong(0) + assert c_ulonglong(2L**42) + + assert c_char_p("hello") + assert c_char_p("") + assert not c_char_p(None) + + assert not c_void_p() Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Wed Apr 19 13:22:22 2006 @@ -104,6 +104,12 @@ res = interpret(func, []) assert ''.join(res.chars) == "hello" + def test_truth_value(self): + def func(): + assert c_char_p("hello") + assert c_char_p("") + assert not c_char_p(None) + interpret(func, []) class Test_compilation: def test_compile_c_char_p(self): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Wed Apr 19 13:22:22 2006 @@ -14,6 +14,7 @@ from ctypes import cdll from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer +from ctypes import POINTER # __________ the standard C library __________ @@ -53,6 +54,10 @@ return result atoi.llinterp_friendly_version = ll_atoi +time_ = mylib.time +time_.restype = c_long # should rather use ctypes_platform.getsimpletype() +time_.argtypes = [POINTER(c_long)] + def test_labs(n=6): assert labs(n) == abs(n) @@ -82,6 +87,13 @@ assert ll_atoi(str2subarray("blah")) == 0 assert ll_atoi(str2subarray("18238")) == 18238 +def test_time(): + import time + t1 = time.time() + t2 = time_(None) + t3 = time.time() + assert int(t1) <= t2 <= int(t3 + 1.0) + class Test_annotation: def test_annotate_labs(self): a = RPythonAnnotator() @@ -141,3 +153,13 @@ fn = compile(test_labs, [int]) res = fn(-11) assert res == 11 + + def test_compile_time(self): + import time + def fn1(): + return time_(None) + fn = compile(fn1, []) + t1 = time.time() + t2 = fn() + t3 = time.time() + assert int(t1) <= t2 <= int(t3 + 1.0) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpointer.py Wed Apr 19 13:22:22 2006 @@ -267,6 +267,14 @@ res = interpret(fn, []) assert res == 5 + def test_specialize_null_pointer(self): + def fn(): + p = POINTER(c_int)() + assert not p + p.contents = c_int(12) + assert p + interpret(fn, []) + class Test_compilation: def test_compile_getitem_nonzero_index(self): A = c_int * 10 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rprimitive.py Wed Apr 19 13:22:22 2006 @@ -253,6 +253,21 @@ assert x.value == u'B' interpret(func, []) + def test_truth_value(self): + bigzero = r_ulonglong(0) + big = r_ulonglong(2L**42) + def func(n, z): + assert c_int(n) + assert not c_int(z) + assert c_int(-1) + assert not c_byte(z) + assert not c_char(chr(z)) + assert not c_float(z) + assert not c_double(z) + assert not c_ulonglong(bigzero) + assert c_ulonglong(big) + interpret(func, [17, 0]) + class Test_compilation: def test_compile_c_int(self): def create_c_int(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Wed Apr 19 13:22:22 2006 @@ -43,6 +43,11 @@ assert lltype.typeOf(res.item0.c_data[0]) == llmemory.Address assert res.item1 == 12 + def test_truth_value(self): + def func(): + assert not c_void_p() + interpret(func, []) + class Test_compilation: def test_compile_c_char_p(self): def func(): From arigo at codespeak.net Wed Apr 19 13:47:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 13:47:18 +0200 (CEST) Subject: [pypy-svn] r25980 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419114718.1079A100B6@code0.codespeak.net> Author: arigo Date: Wed Apr 19 13:47:17 2006 New Revision: 25980 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: My details. Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 13:47:17 2006 @@ -11,13 +11,13 @@ Armin Rigo ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: +preferred arrival date: 19th acceptable arrival dates: any +preferred departure date: 29th acceptable departure dates: any +start city / airport: Gothenburg / Landvetter (GOT) or City Airport (GSE) mail: arigo at tunes.org -birthdate: -address: -organisation: +birthdate: 26 sep 1976 +address: App 22, c/o Hall?n/Creighton, Linn?gatan 47, 413 08 G?teborg, Sweden +organisation: Heinrich-Heine Universit?ts D?sseldorf, Germany comments: Beatrice During From pedronis at codespeak.net Wed Apr 19 13:51:45 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 19 Apr 2006 13:51:45 +0200 (CEST) Subject: [pypy-svn] r25982 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419115145.AF736100B6@code0.codespeak.net> Author: pedronis Date: Wed Apr 19 13:51:44 2006 New Revision: 25982 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: arre's and mine data. I cannot travel on the 20th, needs to be in Gothenburg Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 13:51:44 2006 @@ -48,26 +48,27 @@ Samuele Pedroni ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: +preferred arrival date: 21st acceptable arrival dates: 21st-22nd +preferred departure date: 29th acceptable departure dates: 28th-31th +start city / airport: Gothenburg / GOT or GSE mail: pedronis at strakt.com -birthdate: -address: +birthdate: 22nd dec 1974 +address: Norra Agatan 10A, 416 64 Gothenburg, Sweden organisation: AB Strakt, -comments: +comments: need to be in Gothenburg on the 20th so cannot travel earlier, + direct flight preferred Anders Chrigstroem ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: -mail: pedronis at strakt.com -birthdate: -address: +preferred arrival date: 21st acceptable arrival dates: 19th-22nd +preferred departure date: 29th acceptable departure dates: 28th-31th +start city / airport: Gothenburg / GOT or GSE +mail: ac at strakt.com +birthdate: 10th dec 1966 +address: Norra Agatan 10A, 416 64 Gothenburg, Sweden organisation: AB Strakt, -comments: +comments: direct flight preferred Jan Balster ------------------------------ From antocuni at codespeak.net Wed Apr 19 13:56:00 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 19 Apr 2006 13:56:00 +0200 (CEST) Subject: [pypy-svn] r25983 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419115600.94B79100B5@code0.codespeak.net> Author: antocuni Date: Wed Apr 19 13:55:50 2006 New Revision: 25983 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: My details. Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 13:55:50 2006 @@ -99,15 +99,15 @@ Antonio Cuni ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: -mail: -birthdate: -address: +preferred arrival date: 20th acceptable arrival dates: 19th, 21st +preferred departure date: 26th acceptable departure dates: 27th +start city / airport: Genova / Genova (GOA) +mail: anto.cuni at gmail.com +birthdate: 13th may 1982 +address: Via Perniciaro 70, 16010 Mele(GE), Italy organisation: -comments: - +comments: preferred start airport is Genova (GOA), but Milano Malpensa + (MXP) is acceptable, too. Need to be in Genova on the 28th. (Wanja Saatkamp) ------------------------------ From arigo at codespeak.net Wed Apr 19 15:49:14 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 15:49:14 +0200 (CEST) Subject: [pypy-svn] r25990 - pypy/dist/pypy/module/_demo Message-ID: <20060419134914.73253100B9@code0.codespeak.net> Author: arigo Date: Wed Apr 19 15:49:12 2006 New Revision: 25990 Added: pypy/dist/pypy/module/_demo/ (props changed) pypy/dist/pypy/module/_demo/__init__.py (contents, props changed) pypy/dist/pypy/module/_demo/app_demo.py (contents, props changed) pypy/dist/pypy/module/_demo/demo.py (contents, props changed) Log: A demo built-in module using ctypes. Works with either py.py --usemodules=_demo or translate.py targetpypystandalone --usemodules=_demo Added: pypy/dist/pypy/module/_demo/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_demo/__init__.py Wed Apr 19 15:49:12 2006 @@ -0,0 +1,12 @@ +from pypy.interpreter.mixedmodule import MixedModule + +class Module(MixedModule): + """A demo built-in module based on ctypes.""" + + interpleveldefs = { + 'measuretime' : 'demo.measuretime', + } + + appleveldefs = { + 'DemoError' : 'app_demo.DemoError', + } Added: pypy/dist/pypy/module/_demo/app_demo.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_demo/app_demo.py Wed Apr 19 15:49:12 2006 @@ -0,0 +1,3 @@ + +class DemoError(Exception): + pass Added: pypy/dist/pypy/module/_demo/demo.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_demo/demo.py Wed Apr 19 15:49:12 2006 @@ -0,0 +1,49 @@ +from pypy.interpreter.error import OperationError +from pypy.interpreter.baseobjspace import ObjSpace, W_Root +from pypy.rpython.rctypes import implementation, ctypes_platform +import sys +from ctypes import * + +# __________ the standard C library __________ + +# LoadLibrary is deprecated in ctypes, this should be removed at some point +if "load" in dir(cdll): + cdll_load = cdll.load +else: + cdll_load = cdll.LoadLibrary + +if sys.platform == 'win32': + mylib = cdll_load('msvcrt.dll') +elif sys.platform == 'linux2': + mylib = cdll_load('libc.so.6') +elif sys.platform == 'darwin': + mylib = cdll.c +else: + py.test.skip("don't know how to load the c lib for %s" % + sys.platform) +# ____________________________________________ + + +time_t = ctypes_platform.getsimpletype('time_t', '#include ', c_long) + +time = mylib.time +time.argtypes = [POINTER(time_t)] +time.restype = time_t + + +def get(space, name): + w_module = space.getbuiltinmodule('_demo') + return space.getattr(w_module, space.wrap(name)) + + +def measuretime(space, repetitions, w_callable): + if repetitions <= 0: + w_DemoError = get(space, 'DemoError') + msg = "repetition count must be > 0" + raise OperationError(w_DemoError, space.wrap(msg)) + starttime = time(None) + for i in range(repetitions): + space.call_function(w_callable) + endtime = time(None) + return space.wrap(endtime - starttime) +measuretime.unwrap_spec = [ObjSpace, int, W_Root] From wanja at codespeak.net Wed Apr 19 17:53:46 2006 From: wanja at codespeak.net (wanja at codespeak.net) Date: Wed, 19 Apr 2006 17:53:46 +0200 (CEST) Subject: [pypy-svn] r25993 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060419155346.E2471100B9@code0.codespeak.net> Author: wanja Date: Wed Apr 19 17:53:34 2006 New Revision: 25993 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: added myself to the list, although my attendance is not 100% decided yet. Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Wed Apr 19 17:53:34 2006 @@ -112,11 +112,12 @@ (Wanja Saatkamp) ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: -mail: wanja at merlinux.de -birthdate: -address: -organisation: -comments: +preferred arrival date: 20th acceptable arrival dates: 19th, 21st +preferred departure date: 28th acceptable departure dates: 27th,29th +start city / airport: Hildesheim / Frankfurt (FRA), Hannover, Berlin or Hamburg +mail: wanja at merlinux.de +birthdate: 22.02.1975 +address: Wrangelstr. 67, 10997 Berlin, Germany +organisation: Merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany +comments: - no flight departure before 12AM because of travel time to the airport; +- my attendance is not decided yet (and i will not be coding but documenting the activities with the camera) From arigo at codespeak.net Wed Apr 19 18:46:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 18:46:18 +0200 (CEST) Subject: [pypy-svn] r25994 - pypy/dist/pypy/translator/goal Message-ID: <20060419164618.660CD10086@code0.codespeak.net> Author: arigo Date: Wed Apr 19 18:46:17 2006 New Revision: 25994 Added: pypy/dist/pypy/translator/goal/targetdemomodule.py (contents, props changed) Log: In-progress target: compiling pypy/module/_demo/ as a CPython extension module. Added: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Wed Apr 19 18:46:17 2006 @@ -0,0 +1,80 @@ +from pypy.module._demo import demo + +from ctypes import * + +Py_ssize_t = c_int # XXX changes in Python 2.5 + +PyObject_GetAttr = pythonapi.PyObject_GetAttr +PyObject_GetAttr.argtypes = [py_object, py_object] +PyObject_GetAttr.restype = py_object + +PyImport_ImportModule = pythonapi.PyImport_ImportModule +PyImport_ImportModule.argtypes = [c_char_p] +PyImport_ImportModule.restype = py_object + +PyInt_FromLong = pythonapi.PyInt_FromLong +PyInt_FromLong.argtypes = [c_long] +PyInt_FromLong.restype = py_object + +PyString_FromStringAndSize = pythonapi.PyString_FromStringAndSize +PyString_FromStringAndSize.argtypes = [c_char_p, Py_ssize_t] +PyString_FromStringAndSize.restype = py_object + +PyObject_Call = pythonapi.PyObject_Call +PyObject_Call.argtypes = [py_object, py_object, py_object] +PyObject_Call.restype = py_object + +PyTuple_New = pythonapi.PyTuple_New +PyTuple_New.argtypes = [Py_ssize_t] +PyTuple_New.restype = py_object + +PyDict_New = pythonapi.PyDict_New +PyDict_New.argtypes = [] +PyDict_New.restype = py_object + + +class CPyObjSpace: + + def __init__(self): + self.w_int = py_object(int) + self.w_None = py_object(None) + + def getbuiltinmodule(self, name): + return PyImport_ImportModule(name) + + def wrap(self, x): + if x is None: + return self.w_None + if isinstance(x, int): + return PyInt_FromLong(x) + if isinstance(x, str): + return PyString_FromStringAndSize(x, len(x)) + raise TypeError("wrap(%r)" % (x,)) + wrap._annspecialcase_ = "specialize:wrap" + + def getattr(self, w_obj, w_attr): + return PyObject_GetAttr(w_obj, w_attr) + + def call_function(self, w_callable): + return PyObject_Call(w_callable, PyTuple_New(0), PyDict_New()) + + +space = CPyObjSpace() + +def entry_point(n): + return demo.measuretime(space, n, space.w_int) + +# _____ Define and setup target ___ + +def target(*args): + return entry_point, [int] + + +if __name__ == '__main__': + import sys + if len(sys.argv) <= 1: + N = 500000 + else: + N = int(sys.argv[1]) + print 'Timing for %d iterations...' % N + print entry_point(N), 'seconds' From arigo at codespeak.net Wed Apr 19 19:05:09 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 19:05:09 +0200 (CEST) Subject: [pypy-svn] r25995 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060419170509.D0B0610087@code0.codespeak.net> Author: arigo Date: Wed Apr 19 19:05:02 2006 New Revision: 25995 Added: pypy/dist/pypy/rpython/rctypes/apyobject.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py Log: Starting ctypes.py_object support: annotating calls to py_object(). Added: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Wed Apr 19 19:05:02 2006 @@ -0,0 +1,11 @@ +from ctypes import py_object +from pypy.annotation.model import SomeCTypesObject +from pypy.rpython import extregistry + + +def py_object_compute_result_annotation(s_obj=None): + return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) + +extregistry.register_value(py_object, + compute_result_annotation=py_object_compute_result_annotation, + )#specialize_call=py_object_specialize_call) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Apr 19 19:05:02 2006 @@ -7,6 +7,7 @@ import pypy.rpython.rctypes.astruct import pypy.rpython.rctypes.avoid_p import pypy.rpython.rctypes.astringbuf +import pypy.rpython.rctypes.apyobject # Register the correspondance between SomeCTypesObject and the get_repr() Added: pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Wed Apr 19 19:05:02 2006 @@ -0,0 +1,33 @@ +""" +Test the special py_object support. +""" + +import py +import pypy.rpython.rctypes.implementation +from pypy.annotation import model as annmodel +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy import conftest +from pypy.translator.c.test.test_genc import compile +import sys +from pypy.rpython.test.test_llinterp import interpret + +from ctypes import py_object + + +class Test_annotation: + def test_annotate_wrapping(self): + def wrap(x): + return py_object(x) + + a = RPythonAnnotator() + s = a.build_types(wrap, [int]) + assert s.knowntype == py_object + if conftest.option.view: + a.translator.view() + + a = RPythonAnnotator() + s = a.build_types(wrap, [str]) + assert s.knowntype == py_object + if conftest.option.view: + a.translator.view() From auc at codespeak.net Wed Apr 19 19:06:04 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 19 Apr 2006 19:06:04 +0200 (CEST) Subject: [pypy-svn] r25996 - in pypy/dist/pypy/objspace/constraint: . test Message-ID: <20060419170604.479A110087@code0.codespeak.net> Author: auc Date: Wed Apr 19 19:05:59 2006 New Revision: 25996 Added: pypy/dist/pypy/objspace/constraint/btree.py pypy/dist/pypy/objspace/constraint/test/test_btree.py pypy/dist/pypy/objspace/constraint/util.py Modified: pypy/dist/pypy/objspace/constraint/computationspace.py pypy/dist/pypy/objspace/constraint/constraint.py pypy/dist/pypy/objspace/constraint/distributor.py pypy/dist/pypy/objspace/constraint/domain.py pypy/dist/pypy/objspace/constraint/test/test_computationspace.py pypy/dist/pypy/objspace/constraint/test/test_constraint.py pypy/dist/pypy/objspace/constraint/test/test_distributor.py pypy/dist/pypy/objspace/constraint/test/test_solver.py Log: bunch of changes to help translatability Added: pypy/dist/pypy/objspace/constraint/btree.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/btree.py Wed Apr 19 19:05:59 2006 @@ -0,0 +1,72 @@ + +class Found(Exception): pass + +class BTree(object): + """binary tree""" + + def __init__(self): + self.leaf = True + + def _populate(self, key, val): + self.leaf = False + self.key = key + self.val = val + self.left = BTree() + self.right = BTree() + + def add(self, key, val): + if self.leaf: + self._populate(key, val) + else: + if key > self.key: + self.right.add(key, val) + else: + self.left.add(key, val) + + def _in(self, key): + if self.leaf: + return False + if self.key == key: raise Found + self.left._in(key) + self.right._in(key) + + def __contains__(self, key): + if self.leaf: return False + if self.key == key: return True + try: + self._in(key) + self._in(key) + return False + except Found: + return True + + def _infixly(self, output, ret=0): + """depending on the value of ret: + 0 : keys + 1 : values + 2 : both (items) + """ + if self.leaf: return + self.left._infixly(output, ret) + if ret == 0: + output.append(self.key) + elif ret == 1: + output.append(self.val) + else: + output.append((self.key, self.val)) + self.right._infixly(output, ret) + + def keys(self): + out = [] + self._infixly(out, ret=0) + return out + + def values(self): + out = [] + self._infixly(out, ret=1) + return out + + def items(self): + out = [] + self._infixly(out, ret=2) + return out Modified: pypy/dist/pypy/objspace/constraint/computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/computationspace.py Wed Apr 19 19:05:59 2006 @@ -6,7 +6,7 @@ from pypy.objspace.std.stringobject import W_StringObject from pypy.objspace.constraint.domain import W_AbstractDomain - +from pypy.objspace.constraint.util import sort, reverse all_mms = {} @@ -15,13 +15,13 @@ class W_Variable(Wrappable): def __init__(self, obj_space, w_name): self._space = obj_space - self.w_name = w_name + self._name = w_name def name_w(self): - return self._space.str_w(self.w_name) + return self._space.str_w(self._name) def w_name(self): - return self.w_name + return self._name W_Variable.typedef = typedef.TypeDef( "W_Variable", @@ -33,33 +33,18 @@ def __init__(self, object_space): self._space = object_space - def w_affected_variables(self): - pass - - def w_is_variable_relevant(self, w_var): - pass - - def w_estimate_cost(self, w_cs): - pass - - def w_revise(self, w_cs): - pass - W_Constraint.typedef = typedef.TypeDef( - "W_Constraint", - affected_variables = interp2app(W_Constraint.w_affected_variables), - is_variable_relevant = interp2app(W_Constraint.w_is_variable_relevant), - estimate_cost = interp2app(W_Constraint.w_estimate_cost), - revise = interp2app(W_Constraint.w_revise)) + "W_Constraint") #-- Distributors (standards) ------------ class W_Distributor(Wrappable): - def __init__(self, object_space, fanout_w): + def __init__(self, object_space, fanout): + assert isinstance(fanout, int) self._space = object_space - self.fanout = fanout_w + self._fanout = fanout W_Distributor.typedef = typedef.TypeDef("W_Distributor") @@ -99,7 +84,7 @@ except: # FIXME: indexError ? self.status = self._space.newint(1) return self.status - self.status = self._space.newint(self.distributor.fanout) + self.status = self.distributor.w_fanout() return self.status def w_clone(self): @@ -148,7 +133,9 @@ return self._space.newlist(res) def w_dom(self, w_variable): - assert isinstance(w_variable, W_Variable) + if not isinstance(w_variable, W_Variable): + raise OperationError(self._space.w_RuntimeError, + self._space.wrap("dom() takes a variable")) return self.var_dom[w_variable] def w_tell(self, w_constraint): @@ -172,22 +159,24 @@ return self._space.newtuple(res) def w_test_solution(self, w_sol): - varset = {} - for var, val in zip(self.sol_set.wrappeditems, - w_sol.wrappeditems): - varset[var.w_name] = val - for _const in self.constraints: - if not _const.test_solution(varset): - print "Solution", sol, "doesn't satisfy", _const - return self._space.newbool(False) - return self._space.newbool(True) + pass +## varset = {} +## for var, val in zip(self.sol_set.wrappeditems, +## w_sol.wrappeditems): +## varset[var.w_name()] = val +## for _const in self.constraints: +## if not _const.test_solution(varset): +## #print "Solution", sol, "doesn't satisfy", _const +## return self._space.newbool(False) +## return self._space.newbool(True) def w_print_state(self): - print "VARS :", self.name_var.keys() - print "CONST :", self.var_const.values() - print "DOMS :", self.var_dom.values() - print "CHK :", self.to_check + pass +## print "VARS :", self.name_var.keys() +## print "CONST :", self.var_const.values() +## print "DOMS :", self.var_dom.values() +## print "CHK :", self.to_check #-- everything else --------------- @@ -202,8 +191,8 @@ for const in self.to_check] self.to_check = {} assert const_q != [] - const_q.sort() - const_q.reverse() # for pop() friendlyness + sort(const_q) + reverse(const_q) # for pop() friendlyness affected_constraints = {} while True: if not const_q: Modified: pypy/dist/pypy/objspace/constraint/constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/constraint.py Wed Apr 19 19:05:59 2006 @@ -2,14 +2,23 @@ from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter import baseobjspace, typedef, gateway from pypy.interpreter.gateway import interp2app +from pypy.interpreter.function import Function from pypy.objspace.std.listobject import W_ListObject +from pypy.objspace.std.stringobject import W_StringObject from pypy.objspace.constraint.computationspace import W_ComputationSpace -from pypy.objspace.constraint.computationspace import W_Constraint +from pypy.objspace.constraint.computationspace import W_Constraint, W_Variable + +from pypy.objspace.std.model import StdObjSpaceMultiMethod + +from pypy.objspace.constraint.btree import BTree import operator +all_mms = {} + + #-- Exceptions --------------------------------------- class ConsistencyFailure(Exception): @@ -23,7 +32,6 @@ #-- Constraints ------------------------------------------ - class W_AbstractConstraint(W_Constraint): def __init__(self, object_space, w_variables): @@ -45,7 +53,8 @@ return self._variables def w_knows_var(self, w_variable): - return self._space.newbool(variable in self._variables) + assert isinstance(w_variable, W_Variable) + return self._space.newbool(w_variable in self._variables) def w_estimate_cost(self, w_cs): """Return an estimate of the cost of the narrowing of the constraint""" @@ -58,15 +67,20 @@ def estimate_cost_w(self, w_cs): assert isinstance(w_cs, W_ComputationSpace) - return reduce(operator.mul, - [w_cs.w_dom(var).size() - for var in self._variables]) - - def __eq__(self, other): #FIXME and parent - if not isinstance(other, self.__class__): return False - return self._variables == other._variables + v = 1 + for var in self._variables: + v = v * w_cs.w_dom(var).size() + return v + + + +## def __eq__(self, other): #FIXME and parent +## if not isinstance(other, self.__class__): return False +## return self._variables == other._variables -W_AbstractConstraint.typedef = typedef.TypeDef("W_AbstractConstraint", +W_AbstractConstraint.typedef = typedef.TypeDef( + "W_AbstractConstraint", + W_Constraint.typedef, affected_variables = interp2app(W_AbstractConstraint.w_affected_variables), knows_var = interp2app(W_AbstractConstraint.w_knows_var), estimate_cost = interp2app(W_AbstractConstraint.w_estimate_cost), @@ -95,17 +109,25 @@ def revise(self, w_cs): _spc = self._space assert isinstance(w_cs, W_ComputationSpace) - variables = [(_spc.int_w(w_cs.w_dom(variable).w_size()), - variable, w_cs.w_dom(variable)) - for variable in self._variables] + + ord_vars = BTree() + for variable in self._variables: + ord_vars.add(_spc.int_w(w_cs.w_dom(variable).w_size()), + (variable, w_cs.w_dom(variable))) + + variables = ord_vars.values() + +## variables = [(_spc.int_w(w_cs.w_dom(variable).w_size()), +## variable, w_cs.w_dom(variable)) +## for variable in self._variables] +## variables.sort() - variables.sort() # if a domain has a size of 1, # then the value must be removed from the other domains - for size, var, dom in variables: + for var, dom in variables: if _spc.eq_w(dom.w_size(), _spc.newint(1)): #print "AllDistinct removes values" - for _siz, _var, _dom in variables: + for _var, _dom in variables: if not _spc.eq_w(_var, var): try: _dom.w_remove_value(dom.w_get_values().wrappeditems[0]) @@ -116,7 +138,7 @@ # if there are less values than variables, the constraint fails values = {} - for size, var, dom in variables: + for var, dom in variables: for val in dom.w_get_values().wrappeditems: values[val] = 0 @@ -126,8 +148,8 @@ _spc.wrap("ConsistencyFailure")) # the constraint is entailed if all domains have a size of 1 - for variable in variables: - if not _spc.eq_w(variable[2].w_size(), + for _var, dom in variables: + if not _spc.eq_w(dom.w_size(), _spc.newint(1)): return False @@ -138,23 +160,33 @@ W_AllDistinct.typedef = typedef.TypeDef( "W_AllDistinct", W_AbstractConstraint.typedef, - estimate_cost = interp2app(W_AllDistinct.w_estimate_cost), - revise = interp2app(W_AllDistinct.w_revise)) + estimate_cost = interp2app(W_AllDistinct.w_estimate_cost)) +# revise = interp2app(W_AllDistinct.w_revise)) # function bolted into the space to serve as constructor def make_alldistinct(object_space, w_variables): + assert len(w_variables.wrappeditems) > 0 return object_space.wrap(W_AllDistinct(object_space, w_variables)) app_make_alldistinct = gateway.interp2app(make_alldistinct) -def make_filter(object_space, w_variables, w_formula): +def make_filter__List_String(object_space, w_variables, w_formula): """NOT RPYTHON""" - var_ids = ','.join([object_space.str_w(var.w_name) + assert isinstance(w_variables, W_ListObject) + assert isinstance(w_formula, W_StringObject) + var_ids = ','.join([var.name_w() for var in w_variables.wrappeditems]) func_head = 'lambda ' + var_ids + ':' - func_obj = eval(func_head + object_space.str_w(w_formula), {}, {}) + expr = func_head + object_space.str_w(w_formula) + func_obj = object_space.eval(expr, object_space.newdict({}), + object_space.newdict({})) + assert isinstance(func_obj, Function) return func_obj -app_make_filter = gateway.interp2app(make_filter) + +make_filter_mm = StdObjSpaceMultiMethod('make_filter', 2) +make_filter_mm.register(make_filter__List_String, W_ListObject, W_StringObject) +all_mms['make_filter'] = make_filter_mm + class W_Expression(W_AbstractConstraint): """A constraint represented as a python expression.""" @@ -164,21 +196,21 @@ formula is a python expression that will be evaluated as a boolean""" W_AbstractConstraint.__init__(self, object_space, w_variables) self.formula = self._space.str_w(w_formula) - self.filter_func = make_filter(self._space, w_variables, w_formula) + self.filter_func = self._space.make_filter(w_variables, w_formula) def test_solution(self, sol_dict): """test a solution against this constraint accept a mapping of variable names to value""" args = [] for var in self._variables: - args.append(sol_dict[var.w_name]) + args.append(sol_dict[var.w_name()]) return self.filter_func(*args) def _init_result_cache(self): """key = (variable,value), value = [has_success,has_failure]""" result_cache = self._space.newdict({}) for var in self._variables: - result_cache.content[var.w_name] = self._space.newdict({}) + result_cache.content[var.w_name()] = self._space.newdict({}) return result_cache def _assign_values(self, w_cs): @@ -190,7 +222,7 @@ variables.append((self._space.int_w(domain.w_size()), [variable, values, self._space.newint(0), self._space.len(values)])) - kwargs.content[variable.w_name] = values.wrappeditems[0] + kwargs.content[variable.w_name()] = values.wrappeditems[0] # sort variables to instanciate those with fewer possible values first variables.sort() @@ -201,11 +233,11 @@ for size, curr in variables: if self._space.int_w(curr[2]) + 1 < self._space.int_w(curr[-1]): curr[2] = self._space.add(curr[2], self._space.newint(1)) - kwargs.content[curr[0].w_name] = curr[1].wrappeditems[self._space.int_w(curr[2])] + kwargs.content[curr[0].w_name()] = curr[1].wrappeditems[self._space.int_w(curr[2])] break else: curr[2] = self._space.newint(0) - kwargs.content[curr[0].w_name] = curr[1].wrappeditems[0] + kwargs.content[curr[0].w_name()] = curr[1].wrappeditems[0] else: # it's over go_on = 0 @@ -250,8 +282,8 @@ return '<%s>' % self.formula W_Expression.typedef = typedef.TypeDef("W_Expression", - W_AbstractConstraint.typedef, - revise = interp2app(W_Expression.w_revise)) + W_AbstractConstraint.typedef) +# revise = interp2app(W_Expression.w_revise)) # completely unported @@ -305,9 +337,6 @@ except ConsistencyFailure: raise ConsistencyFailure('Inconsistency while applying %s' % \ repr(self)) - except Exception: - print self, kwargs - raise return maybe_entailed @@ -315,6 +344,7 @@ """create a new constraint of type Expression or BinaryExpression The chosen class depends on the number of variables in the constraint""" # encode unicode + assert len(w_variables.wrappeditems) > 0 return W_Expression(o_space, w_variables, w_formula) if o_space.eq_w(o_space.len(w_variables), o_space.newint(2)): return W_BinaryExpression(o_space, w_variables, w_formula) Modified: pypy/dist/pypy/objspace/constraint/distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/distributor.py Wed Apr 19 19:05:59 2006 @@ -21,12 +21,11 @@ class W_AbstractDistributor(W_Distributor): """_distribute is left unimplemented.""" - def w_fanout(self): - return self._space.newint(self.fanout) + def __init__(self, objspace, fanout): + W_Distributor.__init__(objspace, fanout) - def fanout(self): - """return number of possible splits""" - return self.fanout + def w_fanout(self): + return self._space.newint(self._fanout) def _find_smallest_domain(self, w_cs): """returns the variable having the smallest domain. @@ -34,6 +33,7 @@ """ vars_ = [var for var, dom in w_cs.var_dom.items() if dom.size() > 1] + assert len(vars_) > 0 best = vars_[0] for var in vars_: if w_cs.var_dom[var].size() < w_cs.var_dom[best].size(): @@ -54,7 +54,7 @@ def find_distribution_variable(self, w_cs): return self._find_smallest_domain(w_cs) - def _do_distribute(self, domain, choice): + def _do_distribute(self, w_cs, domain, choice): """remove values from domain depending on choice""" raise NotImplementedError @@ -69,14 +69,15 @@ The first new domain has a size of one, and the second has all the other values""" - def __init__(self, object_space, fanout_w): + def __init__(self, object_space, fanout): # default fanout is 2, see make_naive_distributor - W_Distributor.__init__(self, object_space, fanout_w) + W_Distributor.__init__(self, object_space, fanout) def _do_distribute(self, w_cs, domain, choice): values = domain.get_values() + #assert len(values) > 0 if choice == 0: - domain.w_remove_values(values[1:]) + domain.remove_values(values[1:]) else: domain.w_remove_value(values[0]) @@ -85,6 +86,9 @@ W_AbstractDistributor.typedef) def make_naive_distributor(object_space, fanout=2): + if not isinstance(fanout, int): + raise OperationError(object_space.w_RuntimeError, + object_space.wrap("fanout must be a positive integer")) return object_space.wrap(W_NaiveDistributor(object_space, fanout)) app_make_naive_distributor = interp2app(make_naive_distributor, unwrap_spec = [baseobjspace.ObjSpace, int]) @@ -96,16 +100,16 @@ If nb_subspaces is 0, then the smallest domain is split in domains of size 1""" - def __init__(self, object_space, fanout_w): + def __init__(self, object_space, fanout): # default fanout is 3, see make_split_distributor - W_Distributor.__init__(self, object_space, fanout_w) + W_Distributor.__init__(self, object_space, fanout) def _subdomains(self, w_cs): """returns the min number of partitions for a domain to be distributed""" to_split = self._find_smallest_domain(w_cs) - if self.fanout: - return min(self.fanout, + if self._fanout > 0: + return min(self._fanout, w_cs.w_dom(to_split).size()) else: return w_cs.w_dom(to_split).size() @@ -119,6 +123,9 @@ domain.remove_values(values[end:]) def make_split_distributor(object_space, fanout=3): + if not isinstance(fanout, int): + raise OperationError(object_space.w_RuntimeError, + object_space.wrap("fanout must be a positive integer")) return object_space.wrap(W_SplitDistributor(object_space, fanout)) app_make_split_distributor = interp2app(make_split_distributor, unwrap_spec = [baseobjspace.ObjSpace, int]) @@ -133,4 +140,3 @@ def make_dichotomy_distributor(object_space): return make_split_distributor(object_space, 2) app_make_dichotomy_distributor = interp2app(make_dichotomy_distributor) - Modified: pypy/dist/pypy/objspace/constraint/domain.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/domain.py (original) +++ pypy/dist/pypy/objspace/constraint/domain.py Wed Apr 19 19:05:59 2006 @@ -8,7 +8,6 @@ from pypy.objspace.std.model import StdObjSpaceMultiMethod - all_mms = {} class ConsistencyFailure(Exception): @@ -33,12 +32,6 @@ def has_changed(self): return self.__changed - def w_size(self): - return self._space.newint(self.size()) - - def size(self): - pass - def _value_removed(self): """The implementation of remove_value should call this method""" self.__changed = True @@ -78,14 +71,23 @@ def w_remove_values(self, w_values): """Remove values of domain and check for consistency""" + assert isinstance(w_values, W_ListObject) self.remove_values(w_values.wrappeditems) def remove_values(self, values): - if len(values) > 0: - for w_val in values: - del self._values[w_val] - self._value_removed() - + assert isinstance(values, list) + try: + if len(values) > 0: + for w_val in values: + del self._values[w_val] + self._value_removed() + except: + raise OperationError(self._space.w_RuntimeError, + self._space.wrap("attempt to remove unkown value from domain")) + + def w_size(self): + return self._space.newint(self.size()) + def size(self): """computes the size of a finite domain""" return len(self._values) Added: pypy/dist/pypy/objspace/constraint/test/test_btree.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/test/test_btree.py Wed Apr 19 19:05:59 2006 @@ -0,0 +1,18 @@ +class AppTest_BTree(object): + + + def test_everything_at_once(self): + from pypy.objspace.constraint.btree import BTree + b = BTree() + l = [(-1, 'egg'), (-7, 'spam'), (3, 'bacon'), + (99, 'ham'), (77, 'cheese'), (7, 'tomato'), + (5, 'chicken'), (9, 'noodles')] + for k, v in l: b.add(k, v) + assert 77 in b + assert 66 not in b + assert b.values() == ['spam', 'egg', 'bacon', 'chicken', + 'tomato', 'noodles', 'cheese', 'ham'] + assert b.keys() == [-7, -1, 3, 5, 7, 9, 77, 99] + assert b.items() == [(-7, 'spam'), (-1, 'egg'), (3, 'bacon'), + (5, 'chicken'), (7, 'tomato'), (9, 'noodles'), + (77, 'cheese'), (99, 'ham')] Modified: pypy/dist/pypy/objspace/constraint/test/test_computationspace.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_computationspace.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_computationspace.py Wed Apr 19 19:05:59 2006 @@ -128,7 +128,6 @@ csp.tell(AllDistinct([x, y, z])) return [x, y, z] csp.define_problem(problem) - csp.print_state() csp = csp.clone() assert csp.ask() == 1 assert csp.merge() == (1, 2, 3) Modified: pypy/dist/pypy/objspace/constraint/test/test_constraint.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_constraint.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_constraint.py Wed Apr 19 19:05:59 2006 @@ -16,6 +16,8 @@ assert len(variables) == 2 assert v1 in variables assert v2 in variables + assert cstr.knows_var(v1) + assert cstr.knows_var(v2) def test_estimated_cost(self): csp = newspace() Modified: pypy/dist/pypy/objspace/constraint/test/test_distributor.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_distributor.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_distributor.py Wed Apr 19 19:05:59 2006 @@ -22,6 +22,10 @@ d = NaiveDistributor() d.distribute(spc, 2) assert spc.dom(y) == FiniteDomain([2]) + d.distribute(spc, 1) + assert spc.dom(y) == FiniteDomain([2]) + assert spc.dom(z) == FiniteDomain([1]) + def test_split_distributor(self): spc = newspace() Modified: pypy/dist/pypy/objspace/constraint/test/test_solver.py ============================================================================== --- pypy/dist/pypy/objspace/constraint/test/test_solver.py (original) +++ pypy/dist/pypy/objspace/constraint/test/test_solver.py Wed Apr 19 19:05:59 2006 @@ -21,10 +21,7 @@ spc.define_problem(problems.conference_scheduling) sols = solver.solve(spc) - count = 0 solutions = set() for sol in sols: - assert sol not in solutions solutions.add(sol) - count += 1 - assert count == 64 + assert len(solutions) == 64 Added: pypy/dist/pypy/objspace/constraint/util.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/constraint/util.py Wed Apr 19 19:05:59 2006 @@ -0,0 +1,37 @@ + +def sort(l): + """in place merge sort""" + _sort(l, 0, len(l) - 1) + +def _sort(l, low, high): + if low >= high: return + mid = (low + high) / 2 + _sort(l, low, mid) + _sort(l, mid + 1, high) + stop_low = mid + start_high = mid + 1 + while low <= stop_low and start_high <= high: + if l[low] < l[start_high]: + low += 1 + else: + t = l[start_high] + k = start_high - 1 + while k >= low: + l[k+1] = l[k] + k -= 1 + l[low] = t + low += 1 + stop_low += 1 + start_high += 1 + + +def reverse(l): + """reverse the content of a vector""" + length = len(l) + mid = length / 2 + max = length - 1 + idx = 0 + while idx < mid: + antidx = max-idx + l[idx], l[antidx] = l[antidx], l[idx] + idx += 1 From arigo at codespeak.net Wed Apr 19 19:07:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 19:07:48 +0200 (CEST) Subject: [pypy-svn] r25997 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060419170748.E8B0B10089@code0.codespeak.net> Author: arigo Date: Wed Apr 19 19:07:47 2006 New Revision: 25997 Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Log: Annotate prebuilt py_object instances. Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Wed Apr 19 19:07:47 2006 @@ -3,9 +3,19 @@ from pypy.rpython import extregistry +# __________ py_object() calls __________ + def py_object_compute_result_annotation(s_obj=None): return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) extregistry.register_value(py_object, compute_result_annotation=py_object_compute_result_annotation, )#specialize_call=py_object_specialize_call) + +# __________ prebuilt py_object instances __________ + +def py_object_instance_compute_annotation(type, instance): + return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) + +extregistry.register_type(py_object, + compute_annotation=py_object_instance_compute_annotation) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Wed Apr 19 19:07:47 2006 @@ -31,3 +31,16 @@ assert s.knowntype == py_object if conftest.option.view: a.translator.view() + + def test_annotate_prebuilt(self): + five = py_object(5) + hello = py_object("hello") + + def fn(i): + return [five, hello][i] + + a = RPythonAnnotator() + s = a.build_types(fn, [int]) + assert s.knowntype == py_object + if conftest.option.view: + a.translator.view() From arigo at codespeak.net Wed Apr 19 20:37:14 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 20:37:14 +0200 (CEST) Subject: [pypy-svn] r25998 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060419183714.AC1A110087@code0.codespeak.net> Author: arigo Date: Wed Apr 19 20:37:12 2006 New Revision: 25998 Added: pypy/dist/pypy/rpython/rctypes/rpyobject.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Log: Specialize py_object() calls. Easy enough because we can reuse the many existing conversions from various RPython types to PyObjRepr, even without reusing more of PyObjRepr. Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Wed Apr 19 20:37:12 2006 @@ -1,6 +1,8 @@ from ctypes import py_object from pypy.annotation.model import SomeCTypesObject from pypy.rpython import extregistry +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.robject import pyobj_repr # __________ py_object() calls __________ @@ -8,14 +10,28 @@ def py_object_compute_result_annotation(s_obj=None): return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) +def py_object_specialize_call(hop): + r_pyobject = hop.r_result + v_result = r_pyobject.allocate_instance(hop.llops) + if len(hop.args_s): + [v_input] = hop.inputargs(pyobj_repr) + r_pyobject.setvalue(hop.llops, v_result, v_input) + return v_result + extregistry.register_value(py_object, compute_result_annotation=py_object_compute_result_annotation, - )#specialize_call=py_object_specialize_call) + specialize_call=py_object_specialize_call) # __________ prebuilt py_object instances __________ def py_object_instance_compute_annotation(type, instance): return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) +def py_object_instance_get_repr(rtyper, s_pyobject): + from pypy.rpython.rctypes.rpyobject import CTypesPyObjRepr + lowleveltype = lltype.Ptr(lltype.PyObject) + return CTypesPyObjRepr(rtyper, s_pyobject, lowleveltype) + extregistry.register_type(py_object, - compute_annotation=py_object_instance_compute_annotation) + compute_annotation=py_object_instance_compute_annotation, + get_repr=py_object_instance_get_repr) Added: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Wed Apr 19 20:37:12 2006 @@ -0,0 +1,7 @@ +from pypy.rpython.rmodel import inputconst +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.rctypes.rmodel import CTypesValueRepr + + +class CTypesPyObjRepr(CTypesValueRepr): + pass Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Wed Apr 19 20:37:12 2006 @@ -155,3 +155,11 @@ assert not c_char_p(None) assert not c_void_p() + +def test_py_object(): + class TAG: + pass + x = py_object(TAG) + assert x.value is TAG + x.value = 42 + assert x.value == 42 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Wed Apr 19 20:37:12 2006 @@ -11,6 +11,7 @@ from pypy.translator.c.test.test_genc import compile import sys from pypy.rpython.test.test_llinterp import interpret +from pypy.rpython.lltypesystem import lltype from ctypes import py_object @@ -44,3 +45,12 @@ assert s.knowntype == py_object if conftest.option.view: a.translator.view() + +class Test_specialization: + def test_specialize_wrapping(self): + def wrap(x): + return py_object(x) + + res = interpret(wrap, [9]) + assert lltype.typeOf(res.c_data[0]) == lltype.Ptr(lltype.PyObject) + assert res.c_data[0]._obj.value == 9 From arigo at codespeak.net Wed Apr 19 20:50:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 20:50:55 +0200 (CEST) Subject: [pypy-svn] r25999 - in pypy/dist/pypy/rpython: . rctypes rctypes/test Message-ID: <20060419185055.58B4110087@code0.codespeak.net> Author: arigo Date: Wed Apr 19 20:50:53 2006 New Revision: 25999 Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py pypy/dist/pypy/rpython/rfloat.py Log: Assigning to .value on py_object instances. Cannot read .value: this couldn't be properly annotated without falling back to SomeObjects! Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Wed Apr 19 20:50:53 2006 @@ -1,7 +1,17 @@ from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesValueRepr +from pypy.rpython.robject import pyobj_repr class CTypesPyObjRepr(CTypesValueRepr): - pass + + # reading .value is not allowed, as it can't be annotated! + + def rtype_setattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + v_pyobj, v_attr, v_newvalue = hop.inputargs(self, lltype.Void, + pyobj_repr) + self.setvalue(hop.llops, v_pyobj, v_newvalue) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Wed Apr 19 20:50:53 2006 @@ -54,3 +54,12 @@ res = interpret(wrap, [9]) assert lltype.typeOf(res.c_data[0]) == lltype.Ptr(lltype.PyObject) assert res.c_data[0]._obj.value == 9 + + def test_specialize_ass_value(self): + # reading .value is not allowed, as it can't be annotated! + def func(x): + o = py_object() + o.value = x + + interpret(func, [9]) + interpret(func, [9.2]) Modified: pypy/dist/pypy/rpython/rfloat.py ============================================================================== --- pypy/dist/pypy/rpython/rfloat.py (original) +++ pypy/dist/pypy/rpython/rfloat.py Wed Apr 19 20:50:53 2006 @@ -1,7 +1,7 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem.lltype import \ - Signed, Unsigned, Bool, Float, Void + Signed, Unsigned, Bool, Float, Void, pyobjectptr from pypy.rpython.error import TyperError from pypy.rpython.rmodel import FloatRepr from pypy.rpython.rmodel import IntegerRepr, BoolRepr, StringRepr @@ -198,12 +198,14 @@ def convert_from_to((r_from, r_to), v, llops): if r_to.lowleveltype == Float: return llops.gencapicall('PyFloat_AsDouble', [v], - resulttype=Float) + resulttype=Float, + _callable=lambda pyo: float(pyo._obj.value)) return NotImplemented class __extend__(pairtype(FloatRepr, PyObjRepr)): def convert_from_to((r_from, r_to), v, llops): if r_from.lowleveltype == Float: return llops.gencapicall('PyFloat_FromDouble', [v], - resulttype=pyobj_repr) + resulttype=pyobj_repr, + _callable=lambda x: pyobjectptr(x)) return NotImplemented From arigo at codespeak.net Wed Apr 19 20:55:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 19 Apr 2006 20:55:48 +0200 (CEST) Subject: [pypy-svn] r26000 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060419185548.9A97310092@code0.codespeak.net> Author: arigo Date: Wed Apr 19 20:55:47 2006 New Revision: 26000 Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Log: Specialize prebuilt py_object instances. Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Wed Apr 19 20:55:47 2006 @@ -6,6 +6,11 @@ class CTypesPyObjRepr(CTypesValueRepr): + def initialize_const(self, p, value): + if isinstance(value, self.ctype): + value = value.value + p.c_data[0] = lltype.pyobjectptr(value) + # reading .value is not allowed, as it can't be annotated! def rtype_setattr(self, hop): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rpyobject.py Wed Apr 19 20:55:47 2006 @@ -63,3 +63,15 @@ interpret(func, [9]) interpret(func, [9.2]) + + def test_specialize_prebuilt(self): + five = py_object(5) + hello = py_object("hello") + + def fn(i): + return [five, hello][i] + + res = interpret(fn, [0]) + assert res.c_data[0]._obj.value == 5 + res = interpret(fn, [1]) + assert res.c_data[0]._obj.value == "hello" From antocuni at codespeak.net Wed Apr 19 21:05:48 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 19 Apr 2006 21:05:48 +0200 (CEST) Subject: [pypy-svn] r26001 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060419190548.AC6341009C@code0.codespeak.net> Author: antocuni Date: Wed Apr 19 21:05:16 2006 New Revision: 26001 Added: pypy/dist/pypy/translator/cli/oopspec.py (contents, props changed) Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/compile.py pypy/dist/pypy/translator/cli/test/test_list.py Log: Added support for target-specific specialization based on the oopspec attribute. The CLI backend can now decide to call a builtin method instead of a ll_* helper function, thus bypassing it. As an example the ll_append function is converted to the List.append method. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Wed Apr 19 21:05:16 2006 @@ -8,6 +8,7 @@ from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod, List from pypy.translator.cli.option import getoption +from pypy.translator.cli import oopspec from pypy.tool.ansi_print import ansi_log import py @@ -100,7 +101,7 @@ return self.graph_to_signature(meth.graph, True, full_name), True elif isinstance(obj, List): - meth = obj._GENERIC_METHODS[name] + meth = oopspec.get_method(obj, name) class_name = self.lltype_to_cts(obj) ret_type = self.lltype_to_cts(meth.RESULT) arg_types = [self.lltype_to_cts(arg) for arg in meth.ARGS] Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Wed Apr 19 21:05:16 2006 @@ -1,3 +1,5 @@ +from pypy.translator.cli import oopspec + class Generator(object): def function_signature(self, graph): pass @@ -54,9 +56,11 @@ class _Call(MicroInstruction): def render(self, generator, op): graph = op.args[0].value.graph - cls = getattr(graph.func, 'class_', None) - - self._render_function(generator, graph, op.args) + method_name = oopspec.get_method_name(graph, op) + if method_name is None: + self._render_function(generator, graph, op.args) + else: + self._render_method(generator, method_name, op.args[1:]) def _render_function(self, generator, graph, args): func_sig = generator.function_signature(graph) @@ -64,6 +68,18 @@ generator.load(func_arg) generator.call(graph, func_sig) + def _render_method(self, generator, method_name, args): + this = args[0] + for arg in args: # push parametes + generator.load(arg) + generator.call_method(this.concretetype, method_name) + + +class _CallMethod(_Call): + def render(self, generator, op): + method = op.args[0] + self._render_method(generator, method.value, op.args[1:]) + class _New(MicroInstruction): def render(self, generator, op): generator.new(op.args[0].value) @@ -84,17 +100,6 @@ generator.load(this) generator.get_field(this.concretetype, field.value) -class _CallMethod(MicroInstruction): - def render(self, generator, op): - method = op.args[0] - this = op.args[1] - - # push parameters - for func_arg in op.args[1:]: - generator.load(func_arg) - - generator.call_method(this.concretetype, method.value) - class _RuntimeNew(MicroInstruction): def render(self, generator, op): generator.load(op.args[0]) Added: pypy/dist/pypy/translator/cli/oopspec.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/oopspec.py Wed Apr 19 21:05:16 2006 @@ -0,0 +1,41 @@ +from pypy.rpython.ootypesystem.ootype import List, Meth, Void + +def get_method_name(graph, op): + try: + oopspec = graph.func.oopspec + except AttributeError: + return None + + # TODO: handle parsing of arguments; by now it is assumed that + # builtin methods take the same arguments of the corresponding + # ll_* function. + full_name, _ = oopspec.split('(', 1) + type_name, method_name = full_name.split('.') + + try: + type_ = BUILTIN_TYPES[type_name] + except KeyError: + return None + + this = op.args[1] + if isinstance(this.concretetype, type_) and method_name in BUILTIN_METHODS[type_]: + return method_name + else: + return None # explicit is better than implicit :-) + +def get_method(obj, name): + try: + return obj._GENERIC_METHODS[name] + except KeyError: + t = type(obj) + return BUILTIN_METHODS[t][name] + +BUILTIN_TYPES = { + 'list': List + } + +BUILTIN_METHODS = { + List : { + 'append': Meth([List.ITEMTYPE_T], Void) + } + } Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Wed Apr 19 21:05:16 2006 @@ -58,12 +58,12 @@ } } - /* public void append(T item) { this.Add(item); } + /* public void extend(List other) { this.AddRange(other); Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Wed Apr 19 21:05:16 2006 @@ -24,20 +24,11 @@ print 'OK' -def sum_(lst): - total = 0 - i = 0 - while i < len(lst): - total += lst[i] - i += 1 - return total - -class A: - pass - -##def bar(x, y): -## lst = [A(), A()] -## return lst.pop(1) + +def bar(x, y): + lst = [1,2] + lst.append(x) + return lst[2] f = compile_function(bar, [int, int]) Modified: pypy/dist/pypy/translator/cli/test/test_list.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_list.py (original) +++ pypy/dist/pypy/translator/cli/test/test_list.py Wed Apr 19 21:05:16 2006 @@ -23,6 +23,11 @@ def list_sum(x, y): return sum_(create(x, y)) +def list_append(x, y): + lst = create(x, y) + lst.append(x) + return sum_(lst) + def list_setitem(x, y): lst = create(x, y) lst[0] = 0 From cfbolz at codespeak.net Wed Apr 19 21:21:16 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 19 Apr 2006 21:21:16 +0200 (CEST) Subject: [pypy-svn] r26002 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060419192116.E9417100B3@code0.codespeak.net> Author: cfbolz Date: Wed Apr 19 21:21:14 2006 New Revision: 26002 Modified: pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: use a correctly type NULL constant Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Wed Apr 19 21:21:14 2006 @@ -169,7 +169,7 @@ node = self.getcontainernode(obj._obj) return node.ptrname else: - return 'NULL' + return '((%s) NULL)' % (cdecl(self.gettype(T), ''), ) else: raise Exception("don't know about %r" % (obj,)) Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Wed Apr 19 21:21:14 2006 @@ -83,14 +83,13 @@ res = fn(1) assert res == 42 - class TestTypedOptimizedSwitchTestCase: class CodeGenerator(_TestTypedTestCase): def process(self, t): _TestTypedTestCase.process(self, t) self.t = t - backend_optimizations(t, merge_if_blocks_to_switch=True) + backend_optimizations(t, merge_if_blocks_to_switch=True, propagate=True) def test_int_switch(self): def f(x=int): @@ -180,6 +179,28 @@ y = ord(x) assert fn(y) == f(y) + def test_typed_NULL(self): + class A(object): + def __init__(self): + self.next = None + def g(x): + a = A() + a.next = A() + a.x = x + a.next.x = x + 1 + def f(): + g(42) + a = A() + a.x = 43 + if a.next is not None: + return a.next.x + return a.x - 1 + codegenerator = self.CodeGenerator() + fn = codegenerator.getcompiled(f) + res = f() + assert res == 42 + + class TestTypedOptimizedRaisingOps: class CodeGenerator(_TestTypedTestCase): From cfbolz at codespeak.net Wed Apr 19 21:39:22 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 19 Apr 2006 21:39:22 +0200 (CEST) Subject: [pypy-svn] r26004 - pypy/dist/pypy/translator/c/test Message-ID: <20060419193922.8019710090@code0.codespeak.net> Author: cfbolz Date: Wed Apr 19 21:39:20 2006 New Revision: 26004 Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: use a different subclass for the propagate test Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Wed Apr 19 21:39:20 2006 @@ -89,7 +89,7 @@ def process(self, t): _TestTypedTestCase.process(self, t) self.t = t - backend_optimizations(t, merge_if_blocks_to_switch=True, propagate=True) + backend_optimizations(t, merge_if_blocks_to_switch=True) def test_int_switch(self): def f(x=int): @@ -179,6 +179,14 @@ y = ord(x) assert fn(y) == f(y) +class TestOptimizedWithPropagate: + + class CodeGenerator(_TestTypedTestCase): + def process(self, t): + _TestTypedTestCase.process(self, t) + self.t = t + backend_optimizations(t, propagate=True) + def test_typed_NULL(self): class A(object): def __init__(self): @@ -201,6 +209,7 @@ assert res == 42 + class TestTypedOptimizedRaisingOps: class CodeGenerator(_TestTypedTestCase): From tismer at codespeak.net Wed Apr 19 22:26:12 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 19 Apr 2006 22:26:12 +0200 (CEST) Subject: [pypy-svn] r26007 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060419202612.14CD1100BA@code0.codespeak.net> Author: tismer Date: Wed Apr 19 22:26:10 2006 New Revision: 26007 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: testing bare_setfield and setfield of PyObject (maybe extend to gcstruct?) Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Wed Apr 19 22:26:10 2006 @@ -244,7 +244,7 @@ assert push_count == 0 and pop_count == 1 # ____________________________________________________________________ -# testing the protection magic +# testing the protection magic and bare_setfield def protect(obj): RaiseNameError def unprotect(obj): RaiseNameError @@ -269,6 +269,32 @@ ops = getops(graphof(t, f)) assert len(ops.get('direct_call', [])) == ex +def generic_op(*args): RaiseNameError +def rtype_generic_op(hop): + args = hop.inputargs(*hop.args_r) + args.pop(0) + op = hop.args_s[0].const + hop.genop(op, args) + +extregistry.register_value(generic_op, + compute_result_annotation=lambda *args: None, specialize_call=rtype_generic_op) + +def test_bare_setfield(): + class A: + def __init__(self, obj): self.x = obj + def f(v): + inst = A(v) + generic_op('setfield', inst, 'x', v) + generic_op('bare_setfield', inst, 'x', v) + + rgc = gctransform.RefcountingGCTransformer + bgc = gctransform.BoehmGCTransformer + # should not influence PyObject at all + for gc in rgc, bgc: + t, transformer = rtype_and_transform(f, [object], gc, check=False) + ops = getops(graphof(t, f)) + assert len(ops.get('getfield', [])) == 1 + def DONOTtest_protect_unprotect_no_exception_block(): def p(): protect('this is an object') def u(): unprotect('this is an object') @@ -276,7 +302,6 @@ gc = gctransform.RefcountingGCTransformer for f in p, u: t, transformer = rtype_and_transform(f, [], gc, check=False) - has_cleanup = False ops = getops(graphof(t, f)) for op in ops.get('direct_call', []): assert op.cleanup is None or op.cleanup == ((), ()) From pedronis at codespeak.net Wed Apr 19 23:32:54 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 19 Apr 2006 23:32:54 +0200 (CEST) Subject: [pypy-svn] r26010 - pypy/dist/pypy/translator/c Message-ID: <20060419213254.22C26100BA@code0.codespeak.net> Author: pedronis Date: Wed Apr 19 23:32:53 2006 New Revision: 26010 Modified: pypy/dist/pypy/translator/c/exceptiontransform.py Log: the test for exception occurred was a macro before the exception transform. inline the testing code manually now in the transformation. The resulting pypy-c seems slightly smaller and a bit faster. Is probably overkill to do this using inlining, also probably time-wise expensive. Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Wed Apr 19 23:32:53 2006 @@ -6,6 +6,7 @@ from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.memory.lladdress import NULL from pypy.rpython.memory.gctransform import varoftype +from pypy.rpython import rtyper from pypy.rpython import rclass from pypy.rpython.rarithmetic import r_uint, r_longlong, r_ulonglong from pypy.annotation import model as annmodel @@ -114,6 +115,12 @@ mixlevelannotator.finish() + ExcDataDef = translator.annotator.bookkeeper.getuniqueclassdef(ExcData) + self.ExcData_repr = rclass.getinstancerepr(translator.rtyper, ExcDataDef) + exc_data_ptr = self.ExcData_repr.convert_const(exc_data) + self.cexcdata = Constant(exc_data_ptr, self.ExcData_repr.lowleveltype) + self.cnulltype = Constant(null_type, self.lltype_of_exception_type) + self.lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() def transform_completely(self): @@ -249,10 +256,15 @@ return newgraph, SpaceOperation("direct_call", [fptr] + callargs, op.result) def gen_exc_check(self, block, returnblock): - var_exc_occured = Variable() - var_exc_occured.concretetype = lltype.Bool + #var_exc_occured = Variable() + #var_exc_occured.concretetype = lltype.Bool + #block.operations.append(SpaceOperation("safe_call", [self.rpyexc_occured_ptr], var_exc_occured)) + + llops = rtyper.LowLevelOpList(None) + v_exc_type = self.ExcData_repr.getfield(self.cexcdata, 'exc_type', llops) + var_exc_occured = llops.genop('ptr_ne', [v_exc_type, self.cnulltype], lltype.Bool) + block.operations.extend(llops) - block.operations.append(SpaceOperation("safe_call", [self.rpyexc_occured_ptr], var_exc_occured)) block.exitswitch = var_exc_occured #exception occurred case l = Link([error_value(returnblock.inputargs[0].concretetype)], returnblock) From tismer at codespeak.net Thu Apr 20 00:30:54 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 20 Apr 2006 00:30:54 +0200 (CEST) Subject: [pypy-svn] r26011 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060419223054.61522100C0@code0.codespeak.net> Author: tismer Date: Thu Apr 20 00:30:52 2006 New Revision: 26011 Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py Log: - create __self__ attribute also in the absence of wrapper - bugfix for setfield if the wrapper is not in the same class Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Thu Apr 20 00:30:52 2006 @@ -470,7 +470,7 @@ else: if self.classdef is None: raise MissingRTypeAttribute(attr) - self.rbase.setfield(vinst, attr, vvalue, llops, force_cast=True) + self.rbase.setfield(vinst, attr, vvalue, llops, force_cast=True, opname=opname) def new_instance(self, llops): """Build a new instance, without calling __init__.""" @@ -639,13 +639,13 @@ v_inst, c_spec = hop.inputargs(*hop.args_r) repr = c_spec.value v_res = into_cobject(v_inst, repr, hop.llops) + v_cobj = v_res + c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) + c_0 = hop.inputconst(Signed, 0) + v_res = hop.llops.gencapicall('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyobj_repr) + c_self = hop.inputconst(pyobj_repr, '__self__') + hop.genop('setattr', [v_res, c_self, v_cobj], resulttype=pyobj_repr) if repr.has_wrapper: - v_cobj = v_res - c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) - c_0 = hop.inputconst(Signed, 0) - v_res = hop.llops.gencapicall('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyobj_repr) - c_self = hop.inputconst(pyobj_repr, '__self__') - hop.genop('setattr', [v_res, c_self, v_cobj], resulttype=pyobj_repr) repr.setfield(v_inst, '_wrapper_', v_res, hop.llops) hop.genop('gc_unprotect', [v_res]) # yes a weak ref return v_res @@ -702,9 +702,9 @@ r_to.setfield(v_inst, '_wrapper_', v, llops) llops.genop('gc_unprotect', [v]) return v_inst - - c_self = inputconst(pyobj_repr, '__self__') - v = llops.genop('getattr', [v, c_self], resulttype=r_from) + # if we don't have a wrapper field, we just don't support __init__ + c_self = inputconst(pyobj_repr, '__self__') + v = llops.genop('getattr', [v, c_self], resulttype=r_from) return outof_cobject(v, r_to, llops) # ____________________________________________________________ From tismer at codespeak.net Thu Apr 20 00:31:46 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 20 Apr 2006 00:31:46 +0200 (CEST) Subject: [pypy-svn] r26012 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060419223146.8FECF100C6@code0.codespeak.net> Author: tismer Date: Thu Apr 20 00:31:44 2006 New Revision: 26012 Modified: pypy/dist/pypy/rpython/lltypesystem/rtuple.py Log: use the external object representation for wrapping Modified: pypy/dist/pypy/rpython/lltypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rtuple.py Thu Apr 20 00:31:44 2006 @@ -77,7 +77,7 @@ v_item = llops.gencapicall('PyTuple_GetItem_WithIncref', [v, ci], resulttype = pyobj_repr) v_converted = llops.convertvar(v_item, pyobj_repr, - r_to.items_r[i]) + r_to.external_items_r[i]) vlist.append(v_converted) return r_to.newtuple(llops, r_to, vlist) @@ -89,8 +89,8 @@ for i in range(len(r_from.items_r)): cname = inputconst(Void, r_from.fieldnames[i]) v_item = llops.genop('getfield', [v, cname], - resulttype = r_from.items_r[i].lowleveltype) - v_converted = llops.convertvar(v_item, r_from.items_r[i], + resulttype = r_from.external_items_r[i].lowleveltype) + v_converted = llops.convertvar(v_item, r_from.external_items_r[i], pyobj_repr) ci = inputconst(Signed, i) llops.gencapicall('PyTuple_SetItem_WithIncref', [v_result, ci, From mwh at codespeak.net Thu Apr 20 10:15:23 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 20 Apr 2006 10:15:23 +0200 (CEST) Subject: [pypy-svn] r26017 - in pypy/extradoc/talk/accu2006: . accu-2006.key accu-2006.key/thumbs Message-ID: <20060420081523.6B32410089@code0.codespeak.net> Author: mwh Date: Thu Apr 20 10:15:11 2006 New Revision: 26017 Modified: pypy/extradoc/talk/accu2006/accu-2006.key/index.apxl.gz pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-1.tiff pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7.tiff pypy/extradoc/talk/accu2006/accu-2006.pdf Log: the version of the accu talk that actually got presented. only tiny, tiny changes from the last committed version. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/index.apxl.gz ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st3-1.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.key/thumbs/st7.tiff ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/accu2006/accu-2006.pdf ============================================================================== Binary files. No diff available. From ale at codespeak.net Thu Apr 20 10:51:47 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 20 Apr 2006 10:51:47 +0200 (CEST) Subject: [pypy-svn] r26018 - pypy/dist/pypy/translator/microbench/pybench Message-ID: <20060420085147.998C510087@code0.codespeak.net> Author: ale Date: Thu Apr 20 10:51:45 2006 New Revision: 26018 Modified: pypy/dist/pypy/translator/microbench/pybench/pybench.py Log: Some improvements to html generation (like making it work) Modified: pypy/dist/pypy/translator/microbench/pybench/pybench.py ============================================================================== --- pypy/dist/pypy/translator/microbench/pybench/pybench.py (original) +++ pypy/dist/pypy/translator/microbench/pybench/pybench.py Thu Apr 20 10:51:45 2006 @@ -262,14 +262,17 @@ def html_stat(self, compare_to=None, hidenoise=0): + import autopath from py.xml import html if not compare_to: - table = html.table(html.thead( - html.tr([ html.th(x, align='left', mochi_format = y) - for (x,y) in [('Tests','str'), ('per run','float'), - ('per oper.', 'float'), ('overhead', 'float')])), - id = "sortable_table") + table = html.table( + html.thead( + html.tr( + [ html.th(x, align='left', mochi_format = y) + for (x,y) in [('Tests','str'), ('per run','float'), + ('per oper.', 'float'), ('overhead', 'float')]]) + ),id = "sortable_table") tests = self.tests.items() tests.sort() @@ -278,16 +281,18 @@ table.append(html.tr( html.td(name), html.td((avg*1000.0), html.td(op_avg*1000000.0), - html.td(ov_avg*1000.0) - ) - table.append(html.tr('Average round time %s' % (self.roundtime * 1000.0)) + html.td(ov_avg*1000.0)) + )) + table.append(html.tr( + 'Average round time %s' % (self.roundtime * 1000.0)) + ) return table else: table = html.table(html.thead( html.tr([ html.th(x, align='left', mochi_format = y) for (x,y) in [('Tests','str'), ('per run','float'), - ('per oper.', 'float'), ('diff', 'float')])), - id = "sortable_table") + ('per oper.', 'float'), ('diff', 'float')]]), + id = "sortable_table")) tests = self.tests.items() tests.sort() compatible = 1 @@ -311,20 +316,22 @@ table.append(html.tr( html.td(name), html.td((avg*1000.0), html.td(op_avg*1000000.0), - html.td(qop_avg) - ) + html.td(qop_avg) + ))) if compatible and compare_to.roundtime > 0 and \ compare_to.version == self.version: table.append(html.tr( - html.td('Average round time'), - html.td(self.roundtime * 1000.0), - html.td(''), - html.td('%+7.2f%%'% (((self.roundtime*self.warp)/ - (compare_to.roundtime*compare_to.warp)-1.0)*100.0))) + html.td('Average round time'), + html.td(self.roundtime * 1000.0), + html.td(''), + html.td('%+7.2f%%'% (((self.roundtime*self.warp)/ + (compare_to.roundtime*compare_to.warp)-1.0)*100.0) + ))) + else: table.append(html.tr( - html.td('Average round time'), - html.td(self.roundtime * 1000.0))) + html.td('Average round time'), + html.td(self.roundtime * 1000.0))) return table def print_machine(): @@ -353,6 +360,7 @@ ArgumentOption('-w','set warp factor to arg',Setup.Warp_factor), SwitchOption('-d','hide noise in compares', 0), SwitchOption('--no-gc','disable garbage collection', 0), + SwitchOption('-x','write html table', 0), ] about = """\ @@ -376,6 +384,7 @@ hidenoise = self.values['-d'] warp = self.values['-w'] nogc = self.values['--no-gc'] + html = self.values['-x'] # Switch off GC if nogc: @@ -414,7 +423,13 @@ print 'Benchmark: %s (rounds=%i, warp=%i)' % \ (bench.name,bench.rounds,bench.warp) print - bench.print_stat(compare_to, hidenoise) + if html: + table = bench.html_stat(compare_to, hidenoise) + f = open('index.html','w') + print >>f,table.unicode() + f.close() + else: + bench.print_stat(compare_to, hidenoise) except IOError: print '* Error opening/reading file',show_bench print From arigo at codespeak.net Thu Apr 20 10:52:50 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 20 Apr 2006 10:52:50 +0200 (CEST) Subject: [pypy-svn] r26019 - in pypy/dist/pypy: rpython/rctypes translator/c translator/c/test translator/goal Message-ID: <20060420085250.DB2F310087@code0.codespeak.net> Author: arigo Date: Thu Apr 20 10:52:48 2006 New Revision: 26019 Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/c/test/test_lltyped.py pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Minor fixes/tests to make targetdemomodule.py compile. Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Thu Apr 20 10:52:48 2006 @@ -1,7 +1,8 @@ +from pypy.annotation.pairtype import pairtype from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype from pypy.rpython.rctypes.rmodel import CTypesValueRepr -from pypy.rpython.robject import pyobj_repr +from pypy.rpython.robject import PyObjRepr, pyobj_repr class CTypesPyObjRepr(CTypesValueRepr): @@ -20,3 +21,10 @@ v_pyobj, v_attr, v_newvalue = hop.inputargs(self, lltype.Void, pyobj_repr) self.setvalue(hop.llops, v_pyobj, v_newvalue) + + +class __extend__(pairtype(CTypesPyObjRepr, PyObjRepr)): + # conversion used by wrapper.py in genc when returning a py_object + # from a function exposed in a C extension module + def convert_from_to((r_from, r_to), v, llops): + return r_from.getvalue(llops, v) Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Thu Apr 20 10:52:48 2006 @@ -427,18 +427,17 @@ assert isinstance(op.args[1], Constant) STRUCT = self.lltypemap(op.args[0]).TO structdef = self.db.gettypedefnode(STRUCT) - fieldname = structdef.c_struct_field_name(op.args[1].value) - return self.generic_get(op, '%s%s->%s' % (ampersand, - self.expr(op.args[0]), - fieldname)) + expr = ampersand + structdef.ptr_access_expr(self.expr(op.args[0]), + op.args[1].value) + return self.generic_get(op, expr) def OP_SETFIELD(self, op): assert isinstance(op.args[1], Constant) STRUCT = self.lltypemap(op.args[0]).TO structdef = self.db.gettypedefnode(STRUCT) - fieldname = structdef.c_struct_field_name(op.args[1].value) - return self.generic_set(op, '%s->%s' % (self.expr(op.args[0]), - fieldname)) + expr = structdef.ptr_access_expr(self.expr(op.args[0]), + op.args[1].value) + return self.generic_set(op, expr) OP_BARE_SETFIELD = OP_SETFIELD Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Thu Apr 20 10:52:48 2006 @@ -55,13 +55,14 @@ pfname = db.get(pf) self.exports[self.entrypoint.func_name] = pf for obj in exports: - objname = None if type(obj) is tuple: objname, obj = obj + elif hasattr(obj, '__name__'): + objname = obj.__name__ + else: + objname = None po = self.getentrypointptr(obj) poname = db.get(po) - if hasattr(obj, '__name__'): - objname = obj.__name__ objname = objname or poname if objname in self.exports: raise NameError, 'duplicate name in export: %s is %s and %s' % ( Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Thu Apr 20 10:52:48 2006 @@ -102,6 +102,10 @@ fldname = self.c_struct_field_name(fldname) return '%s.%s' % (baseexpr, fldname) + def ptr_access_expr(self, baseexpr, fldname): + fldname = self.c_struct_field_name(fldname) + return '%s->%s' % (baseexpr, fldname) + def definition(self): if self.STRUCT._hints.get('external'): # XXX hack return @@ -187,6 +191,9 @@ def access_expr(self, baseexpr, index): return '%s.items[%d]' % (baseexpr, index) + def ptr_access_expr(self, baseexpr, index): + return '%s->items[%d]' % (baseexpr, index) + def definition(self): gcpolicy = self.db.gcpolicy yield 'struct %s {' % self.name @@ -267,6 +274,8 @@ index = int(index[4:]) return '%s[%d]' % (baseexpr, index) + ptr_access_expr = access_expr + def definition(self): return [] # no declaration is needed Modified: pypy/dist/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/dist/pypy/translator/c/test/test_lltyped.py Thu Apr 20 10:52:48 2006 @@ -36,7 +36,7 @@ a3 = big.a3 a42 = big.a42 a7[0] = -1 - a7[6] = -2 + a7.item6 = -2 a3[0].v = -3 a3[2].v = -4 a42[0][0] = -5 @@ -44,7 +44,7 @@ assert a7[0] == -1 assert a7[6] == -2 assert a3[0].v == -3 - assert a3[2].v == -4 + assert a3.item2.v == -4 assert a42[0][0] == -5 assert a42[5][6] == -6 return len(a42)*100 + len(a42[4]) Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Thu Apr 20 10:52:48 2006 @@ -1,4 +1,5 @@ from pypy.module._demo import demo +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy from ctypes import * @@ -67,7 +68,7 @@ # _____ Define and setup target ___ def target(*args): - return entry_point, [int] + return entry_point, [int], PyPyAnnotatorPolicy() if __name__ == '__main__': From arigo at codespeak.net Thu Apr 20 11:31:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 20 Apr 2006 11:31:01 +0200 (CEST) Subject: [pypy-svn] r26021 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060420093101.6E1DD10089@code0.codespeak.net> Author: arigo Date: Thu Apr 20 11:30:59 2006 New Revision: 26021 Modified: pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: For all calls to pythonapi.PyXxx(), generate a PyErr_Occurred() check in genc. Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Thu Apr 20 11:30:59 2006 @@ -62,12 +62,17 @@ else: RESTYPE = lltype.Void - ll_func = getattr(cfuncptr, 'llinterp_friendly_version', None) - includes = getattr(cfuncptr, 'includes', ()) + kwds = {} + if hasattr(cfuncptr, 'llinterp_friendly_version'): + kwds['_callable'] = cfuncptr.llinterp_friendly_version + if (cfuncptr._flags_ & ctypes._FUNCFLAG_PYTHONAPI) == 0: + kwds['includes'] = getattr(cfuncptr, 'includes', ()) + #else: + # no 'includes': hack to trigger in GenC a PyErr_Occurred() check + v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, resulttype = RESTYPE, - _callable = ll_func, - includes = includes) + **kwds) # XXX hack! hack! temporary! I promize! FUNCTYPE = lltype.FuncType(ARGTYPES, RESTYPE) last_op = hop.llops[-1] Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Thu Apr 20 11:30:59 2006 @@ -163,3 +163,9 @@ assert x.value is TAG x.value = 42 assert x.value == 42 + +def test_pythonapi(): + pythonapi.PyInt_AsLong.argtypes = [py_object] + pythonapi.PyInt_AsLong.restype = c_long + assert pythonapi.PyInt_AsLong(py_object(17L)) == 17 + py.test.raises(TypeError, "pythonapi.PyInt_AsLong(py_object('hello'))") Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Thu Apr 20 11:30:59 2006 @@ -12,9 +12,9 @@ from pypy.rpython.rstr import string_repr from pypy.rpython.lltypesystem import lltype -from ctypes import cdll +from ctypes import cdll, pythonapi, _FUNCFLAG_PYTHONAPI from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer -from ctypes import POINTER +from ctypes import POINTER, py_object # __________ the standard C library __________ @@ -163,3 +163,29 @@ t2 = fn() t3 = time.time() assert int(t1) <= t2 <= int(t3 + 1.0) + + def test_compile_pythonapi(self): + PyInt_AsLong = pythonapi.PyInt_AsLong + PyInt_AsLong.argtypes = [py_object] + PyInt_AsLong.restype = c_long + assert PyInt_AsLong._flags_ & _FUNCFLAG_PYTHONAPI + + PyNumber_Add = pythonapi.PyNumber_Add + PyNumber_Add.argtypes = [py_object, py_object] + PyNumber_Add.restype = py_object + + def fn1(x, crash): + pyobj = py_object(x) + pyobj = PyNumber_Add(pyobj, pyobj) + x = PyInt_AsLong(pyobj) + if crash: + # fn(sys.maxint, 1) should crash on PyInt_AsLong before + # it arrives here. If by mistake it arrives here then + # we get a TypeError instead of the OverflowError + PyNumber_Add(py_object(5), py_object("x")) + return x + + fn = compile(fn1, [int, int]) + res = fn(17, 0) + assert res == 34 + py.test.raises(OverflowError, 'fn(sys.maxint, 1)') From arigo at codespeak.net Thu Apr 20 11:57:53 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 20 Apr 2006 11:57:53 +0200 (CEST) Subject: [pypy-svn] r26025 - pypy/dist/pypy/rpython/test Message-ID: <20060420095753.624E810089@code0.codespeak.net> Author: arigo Date: Thu Apr 20 11:57:52 2006 New Revision: 26025 Modified: pypy/dist/pypy/rpython/test/test_rbuiltin.py Log: Forgot the fix or remove these tests now that the corresponding operations have disappeared. Modified: pypy/dist/pypy/rpython/test/test_rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rbuiltin.py (original) +++ pypy/dist/pypy/rpython/test/test_rbuiltin.py Thu Apr 20 11:57:52 2006 @@ -335,50 +335,3 @@ return lltype.cast_primitive(lltype.UniChar, v) res = interpret(llf, [ord('x')], policy=LowLevelAnnotatorPolicy()) assert res == u'x' - -def test_cast_subarray_pointer(): - from pypy.rpython.lltypesystem.lltype import malloc, GcArray, Signed - from pypy.rpython.lltypesystem.lltype import FixedSizeArray, Ptr - for a in [malloc(GcArray(Signed), 5), - malloc(FixedSizeArray(Signed, 5), immortal=True)]: - a[0] = 0 - a[1] = 10 - a[2] = 20 - a[3] = 30 - a[4] = 40 - BOX = Ptr(FixedSizeArray(Signed, 2)) - b01 = lltype.cast_subarray_pointer(BOX, a, 0) - b12 = lltype.cast_subarray_pointer(BOX, a, 1) - b23 = lltype.cast_subarray_pointer(BOX, a, 2) - b34 = lltype.cast_subarray_pointer(BOX, a, 3) - def llf(n): - saved = a[n] - a[n] = 1000 - try: - return b01[0] + b12[0] + b23[1] + b34[1] - finally: - a[n] = saved - - res = interpret(llf, [0]) - assert res == 1000 + 10 + 30 + 40 - res = interpret(llf, [1]) - assert res == 0 + 1000 + 30 + 40 - res = interpret(llf, [2]) - assert res == 0 + 10 + 30 + 40 - res = interpret(llf, [3]) - assert res == 0 + 10 + 1000 + 40 - res = interpret(llf, [4]) - assert res == 0 + 10 + 30 + 1000 - -def test_cast_structfield_pointer(): - S = lltype.GcStruct('S', ('x', lltype.Signed), ('y', lltype.Signed)) - SUBARRAY = lltype.FixedSizeArray(lltype.Signed, 1) - P = lltype.Ptr(SUBARRAY) - def llf(n): - s = lltype.malloc(S) - a = lltype.cast_structfield_pointer(P, s, 'y') - a[0] = n - return s.y - - res = interpret(llf, [34]) - assert res == 34 From mwh at codespeak.net Thu Apr 20 12:19:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 20 Apr 2006 12:19:53 +0200 (CEST) Subject: [pypy-svn] r26028 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060420101953.4F5361008D@code0.codespeak.net> Author: mwh Date: Thu Apr 20 12:19:51 2006 New Revision: 26028 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: my iceland details Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Thu Apr 20 12:19:51 2006 @@ -87,14 +87,17 @@ Michael Hudson ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: -mail: arigo at tunes.org -birthdate: -address: -organisation: -comments: +preferred arrival date: 21 acceptable arrival dates: 20-22 +preferred departure date: 28 acceptable departure dates: 27-29 +start city / airport: bristol, cardiff, birmingham, LHR, LGW + (in roughly that order) +mail: mwh at python.net +birthdate: 1978-10-07 +address: Flat 4 Plimsoll House, Ashgrove Road, Bristol BS6 6LZ, UK +organisation: Heinrich-Heine Universitat Dusseldorf +comments: I'd much rather travel up to london and take a direct + flight than travel to a closer airport and have to change + planes Antonio Cuni ------------------------------ From ale at codespeak.net Thu Apr 20 13:30:26 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 20 Apr 2006 13:30:26 +0200 (CEST) Subject: [pypy-svn] r26037 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060420113026.3C19F10088@code0.codespeak.net> Author: ale Date: Thu Apr 20 13:30:23 2006 New Revision: 26037 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: My details/wishes Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Thu Apr 20 13:30:23 2006 @@ -122,5 +122,19 @@ birthdate: 22.02.1975 address: Wrangelstr. 67, 10997 Berlin, Germany organisation: Merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany -comments: - no flight departure before 12AM because of travel time to the airport; +comments: - no flight departure before 12AM because of travel time to the airport; + - my attendance is not decided yet (and i will not be coding but documenting the activities with the camera) + +Anders Lehmann +------------------------------ + +preferred arrival date: 20.05.06 acceptable arrival dates: 21.05.06 +preferred departure date: 28.05.06 acceptable departure dates: 29.05.06 +start city / airport: Holstebro, Denmark / Copenhagen +mail: anders.lehmann at dfki.de +birthdate: 30.11.1962 +address: Anna Anchersvej 206, DK - 7500 Denmark +organisation: DFKI Gmbh +comments: + From ericvrp at codespeak.net Thu Apr 20 16:18:07 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 20 Apr 2006 16:18:07 +0200 (CEST) Subject: [pypy-svn] r26043 - in pypy/dist/pypy/translator/llvm/wrapable: . test Message-ID: <20060420141807.CBAA510078@code0.codespeak.net> Author: ericvrp Date: Thu Apr 20 16:17:39 2006 New Revision: 26043 Added: pypy/dist/pypy/translator/llvm/wrapable/ pypy/dist/pypy/translator/llvm/wrapable/__init__.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/libllvm.cpp pypy/dist/pypy/translator/llvm/wrapable/libllvm.h pypy/dist/pypy/translator/llvm/wrapable/llvm.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/llvm_helper.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/setup.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/test/ pypy/dist/pypy/translator/llvm/wrapable/test/test_llvm.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/wrap_llvm.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/wrapable.py (contents, props changed) pypy/dist/pypy/translator/llvm/wrapable/wrapper.py (contents, props changed) Log: Code I have been working on for to long without staying in contact with the other PyPy developers. It's purpose is to use C++ libraries with (r)ctypes. The code will generate .h, .cpp and .py files out of a class specification. This specs look similar to boost.Python . From my experience with prior project I know that C++ header files with all it's inline function, templates, etc. quickly become to complicated to handle automaticly. Even with a tool such as gccxml there are often no clear cut solutions. That is the why. As an example you can look at wrap_llvm.py , which when executed creates libllvm.h libllvm.cpp (the C API) and llvm.py (the Python classes which mimic the C++ API's behaviour) A pointer to the original C++ object is stored in the 'instance' attribute of the Python class. Whenever one of these Python objects is used as a parameter ctypes automatically uses the objects 'instance' attribute instead. (see wrapper.py from_param) I am no sure if we (can) support this in rctypes as well, otherwise this tool could do it for us. I hope this code can be of use and that I did not isolate me too much for too long! Added: pypy/dist/pypy/translator/llvm/wrapable/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/llvm/wrapable/libllvm.cpp ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/libllvm.cpp Thu Apr 20 16:17:39 2006 @@ -0,0 +1,116 @@ +// llvm.cpp + +#include "libllvm.h" + +// Helper functions (mostly for casting returnvalues and parameters) +char* as_c_string(std::string str) { + return (char*)str.c_str(); +} + +std::string as_std_string(char* p) { + return std::string(p); +} + +std::vector as_vector_GenericValue(GenericValue* gv) { + std::vector gvv; + gvv.push_back(*gv); + return gvv; +} + +// ExecutionEngine +ExecutionEngine* ExecutionEngine_create(ModuleProvider* P0,long P1) { + return (ExecutionEngine*)ExecutionEngine::create(P0,P1); +} + +Module& ExecutionEngine_getModule(ExecutionEngine* P0) { + return (Module&)P0->getModule(); +} + +void ExecutionEngine_freeMachineCodeForFunction(ExecutionEngine* P0,Function* P1) { + return (void)P0->freeMachineCodeForFunction(P1); +} + +GenericValue ExecutionEngine_runFunction(ExecutionEngine* P0,Function* P1,GenericValue* P2) { + return (GenericValue)P0->runFunction(P1,as_vector_GenericValue(P2)); +} + + +// ModuleProvider + +// ExistingModuleProvider +ExistingModuleProvider* ExistingModuleProvider___init__(Module* P0) { + return new ExistingModuleProvider(P0); +} + + +// Function +void Function_eraseFromParent(Function* P0) { + return (void)P0->eraseFromParent(); +} + +FunctionType* Function_getFunctionType(Function* P0) { + return (FunctionType*)P0->getFunctionType(); +} + + +// FunctionType + +// GenericValue + +// TypeID + +// Type +TypeID* Type_getTypeID(Type* P0) { + return (TypeID*)P0->getTypeID(); +} + +Type* Type_getContainedType(Type* P0,long P1) { + return (Type*)P0->getContainedType(P1); +} + +char* Type_getDescription(Type* P0) { + return (char*)as_c_string(P0->getDescription()); +} + + +// Module +Module* Module___init__(char* P0) { + return new Module(as_std_string(P0)); +} + +char* Module_getModuleIdentifier(Module* P0) { + return (char*)as_c_string(P0->getModuleIdentifier()); +} + +void Module_setModuleIdentifier(Module* P0,char* P1) { + return (void)P0->setModuleIdentifier(P1); +} + +char* Module_getTargetTriple(Module* P0) { + return (char*)as_c_string(P0->getTargetTriple()); +} + +void Module_setTargetTriple(Module* P0,char* P1) { + return (void)P0->setTargetTriple(P1); +} + +char* Module_getModuleInlineAsm(Module* P0) { + return (char*)as_c_string(P0->getModuleInlineAsm()); +} + +void Module_setModuleInlineAsm(Module* P0,char* P1) { + return (void)P0->setModuleInlineAsm(P1); +} + +Function* Module_getNamedFunction(Module* P0,char* P1) { + return (Function*)P0->getNamedFunction(P1); +} + +Function* Module_getOrInsertFunction(Module* P0,char* P1,FunctionType* P2) { + return (Function*)P0->getOrInsertFunction(P1,P2); +} + + + + +// end of llvm.cpp Added: pypy/dist/pypy/translator/llvm/wrapable/libllvm.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/libllvm.h Thu Apr 20 16:17:39 2006 @@ -0,0 +1,63 @@ +#ifndef __llvm_H__ +#define __llvm_H__ + +#include "llvm/Module.h" +#include "llvm/ModuleProvider.h" +#include "llvm/Function.h" +#include "llvm/DerivedTypes.h" +#include "llvm/Type.h" +#include "llvm/Analysis/Verifier.h" +#include "llvm/ExecutionEngine/ExecutionEngine.h" +#include "llvm/ExecutionEngine/GenericValue.h" + +using namespace llvm; +#define TypeID int + +#ifdef __cplusplus +extern "C" { +#endif + +// ExecutionEngine +ExecutionEngine* ExecutionEngine_create(ModuleProvider*,long); +Module& ExecutionEngine_getModule(ExecutionEngine*); +void ExecutionEngine_freeMachineCodeForFunction(ExecutionEngine*,Function*); +GenericValue ExecutionEngine_runFunction(ExecutionEngine*,Function*,GenericValue*); + +// ModuleProvider + +// ExistingModuleProvider +ExistingModuleProvider* ExistingModuleProvider___init__(Module*); + +// Function +void Function_eraseFromParent(Function*); +FunctionType* Function_getFunctionType(Function*); + +// FunctionType + +// GenericValue + +// TypeID + +// Type +TypeID* Type_getTypeID(Type*); +Type* Type_getContainedType(Type*,long); +char* Type_getDescription(Type*); + +// Module +Module* Module___init__(char*); +char* Module_getModuleIdentifier(Module*); +void Module_setModuleIdentifier(Module*,char*); +char* Module_getTargetTriple(Module*); +void Module_setTargetTriple(Module*,char*); +char* Module_getModuleInlineAsm(Module*); +void Module_setModuleInlineAsm(Module*,char*); +Function* Module_getNamedFunction(Module*,char*); +Function* Module_getOrInsertFunction(Module*,char*,FunctionType*); + + + +#ifdef __cplusplus +}; +#endif + +#endif // __llvm_H__ Added: pypy/dist/pypy/translator/llvm/wrapable/llvm.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/llvm.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,154 @@ +# llvm.py + +from wrapper import Wrapper +from ctypes import * +import llvm_helper + + +llvm = cdll.load('libllvm.so') +STRING = c_char_p + +class ExecutionEngine(Wrapper): + def __init__(P0,P1,P2=False): + P0.instance = llvm_helper.ExecutionEngine___init__(P0,P1,P2) + + def create(P0,P1=False): + return llvm.ExecutionEngine_create(P0,P1) + create = staticmethod(create) + + def getModule(P0): + return llvm.ExecutionEngine_getModule(P0) + + def freeMachineCodeForFunction(P0,P1): + return llvm.ExecutionEngine_freeMachineCodeForFunction(P0,P1) + + def runFunction(P0,P1,P2): + return llvm.ExecutionEngine_runFunction(P0,P1,P2) + +class ModuleProvider(Wrapper): + pass + +class ExistingModuleProvider(Wrapper): + def __init__(P0,P1): + P0.instance = llvm.ExistingModuleProvider___init__(P1) + +class Function(Wrapper): + def eraseFromParent(P0): + return llvm.Function_eraseFromParent(P0) + + def getFunctionType(P0): + return llvm.Function_getFunctionType(P0) + +class FunctionType(Wrapper): + pass + +class GenericValue(Wrapper): + pass + +class TypeID(Wrapper): + pass + +class Type(Wrapper): + def getTypeID(P0): + return llvm.Type_getTypeID(P0) + + def getContainedType(P0,P1): + return llvm.Type_getContainedType(P0,P1) + + def getDescription(P0): + return llvm.Type_getDescription(P0) + +class Module(Wrapper): + def __init__(P0,P1='mymodule'): + P0.instance = llvm.Module___init__(P1) + + def getModuleIdentifier(P0): + return llvm.Module_getModuleIdentifier(P0) + + def setModuleIdentifier(P0,P1='myothermodule'): + return llvm.Module_setModuleIdentifier(P0,P1) + + def getTargetTriple(P0): + return llvm.Module_getTargetTriple(P0) + + def setTargetTriple(P0,P1): + return llvm.Module_setTargetTriple(P0,P1) + + def getModuleInlineAsm(P0): + return llvm.Module_getModuleInlineAsm(P0) + + def setModuleInlineAsm(P0,P1): + return llvm.Module_setModuleInlineAsm(P0,P1) + + def getNamedFunction(P0,P1): + return llvm.Module_getNamedFunction(P0,P1) + + def getOrInsertFunction(P0,P1,P2): + return llvm.Module_getOrInsertFunction(P0,P1,P2) + + def n_functions(P0): + return llvm_helper.n_functions(P0) + + def function_exists(P0,P1='myfunction'): + return llvm_helper.function_exists(P0,P1) + +llvm.ExecutionEngine_create.restype = ExecutionEngine +llvm.ExecutionEngine_create.argtypes = [ModuleProvider,c_long] + +llvm.ExecutionEngine_getModule.restype = Module +llvm.ExecutionEngine_getModule.argtypes = [ExecutionEngine] + +llvm.ExecutionEngine_freeMachineCodeForFunction.restype = None +llvm.ExecutionEngine_freeMachineCodeForFunction.argtypes = [ExecutionEngine,Function] + +llvm.ExecutionEngine_runFunction.restype = GenericValue +llvm.ExecutionEngine_runFunction.argtypes = [ExecutionEngine,Function,GenericValue] + +llvm.ExistingModuleProvider___init__.restype = ExistingModuleProvider +llvm.ExistingModuleProvider___init__.argtypes = [Module] + +llvm.Function_eraseFromParent.restype = None +llvm.Function_eraseFromParent.argtypes = [Function] + +llvm.Function_getFunctionType.restype = FunctionType +llvm.Function_getFunctionType.argtypes = [Function] + +llvm.Type_getTypeID.restype = TypeID +llvm.Type_getTypeID.argtypes = [Type] + +llvm.Type_getContainedType.restype = Type +llvm.Type_getContainedType.argtypes = [Type,c_long] + +llvm.Type_getDescription.restype = STRING +llvm.Type_getDescription.argtypes = [Type] + +llvm.Module___init__.restype = Module +llvm.Module___init__.argtypes = [STRING] + +llvm.Module_getModuleIdentifier.restype = STRING +llvm.Module_getModuleIdentifier.argtypes = [Module] + +llvm.Module_setModuleIdentifier.restype = None +llvm.Module_setModuleIdentifier.argtypes = [Module,STRING] + +llvm.Module_getTargetTriple.restype = STRING +llvm.Module_getTargetTriple.argtypes = [Module] + +llvm.Module_setTargetTriple.restype = None +llvm.Module_setTargetTriple.argtypes = [Module,STRING] + +llvm.Module_getModuleInlineAsm.restype = STRING +llvm.Module_getModuleInlineAsm.argtypes = [Module] + +llvm.Module_setModuleInlineAsm.restype = None +llvm.Module_setModuleInlineAsm.argtypes = [Module,STRING] + +llvm.Module_getNamedFunction.restype = Function +llvm.Module_getNamedFunction.argtypes = [Module,STRING] + +llvm.Module_getOrInsertFunction.restype = Function +llvm.Module_getOrInsertFunction.argtypes = [Module,STRING,FunctionType] + + + +# end of llvm.py Added: pypy/dist/pypy/translator/llvm/wrapable/llvm_helper.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/llvm_helper.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,16 @@ +#Module helper methods +def n_functions(self): #TODO + return 0 + +def function_exists(self, name): #TODO + return False + +#ExecutionEngine helper methods +def ExecutionEngine___init__(self, mp, ForceInterpreter=False): + return self.create(mp, ForceInterpreter) + +def delete(self, fnname): + mod = self.getModule() + f = mod.getNamedFunction(fnname) # XXX handle fnname not found? + self.freeMachineCodeForFunction(f) # still no-op on march 27th 2006 + f.eraseFromParent() Added: pypy/dist/pypy/translator/llvm/wrapable/setup.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/setup.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +from distutils.core import setup +from distutils.extension import Extension +from os import popen +import sys + + +if len(sys.argv) <= 1: + print 'usage: setup.py .... (the usual setup.py options)' + sys.exit() +module_name = sys.argv[1] +del sys.argv[1] + +cxxflags = popen('llvm-config --cxxflags').readline().split() +ldflags = popen('llvm-config --ldflags').readline().split() +libs = popen('llvm-config --libs all').readline().split() + +opts = dict(name=module_name, + sources=[module_name + '.cpp'], + libraries=[], + include_dirs =["include"] + [f[2:] for f in cxxflags if f.startswith('-I')], + library_dirs =[f[2:] for f in ldflags if f.startswith('-L')], + define_macros=[(f[2:], None) for f in cxxflags if f.startswith('-D')], + extra_objects=libs) + +ext_modules = Extension(**opts) + +setup(name=opts['name'], ext_modules=[ext_modules]) Added: pypy/dist/pypy/translator/llvm/wrapable/test/test_llvm.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/test/test_llvm.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,99 @@ +import py +from pypy.translator.llvm.wrapable.llvm import * + + +#XXX The tests below were shamefully never written when writing the rest of the code and +# should therefor be filled in by Eric asap. + +def test_misc(): + print 'Module("mine")' + mod = Module('mine') + print 'mod=', mod + print 'mod.instance=', mod.instance + print + assert mod.n_functions() == 0 + + print 'ExistingModuleProvider(mod)' + mp = ExistingModuleProvider(mod) + print 'mp=', mp + print 'mp.instance=', mp.instance + print + + ee = ExecutionEngine(mp) + #ee = ExecutionEngine.create(mp) #same + print 'ee=', ee + #print 'ee.instance=', ee.instance #can't do here because ee.create is staticmethod + mod2 = ee.getModule() + print mod2 + + print 'mod.getModuleIdentifier()' + modId = mod.getModuleIdentifier() + print 'modId=', modId + assert modId == 'mine' + assert modId == mod2.getModuleIdentifier() + +def test_wrapable_create(): + pass + +def test_global_function(): + pass + +def test_class(): + pass + +def test_class_ctor(): + pass + +def test_class_dtor(): + pass + +def test_method(): + pass + +def test_staticmethod(): + pass + +def test_staticmethod_as_factory(): #self.instance can not be set in this case + pass + +def test_classmethod(): + pass + +def test_include(): + pass + +def test_include_py(): + pass + +def test_call_pymethod(): + pass + +def test_call_pyctor(): + pass + +def test_call_pystaticmethod(): + pass + +def test_call_pyclassmethod(): + pass + +def test_call_pyfunction(): + pass + +def test_cpp_cast(): + pass + +def test_py_cast(): + pass + +def test_enum(): + pass + +def test_structure(): + pass + +def test_ref_ptr_inst(): + pass + +def test_default_arguments(): + pass Added: pypy/dist/pypy/translator/llvm/wrapable/wrap_llvm.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/wrap_llvm.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,84 @@ +#!/usr/bin/env python + +from wrapable import * +from ctypes import * + + +# All the classes we wrap in the module... +Global = Wglobal() +ExecutionEngine = Wclass('ExecutionEngine' , []) +ModuleProvider = Wclass('ModuleProvider' , []) +ExistingModuleProvider = Wclass('ExistingModuleProvider', [ModuleProvider]) +Function = Wclass('Function' , []) +FunctionType = Wclass('FunctionType' , []) +GenericValue = Wclass('GenericValue' , []) +TypeID = Enum('TypeID' , []) +Type = Wclass('Type' , []) +Module = Wclass('Module' , []) + +# Helper functions +as_c_string = CppCast('as_c_string', STRING) +as_std_string = CppCast('as_std_string', STRING) +as_vector_GenericValue = CppCast('as_vector_GenericValue', GenericValue) + +# All the functions (XXX not working/output yet) +Global\ + .add(Wfunction(None , 'ParseAssemblyString', [Module, STRING]))\ + .add(Wfunction(c_int, 'verifyModule' , [Module])) + +# All the methods each class contains... +ExecutionEngine\ + .add(Wctor(ExecutionEngine, [ModuleProvider,c_int], [False],\ + 'llvm_helper.ExecutionEngine___init__'))\ + .add(Wstaticmethod(ExecutionEngine, 'create', [ModuleProvider,c_int], [False]))\ + .add(Wmethod(Ref(Module) , 'getModule', []))\ + .add(Wmethod(None , 'freeMachineCodeForFunction', [Function]))\ + .add(Wmethod(Inst(GenericValue) , 'runFunction', [Function,as_vector_GenericValue])) + +ExistingModuleProvider\ + .add(Wctor(ExistingModuleProvider, [Module])) + +Function\ + .add(Wmethod(None , 'eraseFromParent', []))\ + .add(Wmethod(FunctionType, 'getFunctionType', [])) + +Type\ + .add(Wmethod(TypeID , 'getTypeID', []))\ + .add(Wmethod(Type , 'getContainedType', [c_int]))\ + .add(Wmethod(as_c_string , 'getDescription', [])) + +Module\ + .add(Wctor(Module, [as_std_string], ['mymodule']))\ + .add(Wmethod(as_c_string , 'getModuleIdentifier', []))\ + .add(Wmethod(None , 'setModuleIdentifier', [STRING], ['myothermodule']))\ + .add(Wmethod(as_c_string , 'getTargetTriple', []))\ + .add(Wmethod(None , 'setTargetTriple', [STRING]))\ + .add(Wmethod(as_c_string , 'getModuleInlineAsm', []))\ + .add(Wmethod(None , 'setModuleInlineAsm', [STRING]))\ + .add(Wmethod(Function , 'getNamedFunction', [STRING]))\ + .add(Wmethod(Function , 'getOrInsertFunction', [STRING, FunctionType]))\ + .add(Wmethod(c_int , 'llvm_helper.n_functions', []))\ + .add(Wmethod(c_int , 'llvm_helper.function_exists', [STRING], ['myfunction'])) + +# Create the module... +Wroot('llvm')\ + .include('llvm/Module.h')\ + .include('llvm/ModuleProvider.h')\ + .include('llvm/Function.h')\ + .include('llvm/DerivedTypes.h')\ + .include('llvm/Type.h')\ + .include('llvm/Analysis/Verifier.h')\ + .include('llvm/ExecutionEngine/ExecutionEngine.h')\ + .include('llvm/ExecutionEngine/GenericValue.h')\ + .include('llvm_helper.py')\ + .add(Global)\ + .add(ExecutionEngine)\ + .add(ModuleProvider)\ + .add(ExistingModuleProvider)\ + .add(Function)\ + .add(FunctionType)\ + .add(GenericValue)\ + .add(TypeID)\ + .add(Type)\ + .add(Module)\ + .create() Added: pypy/dist/pypy/translator/llvm/wrapable/wrapable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/wrapable.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,396 @@ +import types +from ctypes import * + +STRING = c_char_p + +def log(msg): + print msg + pass + + +class T(object): + def __init__(self, item): + self.item = item +class Inst(T): pass +class Ref(T): pass +#class Ptr(T): pass + + +class Cast(object): + def __init__(self, cppfunc, item): + self.cppfunc = cppfunc + self.item = item +class CppCast(Cast): pass +#class PyCast(Cast): pass + + +class _CTypesInfo(object): + def __init__(self, ctypesname, cppname): + self.ctypesname = ctypesname + self.cppname = cppname + +_ctypesinfo = { + None : _CTypesInfo('None' , 'void'), + c_char : _CTypesInfo('c_char' , 'char'), + c_byte : _CTypesInfo('c_byte' , 'char'), + c_ubyte : _CTypesInfo('c_ubyte' , 'unsigned char'), + c_short : _CTypesInfo('c_short' , 'short'), + c_ushort : _CTypesInfo('c_ushort' , 'unsigned short'), + c_int : _CTypesInfo('c_int' , 'int'), + c_uint : _CTypesInfo('c_uint' , 'unsigned int'), + c_long : _CTypesInfo('c_long' , 'long'), + c_ulong : _CTypesInfo('c_ulong' , 'unsigned long'), + c_longlong : _CTypesInfo('c_longlong' , 'long long'), + c_ulonglong : _CTypesInfo('c_ulonglong', 'unsigned long long'), + c_float : _CTypesInfo('c_float' , 'float'), + c_double : _CTypesInfo('c_double' , 'double'), + c_char_p : _CTypesInfo('STRING' , 'char*'), + c_wchar_p : _CTypesInfo('c_wchar_p' , 'wchar_t*'), + c_void_p : _CTypesInfo('c_void_p' , 'void*'), + } + +def _ctypes_type(t): + if isinstance(t, Cast): + t = t.item + if t in _ctypesinfo: + return _ctypesinfo[t].ctypesname + if isinstance(t, Wclass): + return t.name + if isinstance(t, (Inst, Ref)) and isinstance(t.item, Wclass): + return t.item.name + raise TypeError('unknown type %s' % t) + + +def _cpp_type(t): + if isinstance(t, Cast): + t = t.item + if t in _ctypesinfo: + return _ctypesinfo[t].cppname + if isinstance(t, Wclass): + return t.name + '*' + if isinstance(t, Inst): + return t.item.name + if isinstance(t, Ref) and isinstance(t.item, Wclass): + return t.item.name + '&' + raise TypeError('unknown type %s' % t) +h_type = _cpp_type + + +class _Witem(object): + def __init__(self): + self.items = [] + + def add(self, item): + self.items.append(item) + return self + + def header(self): + pass + + def cpp(self): + pass + + def py(self): + pass + + def finish_declaration(self, parent_item): + for item in self.items: + item.finish_declaration(self) + + +class Wfunction(_Witem): + def __init__(self, ret, name, param): + super(Wfunction, self).__init__() + self.ret = ret + self.name = name + self.param = param + + +class Wmethod(_Witem): + def __init__(self, ret, name, param, defaults=[]): + super(Wmethod, self).__init__() + self.ret = ret + self.name = name + self.param = param + self.defaults = defaults + self._self_was_added = False + + def finish_declaration(self, parent_item): + super(Wmethod, self).finish_declaration(parent_item) + if not self._self_was_added and not isinstance(self, Wstaticmethod): + self.param.insert(0, parent_item) + self._self_was_added = True + + def _is_pyfunc(self): + return '.' in self.name + + def _plain_name(self): + if self._is_pyfunc(): + return self.name.split('.',1)[1] + return self.name + + def _callee_name(self, namespace, funcname): + if self._is_pyfunc(): + return self.name + return '%s.%s_%s' % (namespace, funcname, self.name) + + +class Wstaticmethod(Wmethod): + pass + + +class Wctor(Wmethod): #Wrapped constructor + #XXX I don't like the fact that the returntype needs to be specified. + # We need to somehow keep a link to the parent! + # (maybe set a parent attribute in the .add methods?) + def __init__(self, ret, param, defaults=[], name='__init__'): + #note: The order is different from Wmethod because name would seldomly be supplied. + # Probably this will only happen when a Python function should be called instead. + super(Wctor, self).__init__(ret, name, param, defaults) + + +class Wclass(_Witem): + def __init__(self, name, parents=[]): + super(Wclass, self).__init__() + self.name = name + self.parents = parents +Enum = Wclass #XXX temporary hack + + +class Wglobal(_Witem): + pass + + +class Wroot(_Witem): #root wrapable object + + header_template = '''#ifndef __%(name)s_H__ +#define __%(name)s_H__ + +%(includes)s +using namespace %(namespace)s; +#define TypeID int + +#ifdef __cplusplus +extern "C" { +#endif + +%(code)s + +#ifdef __cplusplus +}; +#endif + +#endif // __%(name)s_H__ +''' + + cpp_template = '''// %(name)s.cpp + +#include "lib%(name)s.h" + +// Helper functions (mostly for casting returnvalues and parameters) +char* as_c_string(std::string str) { + return (char*)str.c_str(); +} + +std::string as_std_string(char* p) { + return std::string(p); +} + +std::vector as_vector_GenericValue(GenericValue* gv) { + std::vector gvv; + gvv.push_back(*gv); + return gvv; +} + +%(code)s + +// end of %(name)s.cpp +''' + + py_template = '''# %(name)s.py + +from wrapper import Wrapper +from ctypes import * +%(includes)s + +%(name)s = cdll.load('lib%(name)s.so') +STRING = c_char_p + +%(code)s + +# end of %(name)s.py +''' + + def __init__(self, name): + super(Wroot, self).__init__() + self.name = name + self.includes = [] + log('Creating %s module' % name) + + def include(self, include): + self.includes.append(include) + return self #to allow chaining + + def header(self): + name = self.name + code = '' + + includes = '' + for include in self.includes: + if include.endswith('.h'): + includes += '#include "%s"\n' % include + + namespace = self.name #XXX + + for item in self.items: + if isinstance(item, Wglobal): + continue + code += '// %s\n' % item.name + for method in item.items: + if method._is_pyfunc(): + continue + restype = h_type(method.ret) + argtypes = [h_type(p) for p in method.param] + if isinstance(method, Wctor): + del argtypes[0] + code += '%s %s_%s(%s);\n' % (restype, item.name, method.name, ','.join(argtypes)) + code += '\n' + + return self.header_template % locals() + + def _casted_arg(self, i, arg): + if isinstance(arg, CppCast): + return '%s(P%d)' % (arg.cppfunc, i) + return 'P%d' % i + + def cpp(self): + name = self.name + code = '' + for item in self.items: + if isinstance(item, Wglobal): + continue + code += '// %s\n' % item.name + for method in item.items: + if method._is_pyfunc(): + continue + restype = _cpp_type(method.ret) + param = method.param[isinstance(method, Wctor):] + argtypes = ['%s P%d' % (_cpp_type(p),i) for i,p in enumerate(param)] + code += '%s %s_%s(%s) {\n' % (restype, item.name, method.name, ','.join(argtypes)) + #for i,p in enumerate(method.param): + # code += ' %s p%d = (%s)P%d;\n' % (_cpp_type(p), i, _cpp_type(p), i) + retcast = '(%s)' % restype + if isinstance(method.ret, CppCast): + retcast_pre = '%s(' % method.ret.cppfunc + retcast_post = ')' + else: + retcast_pre = '' + retcast_post = '' + if isinstance(method, Wstaticmethod): + args = [self._casted_arg(i,p) for i,p in enumerate(method.param)] + code += ' return %s%s%s::%s(%s)%s;\n}\n\n' % \ + (retcast, retcast_pre, item.name, method.name, + ','.join(args), retcast_post) + elif isinstance(method, Wctor): + #args = [self._casted_arg(i+1,p) for i,p in enumerate(method.param[1:])] + args = [self._casted_arg(i,p) for i,p in enumerate(param)] + code += ' return %snew %s(%s)%s;\n}\n\n' % \ + (retcast_pre, item.name, ','.join(args), retcast_post) + else: + args = [self._casted_arg(i+1,p) for i,p in enumerate(method.param[1:])] + code += ' return %s%sP0->%s(%s)%s;\n}\n\n' % \ + (retcast, retcast_pre, method.name, + ','.join(args), retcast_post) + code += '\n' + return self.cpp_template % locals() + + def py(self): + name = self.name + code = '' + + includes = '' + for include in self.includes: + if include.endswith('.py'): + includes += 'import %s\n' % include[:-3] + + # output classes, methods, etc. + for item in self.items: + if isinstance(item, Wglobal): + continue + code += 'class %s(Wrapper):\n' % item.name + for method in item.items: + param = ['P%d' % i for i,p in enumerate(method.param)] + plain_name = method._plain_name() + callee_name = method._callee_name(self.name, item.name) + params = '' + for i, p in enumerate(param): + if params: + params += ',' + params += p + n = len(method.defaults) - len(param) + i + if n >= 0: + params += '=' + repr(method.defaults[n]) + if isinstance(method, Wctor): + #We made sure the header and cpp file constructors don't get the python 'self'. + #Now we have to make sure we only pass 'self' to python helpers. + code += ' def __init__(%s):\n' % params + code += ' P0.instance = %s(%s)\n' % ( + callee_name, ','.join(param[not method._is_pyfunc():])) + else: + code += ' def %s(%s):\n' % (plain_name, params) + code += ' return %s(%s)\n' % ( + callee_name, ','.join(param)) + if isinstance(method, Wstaticmethod): + code += ' %s = staticmethod(%s)\n' % (plain_name, plain_name) + code += '\n' + if not item.items: + code += ' pass\n\n' + + # output return and parameter (c)types + for item in self.items: + if isinstance(item, Wglobal): + continue + for method in item.items: + if method._is_pyfunc(): + continue + n = int(isinstance(method, Wctor)) + restype = _ctypes_type(method.ret) + argtypes = ','.join([_ctypes_type(p) for p in method.param[n:]]) + if isinstance(method, Wstaticmethod): + im_func = '' + else: + im_func = 'im_func.' + code += 'llvm.%s_%s.restype = %s\n' % ( + item.name, method.name, restype) + code += 'llvm.%s_%s.argtypes = [%s]\n\n' % ( + item.name, method.name, argtypes) + + return self.py_template % locals() + + def create_files(self): + log('Creating ' + self.name + '.py') + f = open(self.name + '.py', 'w') + code = self.py() + f.write(code) + f.close() + + log('Creating lib' + self.name + '.h') + f = open('lib' + self.name + '.h', 'w') + code = self.header() + f.write(code) + f.close() + + log('Creating lib' + self.name + '.cpp') + f = open('lib' + self.name + '.cpp', 'w') + code = self.cpp() + f.write(code) + f.close() + + def create_library(self): + from os import system + system('python setup.py lib%s build_ext -i' % self.name) + + def create(self): + self.finish_declaration(None) + self.create_files() + self.create_library() Added: pypy/dist/pypy/translator/llvm/wrapable/wrapper.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/wrapable/wrapper.py Thu Apr 20 16:17:39 2006 @@ -0,0 +1,19 @@ +from ctypes import c_int + + +class Wrapper(c_int): #c_int because it needs to be a ctypes datatype for from_param to work + def from_param(cls, p): + try: + return p.instance + except: + #XXX This might happen if we use a staticmethod (factory) to create a function. + # In that case we have no 'self' to store the instance in. + # An example of this can be found in wrap_llvm.py/ExecutionEngine.create + return p + from_param = classmethod(from_param) + + #def errcheck(retval, function, arguments): + # print 'Wrapper.errcheck: retval(instance)=%s, function=%s %s, arguments=%s' % (retval,function,dir(function.__class__),arguments) + # self.instance = retval + # return retval + #errcheck = staticmethod(errcheck) From auc at codespeak.net Thu Apr 20 17:04:38 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 20 Apr 2006 17:04:38 +0200 (CEST) Subject: [pypy-svn] r26044 - pypy/dist/pypy/doc Message-ID: <20060420150438.0E74F10084@code0.codespeak.net> Author: auc Date: Thu Apr 20 17:04:37 2006 New Revision: 26044 Added: pypy/dist/pypy/doc/gsoc_ideas_WP9-10.txt Log: proposals for google summer of code Added: pypy/dist/pypy/doc/gsoc_ideas_WP9-10.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/gsoc_ideas_WP9-10.txt Thu Apr 20 17:04:37 2006 @@ -0,0 +1,206 @@ +Two possible proposals : + +I. Specialized propagators for specialized finite domains +========================================================= + +Concerns WP9 (concurrent constraint & logic programming workpackage). + +We need specialized propagators for specialized finite domains (over +numbers, sets, etc.). There are two ways to provide them : + +* RPython : which means implementing and debugging a lot of stuff, + with possibly subotpimal runtime performance but no external + dependency, + +* wrapping Gecode : first finish the c-wrapper around Gecode, then use + rctypes to provide interpreter access to low-level Gecode + functionality, and finally integrate it into the existing + interpreter-level constraint solver. + +Rctypes is what PyPy offers, mimicking python ctypes, to interface the +interpreter with C code. The machinery is still very young and the +advancement of the project might be hindered by defects or +incompleteness of rctypes. The point is that, even if the actual +wrapping of gecode is incomplete, the effort to build it at least +should yield a better rctypes. + +One modest, reasonable target could be running the simple +send-more-money problem with very few actual search steps due to the +efficient propagation happening on integer finite domains. + + +II. Provide efficient builtin numeric arrays +============================================ + +Implements numeric arrays (as in numeric/numpy/numarray) as builtin +types of pypy so that numeric operations on arrays are recognized by +the annotator, rtyper and backends and translated optimally. + + +III. A template system for Python code +===================================== + +Related mainly to WP10 (aspects ...), but of WP9 interest. + +Use case +-------- + +We might need a way to extend python with macro-like features, so +as to be able to provide the choice/or operator (and others) without +touching the grammar file (it means in short : making available the new +operator at runtime by way of importing some module). + +Examples of choice usage : + + def soft_color(): + choice: + return 'beige' + or: + return 'coral' + + def hard_color(): + choice: + return 'mauve' + or: + return 'ochre' + + def contrast_colors(C1, C2): + choice: + C1 :=: soft_color() + C2 :=: hard_color() + or: + C1 :=: hard_color() + C2 :=: soft_color() + + def suit(): + Shirt, Pants, Socks = ?, ?, ? + contrast_colors(Shirt, Pants) + contrast_colors(Pants, Socks) + if Shirt == Socks: + fail() + return (Shirt, Pants, Socks) + + +If we give the suit entry point to the solver built for WP9, we can +(lazily) get an enumeration of all suits respecting our ruleset. + +Some explanation for choice/or, ?, :=: and fail() : + +* choice/or : this construct places a so-called non-deterministic + choice point into a program. It is different from an 'if' in the + sense that for one complete run of the program, only one choice + point may be chosen. The solver is practically responsible for + exploring the space of 'worlds' in which the program go through the + different choice points. Some of these worlds will fail (they don't + belong to a solution), some other will yield a valid solution of our + program. Each choice point can be made of arbitrary Python code. + + The important thing, wrt WP10, is to understand that choice is + merely syntactic sugar over the primitive choose() operator. Any + choice/or construct can be rewritten in terms of choose, as in : + + choice: or: or: ... or: + + which has to be rewritten as : + + choice = choose(N) + if choice == 1: + + elif choice == 2: + + ... + else: # choice == N + + + This is possible because all choice points are known at compile + time. + + See the Annex for an example on how (shortly) this could be done in + Lisp. + +* ? denotes a logic variable, i.e a variable which has no value at + creation-time (not even None) and can be bound only once. Currently, + the newvar() builtin is used instead of it in PyPy + +* :=: is a short-cut notation we might want instead of merely calling + unify, as in : unify(Term1, Term2) + +* fail() makes the current computation space fail: it means that the + current computation reached an inconsistent (from a logic point of + view) state and cannot yield a solution. + +Rationale +--------- + +Macros are functions that execute at compile-time, so as to provide +language extensions usable at run-time, by way of source code (or AST) +transformations. Tens of years of work in the field have make it clear +that many macros are infinitely easier to write provided one has means +to express source code as templates in which compile-time computed +information can be injected. + +Some easy example for Python : let's say we want an 'unless operator', +which writes as : + + unless : + ... op sequence ... + +and should be translated back to the following legal CPython code : + + if not : + ... op sequence ... + +One could define unless as a macro which takes the test and operation +sequence as parameters and returns an AST which conforms to plain Python. + + def test(test, op_seq): + templ = `if not ~test: ~op_seq` + return templ + +Here we use a set of new operators, which provide a functionality +sometimes called quasiquotation. + +The `...` (backquote) syntax allows to embed literal pieces of Python +code ; the ~ (called unquote, and others, to be defined) allows to +inject (eventually computed) data into the template. + +This is extremely rough and basic but can give an idea. Mature ways +to do it are exposed in the programming languages Dylan +(http://people.csail.mit.edu/jrb/Projects/dexprs.pdf) and, maybe more +in touch with the Python world, logix (http://livelogix.net/logix/). + +Todo +---- + +Investigate ways to provide a quasiquotation/templating mechanism +suitable for a language like Python. + + +Annex +----- + + A Common Lisp programmer would have a 'choice' operator like this : + + (choice ) + + ... for instance : + + (defun contrast (C1 C2) + (choice + ((unify C1 (soft-color)) + (unify C2 (hard-color))) + ((unify C1 (hard-color)) + (unify C2 (soft-color))))) + + He would define choice as a macro, as follows : + + (defmacro choice (&body choice-points) + (let ((choices (length choice-points)) + (choice (gensym "choice"))) + `(let ((,choice (choose ,choices))) + (cond + ,(loop for alternative in choice-points + for i from 1 upto choices + collect `((= ,i ,choice) + (progn ,alternative))))))) + From cfbolz at codespeak.net Thu Apr 20 17:08:56 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 20 Apr 2006 17:08:56 +0200 (CEST) Subject: [pypy-svn] r26045 - pypy/extradoc/pypy.org Message-ID: <20060420150856.3D8E810086@code0.codespeak.net> Author: cfbolz Date: Thu Apr 20 17:08:53 2006 New Revision: 26045 Added: pypy/extradoc/pypy.org/links.txt Modified: pypy/extradoc/pypy.org/confrest.py pypy/extradoc/pypy.org/revision_march2006.txt Log: finally add the links page. sorry that it took so long Modified: pypy/extradoc/pypy.org/confrest.py ============================================================================== --- pypy/extradoc/pypy.org/confrest.py (original) +++ pypy/extradoc/pypy.org/confrest.py Thu Apr 20 17:08:53 2006 @@ -7,8 +7,7 @@ html.a("home", href="index.html", class_="menu"), " ", html.a("news", href="news.html", class_="menu"), " ", html.a("consortium", href="consortium.html", class_="menu"), " ", - #html.a("EU-Issues", href="https://codespeak.net/issue/pypy-eu/", - # class_="menu"), " ", + html.a("links", href="links.html", class_="menu"), " ", html.a("community/coding", href="http://codespeak.net/pypy/dist/pypy/doc/index.html", class_="menu"), " ", Added: pypy/extradoc/pypy.org/links.txt ============================================================================== --- (empty file) +++ pypy/extradoc/pypy.org/links.txt Thu Apr 20 17:08:53 2006 @@ -0,0 +1,49 @@ + +Related EU projects and EU pages: +================================= + + * `6th Framework Programme`_ + + * `The Calibre Project`_ + + * `Edos`_ + + * `Amigo`_ + + * `tOSSad`_ + + * `Cospa`_ + + * `Floss`_ + +.. _`6th Framework Programme`: http://fp6.cordis.lu/index.cfm?fuseaction=UserSite.FP6HomePage + +.. _`The Calibre Project`: http://www.calibre.ie + +.. _`Edos`: http://www.pps.jussieu.fr/~dicosmo/EDOS/index.html + +.. _`Amigo`: http://www.hitech-projects.com/euprojects/amigo/ + +.. _`tOSSad`: http://www.tossad.org/ + +.. _`Cospa`: http://www.cospa-project.org/ + +.. _`Floss`: http://www.infonomics.nl/FLOSS/ + +Related Python F/OSS projects: +============================== + + * `CPython`_ + +.. _`CPython`: http://www.python.org + +.. bea put the following into the list as well + Ironpython? + Jython? + Twisted? + + Related F/OSS projects: + Oz? + ..? + ..? + Modified: pypy/extradoc/pypy.org/revision_march2006.txt ============================================================================== --- pypy/extradoc/pypy.org/revision_march2006.txt (original) +++ pypy/extradoc/pypy.org/revision_march2006.txt Thu Apr 20 17:08:53 2006 @@ -6,7 +6,6 @@ The following changes/adds needs to be done: -- change eu logo to IST logo (done) - change navigation to links about: Home, News, Consortium, Links, Documentation, Community/Coding - Links being a page with links to sisterprojects, to commission pages etc @@ -17,37 +16,6 @@ do we target - who will have use of PyPy???? This info could be on the main page (HOME) ----------------------------------------------------------------------------------------------------------- - -Here are the urls for the Link page: - -Related EU projects and url:s: -6th Framework Programme -http://fp6.cordis.lu/index.cfm?fuseaction=UserSite.FP6HomePage -Calibre: -http://www.calibre.ie -Edos: -http://www.pps.jussieu.fr/~dicosmo/EDOS/index.html -Amigo: -http://www.hitech-projects.com/euprojects/amigo/ -tOSSad -http://www.tossad.org/ -Cospa -http://www.cospa-project.org/ -Floss -http://www.infonomics.nl/FLOSS/ - -Related Python F/OSS projects: -Cpython? -Ironpython? -Jython? -Twisted? -Plone? - -Related F/OSS projects: -Oz? -..? -..? --------------------------- Do we want to have a link to universities, conferences and companies we are in touch with/cooperating with? -Such as UIB, UCD, Iona, x, y, z... \ No newline at end of file +Such as UIB, UCD, Iona, x, y, z... From hpk at codespeak.net Thu Apr 20 17:12:35 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 17:12:35 +0200 (CEST) Subject: [pypy-svn] r26046 - pypy/extradoc/pypy.org Message-ID: <20060420151235.2CC3010087@code0.codespeak.net> Author: hpk Date: Thu Apr 20 17:12:34 2006 New Revision: 26046 Modified: pypy/extradoc/pypy.org/revision_march2006.txt Log: fixing ReST in bea's todo file Modified: pypy/extradoc/pypy.org/revision_march2006.txt ============================================================================== --- pypy/extradoc/pypy.org/revision_march2006.txt (original) +++ pypy/extradoc/pypy.org/revision_march2006.txt Thu Apr 20 17:12:34 2006 @@ -16,6 +16,5 @@ do we target - who will have use of PyPy???? This info could be on the main page (HOME) ---------------------------- Do we want to have a link to universities, conferences and companies we are in touch with/cooperating with? Such as UIB, UCD, Iona, x, y, z... From hpk at codespeak.net Thu Apr 20 18:02:37 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 18:02:37 +0200 (CEST) Subject: [pypy-svn] r26048 - pypy/extradoc/soc-2006 Message-ID: <20060420160237.7B46C10086@code0.codespeak.net> Author: hpk Date: Thu Apr 20 18:02:36 2006 New Revision: 26048 Added: pypy/extradoc/soc-2006/ pypy/extradoc/soc-2006/planning.txt Log: as decided on pypy-sync, a soc-2006 directory Added: pypy/extradoc/soc-2006/planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/soc-2006/planning.txt Thu Apr 20 18:02:36 2006 @@ -0,0 +1,28 @@ +PyPy Summer of Code mentors and plans +============================================== + +mentors +------------------------ + +Armin Rigo +Michael Hudson +Carl Friedrich Bolz +Christian Tismer +Samuele Pedroni +Anders Chrigstroem +Holger Krekel +Aurelien Campeas + + +possible topics by keyword +---------------------------------------- + +- orthogonal persistence +- implementing security models, e.g. sandboxing +- work on high level backends +- transactional memory +- writing extension modules +- various optimization approaches +- extending py.test to support distributed testing + + From hpk at codespeak.net Thu Apr 20 18:04:31 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 18:04:31 +0200 (CEST) Subject: [pypy-svn] r26049 - pypy/extradoc/soc-2006 Message-ID: <20060420160431.8317D10087@code0.codespeak.net> Author: hpk Date: Thu Apr 20 18:04:30 2006 New Revision: 26049 Added: pypy/extradoc/soc-2006/wp09_10_ideas.txt - copied unchanged from r26048, pypy/dist/pypy/doc/gsoc_ideas_WP9-10.txt Log: moving Aurelien's proposal ideas to the soc-2006 directory (not sure we are to submit such exact proposal ideas to the SoC guys) From hpk at codespeak.net Thu Apr 20 18:04:57 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 18:04:57 +0200 (CEST) Subject: [pypy-svn] r26050 - pypy/dist/pypy/doc Message-ID: <20060420160457.5C12E10084@code0.codespeak.net> Author: hpk Date: Thu Apr 20 18:04:56 2006 New Revision: 26050 Removed: pypy/dist/pypy/doc/gsoc_ideas_WP9-10.txt Log: removing the old soc file (now in extradoc/soc-2006) From mwh at codespeak.net Thu Apr 20 18:11:27 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 20 Apr 2006 18:11:27 +0200 (CEST) Subject: [pypy-svn] r26051 - pypy/extradoc/soc-2006 Message-ID: <20060420161127.B3CFF10084@code0.codespeak.net> Author: mwh Date: Thu Apr 20 18:11:23 2006 New Revision: 26051 Modified: pypy/extradoc/soc-2006/planning.txt Log: some planning Modified: pypy/extradoc/soc-2006/planning.txt ============================================================================== --- pypy/extradoc/soc-2006/planning.txt (original) +++ pypy/extradoc/soc-2006/planning.txt Thu Apr 20 18:11:23 2006 @@ -13,6 +13,12 @@ Holger Krekel Aurelien Campeas +People already registered as a PSF mentor: + +Michael Hudson + +On the 27th of April or so, mwh will send a list of all +not-yet-registered mentors to Neal Norwitz. possible topics by keyword ---------------------------------------- From hpk at codespeak.net Thu Apr 20 18:19:35 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 18:19:35 +0200 (CEST) Subject: [pypy-svn] r26052 - pypy/extradoc/minute Message-ID: <20060420161935.77C9C10089@code0.codespeak.net> Author: hpk Date: Thu Apr 20 18:19:34 2006 New Revision: 26052 Added: pypy/extradoc/minute/pypy-sync-04-20-2006.txt - copied, changed from r26047, pypy/extradoc/minute/invite-pypy-sync-04-20-2006.txt Removed: pypy/extradoc/minute/invite-pypy-sync-04-20-2006.txt Log: pypy-sync minutes draft, please check From hpk at codespeak.net Thu Apr 20 18:20:33 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 18:20:33 +0200 (CEST) Subject: [pypy-svn] r26053 - pypy/extradoc/minute Message-ID: <20060420162033.EF4E910089@code0.codespeak.net> Author: hpk Date: Thu Apr 20 18:20:32 2006 New Revision: 26053 Added: pypy/extradoc/minute/pypy-sync-2006-04-20.txt - copied unchanged from r26052, pypy/extradoc/minute/pypy-sync-04-20-2006.txt Removed: pypy/extradoc/minute/pypy-sync-04-20-2006.txt Log: adhere to naming convention in this directory From hpk at codespeak.net Thu Apr 20 18:21:05 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 20 Apr 2006 18:21:05 +0200 (CEST) Subject: [pypy-svn] r26054 - pypy/extradoc/minute Message-ID: <20060420162105.7A2B01009A@code0.codespeak.net> Author: hpk Date: Thu Apr 20 18:21:03 2006 New Revision: 26054 Modified: pypy/extradoc/minute/pypy-sync-2005-09-22.txt (props changed) pypy/extradoc/minute/pypy-sync-2005-09-29.txt (props changed) pypy/extradoc/minute/pypy-sync-2005-10-20.txt (props changed) pypy/extradoc/minute/pypy-sync-2005-10-27.txt (props changed) pypy/extradoc/minute/pypy-sync-2005-11-03.txt (props changed) pypy/extradoc/minute/pypy-sync-2005-11-10.txt (props changed) pypy/extradoc/minute/pypy-sync-2006-03-09.txt (props changed) pypy/extradoc/minute/pypy-sync-2006-03-30.txt (props changed) pypy/extradoc/minute/pypy-sync-2006-04-20.txt (props changed) Log: fixeol From antocuni at codespeak.net Fri Apr 21 00:11:31 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 21 Apr 2006 00:11:31 +0200 (CEST) Subject: [pypy-svn] r26070 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060420221131.B4BCF10053@code0.codespeak.net> Author: antocuni Date: Fri Apr 21 00:11:02 2006 New Revision: 26070 Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py Log: Fixed a bug caused by the change of rlist's inteface. Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Fri Apr 21 00:11:02 2006 @@ -40,14 +40,19 @@ v_tup = hop.inputarg(self, 0) LIST = hop.r_result.lowleveltype c_list = inputconst(ootype.Void, LIST) - v_list = hop.gendirectcall(rlist.ll_newlist, c_list) + v_list = hop.genop('new', [c_list], resulttype=LIST) + c_resize = inputconst(ootype.Void, '_ll_resize') + c_setitem = inputconst(ootype.Void, 'll_setitem_fast') + c_length = inputconst(ootype.Signed, len(self.items_r)) + hop.genop('oosend', [c_resize, v_list, c_length], resulttype=ootype.Void) for index in range(len(self.items_r)): name = self.fieldnames[index] r_item = self.items_r[index] c_name = hop.inputconst(ootype.Void, name) v_item = hop.genop("oogetfield", [v_tup, c_name], resulttype=r_item) v_item = hop.llops.convertvar(v_item, r_item, hop.r_result.item_repr) - hop.gendirectcall(rlist.ll_append, v_list, v_item) + c_index = inputconst(ootype.Signed, index) + hop.genop('oosend', [c_setitem, v_list, c_index, v_item], resulttype=ootype.Void) return v_list _tuple_types = {} From antocuni at codespeak.net Fri Apr 21 00:17:56 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 21 Apr 2006 00:17:56 +0200 (CEST) Subject: [pypy-svn] r26071 - in pypy/dist/pypy: jit/timeshifter rpython rpython/lltypesystem rpython/ootypesystem rpython/ootypesystem/test rpython/test Message-ID: <20060420221756.3701D1007D@code0.codespeak.net> Author: antocuni Date: Fri Apr 21 00:17:24 2006 New Revision: 26071 Added: pypy/dist/pypy/rpython/lltypesystem/rdict.py (contents, props changed) pypy/dist/pypy/rpython/ootypesystem/rdict.py (contents, props changed) pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (contents, props changed) Modified: pypy/dist/pypy/jit/timeshifter/timeshift.py pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rdict.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/rpython/test/test_rdict.py pypy/dist/pypy/rpython/typesystem.py Log: - rdict has been made type-system depended. Almost all old code has been moved to lltypesystem/rdict.py. - some files has been fixed because they imported directly rpython.rdict. - a new Dict definition has been added to ootype.py. Three tests in test_rtuple.py fail because by now ootypesystem lacks a DictRepr, but they should automatically pass as soon as ootypesystem.rdict.DictRepr will be added. Modified: pypy/dist/pypy/jit/timeshifter/timeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/timeshift.py (original) +++ pypy/dist/pypy/jit/timeshifter/timeshift.py Fri Apr 21 00:17:24 2006 @@ -5,8 +5,8 @@ from pypy.annotation import listdef, dictdef from pypy.jit.timeshifter import rvalue from pypy.jit.timeshifter.rtimeshift import JITState -from pypy.rpython import rmodel, rdict, rgenop, annlowlevel -from pypy.rpython.lltypesystem import rtuple, rlist +from pypy.rpython import rmodel, rgenop, annlowlevel +from pypy.rpython.lltypesystem import rtuple, rlist, rdict from pypy.jit.timeshifter import rtimeshift from pypy.jit.timeshifter.rtyper import HintRTyper, originalconcretetype from pypy.jit.timeshifter.rtyper import GreenRepr, RedRepr, HintLowLevelOpList Added: pypy/dist/pypy/rpython/lltypesystem/rdict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/rdict.py Fri Apr 21 00:17:24 2006 @@ -0,0 +1,773 @@ +from pypy.annotation.pairtype import pairtype +from pypy.annotation import model as annmodel +from pypy.objspace.flow.model import Constant +from pypy.rpython.rdict import AbstractDictRepr, rtype_newdict +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.rarithmetic import r_uint +from pypy.rpython.objectmodel import hlinvoke +from pypy.rpython import robject +from pypy.rpython import objectmodel +from pypy.rpython import rmodel + +# ____________________________________________________________ +# +# generic implementation of RPython dictionary, with parametric DICTKEY and +# DICTVALUE types. +# +# XXX for immutable dicts, the array should be inlined and +# num_pristine_entries and everused are not needed. +# +# struct dictentry { +# DICTKEY key; +# bool f_valid; # (optional) the entry is filled +# bool f_everused; # (optional) the entry is or has ever been filled +# DICTVALUE value; +# int f_hash; # (optional) key hash, if hard to recompute +# } +# +# struct dicttable { +# int num_items; +# int num_pristine_entries; # never used entries +# Array *entries; +# (Function DICTKEY, DICTKEY -> bool) *fnkeyeq; +# (Function DICTKEY -> int) *fnkeyhash; +# } +# +# + +class DictRepr(AbstractDictRepr): + + def __init__(self, rtyper, key_repr, value_repr, dictkey, dictvalue, + custom_eq_hash=None): + self.rtyper = rtyper + self.DICT = lltype.GcForwardReference() + self.lowleveltype = lltype.Ptr(self.DICT) + self.custom_eq_hash = custom_eq_hash is not None + if not isinstance(key_repr, rmodel.Repr): # not computed yet, done by setup() + assert callable(key_repr) + self._key_repr_computer = key_repr + else: + self.external_key_repr, self.key_repr = self.pickkeyrepr(key_repr) + if not isinstance(value_repr, rmodel.Repr): # not computed yet, done by setup() + assert callable(value_repr) + self._value_repr_computer = value_repr + else: + self.external_value_repr, self.value_repr = self.pickrepr(value_repr) + self.dictkey = dictkey + self.dictvalue = dictvalue + self.dict_cache = {} + self._custom_eq_hash_repr = custom_eq_hash + # setup() needs to be called to finish this initialization + + def pickrepr(self, item_repr): + if self.custom_eq_hash: + return item_repr, item_repr + else: + return rmodel.externalvsinternal(self.rtyper, item_repr) + + def pickkeyrepr(self, key_repr): + external, internal = self.pickrepr(key_repr) + if external != internal: + internal = external + while not self.rtyper.needs_hash_support(internal.classdef): + internal = internal.rbase + return external, internal + + def compact_repr(self): + return 'DictR %s %s' % (self.key_repr.compact_repr(), self.value_repr.compact_repr()) + + def _setup_repr(self): + if 'key_repr' not in self.__dict__: + key_repr = self._key_repr_computer() + self.external_key_repr, self.key_repr = self.pickkeyrepr(key_repr) + if 'value_repr' not in self.__dict__: + self.external_value_repr, self.value_repr = self.pickrepr(self._value_repr_computer()) + if isinstance(self.DICT, lltype.GcForwardReference): + self.DICTKEY = self.key_repr.lowleveltype + self.DICTVALUE = self.value_repr.lowleveltype + + # compute the shape of the DICTENTRY structure + entryfields = [] + entrymeths = { + 'must_clear_key': (isinstance(self.DICTKEY, lltype.Ptr) + and self.DICTKEY._needsgc()), + 'must_clear_value': (isinstance(self.DICTVALUE, lltype.Ptr) + and self.DICTVALUE._needsgc()), + } + + # * the key + entryfields.append(("key", self.DICTKEY)) + + # * if NULL is not a valid ll value for the key or the value + # field of the entry, it can be used as a marker for + # never-used entries. Otherwise, we need an explicit flag. + s_key = self.dictkey.s_value + s_value = self.dictvalue.s_value + nullkeymarker = not self.key_repr.can_ll_be_null(s_key) + nullvaluemarker = not self.value_repr.can_ll_be_null(s_value) + + if nullkeymarker: + entrymeths['everused'] = ll_everused_from_key + elif nullvaluemarker: + entrymeths['everused'] = ll_everused_from_value + else: + entryfields.append(("f_everused", lltype.Bool)) + entrymeths['everused'] = ll_everused_from_flag + + # * if the key or the value can also contain a "dummy" non-null + # marker, we use it for deleted entries. + rtyper = self.rtyper + dummy_obj = self.key_repr.get_ll_dummyval_obj(rtyper, s_key) + if dummy_obj: + entrymeths['dummy_obj'] = dummy_obj + entrymeths['valid'] = ll_valid_from_key + entrymeths['mark_deleted'] = ll_mark_deleted_in_key + # the key is overwritten by 'dummy' when the entry is deleted + entrymeths['must_clear_key'] = False + else: + dummy_obj = self.value_repr.get_ll_dummyval_obj(rtyper, + s_value) + if dummy_obj: + entrymeths['dummy_obj'] = dummy_obj + entrymeths['valid'] = ll_valid_from_value + entrymeths['mark_deleted'] = ll_mark_deleted_in_value + # value is overwritten by 'dummy' when entry is deleted + entrymeths['must_clear_value'] = False + else: + entryfields.append(("f_valid", lltype.Bool)) + entrymeths['valid'] = ll_valid_from_flag + entrymeths['mark_deleted'] = ll_mark_deleted_in_flag + + # * the value + entryfields.append(("value", self.DICTVALUE)) + + # * the hash, if needed + if self.custom_eq_hash: + fasthashfn = None + else: + fasthashfn = self.key_repr.get_ll_fasthash_function() + if fasthashfn is None: + entryfields.append(("f_hash", lltype.Signed)) + entrymeths['hash'] = ll_hash_from_cache + else: + entrymeths['hash'] = ll_hash_recomputed + entrymeths['fasthashfn'] = fasthashfn + + # Build the lltype data structures + self.DICTENTRY = lltype.Struct("dictentry", adtmeths=entrymeths, + *entryfields) + self.DICTENTRYARRAY = lltype.GcArray(self.DICTENTRY) + fields = [ ("num_items", lltype.Signed), + ("num_pristine_entries", lltype.Signed), + ("entries", lltype.Ptr(self.DICTENTRYARRAY)) ] + if self.custom_eq_hash: + self.r_rdict_eqfn, self.r_rdict_hashfn = self._custom_eq_hash_repr() + fields.extend([ ("fnkeyeq", self.r_rdict_eqfn.lowleveltype), + ("fnkeyhash", self.r_rdict_hashfn.lowleveltype) ]) + adtmeths = { + 'keyhash': ll_keyhash_custom, + 'keyeq': ll_keyeq_custom, + 'r_rdict_eqfn': self.r_rdict_eqfn, + 'r_rdict_hashfn': self.r_rdict_hashfn, + 'paranoia': True, + } + else: + # figure out which functions must be used to hash and compare + ll_keyhash = self.key_repr.get_ll_hash_function() + ll_keyeq = self.key_repr.get_ll_eq_function() # can be None + ll_keyhash = lltype.staticAdtMethod(ll_keyhash) + if ll_keyeq is not None: + ll_keyeq = lltype.staticAdtMethod(ll_keyeq) + adtmeths = { + 'keyhash': ll_keyhash, + 'keyeq': ll_keyeq, + 'paranoia': False, + } + self.DICT.become(lltype.GcStruct("dicttable", adtmeths=adtmeths, + *fields)) + + def recast_value(self, llops, v): + return llops.convertvar(v, self.value_repr, self.external_value_repr) + + def recast_key(self, llops, v): + return llops.convertvar(v, self.key_repr, self.external_key_repr) + + def convert_const(self, dictobj): + # get object from bound dict methods + #dictobj = getattr(dictobj, '__self__', dictobj) + if dictobj is None: + return lltype.nullptr(self.DICT) + if not isinstance(dictobj, (dict, objectmodel.r_dict)): + raise TyperError("expected a dict: %r" % (dictobj,)) + try: + key = Constant(dictobj) + return self.dict_cache[key] + except KeyError: + self.setup() + l_dict = ll_newdict_size(self.DICT, len(dictobj)) + self.dict_cache[key] = l_dict + r_key = self.key_repr + r_value = self.value_repr + if isinstance(dictobj, objectmodel.r_dict): + if self.r_rdict_eqfn.lowleveltype != lltype.Void: + l_fn = self.r_rdict_eqfn.convert_const(dictobj.key_eq) + l_dict.fnkeyeq = l_fn + if self.r_rdict_hashfn.lowleveltype != lltype.Void: + l_fn = self.r_rdict_hashfn.convert_const(dictobj.key_hash) + l_dict.fnkeyhash = l_fn + + for dictkeycontainer, dictvalue in dictobj._dict.items(): + llkey = r_key.convert_const(dictkeycontainer.key) + llvalue = r_value.convert_const(dictvalue) + ll_dict_insertclean(l_dict, llkey, llvalue, + dictkeycontainer.hash) + return l_dict + + else: + for dictkey, dictvalue in dictobj.items(): + llkey = r_key.convert_const(dictkey) + llvalue = r_value.convert_const(dictvalue) + ll_dict_insertclean(l_dict, llkey, llvalue, + l_dict.keyhash(llkey)) + return l_dict + + def rtype_len(self, hop): + v_dict, = hop.inputargs(self) + return hop.gendirectcall(ll_dict_len, v_dict) + + def rtype_is_true(self, hop): + v_dict, = hop.inputargs(self) + return hop.gendirectcall(ll_dict_is_true, v_dict) + + def make_iterator_repr(self, *variant): + return DictIteratorRepr(self, *variant) + + def rtype_method_get(self, hop): + v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, + self.value_repr) + hop.exception_cannot_occur() + v_res = hop.gendirectcall(ll_get, v_dict, v_key, v_default) + return self.recast_value(hop.llops, v_res) + + def rtype_method_setdefault(self, hop): + v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, + self.value_repr) + hop.exception_cannot_occur() + v_res = hop.gendirectcall(ll_setdefault, v_dict, v_key, v_default) + return self.recast_value(hop.llops, v_res) + + def rtype_method_copy(self, hop): + v_dict, = hop.inputargs(self) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_copy, v_dict) + + def rtype_method_update(self, hop): + v_dic1, v_dic2 = hop.inputargs(self, self) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_update, v_dic1, v_dic2) + + def _rtype_method_kvi(self, hop, spec): + v_dic, = hop.inputargs(self) + r_list = hop.r_result + v_func = hop.inputconst(lltype.Void, spec) + cLIST = hop.inputconst(lltype.Void, r_list.lowleveltype.TO) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_kvi, v_dic, cLIST, v_func) + + def rtype_method_keys(self, hop): + return self._rtype_method_kvi(hop, dum_keys) + + def rtype_method_values(self, hop): + return self._rtype_method_kvi(hop, dum_values) + + def rtype_method_items(self, hop): + return self._rtype_method_kvi(hop, dum_items) + + def rtype_method_iterkeys(self, hop): + hop.exception_cannot_occur() + return DictIteratorRepr(self, "keys").newiter(hop) + + def rtype_method_itervalues(self, hop): + hop.exception_cannot_occur() + return DictIteratorRepr(self, "values").newiter(hop) + + def rtype_method_iteritems(self, hop): + hop.exception_cannot_occur() + return DictIteratorRepr(self, "items").newiter(hop) + + def rtype_method_clear(self, hop): + v_dict, = hop.inputargs(self) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_clear, v_dict) + +class __extend__(pairtype(DictRepr, rmodel.Repr)): + + def rtype_getitem((r_dict, r_key), hop): + v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + if not r_dict.custom_eq_hash: + hop.has_implicit_exception(KeyError) # record that we know about it + hop.exception_is_here() + v_res = hop.gendirectcall(ll_dict_getitem, v_dict, v_key) + return r_dict.recast_value(hop.llops, v_res) + + def rtype_delitem((r_dict, r_key), hop): + v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + if not r_dict.custom_eq_hash: + hop.has_implicit_exception(KeyError) # record that we know about it + hop.exception_is_here() + return hop.gendirectcall(ll_dict_delitem, v_dict, v_key) + + def rtype_setitem((r_dict, r_key), hop): + v_dict, v_key, v_value = hop.inputargs(r_dict, r_dict.key_repr, r_dict.value_repr) + if r_dict.custom_eq_hash: + hop.exception_is_here() + else: + hop.exception_cannot_occur() + hop.gendirectcall(ll_dict_setitem, v_dict, v_key, v_value) + + def rtype_contains((r_dict, r_key), hop): + v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + return hop.gendirectcall(ll_contains, v_dict, v_key) + +class __extend__(pairtype(DictRepr, DictRepr)): + def convert_from_to((r_dict1, r_dict2), v, llops): + # check that we don't convert from Dicts with + # different key/value types + if r_dict1.dictkey is None or r_dict2.dictkey is None: + return NotImplemented + if r_dict1.dictkey is not r_dict2.dictkey: + return NotImplemented + if r_dict1.dictvalue is None or r_dict2.dictvalue is None: + return NotImplemented + if r_dict1.dictvalue is not r_dict2.dictvalue: + return NotImplemented + return v + +# ____________________________________________________________ +# +# Low-level methods. These can be run for testing, but are meant to +# be direct_call'ed from rtyped flow graphs, which means that they will +# get flowed and annotated, mostly with SomePtr. + +def ll_everused_from_flag(entry): + return entry.f_everused + +def ll_everused_from_key(entry): + return bool(entry.key) + +def ll_everused_from_value(entry): + return bool(entry.value) + +def ll_valid_from_flag(entry): + return entry.f_valid + +def ll_mark_deleted_in_flag(entry): + entry.f_valid = False + +def ll_valid_from_key(entry): + ENTRY = lltype.typeOf(entry).TO + dummy = ENTRY.dummy_obj.ll_dummy_value + return entry.everused() and entry.key != dummy + +def ll_mark_deleted_in_key(entry): + ENTRY = lltype.typeOf(entry).TO + dummy = ENTRY.dummy_obj.ll_dummy_value + entry.key = dummy + +def ll_valid_from_value(entry): + ENTRY = lltype.typeOf(entry).TO + dummy = ENTRY.dummy_obj.ll_dummy_value + return entry.everused() and entry.value != dummy + +def ll_mark_deleted_in_value(entry): + ENTRY = lltype.typeOf(entry).TO + dummy = ENTRY.dummy_obj.ll_dummy_value + entry.value = dummy + +def ll_hash_from_cache(entry): + return entry.f_hash + +def ll_hash_recomputed(entry): + ENTRY = lltype.typeOf(entry).TO + return ENTRY.fasthashfn(entry.key) + +def ll_keyhash_custom(d, key): + DICT = lltype.typeOf(d).TO + return hlinvoke(DICT.r_rdict_hashfn, d.fnkeyhash, key) + +def ll_keyeq_custom(d, key1, key2): + DICT = lltype.typeOf(d).TO + return hlinvoke(DICT.r_rdict_eqfn, d.fnkeyeq, key1, key2) + +def dum_keys(): pass +def dum_values(): pass +def dum_items():pass +dum_variant = {"keys": dum_keys, + "values": dum_values, + "items": dum_items} + +def ll_dict_len(d): + return d.num_items + +def ll_dict_is_true(d): + # check if a dict is True, allowing for None + return bool(d) and d.num_items != 0 + +def ll_dict_getitem(d, key): + entry = ll_dict_lookup(d, key, d.keyhash(key)) + if entry.valid(): + return entry.value + else: + raise KeyError + +def ll_dict_setitem(d, key, value): + hash = d.keyhash(key) + entry = ll_dict_lookup(d, key, hash) + everused = entry.everused() + valid = entry.valid() + # set up the new entry + ENTRY = lltype.typeOf(entry).TO + entry.value = value + if valid: + return + entry.key = key + if hasattr(ENTRY, 'f_hash'): entry.f_hash = hash + if hasattr(ENTRY, 'f_valid'): entry.f_valid = True + d.num_items += 1 + if not everused: + if hasattr(ENTRY, 'f_everused'): entry.f_everused = True + d.num_pristine_entries -= 1 + if d.num_pristine_entries <= len(d.entries) / 3: + ll_dict_resize(d) + +def ll_dict_insertclean(d, key, value, hash): + # Internal routine used by ll_dict_resize() to insert an item which is + # known to be absent from the dict. This routine also assumes that + # the dict contains no deleted entries. This routine has the advantage + # of never calling d.keyhash() and d.keyeq(), so it cannot call back + # to user code. ll_dict_insertclean() doesn't resize the dict, either. + entry = ll_dict_lookup_clean(d, hash) + ENTRY = lltype.typeOf(entry).TO + entry.value = value + entry.key = key + if hasattr(ENTRY, 'f_hash'): entry.f_hash = hash + if hasattr(ENTRY, 'f_valid'): entry.f_valid = True + if hasattr(ENTRY, 'f_everused'): entry.f_everused = True + d.num_items += 1 + d.num_pristine_entries -= 1 + +def ll_dict_delitem(d, key): + entry = ll_dict_lookup(d, key, d.keyhash(key)) + if not entry.valid(): + raise KeyError + entry.mark_deleted() + d.num_items -= 1 + # clear the key and the value if they are GC pointers + ENTRY = lltype.typeOf(entry).TO + if ENTRY.must_clear_key: + key = entry.key # careful about destructor side effects: + # keep key alive until entry.value has also + # been zeroed (if it must be) + entry.key = lltype.nullptr(ENTRY.key.TO) + if ENTRY.must_clear_value: + entry.value = lltype.nullptr(ENTRY.value.TO) + num_entries = len(d.entries) + if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4: + ll_dict_resize(d) + +def ll_dict_resize(d): + old_entries = d.entries + old_size = len(old_entries) + # make a 'new_size' estimate and shrink it if there are many + # deleted entry markers + new_size = old_size * 2 + while new_size > DICT_INITSIZE and d.num_items < new_size / 4: + new_size /= 2 + d.entries = lltype.malloc(lltype.typeOf(old_entries).TO, new_size) + d.num_items = 0 + d.num_pristine_entries = new_size + i = 0 + while i < old_size: + entry = old_entries[i] + if entry.valid(): + ll_dict_insertclean(d, entry.key, entry.value, entry.hash()) + i += 1 + +# ------- a port of CPython's dictobject.c's lookdict implementation ------- +PERTURB_SHIFT = 5 + +def ll_dict_lookup(d, key, hash): + DICT = lltype.typeOf(d).TO + entries = d.entries + mask = len(entries) - 1 + i = r_uint(hash & mask) + # do the first try before any looping + entry = entries[i] + if entry.valid(): + checkingkey = entry.key + if checkingkey == key: + return entry # found the entry + if d.keyeq is not None and entry.hash() == hash: + # correct hash, maybe the key is e.g. a different pointer to + # an equal object + found = d.keyeq(checkingkey, key) + if DICT.paranoia: + if (entries != d.entries or + not entry.valid() or entry.key != checkingkey): + # the compare did major nasty stuff to the dict: start over + return ll_dict_lookup(d, key, hash) + if found: + return entry # found the entry + freeslot = lltype.nullptr(lltype.typeOf(entry).TO) + elif entry.everused(): + freeslot = entry + else: + return entry # pristine entry -- lookup failed + + # In the loop, a deleted entry (everused and not valid) is by far + # (factor of 100s) the least likely outcome, so test for that last. + perturb = r_uint(hash) + while 1: + i = ((i << 2) + i + perturb + 1) & mask + entry = entries[i] + if not entry.everused(): + return freeslot or entry + elif entry.valid(): + checkingkey = entry.key + if checkingkey == key: + return entry + if d.keyeq is not None and entry.hash() == hash: + # correct hash, maybe the key is e.g. a different pointer to + # an equal object + found = d.keyeq(checkingkey, key) + if DICT.paranoia: + if (entries != d.entries or + not entry.valid() or entry.key != checkingkey): + # the compare did major nasty stuff to the dict: + # start over + return ll_dict_lookup(d, key, hash) + if found: + return entry # found the entry + elif not freeslot: + freeslot = entry + perturb >>= PERTURB_SHIFT + +def ll_dict_lookup_clean(d, hash): + # a simplified version of ll_dict_lookup() which assumes that the + # key is new, and the dictionary doesn't contain deleted entries. + # It only find the next free slot for the given hash. + entries = d.entries + mask = len(entries) - 1 + i = r_uint(hash & mask) + entry = entries[i] + perturb = r_uint(hash) + while entry.everused(): + i = ((i << 2) + i + perturb + 1) & mask + entry = entries[i] + perturb >>= PERTURB_SHIFT + return entry + +# ____________________________________________________________ +# +# Irregular operations. + +DICT_INITSIZE = 8 + +def ll_newdict(DICT): + d = lltype.malloc(DICT) + d.entries = lltype.malloc(DICT.entries.TO, DICT_INITSIZE) + #d.num_items = 0 -- defaults + d.num_pristine_entries = DICT_INITSIZE + return d + +def ll_newdict_size(DICT, length_estimate): + length_estimate = (length_estimate // 2) * 3 + n = DICT_INITSIZE + while n < length_estimate: + n *= 2 + d = lltype.malloc(DICT) + d.entries = lltype.malloc(DICT.entries.TO, n) + #d.num_items = 0 -- defaults + d.num_pristine_entries = DICT_INITSIZE + return d + + +def rtype_r_dict(hop): + r_dict = hop.r_result + if not r_dict.custom_eq_hash: + raise TyperError("r_dict() call does not return an r_dict instance") + v_eqfn, v_hashfn = hop.inputargs(r_dict.r_rdict_eqfn, + r_dict.r_rdict_hashfn) + cDICT = hop.inputconst(lltype.Void, r_dict.DICT) + hop.exception_cannot_occur() + v_result = hop.gendirectcall(ll_newdict, cDICT) + if r_dict.r_rdict_eqfn.lowleveltype != lltype.Void: + cname = hop.inputconst(lltype.Void, 'fnkeyeq') + hop.genop('setfield', [v_result, cname, v_eqfn]) + if r_dict.r_rdict_hashfn.lowleveltype != lltype.Void: + cname = hop.inputconst(lltype.Void, 'fnkeyhash') + hop.genop('setfield', [v_result, cname, v_hashfn]) + return v_result + +# ____________________________________________________________ +# +# Iteration. + +class DictIteratorRepr(rmodel.IteratorRepr): + + def __init__(self, r_dict, variant="keys"): + self.r_dict = r_dict + self.variant = variant + self.lowleveltype = lltype.Ptr(lltype.GcStruct('dictiter', + ('dict', r_dict.lowleveltype), + ('index', lltype.Signed))) + + def newiter(self, hop): + v_dict, = hop.inputargs(self.r_dict) + citerptr = hop.inputconst(lltype.Void, self.lowleveltype) + return hop.gendirectcall(ll_dictiter, citerptr, v_dict) + + def rtype_next(self, hop): + variant = self.variant + v_iter, = hop.inputargs(self) + v_func = hop.inputconst(lltype.Void, dum_variant[self.variant]) + if variant in ('keys', 'values'): + c1 = hop.inputconst(lltype.Void, None) + else: + c1 = hop.inputconst(lltype.Void, hop.r_result.lowleveltype) + hop.has_implicit_exception(StopIteration) # record that we know about it + hop.exception_is_here() + v = hop.gendirectcall(ll_dictnext, v_iter, v_func, c1) + if variant == 'keys': + return self.r_dict.recast_key(hop.llops, v) + elif variant == 'values': + return self.r_dict.recast_value(hop.llops, v) + else: + return v + +def ll_dictiter(ITERPTR, d): + iter = lltype.malloc(ITERPTR.TO) + iter.dict = d + iter.index = 0 + return iter + +def ll_dictnext(iter, func, RETURNTYPE): + dict = iter.dict + if dict: + entries = dict.entries + index = iter.index + entries_len = len(entries) + while index < entries_len: + entry = entries[index] + index = index + 1 + if entry.valid(): + iter.index = index + if func is dum_items: + r = lltype.malloc(RETURNTYPE.TO) + r.item0 = recast(RETURNTYPE.TO.item0, entry.key) + r.item1 = recast(RETURNTYPE.TO.item1, entry.value) + return r + elif func is dum_keys: + return entry.key + elif func is dum_values: + return entry.value + # clear the reference to the dict and prevent restarts + iter.dict = lltype.nullptr(lltype.typeOf(iter).TO.dict.TO) + raise StopIteration + +# _____________________________________________________________ +# methods + +def ll_get(dict, key, default): + entry = ll_dict_lookup(dict, key, dict.keyhash(key)) + if entry.valid(): + return entry.value + else: + return default + +def ll_setdefault(dict, key, default): + entry = ll_dict_lookup(dict, key, dict.keyhash(key)) + if entry.valid(): + return entry.value + else: + ll_dict_setitem(dict, key, default) + return default + +def ll_copy(dict): + DICT = lltype.typeOf(dict).TO + dictsize = len(dict.entries) + d = lltype.malloc(DICT) + d.entries = lltype.malloc(DICT.entries.TO, dictsize) + d.num_items = dict.num_items + d.num_pristine_entries = dict.num_pristine_entries + if hasattr(DICT, 'fnkeyeq'): d.fnkeyeq = dict.fnkeyeq + if hasattr(DICT, 'fnkeyhash'): d.fnkeyhash = dict.fnkeyhash + i = 0 + while i < dictsize: + d_entry = d.entries[i] + entry = dict.entries[i] + ENTRY = lltype.typeOf(entry).TO + d_entry.key = entry.key + if hasattr(ENTRY, 'f_valid'): d_entry.f_valid = entry.f_valid + if hasattr(ENTRY, 'f_everused'): d_entry.f_everused = entry.f_everused + d_entry.value = entry.value + if hasattr(ENTRY, 'f_hash'): d_entry.f_hash = entry.f_hash + i += 1 + return d + +def ll_clear(d): + if len(d.entries) == d.num_pristine_entries == DICT_INITSIZE: + return + DICT = lltype.typeOf(d).TO + d.entries = lltype.malloc(DICT.entries.TO, DICT_INITSIZE) + d.num_items = 0 + d.num_pristine_entries = DICT_INITSIZE + +def ll_update(dic1, dic2): + entries = dic2.entries + d2len = len(entries) + i = 0 + while i < d2len: + entry = entries[i] + if entry.valid(): + ll_dict_setitem(dic1, entry.key, entry.value) + i += 1 + +# this is an implementation of keys(), values() and items() +# in a single function. +# note that by specialization on func, three different +# and very efficient functions are created. + +def recast(P, v): + if isinstance(P, lltype.Ptr): + return lltype.cast_pointer(P, v) + else: + return v + +def ll_kvi(dic, LIST, func): + res = LIST.ll_newlist(dic.num_items) + entries = dic.entries + dlen = len(entries) + items = res.ll_items() + i = 0 + p = 0 + while i < dlen: + entry = entries[i] + if entry.valid(): + ELEM = lltype.typeOf(items).TO.OF + if func is dum_items: + r = lltype.malloc(ELEM.TO) + r.item0 = recast(ELEM.TO.item0, entry.key) + r.item1 = recast(ELEM.TO.item1, entry.value) + items[p] = r + elif func is dum_keys: + items[p] = recast(ELEM, entry.key) + elif func is dum_values: + items[p] = recast(ELEM, entry.value) + p += 1 + i += 1 + return res + +def ll_contains(d, key): + entry = ll_dict_lookup(d, key, d.keyhash(key)) + return entry.valid() Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Fri Apr 21 00:17:24 2006 @@ -49,15 +49,6 @@ self.listitem = listitem self.list_cache = {} # setup() needs to be called to finish this initialization -## self.ll_concat = ll_concat -## self.ll_extend = ll_extend -## self.ll_listslice_startonly = ll_listslice_startonly -## self.ll_listslice = ll_listslice -## self.ll_listslice_minusone = ll_listslice_minusone -## self.ll_listsetslice = ll_listsetslice -## self.ll_listdelslice_startonly = ll_listdelslice_startonly -## self.ll_listdelslice = ll_listdelslice -## self.ll_listindex = ll_listindex self.list_builder = ListBuilder() def _setup_repr_final(self): Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 21 00:17:24 2006 @@ -181,8 +181,38 @@ StaticMethod.__init__(self, args, result) -class List(OOType): - # placeholders +class BuiltinType(OOType): + + def _setup_methods(self, generic_types): + methods = {} + for name, meth in self._GENERIC_METHODS.iteritems(): + args = [generic_types.get(arg, arg) for arg in meth.ARGS] + result = generic_types.get(meth.RESULT, meth.RESULT) + methods[name] = Meth(args, result) + self._METHODS = frozendict(methods) + + def _lookup(self, meth_name): + METH = self._METHODS.get(meth_name) + meth = None + if METH is not None: + cls = self._get_interp_class() + meth = _meth(METH, _name=meth_name, _callable=getattr(cls, meth_name)) + return self, meth + + def _example(self): + return new(self) + + def _defl(self): + return self._null + + def _get_interp_class(self): + raise NotImplementedError + + +class List(BuiltinType): + # placeholders for types + # make sure that each derived class has his own SELFTYPE_T + # placeholder, because we want backends to distinguish that. SELFTYPE_T = object() ITEMTYPE_T = object() @@ -209,9 +239,6 @@ "_ll_resize_ge": Meth([Signed], Void), "_ll_resize_le": Meth([Signed], Void), "_ll_resize": Meth([Signed], Void), -## "append": Meth([self.ITEMTYPE_T], Void), -## "extend": Meth([self.SELFTYPE_T], Void), -## "remove_range": Meth([Signed, Signed], Void), # remove_range(start, count) }) self._setup_methods(generic_types) @@ -223,14 +250,6 @@ lst._ll_resize_ge(length) return lst - def _setup_methods(self, generic_types): - methods = {} - for name, meth in self._GENERIC_METHODS.iteritems(): - args = [generic_types.get(arg, arg) for arg in meth.ARGS] - result = generic_types.get(meth.RESULT, meth.RESULT) - methods[name] = Meth(args, result) - self._METHODS = frozendict(methods) - # NB: We are expecting Lists of the same ITEMTYPE to compare/hash # equal. We don't redefine __eq__/__hash__ since the implementations # from LowLevelType work fine, especially in the face of recursive @@ -241,19 +260,48 @@ return '%s(%s)' % (self.__class__.__name__, saferecursive(str, "...")(self._ITEMTYPE)) - def _lookup(self, meth_name): - METH = self._METHODS.get(meth_name) - meth = None - if METH is not None: - meth = _meth(METH, _name=meth_name, - _callable=getattr(_list, meth_name)) - return self, meth + def _get_interp_class(self): + return _list - def _example(self): - return new(self) - def _defl(self): - return self._null +class Dict(BuiltinType): + # placeholders for types + SELFTYPE_T = object() + KEYTYPE_T = object() + VALUETYPE_T = object() + + def __init__(self, KEYTYPE, VALUETYPE): + self._KEYTYPE = KEYTYPE + self._VALUETYPE = VALUETYPE + self._null = _null_dict(self) + + generic_types = { + self.SELFTYPE_T: self, + self.KEYTYPE_T: KEYTYPE, + self.VALUETYPE_T: VALUETYPE + } + + self._GENERIC_METHODS = frozendict({ + "ll_length": Meth([], Signed), + "ll_getitem": Meth([self.KEYTYPE_T], self.VALUETYPE_T), + "ll_setitem": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), + }) + + self._setup_methods(generic_types) + + # NB: We are expecting Dicts of the same KEYTYPE, VALUETYPE to + # compare/hash equal. We don't redefine __eq__/__hash__ since the + # implementations from LowLevelType work fine, especially in the + # face of recursive data structures. But it is important to make + # sure that attributes of supposedly equal Dicts compare/hash + # equal. + + def __str__(self): + return '%s(%s)' % (self.__class__.__name__, + saferecursive(str, "...")(self._KEYTYPE, self._VALUETYPE)) + + def _get_interp_class(self): + return _dict class ForwardReference(OOType): @@ -523,13 +571,7 @@ callb, checked_args = self.meth._checkargs(args) return callb(self.inst, *checked_args) - -class _list(object): - - def __init__(self, LIST): - self._TYPE = LIST - self._list = [] - +class _builtin_type(object): def __getattribute__(self, name): TYPE = object.__getattribute__(self, "_TYPE") _, meth = TYPE._lookup(name) @@ -537,6 +579,13 @@ return meth._bound(TYPE, self) return object.__getattribute__(self, name) + + +class _list(_builtin_type): + + def __init__(self, LIST): + self._TYPE = LIST + self._list = [] # The following are implementations of the abstract list interface for # use by the llinterpreter and ootype tests. There are NOT_RPYTHON @@ -580,30 +629,40 @@ assert index >= 0 self._list[index] = item -## def append(self, item): -## # NOT_RPYTHON -## assert typeOf(item) == self._TYPE._ITEMTYPE -## self._list.append(item) - -## def extend(self, other): -## # NOT_RPYTHON -## assert typeOf(other) == typeOf(self) -## self._list.extend(other._list) - -## def remove_range(self, start, count): -## # NOT_RPYTHON -## del self._list[start:start+count] - class _null_list(_null_mixin(_list), _list): def __init__(self, LIST): self.__dict__["_TYPE"] = LIST +class _dict(object): + def __init__(self, DICT): + self._TYPE = DICT + self._dict = {} + + def ll_length(self): + # NOT_RPYTHON + return len(self._dict) + + def ll_getitem(self, key): + assert typeOf(key) == self._TYPE._KEYTYPE + return self._dict[key] + + def ll_setitem(self, key, value): + assert typeOf(key) == self._TYPE._KEYTYPE + assert typeOf(value) == self._TYPE._VALUETYPE + self._dict[key] = value + +class _null_dict(_null_mixin(_dict), _dict): + + def __init__(self, DICT): + self.__dict__["_TYPE"] = DICT + + def new(TYPE): if isinstance(TYPE, Instance): return make_instance(TYPE) - elif isinstance(TYPE, List): - return _list(TYPE) + elif isinstance(TYPE, BuiltinType): + return TYPE._get_interp_class()(TYPE) def runtimenew(class_): assert isinstance(class_, _class) Added: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Fri Apr 21 00:17:24 2006 @@ -0,0 +1 @@ +# TODO Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Fri Apr 21 00:17:24 2006 @@ -25,15 +25,6 @@ self.lowleveltype = self.LIST self.listitem = listitem self.list_cache = {} -## self.ll_concat = ll_concat -## self.ll_extend = ll_extend -## self.ll_listslice_startonly = ll_listslice_startonly -## self.ll_listslice = ll_listslice -## self.ll_listslice_minusone = ll_listslice_minusone -## self.ll_listsetslice = ll_listsetslice -## self.ll_listdelslice_startonly = ll_listdelslice_startonly -## self.ll_listdelslice = ll_listdelslice -## self.ll_listindex = ll_listindex # setup() needs to be called to finish this initialization def _setup_repr(self): Added: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Fri Apr 21 00:17:24 2006 @@ -0,0 +1,20 @@ +import py +from pypy.rpython.test.test_llinterp import interpret +from pypy.rpython.ootypesystem.ootype import Signed, Float, Dict, new, typeOf + +def test_new(): + DT = Dict(Signed, Float) + d = new(DT) + assert typeOf(d) == DT + +def test_length(): + DT = Dict(Signed, Float) + d = new(DT) + d.ll_setitem(42, 123.45) + assert d.ll_length() == 1 + +def test_setitem_getitem(): + DT = Dict(Signed, Float) + d = new(DT) + d.ll_setitem(42, 123.45) + assert d.ll_getitem(42) == 123.45 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Fri Apr 21 00:17:24 2006 @@ -31,20 +31,6 @@ lst._ll_resize_le(10) assert lst.ll_length() <= 10 -##def test_append(): -## LT = List(Signed) -## l = new(LT) -## l.append(1) -## assert l.ll_length() == 1 - -##def test_extend(): -## LT = List(Signed) -## l1 = new(LT) -## l2 = new(LT) -## l1.append(1) -## l2.append(2) -## l1.extend(l2) -## assert l1.ll_length() == 2 def test_setitem_getitem(): LT = List(Signed) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Fri Apr 21 00:17:24 2006 @@ -9,7 +9,7 @@ from pypy.rpython import rstr from pypy.rpython import rptr from pypy.rpython.robject import pyobj_repr -from pypy.rpython.rdict import rtype_r_dict +from pypy.rpython.lltypesystem.rdict import rtype_r_dict # TODO: typesystem? from pypy.tool import sourcetools from pypy.rpython import extregistry Modified: pypy/dist/pypy/rpython/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/rdict.py (original) +++ pypy/dist/pypy/rpython/rdict.py Fri Apr 21 00:17:24 2006 @@ -6,33 +6,8 @@ from pypy.rpython.objectmodel import hlinvoke from pypy.rpython import robject from pypy.rpython import objectmodel -from pypy.rpython import rmodel +from pypy.rpython.rmodel import Repr -# ____________________________________________________________ -# -# generic implementation of RPython dictionary, with parametric DICTKEY and -# DICTVALUE types. -# -# XXX for immutable dicts, the array should be inlined and -# num_pristine_entries and everused are not needed. -# -# struct dictentry { -# DICTKEY key; -# bool f_valid; # (optional) the entry is filled -# bool f_everused; # (optional) the entry is or has ever been filled -# DICTVALUE value; -# int f_hash; # (optional) key hash, if hard to recompute -# } -# -# struct dicttable { -# int num_items; -# int num_pristine_entries; # never used entries -# Array *entries; -# (Function DICTKEY, DICTKEY -> bool) *fnkeyeq; -# (Function DICTKEY -> int) *fnkeyhash; -# } -# -# class __extend__(annmodel.SomeDict): def rtyper_makerepr(self, rtyper): @@ -49,572 +24,21 @@ rtyper.getrepr(dictkey.s_rdict_hashfn)) else: custom_eq_hash = None - return DictRepr(rtyper, - lambda: rtyper.getrepr(s_key), - lambda: rtyper.getrepr(s_value), - dictkey, - dictvalue, - custom_eq_hash) + return rtyper.type_system.rdict.DictRepr(rtyper, + lambda: rtyper.getrepr(s_key), + lambda: rtyper.getrepr(s_value), + dictkey, + dictvalue, + custom_eq_hash) def rtyper_makekey(self): return (self.__class__, self.dictdef.dictkey, self.dictdef.dictvalue) -class DictRepr(rmodel.Repr): - def __init__(self, rtyper, key_repr, value_repr, dictkey, dictvalue, - custom_eq_hash=None): - self.rtyper = rtyper - self.DICT = lltype.GcForwardReference() - self.lowleveltype = lltype.Ptr(self.DICT) - self.custom_eq_hash = custom_eq_hash is not None - if not isinstance(key_repr, rmodel.Repr): # not computed yet, done by setup() - assert callable(key_repr) - self._key_repr_computer = key_repr - else: - self.external_key_repr, self.key_repr = self.pickkeyrepr(key_repr) - if not isinstance(value_repr, rmodel.Repr): # not computed yet, done by setup() - assert callable(value_repr) - self._value_repr_computer = value_repr - else: - self.external_value_repr, self.value_repr = self.pickrepr(value_repr) - self.dictkey = dictkey - self.dictvalue = dictvalue - self.dict_cache = {} - self._custom_eq_hash_repr = custom_eq_hash - # setup() needs to be called to finish this initialization - - def pickrepr(self, item_repr): - if self.custom_eq_hash: - return item_repr, item_repr - else: - return rmodel.externalvsinternal(self.rtyper, item_repr) - - def pickkeyrepr(self, key_repr): - external, internal = self.pickrepr(key_repr) - if external != internal: - internal = external - while not self.rtyper.needs_hash_support(internal.classdef): - internal = internal.rbase - return external, internal - - def compact_repr(self): - return 'DictR %s %s' % (self.key_repr.compact_repr(), self.value_repr.compact_repr()) - - def _setup_repr(self): - if 'key_repr' not in self.__dict__: - key_repr = self._key_repr_computer() - self.external_key_repr, self.key_repr = self.pickkeyrepr(key_repr) - if 'value_repr' not in self.__dict__: - self.external_value_repr, self.value_repr = self.pickrepr(self._value_repr_computer()) - if isinstance(self.DICT, lltype.GcForwardReference): - self.DICTKEY = self.key_repr.lowleveltype - self.DICTVALUE = self.value_repr.lowleveltype - - # compute the shape of the DICTENTRY structure - entryfields = [] - entrymeths = { - 'must_clear_key': (isinstance(self.DICTKEY, lltype.Ptr) - and self.DICTKEY._needsgc()), - 'must_clear_value': (isinstance(self.DICTVALUE, lltype.Ptr) - and self.DICTVALUE._needsgc()), - } - - # * the key - entryfields.append(("key", self.DICTKEY)) - - # * if NULL is not a valid ll value for the key or the value - # field of the entry, it can be used as a marker for - # never-used entries. Otherwise, we need an explicit flag. - s_key = self.dictkey.s_value - s_value = self.dictvalue.s_value - nullkeymarker = not self.key_repr.can_ll_be_null(s_key) - nullvaluemarker = not self.value_repr.can_ll_be_null(s_value) - - if nullkeymarker: - entrymeths['everused'] = ll_everused_from_key - elif nullvaluemarker: - entrymeths['everused'] = ll_everused_from_value - else: - entryfields.append(("f_everused", lltype.Bool)) - entrymeths['everused'] = ll_everused_from_flag - - # * if the key or the value can also contain a "dummy" non-null - # marker, we use it for deleted entries. - rtyper = self.rtyper - dummy_obj = self.key_repr.get_ll_dummyval_obj(rtyper, s_key) - if dummy_obj: - entrymeths['dummy_obj'] = dummy_obj - entrymeths['valid'] = ll_valid_from_key - entrymeths['mark_deleted'] = ll_mark_deleted_in_key - # the key is overwritten by 'dummy' when the entry is deleted - entrymeths['must_clear_key'] = False - else: - dummy_obj = self.value_repr.get_ll_dummyval_obj(rtyper, - s_value) - if dummy_obj: - entrymeths['dummy_obj'] = dummy_obj - entrymeths['valid'] = ll_valid_from_value - entrymeths['mark_deleted'] = ll_mark_deleted_in_value - # value is overwritten by 'dummy' when entry is deleted - entrymeths['must_clear_value'] = False - else: - entryfields.append(("f_valid", lltype.Bool)) - entrymeths['valid'] = ll_valid_from_flag - entrymeths['mark_deleted'] = ll_mark_deleted_in_flag - - # * the value - entryfields.append(("value", self.DICTVALUE)) - - # * the hash, if needed - if self.custom_eq_hash: - fasthashfn = None - else: - fasthashfn = self.key_repr.get_ll_fasthash_function() - if fasthashfn is None: - entryfields.append(("f_hash", lltype.Signed)) - entrymeths['hash'] = ll_hash_from_cache - else: - entrymeths['hash'] = ll_hash_recomputed - entrymeths['fasthashfn'] = fasthashfn - - # Build the lltype data structures - self.DICTENTRY = lltype.Struct("dictentry", adtmeths=entrymeths, - *entryfields) - self.DICTENTRYARRAY = lltype.GcArray(self.DICTENTRY) - fields = [ ("num_items", lltype.Signed), - ("num_pristine_entries", lltype.Signed), - ("entries", lltype.Ptr(self.DICTENTRYARRAY)) ] - if self.custom_eq_hash: - self.r_rdict_eqfn, self.r_rdict_hashfn = self._custom_eq_hash_repr() - fields.extend([ ("fnkeyeq", self.r_rdict_eqfn.lowleveltype), - ("fnkeyhash", self.r_rdict_hashfn.lowleveltype) ]) - adtmeths = { - 'keyhash': ll_keyhash_custom, - 'keyeq': ll_keyeq_custom, - 'r_rdict_eqfn': self.r_rdict_eqfn, - 'r_rdict_hashfn': self.r_rdict_hashfn, - 'paranoia': True, - } - else: - # figure out which functions must be used to hash and compare - ll_keyhash = self.key_repr.get_ll_hash_function() - ll_keyeq = self.key_repr.get_ll_eq_function() # can be None - ll_keyhash = lltype.staticAdtMethod(ll_keyhash) - if ll_keyeq is not None: - ll_keyeq = lltype.staticAdtMethod(ll_keyeq) - adtmeths = { - 'keyhash': ll_keyhash, - 'keyeq': ll_keyeq, - 'paranoia': False, - } - self.DICT.become(lltype.GcStruct("dicttable", adtmeths=adtmeths, - *fields)) - - def recast_value(self, llops, v): - return llops.convertvar(v, self.value_repr, self.external_value_repr) - - def recast_key(self, llops, v): - return llops.convertvar(v, self.key_repr, self.external_key_repr) - - def convert_const(self, dictobj): - # get object from bound dict methods - #dictobj = getattr(dictobj, '__self__', dictobj) - if dictobj is None: - return lltype.nullptr(self.DICT) - if not isinstance(dictobj, (dict, objectmodel.r_dict)): - raise TyperError("expected a dict: %r" % (dictobj,)) - try: - key = Constant(dictobj) - return self.dict_cache[key] - except KeyError: - self.setup() - l_dict = ll_newdict_size(self.DICT, len(dictobj)) - self.dict_cache[key] = l_dict - r_key = self.key_repr - r_value = self.value_repr - if isinstance(dictobj, objectmodel.r_dict): - if self.r_rdict_eqfn.lowleveltype != lltype.Void: - l_fn = self.r_rdict_eqfn.convert_const(dictobj.key_eq) - l_dict.fnkeyeq = l_fn - if self.r_rdict_hashfn.lowleveltype != lltype.Void: - l_fn = self.r_rdict_hashfn.convert_const(dictobj.key_hash) - l_dict.fnkeyhash = l_fn - - for dictkeycontainer, dictvalue in dictobj._dict.items(): - llkey = r_key.convert_const(dictkeycontainer.key) - llvalue = r_value.convert_const(dictvalue) - ll_dict_insertclean(l_dict, llkey, llvalue, - dictkeycontainer.hash) - return l_dict - - else: - for dictkey, dictvalue in dictobj.items(): - llkey = r_key.convert_const(dictkey) - llvalue = r_value.convert_const(dictvalue) - ll_dict_insertclean(l_dict, llkey, llvalue, - l_dict.keyhash(llkey)) - return l_dict - - def rtype_len(self, hop): - v_dict, = hop.inputargs(self) - return hop.gendirectcall(ll_dict_len, v_dict) - - def rtype_is_true(self, hop): - v_dict, = hop.inputargs(self) - return hop.gendirectcall(ll_dict_is_true, v_dict) - - def make_iterator_repr(self, *variant): - return DictIteratorRepr(self, *variant) - - def rtype_method_get(self, hop): - v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, - self.value_repr) - hop.exception_cannot_occur() - v_res = hop.gendirectcall(ll_get, v_dict, v_key, v_default) - return self.recast_value(hop.llops, v_res) - - def rtype_method_setdefault(self, hop): - v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, - self.value_repr) - hop.exception_cannot_occur() - v_res = hop.gendirectcall(ll_setdefault, v_dict, v_key, v_default) - return self.recast_value(hop.llops, v_res) - - def rtype_method_copy(self, hop): - v_dict, = hop.inputargs(self) - hop.exception_cannot_occur() - return hop.gendirectcall(ll_copy, v_dict) - - def rtype_method_update(self, hop): - v_dic1, v_dic2 = hop.inputargs(self, self) - hop.exception_cannot_occur() - return hop.gendirectcall(ll_update, v_dic1, v_dic2) - - def _rtype_method_kvi(self, hop, spec): - v_dic, = hop.inputargs(self) - r_list = hop.r_result - v_func = hop.inputconst(lltype.Void, spec) - cLIST = hop.inputconst(lltype.Void, r_list.lowleveltype.TO) - hop.exception_cannot_occur() - return hop.gendirectcall(ll_kvi, v_dic, cLIST, v_func) - - def rtype_method_keys(self, hop): - return self._rtype_method_kvi(hop, dum_keys) - - def rtype_method_values(self, hop): - return self._rtype_method_kvi(hop, dum_values) - - def rtype_method_items(self, hop): - return self._rtype_method_kvi(hop, dum_items) - - def rtype_method_iterkeys(self, hop): - hop.exception_cannot_occur() - return DictIteratorRepr(self, "keys").newiter(hop) - - def rtype_method_itervalues(self, hop): - hop.exception_cannot_occur() - return DictIteratorRepr(self, "values").newiter(hop) - - def rtype_method_iteritems(self, hop): - hop.exception_cannot_occur() - return DictIteratorRepr(self, "items").newiter(hop) - - def rtype_method_clear(self, hop): - v_dict, = hop.inputargs(self) - hop.exception_cannot_occur() - return hop.gendirectcall(ll_clear, v_dict) - -class __extend__(pairtype(DictRepr, rmodel.Repr)): - - def rtype_getitem((r_dict, r_key), hop): - v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) - if not r_dict.custom_eq_hash: - hop.has_implicit_exception(KeyError) # record that we know about it - hop.exception_is_here() - v_res = hop.gendirectcall(ll_dict_getitem, v_dict, v_key) - return r_dict.recast_value(hop.llops, v_res) - - def rtype_delitem((r_dict, r_key), hop): - v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) - if not r_dict.custom_eq_hash: - hop.has_implicit_exception(KeyError) # record that we know about it - hop.exception_is_here() - return hop.gendirectcall(ll_dict_delitem, v_dict, v_key) - - def rtype_setitem((r_dict, r_key), hop): - v_dict, v_key, v_value = hop.inputargs(r_dict, r_dict.key_repr, r_dict.value_repr) - if r_dict.custom_eq_hash: - hop.exception_is_here() - else: - hop.exception_cannot_occur() - hop.gendirectcall(ll_dict_setitem, v_dict, v_key, v_value) - - def rtype_contains((r_dict, r_key), hop): - v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) - return hop.gendirectcall(ll_contains, v_dict, v_key) - -class __extend__(pairtype(DictRepr, DictRepr)): - def convert_from_to((r_dict1, r_dict2), v, llops): - # check that we don't convert from Dicts with - # different key/value types - if r_dict1.dictkey is None or r_dict2.dictkey is None: - return NotImplemented - if r_dict1.dictkey is not r_dict2.dictkey: - return NotImplemented - if r_dict1.dictvalue is None or r_dict2.dictvalue is None: - return NotImplemented - if r_dict1.dictvalue is not r_dict2.dictvalue: - return NotImplemented - return v - -# ____________________________________________________________ -# -# Low-level methods. These can be run for testing, but are meant to -# be direct_call'ed from rtyped flow graphs, which means that they will -# get flowed and annotated, mostly with SomePtr. - -def ll_everused_from_flag(entry): - return entry.f_everused - -def ll_everused_from_key(entry): - return bool(entry.key) - -def ll_everused_from_value(entry): - return bool(entry.value) +class AbstractDictRepr(Repr): + pass -def ll_valid_from_flag(entry): - return entry.f_valid - -def ll_mark_deleted_in_flag(entry): - entry.f_valid = False - -def ll_valid_from_key(entry): - ENTRY = lltype.typeOf(entry).TO - dummy = ENTRY.dummy_obj.ll_dummy_value - return entry.everused() and entry.key != dummy - -def ll_mark_deleted_in_key(entry): - ENTRY = lltype.typeOf(entry).TO - dummy = ENTRY.dummy_obj.ll_dummy_value - entry.key = dummy - -def ll_valid_from_value(entry): - ENTRY = lltype.typeOf(entry).TO - dummy = ENTRY.dummy_obj.ll_dummy_value - return entry.everused() and entry.value != dummy - -def ll_mark_deleted_in_value(entry): - ENTRY = lltype.typeOf(entry).TO - dummy = ENTRY.dummy_obj.ll_dummy_value - entry.value = dummy - -def ll_hash_from_cache(entry): - return entry.f_hash - -def ll_hash_recomputed(entry): - ENTRY = lltype.typeOf(entry).TO - return ENTRY.fasthashfn(entry.key) - -def ll_keyhash_custom(d, key): - DICT = lltype.typeOf(d).TO - return hlinvoke(DICT.r_rdict_hashfn, d.fnkeyhash, key) - -def ll_keyeq_custom(d, key1, key2): - DICT = lltype.typeOf(d).TO - return hlinvoke(DICT.r_rdict_eqfn, d.fnkeyeq, key1, key2) - -def dum_keys(): pass -def dum_values(): pass -def dum_items():pass -dum_variant = {"keys": dum_keys, - "values": dum_values, - "items": dum_items} - -def ll_dict_len(d): - return d.num_items - -def ll_dict_is_true(d): - # check if a dict is True, allowing for None - return bool(d) and d.num_items != 0 - -def ll_dict_getitem(d, key): - entry = ll_dict_lookup(d, key, d.keyhash(key)) - if entry.valid(): - return entry.value - else: - raise KeyError - -def ll_dict_setitem(d, key, value): - hash = d.keyhash(key) - entry = ll_dict_lookup(d, key, hash) - everused = entry.everused() - valid = entry.valid() - # set up the new entry - ENTRY = lltype.typeOf(entry).TO - entry.value = value - if valid: - return - entry.key = key - if hasattr(ENTRY, 'f_hash'): entry.f_hash = hash - if hasattr(ENTRY, 'f_valid'): entry.f_valid = True - d.num_items += 1 - if not everused: - if hasattr(ENTRY, 'f_everused'): entry.f_everused = True - d.num_pristine_entries -= 1 - if d.num_pristine_entries <= len(d.entries) / 3: - ll_dict_resize(d) - -def ll_dict_insertclean(d, key, value, hash): - # Internal routine used by ll_dict_resize() to insert an item which is - # known to be absent from the dict. This routine also assumes that - # the dict contains no deleted entries. This routine has the advantage - # of never calling d.keyhash() and d.keyeq(), so it cannot call back - # to user code. ll_dict_insertclean() doesn't resize the dict, either. - entry = ll_dict_lookup_clean(d, hash) - ENTRY = lltype.typeOf(entry).TO - entry.value = value - entry.key = key - if hasattr(ENTRY, 'f_hash'): entry.f_hash = hash - if hasattr(ENTRY, 'f_valid'): entry.f_valid = True - if hasattr(ENTRY, 'f_everused'): entry.f_everused = True - d.num_items += 1 - d.num_pristine_entries -= 1 - -def ll_dict_delitem(d, key): - entry = ll_dict_lookup(d, key, d.keyhash(key)) - if not entry.valid(): - raise KeyError - entry.mark_deleted() - d.num_items -= 1 - # clear the key and the value if they are GC pointers - ENTRY = lltype.typeOf(entry).TO - if ENTRY.must_clear_key: - key = entry.key # careful about destructor side effects: - # keep key alive until entry.value has also - # been zeroed (if it must be) - entry.key = lltype.nullptr(ENTRY.key.TO) - if ENTRY.must_clear_value: - entry.value = lltype.nullptr(ENTRY.value.TO) - num_entries = len(d.entries) - if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4: - ll_dict_resize(d) - -def ll_dict_resize(d): - old_entries = d.entries - old_size = len(old_entries) - # make a 'new_size' estimate and shrink it if there are many - # deleted entry markers - new_size = old_size * 2 - while new_size > DICT_INITSIZE and d.num_items < new_size / 4: - new_size /= 2 - d.entries = lltype.malloc(lltype.typeOf(old_entries).TO, new_size) - d.num_items = 0 - d.num_pristine_entries = new_size - i = 0 - while i < old_size: - entry = old_entries[i] - if entry.valid(): - ll_dict_insertclean(d, entry.key, entry.value, entry.hash()) - i += 1 - -# ------- a port of CPython's dictobject.c's lookdict implementation ------- -PERTURB_SHIFT = 5 - -def ll_dict_lookup(d, key, hash): - DICT = lltype.typeOf(d).TO - entries = d.entries - mask = len(entries) - 1 - i = r_uint(hash & mask) - # do the first try before any looping - entry = entries[i] - if entry.valid(): - checkingkey = entry.key - if checkingkey == key: - return entry # found the entry - if d.keyeq is not None and entry.hash() == hash: - # correct hash, maybe the key is e.g. a different pointer to - # an equal object - found = d.keyeq(checkingkey, key) - if DICT.paranoia: - if (entries != d.entries or - not entry.valid() or entry.key != checkingkey): - # the compare did major nasty stuff to the dict: start over - return ll_dict_lookup(d, key, hash) - if found: - return entry # found the entry - freeslot = lltype.nullptr(lltype.typeOf(entry).TO) - elif entry.everused(): - freeslot = entry - else: - return entry # pristine entry -- lookup failed - - # In the loop, a deleted entry (everused and not valid) is by far - # (factor of 100s) the least likely outcome, so test for that last. - perturb = r_uint(hash) - while 1: - i = ((i << 2) + i + perturb + 1) & mask - entry = entries[i] - if not entry.everused(): - return freeslot or entry - elif entry.valid(): - checkingkey = entry.key - if checkingkey == key: - return entry - if d.keyeq is not None and entry.hash() == hash: - # correct hash, maybe the key is e.g. a different pointer to - # an equal object - found = d.keyeq(checkingkey, key) - if DICT.paranoia: - if (entries != d.entries or - not entry.valid() or entry.key != checkingkey): - # the compare did major nasty stuff to the dict: - # start over - return ll_dict_lookup(d, key, hash) - if found: - return entry # found the entry - elif not freeslot: - freeslot = entry - perturb >>= PERTURB_SHIFT - -def ll_dict_lookup_clean(d, hash): - # a simplified version of ll_dict_lookup() which assumes that the - # key is new, and the dictionary doesn't contain deleted entries. - # It only find the next free slot for the given hash. - entries = d.entries - mask = len(entries) - 1 - i = r_uint(hash & mask) - entry = entries[i] - perturb = r_uint(hash) - while entry.everused(): - i = ((i << 2) + i + perturb + 1) & mask - entry = entries[i] - perturb >>= PERTURB_SHIFT - return entry - -# ____________________________________________________________ -# -# Irregular operations. - -DICT_INITSIZE = 8 - -def ll_newdict(DICT): - d = lltype.malloc(DICT) - d.entries = lltype.malloc(DICT.entries.TO, DICT_INITSIZE) - #d.num_items = 0 -- defaults - d.num_pristine_entries = DICT_INITSIZE - return d - -def ll_newdict_size(DICT, length_estimate): - length_estimate = (length_estimate // 2) * 3 - n = DICT_INITSIZE - while n < length_estimate: - n *= 2 - d = lltype.malloc(DICT) - d.entries = lltype.malloc(DICT.entries.TO, n) - #d.num_items = 0 -- defaults - d.num_pristine_entries = DICT_INITSIZE - return d def rtype_newdict(hop): hop.inputargs() # no arguments expected @@ -623,185 +47,5 @@ cdict = hop.inputconst(robject.pyobj_repr, dict) return hop.genop('simple_call', [cdict], resulttype = robject.pyobj_repr) cDICT = hop.inputconst(lltype.Void, r_dict.DICT) - v_result = hop.gendirectcall(ll_newdict, cDICT) - return v_result - -def rtype_r_dict(hop): - r_dict = hop.r_result - if not r_dict.custom_eq_hash: - raise TyperError("r_dict() call does not return an r_dict instance") - v_eqfn, v_hashfn = hop.inputargs(r_dict.r_rdict_eqfn, - r_dict.r_rdict_hashfn) - cDICT = hop.inputconst(lltype.Void, r_dict.DICT) - hop.exception_cannot_occur() - v_result = hop.gendirectcall(ll_newdict, cDICT) - if r_dict.r_rdict_eqfn.lowleveltype != lltype.Void: - cname = hop.inputconst(lltype.Void, 'fnkeyeq') - hop.genop('setfield', [v_result, cname, v_eqfn]) - if r_dict.r_rdict_hashfn.lowleveltype != lltype.Void: - cname = hop.inputconst(lltype.Void, 'fnkeyhash') - hop.genop('setfield', [v_result, cname, v_hashfn]) + v_result = hop.gendirectcall(hop.rtyper.type_system.rdict.ll_newdict, cDICT) return v_result - -# ____________________________________________________________ -# -# Iteration. - -class DictIteratorRepr(rmodel.IteratorRepr): - - def __init__(self, r_dict, variant="keys"): - self.r_dict = r_dict - self.variant = variant - self.lowleveltype = lltype.Ptr(lltype.GcStruct('dictiter', - ('dict', r_dict.lowleveltype), - ('index', lltype.Signed))) - - def newiter(self, hop): - v_dict, = hop.inputargs(self.r_dict) - citerptr = hop.inputconst(lltype.Void, self.lowleveltype) - return hop.gendirectcall(ll_dictiter, citerptr, v_dict) - - def rtype_next(self, hop): - variant = self.variant - v_iter, = hop.inputargs(self) - v_func = hop.inputconst(lltype.Void, dum_variant[self.variant]) - if variant in ('keys', 'values'): - c1 = hop.inputconst(lltype.Void, None) - else: - c1 = hop.inputconst(lltype.Void, hop.r_result.lowleveltype) - hop.has_implicit_exception(StopIteration) # record that we know about it - hop.exception_is_here() - v = hop.gendirectcall(ll_dictnext, v_iter, v_func, c1) - if variant == 'keys': - return self.r_dict.recast_key(hop.llops, v) - elif variant == 'values': - return self.r_dict.recast_value(hop.llops, v) - else: - return v - -def ll_dictiter(ITERPTR, d): - iter = lltype.malloc(ITERPTR.TO) - iter.dict = d - iter.index = 0 - return iter - -def ll_dictnext(iter, func, RETURNTYPE): - dict = iter.dict - if dict: - entries = dict.entries - index = iter.index - entries_len = len(entries) - while index < entries_len: - entry = entries[index] - index = index + 1 - if entry.valid(): - iter.index = index - if func is dum_items: - r = lltype.malloc(RETURNTYPE.TO) - r.item0 = recast(RETURNTYPE.TO.item0, entry.key) - r.item1 = recast(RETURNTYPE.TO.item1, entry.value) - return r - elif func is dum_keys: - return entry.key - elif func is dum_values: - return entry.value - # clear the reference to the dict and prevent restarts - iter.dict = lltype.nullptr(lltype.typeOf(iter).TO.dict.TO) - raise StopIteration - -# _____________________________________________________________ -# methods - -def ll_get(dict, key, default): - entry = ll_dict_lookup(dict, key, dict.keyhash(key)) - if entry.valid(): - return entry.value - else: - return default - -def ll_setdefault(dict, key, default): - entry = ll_dict_lookup(dict, key, dict.keyhash(key)) - if entry.valid(): - return entry.value - else: - ll_dict_setitem(dict, key, default) - return default - -def ll_copy(dict): - DICT = lltype.typeOf(dict).TO - dictsize = len(dict.entries) - d = lltype.malloc(DICT) - d.entries = lltype.malloc(DICT.entries.TO, dictsize) - d.num_items = dict.num_items - d.num_pristine_entries = dict.num_pristine_entries - if hasattr(DICT, 'fnkeyeq'): d.fnkeyeq = dict.fnkeyeq - if hasattr(DICT, 'fnkeyhash'): d.fnkeyhash = dict.fnkeyhash - i = 0 - while i < dictsize: - d_entry = d.entries[i] - entry = dict.entries[i] - ENTRY = lltype.typeOf(entry).TO - d_entry.key = entry.key - if hasattr(ENTRY, 'f_valid'): d_entry.f_valid = entry.f_valid - if hasattr(ENTRY, 'f_everused'): d_entry.f_everused = entry.f_everused - d_entry.value = entry.value - if hasattr(ENTRY, 'f_hash'): d_entry.f_hash = entry.f_hash - i += 1 - return d - -def ll_clear(d): - if len(d.entries) == d.num_pristine_entries == DICT_INITSIZE: - return - DICT = lltype.typeOf(d).TO - d.entries = lltype.malloc(DICT.entries.TO, DICT_INITSIZE) - d.num_items = 0 - d.num_pristine_entries = DICT_INITSIZE - -def ll_update(dic1, dic2): - entries = dic2.entries - d2len = len(entries) - i = 0 - while i < d2len: - entry = entries[i] - if entry.valid(): - ll_dict_setitem(dic1, entry.key, entry.value) - i += 1 - -# this is an implementation of keys(), values() and items() -# in a single function. -# note that by specialization on func, three different -# and very efficient functions are created. - -def recast(P, v): - if isinstance(P, lltype.Ptr): - return lltype.cast_pointer(P, v) - else: - return v - -def ll_kvi(dic, LIST, func): - res = LIST.ll_newlist(dic.num_items) - entries = dic.entries - dlen = len(entries) - items = res.ll_items() - i = 0 - p = 0 - while i < dlen: - entry = entries[i] - if entry.valid(): - ELEM = lltype.typeOf(items).TO.OF - if func is dum_items: - r = lltype.malloc(ELEM.TO) - r.item0 = recast(ELEM.TO.item0, entry.key) - r.item1 = recast(ELEM.TO.item1, entry.value) - items[p] = r - elif func is dum_keys: - items[p] = recast(ELEM, entry.key) - elif func is dum_values: - items[p] = recast(ELEM, entry.value) - p += 1 - i += 1 - return res - -def ll_contains(d, key): - entry = ll_dict_lookup(d, key, d.keyhash(key)) - return entry.valid() Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Fri Apr 21 00:17:24 2006 @@ -522,7 +522,7 @@ return self.type_system.rlist.rtype_newlist(hop) def translate_op_newdict(self, hop): - return rdict.rtype_newdict(hop) + return self.type_system.rdict.rtype_newdict(hop) def translate_op_alloc_and_set(self, hop): return self.type_system.rlist.rtype_alloc_and_set(hop) @@ -860,7 +860,7 @@ from pypy.rpython import robject from pypy.rpython import rint, rbool, rfloat from pypy.rpython import rslice, rrange -from pypy.rpython import rstr, rdict +from pypy.rpython import rstr, rdict from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase from pypy.rpython import rexternalobj from pypy.rpython import rptr Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Fri Apr 21 00:17:24 2006 @@ -1,7 +1,8 @@ from pypy.translator.translator import TranslationContext from pypy.rpython.lltypesystem import lltype from pypy.rpython.test.test_llinterp import interpret -from pypy.rpython import rstr, rint, rdict +from pypy.rpython import rstr, rint +from pypy.rpython.lltypesystem import rdict import py py.log.setconsumer("rtyper", py.log.STDOUT) Modified: pypy/dist/pypy/rpython/typesystem.py ============================================================================== --- pypy/dist/pypy/rpython/typesystem.py (original) +++ pypy/dist/pypy/rpython/typesystem.py Fri Apr 21 00:17:24 2006 @@ -20,7 +20,7 @@ None, None, ['__doc__']) except ImportError: return None - if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', 'rslice', + if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', 'rslice', 'rdict', 'exceptiondata'): mod = load(name) if mod is not None: From ericvrp at codespeak.net Fri Apr 21 03:57:13 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 21 Apr 2006 03:57:13 +0200 (CEST) Subject: [pypy-svn] r26072 - pypy/dist/pypy/translator/llvm/test Message-ID: <20060421015713.211F11007D@code0.codespeak.net> Author: ericvrp Date: Fri Apr 21 03:57:03 2006 New Revision: 26072 Modified: pypy/dist/pypy/translator/llvm/test/test_symbolic.py Log: removed duplicate test Modified: pypy/dist/pypy/translator/llvm/test/test_symbolic.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_symbolic.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_symbolic.py Fri Apr 21 03:57:03 2006 @@ -22,21 +22,6 @@ res = fn() assert res == 12 -def test_offsetof(): - STRUCT = lltype.GcStruct("s", ("x", lltype.Signed), ("y", lltype.Signed)) - offsetx = llmemory.offsetof(STRUCT, 'x') - offsety = llmemory.offsetof(STRUCT, 'y') - def f(): - s = lltype.malloc(STRUCT) - s.x = 1 - adr = llmemory.cast_ptr_to_adr(s) - result = (adr + offsetx).signed[0] - (adr + offsety).signed[0] = 2 - return result * 10 + s.y - fn = compile_function(f, []) - res = fn() - assert res == 12 - def test_sizeof_array_with_no_length(): py.test.skip("inprogress") A = lltype.GcArray(lltype.Signed, hints={'nolength': True}) From ericvrp at codespeak.net Fri Apr 21 04:08:13 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 21 Apr 2006 04:08:13 +0200 (CEST) Subject: [pypy-svn] r26073 - pypy/dist/pypy/translator/js Message-ID: <20060421020813.8E2DD1007B@code0.codespeak.net> Author: ericvrp Date: Fri Apr 21 04:08:04 2006 New Revision: 26073 Modified: pypy/dist/pypy/translator/js/database.py Log: Added basic symbolic/address support. This fixes most of the failing tests. Modified: pypy/dist/pypy/translator/js/database.py ============================================================================== --- pypy/dist/pypy/translator/js/database.py (original) +++ pypy/dist/pypy/translator/js/database.py Fri Apr 21 04:08:04 2006 @@ -6,7 +6,8 @@ from pypy.translator.js.structnode import StructNode, StructVarsizeNode from pypy.translator.js.arraynode import ArrayNode, StrArrayNode, VoidArrayNode from pypy.translator.js.opaquenode import OpaqueNode -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.objectmodel import Symbolic, ComputedIntSymbolic +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.objspace.flow.model import Constant, Variable from pypy.translator.js.log import log @@ -23,7 +24,9 @@ lltype.SignedLongLong: "int", lltype.UnsignedLongLong: "uint", lltype.UniChar: "uint", - lltype.Void: "void"} + lltype.Void: "void", + llmemory.Address: "sbyte*", + } def __init__(self, translator, js): self.translator = translator @@ -233,6 +236,13 @@ repr = self.float_to_str(value) elif type_ is lltype.Void: repr = "undefined" + elif isinstance(value, Symbolic): + if isinstance(value, llmemory.AddressOffset): + return self.offset_str(value) + elif isinstance(value, ComputedIntSymbolic): + repr = '%d' % (value.compute_fn(),) + else: + raise NotImplementedError("symbolic: %r" % (value,)) else: repr = str(value) return repr From bea at codespeak.net Fri Apr 21 05:34:58 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Fri, 21 Apr 2006 05:34:58 +0200 (CEST) Subject: [pypy-svn] r26074 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060421033458.0F02D1007D@code0.codespeak.net> Author: bea Date: Fri Apr 21 05:34:54 2006 New Revision: 26074 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: My info....please see comment and feedback Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Fri Apr 21 05:34:54 2006 @@ -23,14 +23,15 @@ Beatrice During ------------------------------ -preferred arrival date: acceptable arrival dates: -preferred departure date: acceptable departure dates: -start city / airport: +preferred arrival date: 20th acceptable arrival dates: 21th,22nd +preferred departure date: 224th acceptable departure dates: 25th +start city / airport: Gothenburg mail: bea at changemaker.nu -birthdate: -address: -organisation: -comments: +birthdate: 3rd Dec 1973 +address: Eklandagatan5, 412 55 G?teborg, Sweden +organisation: Change Maker +comments: should we prioritize developers being funded whole time instead? +I do not want to stay away from Simone too long.... Holger Krekel ------------------------------ From ericvrp at codespeak.net Fri Apr 21 11:00:31 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 21 Apr 2006 11:00:31 +0200 (CEST) Subject: [pypy-svn] r26077 - pypy/dist/pypy/translator/js/test Message-ID: <20060421090031.CC6C71007A@code0.codespeak.net> Author: ericvrp Date: Fri Apr 21 11:00:20 2006 New Revision: 26077 Added: pypy/dist/pypy/translator/js/test/test_symbolic.py (contents, props changed) Log: Added symbolic tests (taken from llvm). All but one test is still skipped though. Added: pypy/dist/pypy/translator/js/test/test_symbolic.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/js/test/test_symbolic.py Fri Apr 21 11:00:20 2006 @@ -0,0 +1,99 @@ +import py +from pypy.translator.interactive import Translation +from pypy import conftest +from pypy.rpython.lltypesystem import llmemory, lltype +from pypy.rpython.memory import lladdress +from pypy.rpython.objectmodel import ComputedIntSymbolic + +from pypy.translator.js.test.runtest import compile_function + +def test_computed_int_symbolic(): + def compute_fn(): + return 7 + k = ComputedIntSymbolic(compute_fn) + def f(): + return k*6 + + fn = compile_function(f, []) + res = fn() + assert res == 42 + +def test_offsetof(): + py.test.skip("symbolic/address support is in progress") + STRUCT = lltype.GcStruct("s", ("x", lltype.Signed), ("y", lltype.Signed)) + offsetx = llmemory.offsetof(STRUCT, 'x') + offsety = llmemory.offsetof(STRUCT, 'y') + def f(): + s = lltype.malloc(STRUCT) + s.x = 1 + adr = llmemory.cast_ptr_to_adr(s) + result = (adr + offsetx).signed[0] + (adr + offsety).signed[0] = 2 + return result * 10 + s.y + fn = compile_function(f, []) + res = fn() + assert res == 12 + +def test_sizeof_array_with_no_length(): + py.test.skip("inprogress") + A = lltype.GcArray(lltype.Signed, hints={'nolength': True}) + a = lltype.malloc(A, 5) + + arraysize = llmemory.itemoffsetof(A, 10) + signedsize = llmemory.sizeof(lltype.Signed) + def f(): + return a[0] + arraysize-signedsize*10 + fn = compile_function(f, []) + res = fn() + assert res == 0 + +def test_itemoffsetof(): + py.test.skip("symbolic/address support is in progress") + ARRAY = lltype.GcArray(lltype.Signed) + itemoffsets = [llmemory.itemoffsetof(ARRAY, i) for i in range(5)] + def f(): + a = lltype.malloc(ARRAY, 5) + adr = llmemory.cast_ptr_to_adr(a) + result = 0 + for i in range(5): + a[i] = i + 1 + for i in range(5): + result = result * 10 + (adr + itemoffsets[i]).signed[0] + for i in range(5): + (adr + itemoffsets[i]).signed[0] = i + for i in range(5): + result = 10 * result + a[i] + return result + fn = compile_function(f, []) + res = fn() + assert res == 1234501234 + +def test_sizeof_constsize_struct(): + py.test.skip("symbolic/address support is in progress") + # _not_ a GcStruct, since we want to raw_malloc it + STRUCT = lltype.Struct("s", ("x", lltype.Signed), ("y", lltype.Signed)) + STRUCTPTR = lltype.Ptr(STRUCT) + sizeofs = llmemory.sizeof(STRUCT) + offsety = llmemory.offsetof(STRUCT, 'y') + def f(): + adr = lladdress.raw_malloc(sizeofs) + s = llmemory.cast_adr_to_ptr(adr, STRUCTPTR) + s.y = 5 # does not crash + result = (adr + offsety).signed[0] * 10 + int(offsety < sizeofs) + lladdress.raw_free(adr) + return result + + fn = compile_function(f, []) + res = fn() + assert res == 51 + +def test_computed_int_symbolic(): + def compute_fn(): + return 7 + k = ComputedIntSymbolic(compute_fn) + def f(): + return k*6 + + fn = compile_function(f, []) + res = fn() + assert res == 42 From antocuni at codespeak.net Fri Apr 21 12:16:23 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 21 Apr 2006 12:16:23 +0200 (CEST) Subject: [pypy-svn] r26080 - pypy/dist/pypy/rpython Message-ID: <20060421101623.538021007D@code0.codespeak.net> Author: antocuni Date: Fri Apr 21 12:16:19 2006 New Revision: 26080 Modified: pypy/dist/pypy/rpython/llinterp.py Log: Fixed llinterp so that it supports all classes derived by ootype.BuiltinType, not only ootype.List. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Apr 21 12:16:19 2006 @@ -135,7 +135,7 @@ return lltype.typeOf(addr) == llmemory.Address def checkinst(inst): - return isinstance(lltype.typeOf(inst), (ootype.Instance, ootype.List)) + return isinstance(lltype.typeOf(inst), (ootype.Instance, ootype.BuiltinType)) class LLFrame(object): def __init__(self, graph, args, llinterpreter, f_back=None): @@ -889,7 +889,7 @@ #Operation of ootype def op_new(self, INST): - assert isinstance(INST, (ootype.Instance, ootype.List)) + assert isinstance(INST, (ootype.Instance, ootype.BuiltinType)) return ootype.new(INST) def op_runtimenew(self, class_): From antocuni at codespeak.net Fri Apr 21 12:24:07 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 21 Apr 2006 12:24:07 +0200 (CEST) Subject: [pypy-svn] r26081 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060421102407.479EE1007A@code0.codespeak.net> Author: antocuni Date: Fri Apr 21 12:24:03 2006 New Revision: 26081 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: Fixed some functions so that now they handle all subclasses of BuiltinType, not only List. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 21 12:24:03 2006 @@ -297,8 +297,8 @@ # equal. def __str__(self): - return '%s(%s)' % (self.__class__.__name__, - saferecursive(str, "...")(self._KEYTYPE, self._VALUETYPE)) + return '%s%s' % (self.__class__.__name__, + saferecursive(str, "(...)")((self._KEYTYPE, self._VALUETYPE))) def _get_interp_class(self): return _dict @@ -306,7 +306,7 @@ class ForwardReference(OOType): def become(self, real_instance): - if not isinstance(real_instance, (Instance, List)): + if not isinstance(real_instance, (Instance, BuiltinType)): raise TypeError("ForwardReference can only be to an instance, " "not %r" % (real_instance,)) self.__class__ = real_instance.__class__ @@ -558,7 +558,7 @@ _callable.__init__(self, METHOD, **attrs) def _bound(self, DEFINST, inst): - assert isinstance(inst, _instance) or isinstance(inst, _list) + assert isinstance(inst, _instance) or isinstance(inst, _builtin_type) return _bound_meth(DEFINST, inst, self) class _bound_meth(object): @@ -634,7 +634,7 @@ def __init__(self, LIST): self.__dict__["_TYPE"] = LIST -class _dict(object): +class _dict(_builtin_type): def __init__(self, DICT): self._TYPE = DICT self._dict = {} From mwh at codespeak.net Fri Apr 21 12:50:05 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 21 Apr 2006 12:50:05 +0200 (CEST) Subject: [pypy-svn] r26082 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060421105005.D172A10081@code0.codespeak.net> Author: mwh Date: Fri Apr 21 12:50:02 2006 New Revision: 26082 Added: pypy/dist/pypy/translator/stackless/test/__init__.py (contents, props changed) Modified: pypy/dist/pypy/translator/stackless/code.py Log: make pypy.translator.stackless.test a package Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Fri Apr 21 12:50:02 2006 @@ -15,6 +15,44 @@ currentframe.retval_type, currentframe.restartstate) +SWITCH_STATE = lltype.GcStruct('state_switch', + ('header', STATE_HEADER), + ('c', llmemory.Address)) + +def switch(c): + if not global_state.restartstate: + u = UnwindException() + s = lltype.malloc(SWITCH_STATE) + s.c = llmemory.cast_ptr_to_adr(c) + s.header.restartstate = 1 + s.header.function = llmemory.cast_ptr_to_adr(switch) + s.header.retval_type = RETVAL_VOID_P + add_frame_state(u, s.header) + raise u + else: + top = global_state.top + s = lltype.cast_pointer(lltype.Ptr(SWITCH_STATE), top) + global_state.top = s.c + return top.f_back + +def stack_frames_depth(): + if not global_state.restartstate: + u = UnwindException() + s = lltype.malloc(STATE_HEADER) + s.restartstate = 1 + s.function = llmemory.cast_ptr_to_adr(count_stack_depth) + s.retval_type = RETVAL_VOID + add_frame_state(u, s.header) + raise u + else: + cur = global_state.top + global_state.restartstate = 0 + depth = 0 + while cur: + depth += 1 + cur = cur.f_back + return depth + class StacklessData: def __init__(self): self.top = null_state Added: pypy/dist/pypy/translator/stackless/test/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/stackless/test/__init__.py Fri Apr 21 12:50:02 2006 @@ -0,0 +1 @@ +#ra From hpk at codespeak.net Fri Apr 21 13:54:59 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 21 Apr 2006 13:54:59 +0200 (CEST) Subject: [pypy-svn] r26090 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060421115459.A203810080@code0.codespeak.net> Author: hpk Date: Fri Apr 21 13:54:58 2006 New Revision: 26090 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: updated the sprint info, michael and armin will attend the sprint through HHU Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Fri Apr 21 13:54:58 2006 @@ -8,30 +8,17 @@ Sprint core time is 21st till 28th (both including ASFAIK) -Armin Rigo ------------------------------- - -preferred arrival date: 19th acceptable arrival dates: any -preferred departure date: 29th acceptable departure dates: any -start city / airport: Gothenburg / Landvetter (GOT) or City Airport (GSE) -mail: arigo at tunes.org -birthdate: 26 sep 1976 -address: App 22, c/o Hall?n/Creighton, Linn?gatan 47, 413 08 G?teborg, Sweden -organisation: Heinrich-Heine Universit?ts D?sseldorf, Germany -comments: - Beatrice During ------------------------------ preferred arrival date: 20th acceptable arrival dates: 21th,22nd -preferred departure date: 224th acceptable departure dates: 25th +preferred departure date: 24th acceptable departure dates: 25th start city / airport: Gothenburg mail: bea at changemaker.nu birthdate: 3rd Dec 1973 address: Eklandagatan5, 412 55 G?teborg, Sweden organisation: Change Maker -comments: should we prioritize developers being funded whole time instead? -I do not want to stay away from Simone too long.... +comments: I do not want to stay away from Simone (my daughter) too long.... Holger Krekel ------------------------------ @@ -43,8 +30,8 @@ birthdate: 14th nov. 1969, phone number: +49 171 464 8622 address: Steinbergstr. 42, 31139 Hildesheim, Germany -organisation: merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany -comments: not before 11 AM as i need a 3 hour train travel to FRA airport +organisation: merlinux GmbH +comments: departure not before 12 AM as i need a 3 hour train travel to FRA airport Samuele Pedroni ------------------------------ @@ -55,7 +42,7 @@ mail: pedronis at strakt.com birthdate: 22nd dec 1974 address: Norra Agatan 10A, 416 64 Gothenburg, Sweden -organisation: AB Strakt, +organisation: AB Strakt comments: need to be in Gothenburg on the 20th so cannot travel earlier, direct flight preferred @@ -81,25 +68,9 @@ birthdate: 11th jan. 1978, address: Buergerweide 1, 23562 L?beck, Germany -organisation: merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany +organisation: merlinux GmbH comments: - -Michael Hudson ------------------------------- - -preferred arrival date: 21 acceptable arrival dates: 20-22 -preferred departure date: 28 acceptable departure dates: 27-29 -start city / airport: bristol, cardiff, birmingham, LHR, LGW - (in roughly that order) -mail: mwh at python.net -birthdate: 1978-10-07 -address: Flat 4 Plimsoll House, Ashgrove Road, Bristol BS6 6LZ, UK -organisation: Heinrich-Heine Universitat Dusseldorf -comments: I'd much rather travel up to london and take a direct - flight than travel to a closer airport and have to change - planes - Antonio Cuni ------------------------------ @@ -113,19 +84,6 @@ comments: preferred start airport is Genova (GOA), but Milano Malpensa (MXP) is acceptable, too. Need to be in Genova on the 28th. -(Wanja Saatkamp) ------------------------------- - -preferred arrival date: 20th acceptable arrival dates: 19th, 21st -preferred departure date: 28th acceptable departure dates: 27th,29th -start city / airport: Hildesheim / Frankfurt (FRA), Hannover, Berlin or Hamburg -mail: wanja at merlinux.de -birthdate: 22.02.1975 -address: Wrangelstr. 67, 10997 Berlin, Germany -organisation: Merlinux GmbH, Steinbergstr. 42, 31139 Hildesheim, Germany -comments: - no flight departure before 12AM because of travel time to the airport; - -- my attendance is not decided yet (and i will not be coding but documenting the activities with the camera) Anders Lehmann ------------------------------ @@ -139,3 +97,15 @@ organisation: DFKI Gmbh comments: +Wanja Saatkamp +------------------------------ + +preferred arrival date: 20th acceptable arrival dates: 19th, 21st +preferred departure date: 28th acceptable departure dates: 27th,29th +start city / airport: Hildesheim / Frankfurt (FRA), Hannover, Berlin or Hamburg +mail: wanja at merlinux.de +birthdate: 22.02.1975 +address: Wrangelstr. 67, 10997 Berlin, Germany +organisation: Merlinux GmbH +comments: no flight departure before 12AM because of travel time to the airport; + From ericvrp at codespeak.net Fri Apr 21 14:02:52 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 21 Apr 2006 14:02:52 +0200 (CEST) Subject: [pypy-svn] r26091 - in pypy/dist/pypy/translator: js/test llvm/test Message-ID: <20060421120252.C62B41007F@code0.codespeak.net> Author: ericvrp Date: Fri Apr 21 14:02:26 2006 New Revision: 26091 Modified: pypy/dist/pypy/translator/js/test/test_genllvm.py pypy/dist/pypy/translator/llvm/test/test_genllvm.py Log: Support for ** between integers was removed some time ago in the annotator and the rtyper, because in PyPy we didn't use it at all. (thanks Armin for pointing this out) Modified: pypy/dist/pypy/translator/js/test/test_genllvm.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_genllvm.py (original) +++ pypy/dist/pypy/translator/js/test/test_genllvm.py Fri Apr 21 14:02:26 2006 @@ -66,9 +66,6 @@ x += i >= i x += i > i x += x % i - x += x ** 0 - x += x ** 1 - x += x ** 2 x += i + 1 * i // i - 1 return int(x+0.0001) f = compile_function(ops, [int]) @@ -85,9 +82,6 @@ x += i >= i x += i > i x += x % i - x += x ** 0 - x += x ** 1 - x += x ** 2 x += i + 1 * i // i - 1 return int(x+0.0001) f = compile_function(ops, [r_uint]) @@ -103,9 +97,6 @@ x += flt != flt x += flt >= flt x += flt > flt - x += x ** 0 - x += x ** 1 - x += x ** 2 x += int(flt + 1 * flt / flt - 1) return x f = compile_function(ops, [float]) Modified: pypy/dist/pypy/translator/llvm/test/test_genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_genllvm.py Fri Apr 21 14:02:26 2006 @@ -88,12 +88,7 @@ x += i >= i x += i > i x += x % i - x += x ** 0 - x += x ** 1 - x += x ** 2 x += i + 1 * i // i - 1 - #x += i is not None - #x += i is None return x f = compile_function(ops, [int]) assert f(1) == ops(1) @@ -109,12 +104,7 @@ x += i >= i x += i > i x += x % i - x += x ** 0 - x += x ** 1 - x += x ** 2 x += i + 1 * i // i - 1 - #x += i is not None - #x += i is None return x f = compile_function(ops, [r_uint]) assert f(1) == ops(1) @@ -129,12 +119,7 @@ x += flt != flt x += flt >= flt x += flt > flt - x += x ** 0 - x += x ** 1 - x += x ** 2 x += int(flt + 1 * flt / flt - 1) - #x += flt fs not None - #x += flt is None return x f = compile_function(ops, [float]) assert f(1) == ops(1) From ale at codespeak.net Fri Apr 21 14:07:27 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 21 Apr 2006 14:07:27 +0200 (CEST) Subject: [pypy-svn] r26092 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060421120727.E3AF810084@code0.codespeak.net> Author: ale Date: Fri Apr 21 14:07:26 2006 New Revision: 26092 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: I would like to return to Germany instead of Denmark Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Fri Apr 21 14:07:26 2006 @@ -95,7 +95,7 @@ birthdate: 30.11.1962 address: Anna Anchersvej 206, DK - 7500 Denmark organisation: DFKI Gmbh -comments: +comments: I would like to return to Frankfurt, as I am going to Saarbr?cken after the sprint Wanja Saatkamp ------------------------------ From hpk at codespeak.net Fri Apr 21 14:10:52 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Fri, 21 Apr 2006 14:10:52 +0200 (CEST) Subject: [pypy-svn] r26093 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060421121052.EE88D10086@code0.codespeak.net> Author: hpk Date: Fri Apr 21 14:10:51 2006 New Revision: 26093 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: updated to the info i actually send out (just before Anders committed of course ...) Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Fri Apr 21 14:10:51 2006 @@ -14,7 +14,7 @@ preferred arrival date: 20th acceptable arrival dates: 21th,22nd preferred departure date: 24th acceptable departure dates: 25th start city / airport: Gothenburg -mail: bea at changemaker.nu +mail: bea at changemaker.nu birthdate: 3rd Dec 1973 address: Eklandagatan5, 412 55 G?teborg, Sweden organisation: Change Maker @@ -26,7 +26,7 @@ preferred arrival date: 20th acceptable arrival dates: 19th, 21st preferred departure date: 28th acceptable departure dates: 27th,29th start city / airport: Hildesheim / Frankfurt (FRA) -mail: holger at merlinux.de +mail: holger at merlinux.de birthdate: 14th nov. 1969, phone number: +49 171 464 8622 address: Steinbergstr. 42, 31139 Hildesheim, Germany @@ -64,7 +64,7 @@ preferred arrival date: 20th acceptable arrival dates: 19th, 21st preferred departure date: 28th acceptable departure dates: 27th, 29th start city / airport: L?beck / Hamburg -mail: jan at merlinux.de +mail: jan at merlinux.de birthdate: 11th jan. 1978, address: Buergerweide 1, 23562 L?beck, Germany @@ -103,7 +103,7 @@ preferred arrival date: 20th acceptable arrival dates: 19th, 21st preferred departure date: 28th acceptable departure dates: 27th,29th start city / airport: Hildesheim / Frankfurt (FRA), Hannover, Berlin or Hamburg -mail: wanja at merlinux.de +mail: wanja at merlinux.de birthdate: 22.02.1975 address: Wrangelstr. 67, 10997 Berlin, Germany organisation: Merlinux GmbH From arigo at codespeak.net Fri Apr 21 14:12:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 14:12:52 +0200 (CEST) Subject: [pypy-svn] r26094 - in pypy/dist/pypy/rpython: . rctypes rctypes/test Message-ID: <20060421121252.8E54110080@code0.codespeak.net> Author: arigo Date: Fri Apr 21 14:12:50 2006 New Revision: 26094 Modified: pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/aarray.py pypy/dist/pypy/rpython/rctypes/astruct.py pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Support for passing (positional or keyword) arguments in structure construction. Reworked a bit the rtyping of built-in functions taking keyword arguments to not loose any information. Still not completely nice, but given that it's only the 2nd example of a built-in function with keywords... Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Fri Apr 21 14:12:50 2006 @@ -57,44 +57,49 @@ def __init__(self, builtinfunc): self.builtinfunc = builtinfunc - def rtype_simple_call(self, hop): + def findbltintyper(self, rtyper): + "Find the function to use to specialize calls to this built-in func." + try: + return BUILTIN_TYPER[self.builtinfunc] + except (KeyError, TypeError): + pass try: - bltintyper = BUILTIN_TYPER[self.builtinfunc] + return rtyper.type_system.rbuiltin.BUILTIN_TYPER[self.builtinfunc] except (KeyError, TypeError): - try: - rtyper = hop.rtyper - bltintyper = rtyper.type_system.rbuiltin.\ - BUILTIN_TYPER[self.builtinfunc] - except (KeyError, TypeError): -## if hasattr(self.builtinfunc,"specialize"): -## bltintyper = self.builtinfunc.specialize -## else - if extregistry.is_registered(self.builtinfunc): - entry = extregistry.lookup(self.builtinfunc) - bltintyper = entry.specialize_call - else: - raise TyperError("don't know about built-in function %r" % ( - self.builtinfunc,)) + pass + if extregistry.is_registered(self.builtinfunc): + entry = extregistry.lookup(self.builtinfunc) + return entry.specialize_call + raise TyperError("don't know about built-in function %r" % ( + self.builtinfunc,)) + + def rtype_simple_call(self, hop): + bltintyper = self.findbltintyper(hop.rtyper) hop2 = hop.copy() hop2.r_s_popfirstarg() return bltintyper(hop2) def rtype_call_args(self, hop): # calling a built-in function with keyword arguments: - # mostly for rpython.objectmodel.hint() + # mostly for rpython.objectmodel.hint() and for constructing + # rctypes structures from pypy.interpreter.argument import Arguments arguments = Arguments.fromshape(None, hop.args_s[1].const, # shape - hop.args_s[2:]) + range(hop.nb_args-2)) args_s, kwds = arguments.unpack() - # prefix keyword arguments with 's_' - kwds_s = {} - for key, s_value in kwds.items(): - kwds_s['s_'+key] = s_value - bltintyper = BUILTIN_TYPER[self.builtinfunc] + # prefix keyword arguments with 'i_' + kwds_i = {} + for key, index in kwds.items(): + kwds_i['i_'+key] = index + + bltintyper = self.findbltintyper(hop.rtyper) hop2 = hop.copy() hop2.r_s_popfirstarg() hop2.r_s_popfirstarg() - return bltintyper(hop2, **kwds_s) + # the RPython-level keyword args are passed with an 'i_' prefix and + # the corresponding value is an *index* in the hop2 arguments, + # to be used with hop.inputarg(arg=..) + return bltintyper(hop2, **kwds_i) class BuiltinMethodRepr(Repr): @@ -484,12 +489,13 @@ # hint -def rtype_hint(hop, **kwds_s): +def rtype_hint(hop, **kwds_i): hints = {} - for key, s_value in kwds_s.items(): + for key, index in kwds_i.items(): + s_value = hop.args_s[index] if not s_value.is_constant(): raise TyperError("hint %r is not constant" % (key,)) - assert key.startswith('s_') + assert key.startswith('i_') hints[key[2:]] = s_value.const v = hop.inputarg(hop.args_r[0], arg=0) c_hint = hop.inputconst(lltype.Void, hints) Modified: pypy/dist/pypy/rpython/rctypes/aarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/aarray.py Fri Apr 21 14:12:50 2006 @@ -7,10 +7,7 @@ def arraytype_specialize_call(hop): r_array = hop.r_result - return hop.genop("malloc", [ - hop.inputconst(lltype.Void, r_array.lowleveltype.TO), - ], resulttype=r_array.lowleveltype, - ) + return r_array.allocate_instance(hop.llops) def arraytype_compute_annotation(metatype, type): def compute_result_annotation(*arg_s): Modified: pypy/dist/pypy/rpython/rctypes/astruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/astruct.py Fri Apr 21 14:12:50 2006 @@ -6,15 +6,41 @@ StructType = type(Structure) -def structtype_specialize_call(hop): +def structtype_specialize_call(hop, **kwds_i): + from pypy.rpython.error import TyperError r_struct = hop.r_result - return hop.genop("malloc", [ - hop.inputconst(lltype.Void, r_struct.lowleveltype.TO), - ], resulttype=r_struct.lowleveltype, - ) + v_result = r_struct.allocate_instance(hop.llops) + index_by_name = {} + name_by_index = {} + # collect the keyword arguments + for key, index in kwds_i.items(): + assert key.startswith('i_') + name = key[2:] + assert index not in name_by_index + index_by_name[name] = index + name_by_index[index] = name + # add the positional arguments + fieldsiter = iter(r_struct.c_data_type._names) + for i in range(hop.nb_args): + if i not in name_by_index: + try: + name = fieldsiter.next() + except StopIteration: + raise TyperError("too many arguments in struct construction") + if name in index_by_name: + raise TyperError("multiple values for field %r" % (name,)) + index_by_name[name] = i + name_by_index[i] = name + # initialize the fields from the arguments, as far as they are present + for name in r_struct.c_data_type._names: + if name in index_by_name: + index = index_by_name[name] + v_valuebox = hop.inputarg(r_struct.r_fields[name], arg=index) + r_struct.setfield(hop.llops, v_result, name, v_valuebox) + return v_result def structtype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s): + def compute_result_annotation(*arg_s, **kwds_s): return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) return SomeBuiltin(compute_result_annotation, methodname=type.__name__) Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Fri Apr 21 14:12:50 2006 @@ -105,13 +105,17 @@ name = s_attr.const r_field = self.r_fields[name] v_struct, v_attr, v_item = hop.inputargs(self, lltype.Void, r_field) - v_newvalue = r_field.get_c_data_or_value(hop.llops, v_item) + self.setfield(hop.llops, v_struct, name, v_item) + + def setfield(self, llops, v_struct, name, v_item): + r_field = self.r_fields[name] + v_newvalue = r_field.get_c_data_or_value(llops, v_item) # copy the new value (which might be a whole substructure) - v_c_struct = self.get_c_data(hop.llops, v_struct) - genreccopy_structfield(hop.llops, v_newvalue, v_c_struct, name) + v_c_struct = self.get_c_data(llops, v_struct) + genreccopy_structfield(llops, v_newvalue, v_c_struct, name) # copy the keepalive information too - v_newkeepalive = r_field.getkeepalive(hop.llops, v_item) + v_newkeepalive = r_field.getkeepalive(llops, v_item) if v_newkeepalive is not None: - v_keepalive_struct = self.getkeepalive(hop.llops, v_struct) - genreccopy_structfield(hop.llops, v_newkeepalive, + v_keepalive_struct = self.getkeepalive(llops, v_struct) + genreccopy_structfield(llops, v_newkeepalive, v_keepalive_struct, name) Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Fri Apr 21 14:12:50 2006 @@ -84,6 +84,12 @@ s.p.contents.value = 124 assert y.value == 124 + s = tagpoint(x=12) + assert s.x == 12 + s = tagpoint(17, p=pointer(z)) + assert s.x == 17 + assert s.p.contents.value == -33 + def test_void_p(): x = c_int(12) p1 = cast(pointer(x), c_void_p) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Fri Apr 21 14:12:50 2006 @@ -4,6 +4,7 @@ import py.test import pypy.rpython.rctypes.implementation +from pypy.rpython.error import TyperError from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy import conftest @@ -12,6 +13,7 @@ from pypy.rpython.test.test_llinterp import interpret from ctypes import c_int, c_short, Structure, POINTER, pointer, c_char_p +from ctypes import c_char class tagpoint(Structure): _fields_ = [("x", c_int), @@ -167,6 +169,39 @@ res = interpret(func, []) assert res == 289 + def test_specialize_constructor_args(self): + class S(Structure): + _fields_ = [('x', c_int), + ('y', c_char)] + def func(x, y): + s0 = S(x) + s1 = S(x, y) + s2 = S(y=y) + s3 = S(x, y=y) + return (s0, s1, s2, s3) + + res = interpret(func, [4, '?']) + assert res.item0.c_data.x == 4 + assert res.item0.c_data.y == '\x00' + assert res.item1.c_data.x == 4 + assert res.item1.c_data.y == '?' + assert res.item2.c_data.x == 0 + assert res.item2.c_data.y == '?' + assert res.item3.c_data.x == 4 + assert res.item3.c_data.y == '?' + + def test_specialize_bad_constructor_args(self): + class S(Structure): + _fields_ = [('x', c_int), + ('y', c_char)] + def f1(x, y): + S(x, y, 7) + py.test.raises(TyperError, "interpret(f1, [4, '?'])") + + def f2(x): + S(x, x=5) + py.test.raises(TyperError, "interpret(f2, [4])") + class Test_compilation: def test_compile_struct_access(self): def access_struct(n): From arigo at codespeak.net Fri Apr 21 14:16:12 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 14:16:12 +0200 (CEST) Subject: [pypy-svn] r26095 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060421121612.63AE010080@code0.codespeak.net> Author: arigo Date: Fri Apr 21 14:16:11 2006 New Revision: 26095 Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py Log: Oups, an import that brought the whole rtyper in. Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Fri Apr 21 14:16:11 2006 @@ -2,7 +2,6 @@ from pypy.annotation.model import SomeCTypesObject from pypy.rpython import extregistry from pypy.rpython.lltypesystem import lltype -from pypy.rpython.robject import pyobj_repr # __________ py_object() calls __________ @@ -11,6 +10,7 @@ return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) def py_object_specialize_call(hop): + from pypy.rpython.robject import pyobj_repr r_pyobject = hop.r_result v_result = r_pyobject.allocate_instance(hop.llops) if len(hop.args_s): From antocuni at codespeak.net Fri Apr 21 14:27:13 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 21 Apr 2006 14:27:13 +0200 (CEST) Subject: [pypy-svn] r26096 - in pypy/dist/pypy/rpython: . lltypesystem ootypesystem test Message-ID: <20060421122713.63FAF10088@code0.codespeak.net> Author: antocuni Date: Fri Apr 21 14:27:04 2006 New Revision: 26096 Modified: pypy/dist/pypy/rpython/lltypesystem/rdict.py pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: Basic ootypesystem.rdict support Modified: pypy/dist/pypy/rpython/lltypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rdict.py Fri Apr 21 14:27:04 2006 @@ -58,23 +58,6 @@ self.dict_cache = {} self._custom_eq_hash_repr = custom_eq_hash # setup() needs to be called to finish this initialization - - def pickrepr(self, item_repr): - if self.custom_eq_hash: - return item_repr, item_repr - else: - return rmodel.externalvsinternal(self.rtyper, item_repr) - - def pickkeyrepr(self, key_repr): - external, internal = self.pickrepr(key_repr) - if external != internal: - internal = external - while not self.rtyper.needs_hash_support(internal.classdef): - internal = internal.rbase - return external, internal - - def compact_repr(self): - return 'DictR %s %s' % (self.key_repr.compact_repr(), self.value_repr.compact_repr()) def _setup_repr(self): if 'key_repr' not in self.__dict__: @@ -186,11 +169,6 @@ self.DICT.become(lltype.GcStruct("dicttable", adtmeths=adtmeths, *fields)) - def recast_value(self, llops, v): - return llops.convertvar(v, self.value_repr, self.external_value_repr) - - def recast_key(self, llops, v): - return llops.convertvar(v, self.key_repr, self.external_key_repr) def convert_const(self, dictobj): # get object from bound dict methods Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Fri Apr 21 14:27:04 2006 @@ -1 +1,94 @@ -# TODO +from pypy.annotation.pairtype import pairtype +from pypy.annotation import model as annmodel +from pypy.objspace.flow.model import Constant +from pypy.rpython.rdict import AbstractDictRepr, rtype_newdict +from pypy.rpython.ootypesystem import ootype +from pypy.rpython.rarithmetic import r_uint +from pypy.rpython.objectmodel import hlinvoke +from pypy.rpython import robject +from pypy.rpython import objectmodel +from pypy.rpython import rmodel + + +class DictRepr(AbstractDictRepr): + def __init__(self, rtyper, key_repr, value_repr, dictkey, dictvalue, + custom_eq_hash=None): + self.rtyper = rtyper + self.custom_eq_hash = custom_eq_hash is not None + + already_computed = True + if not isinstance(key_repr, rmodel.Repr): # not computed yet, done by setup() + assert callable(key_repr) + self._key_repr_computer = key_repr + already_computed = False + else: + self.external_key_repr, self.key_repr = self.pickkeyrepr(key_repr) + if not isinstance(value_repr, rmodel.Repr): # not computed yet, done by setup() + assert callable(value_repr) + self._value_repr_computer = value_repr + already_computed = False + else: + self.external_value_repr, self.value_repr = self.pickrepr(value_repr) + + if already_computed: + self.DICT = ootype.Dict(key_repr.lowleveltype, value_repr.lowleveltype) + else: + self.DICT = ootype.ForwardReference() + self.lowleveltype = self.DICT + + self.dictkey = dictkey + self.dictvalue = dictvalue + self.dict_cache = {} + self._custom_eq_hash_repr = custom_eq_hash + # setup() needs to be called to finish this initialization + + def _setup_repr(self): + if 'key_repr' not in self.__dict__: + key_repr = self._key_repr_computer() + self.external_key_repr, self.key_repr = self.pickkeyrepr(key_repr) + if 'value_repr' not in self.__dict__: + self.external_value_repr, self.value_repr = self.pickrepr(self._value_repr_computer()) + + if isinstance(self.DICT, ootype.ForwardReference): + self.lowleveltype.become(ootype.Dict(self.key_repr.lowleveltype, + self.value_repr.lowleveltype)) + + def send_message(self, hop, message, can_raise=False, v_args=None): + if v_args is None: + v_args = hop.inputargs(self, *hop.args_r[1:]) + c_name = hop.inputconst(ootype.Void, message) + if can_raise: + hop.exception_is_here() + return hop.genop("oosend", [c_name] + v_args, + resulttype=hop.r_result.lowleveltype) + + +class __extend__(pairtype(DictRepr, rmodel.Repr)): + + def rtype_getitem((r_dict, r_key), hop): + v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + if not r_dict.custom_eq_hash: # TODO: why only in this case? + hop.has_implicit_exception(KeyError) # record that we know about it + hop.exception_is_here() + v_res = r_dict.send_message(hop, 'll_getitem', can_raise=True) + return r_dict.recast_value(hop.llops, v_res) + +## def rtype_delitem((r_dict, r_key), hop): +## v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) +## if not r_dict.custom_eq_hash: +## hop.has_implicit_exception(KeyError) # record that we know about it +## hop.exception_is_here() +## return hop.gendirectcall(ll_dict_delitem, v_dict, v_key) + + def rtype_setitem((r_dict, r_key), hop): + v_dict, v_key, v_value = hop.inputargs(r_dict, r_dict.key_repr, r_dict.value_repr) +## if r_dict.custom_eq_hash: +## hop.exception_is_here() +## else: +## hop.exception_cannot_occur() + hop.exception_is_here() + return r_dict.send_message(hop, 'll_setitem', can_raise=True) + + +def ll_newdict(DICT): + return ootype.new(DICT) Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Fri Apr 21 14:27:04 2006 @@ -40,7 +40,7 @@ def prepare_const(self, n): result = self.LIST.ll_newlist(n) return result - + def send_message(self, hop, message, can_raise=False, v_args=None): if v_args is None: Modified: pypy/dist/pypy/rpython/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/rdict.py (original) +++ pypy/dist/pypy/rpython/rdict.py Fri Apr 21 14:27:04 2006 @@ -6,7 +6,7 @@ from pypy.rpython.objectmodel import hlinvoke from pypy.rpython import robject from pypy.rpython import objectmodel -from pypy.rpython.rmodel import Repr +from pypy.rpython import rmodel class __extend__(annmodel.SomeDict): @@ -36,8 +36,30 @@ -class AbstractDictRepr(Repr): - pass +class AbstractDictRepr(rmodel.Repr): + + def pickrepr(self, item_repr): + if self.custom_eq_hash: + return item_repr, item_repr + else: + return rmodel.externalvsinternal(self.rtyper, item_repr) + + def pickkeyrepr(self, key_repr): + external, internal = self.pickrepr(key_repr) + if external != internal: + internal = external + while not self.rtyper.needs_hash_support(internal.classdef): + internal = internal.rbase + return external, internal + + def compact_repr(self): + return 'DictR %s %s' % (self.key_repr.compact_repr(), self.value_repr.compact_repr()) + + def recast_value(self, llops, v): + return llops.convertvar(v, self.value_repr, self.external_value_repr) + + def recast_key(self, llops, v): + return llops.convertvar(v, self.key_repr, self.external_key_repr) def rtype_newdict(hop): Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Fri Apr 21 14:27:04 2006 @@ -7,7 +7,23 @@ import py py.log.setconsumer("rtyper", py.log.STDOUT) -def test_dict_creation(): +class BaseTestDictRtyping: + def interpret(self, fn, args): + return interpret(fn, args, type_system=self.ts) + + def interpret_raises(self, exc, fn, args): + return interpret_raises(exc, fn, args, type_system=self.ts) + + + def test_dict_creation(self): + def createdict(i): + d = {i: i+1} + return d[i] + res = self.interpret(createdict, [42]) + assert res == 43 + +# XXX: most tests doesn't works because ootypesystem doesn't support strings, yet +def test_dict_creation(): def createdict(i): d = {'hello' : i} return d['hello'] @@ -708,3 +724,11 @@ return d[2] res = interpret(g, [3]) assert res == 77 + + + +class TestLltypeRtyping(BaseTestDictRtyping): + ts = "lltype" + +class TestOotypeRtyping(BaseTestDictRtyping): + ts = "ootype" From arigo at codespeak.net Fri Apr 21 14:28:37 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 14:28:37 +0200 (CEST) Subject: [pypy-svn] r26097 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060421122837.A707810089@code0.codespeak.net> Author: arigo Date: Fri Apr 21 14:28:36 2006 New Revision: 26097 Modified: pypy/dist/pypy/rpython/rctypes/aarray.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Log: rctypes: support for array(..initial values..). Note that it's of limited usefulness because the annotator doesn't support *args where 'args' is a list. Modified: pypy/dist/pypy/rpython/rctypes/aarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/aarray.py Fri Apr 21 14:28:36 2006 @@ -6,8 +6,18 @@ ArrayType = type(ARRAY(c_int, 10)) def arraytype_specialize_call(hop): + from pypy.rpython.error import TyperError + from pypy.rpython.rmodel import inputconst r_array = hop.r_result - return r_array.allocate_instance(hop.llops) + v_result = r_array.allocate_instance(hop.llops) + if hop.nb_args > r_array.length: + raise TyperError("too many arguments for an array of length %d" % ( + r_array.length,)) + for i in range(hop.nb_args): + v_item = hop.inputarg(r_array.r_item, arg=i) + c_index = inputconst(lltype.Signed, i) + r_array.setitem(hop.llops, v_result, c_index, v_item) + return v_result def arraytype_compute_annotation(metatype, type): def compute_result_annotation(*arg_s): Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Fri Apr 21 14:28:36 2006 @@ -87,6 +87,18 @@ ## v_c_array = self.get_c_data(llops, v_array) ## llops.genop('setarrayitem', [v_c_array, v_index, v_newvalue]) + def setitem(self, llops, v_array, v_index, v_item): + v_newvalue = self.r_item.get_c_data_or_value(llops, v_item) + # copy the new value (which might be a whole structure) + v_c_array = self.get_c_data(llops, v_array) + genreccopy_arrayitem(llops, v_newvalue, v_c_array, v_index) + # copy the keepalive information too + v_keepalive_array = self.getkeepalive(llops, v_array) + if v_keepalive_array is not None: + v_newkeepalive = self.r_item.getkeepalive(llops, v_item) + genreccopy_arrayitem(llops, v_newkeepalive, + v_keepalive_array, v_index) + class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_getitem((r_array, r_int), hop): @@ -106,16 +118,7 @@ def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item) - v_newvalue = r_array.r_item.get_c_data_or_value(hop.llops, v_item) - # copy the new value (which might be a whole structure) - v_c_array = r_array.get_c_data(hop.llops, v_array) - genreccopy_arrayitem(hop.llops, v_newvalue, v_c_array, v_index) - # copy the keepalive information too - v_keepalive_array = r_array.getkeepalive(hop.llops, v_array) - if v_keepalive_array is not None: - v_newkeepalive = r_array.r_item.getkeepalive(hop.llops, v_item) - genreccopy_arrayitem(hop.llops, v_newkeepalive, - v_keepalive_array, v_index) + r_array.setitem(hop.llops, v_array, v_index, v_item) class __extend__(pairtype(ArrayRepr, PointerRepr)): Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Fri Apr 21 14:28:36 2006 @@ -139,6 +139,9 @@ s.p.contents.value = 42 assert a[0] == 42 + a = (c_int * 5)(5, 6, 7) + assert list(a) == [5, 6, 7, 0, 0] + def test_truth_value(): p = POINTER(c_int)() assert not p Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Fri Apr 21 14:28:36 2006 @@ -257,6 +257,17 @@ func() interpret(func, []) + def test_specialize_constructor_args(self): + A = c_int * 5 + def func(x, y): + return A(x, y) + res = interpret(func, [123, 456]) + assert res.c_data[0] == 123 + assert res.c_data[1] == 456 + assert res.c_data[2] == 0 + assert res.c_data[3] == 0 + assert res.c_data[4] == 0 + class Test_compilation: def test_compile_array_access(self): def access_array(): From arigo at codespeak.net Fri Apr 21 14:35:03 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 14:35:03 +0200 (CEST) Subject: [pypy-svn] r26098 - in pypy/dist/pypy: rpython/rctypes translator/goal Message-ID: <20060421123503.013D71008E@code0.codespeak.net> Author: arigo Date: Fri Apr 21 14:35:02 2006 New Revision: 26098 Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Wow! Passing a pure Python callback object to a built-in function compiled as a C extension module was incredibly easy. Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Fri Apr 21 14:35:02 2006 @@ -28,3 +28,15 @@ # from a function exposed in a C extension module def convert_from_to((r_from, r_to), v, llops): return r_from.getvalue(llops, v) + +class __extend__(pairtype(PyObjRepr, CTypesPyObjRepr)): + # conversion used by wrapper.py in genc when passing a py_object + # argument into a function exposed in a C extension module + def convert_from_to((r_from, r_to), v, llops): + # allocate a memory-owning box to hold a copy of the 'PyObject*' + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + r_temp.setvalue(llops, v_owned_box, v) + # return this box possibly converted to the expected output repr, + # which might be a memory-aliasing box + return llops.convertvar(v_owned_box, r_temp, r_to) Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Fri Apr 21 14:35:02 2006 @@ -62,13 +62,13 @@ space = CPyObjSpace() -def entry_point(n): - return demo.measuretime(space, n, space.w_int) +def entry_point(n, w_callable): + return demo.measuretime(space, n, w_callable) # _____ Define and setup target ___ def target(*args): - return entry_point, [int], PyPyAnnotatorPolicy() + return entry_point, [int, py_object], PyPyAnnotatorPolicy() if __name__ == '__main__': @@ -78,4 +78,4 @@ else: N = int(sys.argv[1]) print 'Timing for %d iterations...' % N - print entry_point(N), 'seconds' + print entry_point(N, int), 'seconds' From arigo at codespeak.net Fri Apr 21 15:43:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 15:43:16 +0200 (CEST) Subject: [pypy-svn] r26102 - in pypy/dist/pypy: module/_demo rpython/rctypes rpython/rctypes/socketmodule rpython/rctypes/test rpython/rctypes/tool rpython/rctypes/tool/test translator/goal Message-ID: <20060421134316.CCCFD10084@code0.codespeak.net> Author: arigo Date: Fri Apr 21 15:43:13 2006 New Revision: 26102 Added: pypy/dist/pypy/rpython/rctypes/tool/ (props changed) pypy/dist/pypy/rpython/rctypes/tool/__init__.py - copied unchanged from r26097, pypy/dist/pypy/rpython/rctypes/__init__.py pypy/dist/pypy/rpython/rctypes/tool/compilemodule.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py - copied unchanged from r26097, pypy/dist/pypy/rpython/rctypes/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/tool/test/ (props changed) pypy/dist/pypy/rpython/rctypes/tool/test/__init__.py - copied unchanged from r26097, pypy/dist/pypy/rpython/rctypes/test/__init__.py pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py - copied, changed from r26097, pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Removed: pypy/dist/pypy/rpython/rctypes/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes_platform.py Modified: pypy/dist/pypy/module/_demo/demo.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Intermediate check-in: reorganize files, add a dir rctypes/tool/ where things will be moved to when they start working in targetdemomodule.py. The CPyObjSpace is now there. Any ideas for a better place to put the compilemodule.py script? Modified: pypy/dist/pypy/module/_demo/demo.py ============================================================================== --- pypy/dist/pypy/module/_demo/demo.py (original) +++ pypy/dist/pypy/module/_demo/demo.py Fri Apr 21 15:43:13 2006 @@ -1,6 +1,6 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import ObjSpace, W_Root -from pypy.rpython.rctypes import implementation, ctypes_platform +from pypy.rpython.rctypes.tool import ctypes_platform import sys from ctypes import * Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Fri Apr 21 15:43:13 2006 @@ -1,5 +1,5 @@ import os -from pypy.rpython.rctypes import ctypes_platform +from pypy.rpython.rctypes.tool import ctypes_platform from ctypes import * includes = ('sys/types.h', Added: pypy/dist/pypy/rpython/rctypes/tool/compilemodule.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/tool/compilemodule.py Fri Apr 21 15:43:13 2006 @@ -0,0 +1,32 @@ +#! /usr/bin/env python +""" +Usage: compilemodule.py + +Compiles the PyPy extension module from pypy/module// +into a regular CPython extension module. +""" + +import sys +import pypy.rpython.rctypes.implementation +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy +from pypy.rpython.rctypes.tool.cpyobjspace import CPyObjSpace +from pypy.translator.driver import TranslationDriver + + +def compilemodule(modname): + "Compile a PyPy module for CPython." + + space = CPyObjSpace() + ModuleClass = __import__('pypy.module.%s' % modname, + None, None, ['Module']).Module + module = ModuleClass(space, space.wrap(modname)) + w_moduledict = module.getdict() + + XXX in-progress, for now see translator/goal/targetdemomodule.py + + +if __name__ == '__main__': + if len(sys.argv) != 2: + print >> sys.stderr, __doc__ + sys.exit(2) + compilemodule(sys.argv[1]) Added: pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py Fri Apr 21 15:43:13 2006 @@ -0,0 +1,88 @@ +import sys +from ctypes import * + +assert sys.version < (2, 5), "XXX fix Py_ssize_t for Python 2.5" +Py_ssize_t = c_int + +PyObject_GetAttr = pythonapi.PyObject_GetAttr +PyObject_GetAttr.argtypes = [py_object, py_object] +PyObject_GetAttr.restype = py_object + +PyImport_ImportModule = pythonapi.PyImport_ImportModule +PyImport_ImportModule.argtypes = [c_char_p] +PyImport_ImportModule.restype = py_object + +PyInt_FromLong = pythonapi.PyInt_FromLong +PyInt_FromLong.argtypes = [c_long] +PyInt_FromLong.restype = py_object + +PyString_FromStringAndSize = pythonapi.PyString_FromStringAndSize +PyString_FromStringAndSize.argtypes = [c_char_p, Py_ssize_t] +PyString_FromStringAndSize.restype = py_object + +PyString_InternInPlace = pythonapi.PyString_InternInPlace +PyString_InternInPlace.argtypes = [POINTER(py_object)] +PyString_InternInPlace.restype = None + +PyObject_SetItem = pythonapi.PyObject_SetItem +PyObject_SetItem.argtypes = [py_object, py_object, py_object] +PyObject_SetItem.restype = c_int + +PyObject_Call = pythonapi.PyObject_Call +PyObject_Call.argtypes = [py_object, py_object, py_object] +PyObject_Call.restype = py_object + +PyTuple_New = pythonapi.PyTuple_New +PyTuple_New.argtypes = [Py_ssize_t] +PyTuple_New.restype = py_object + +PyDict_New = pythonapi.PyDict_New +PyDict_New.argtypes = [] +PyDict_New.restype = py_object + +PyDict_SetItem = pythonapi.PyDict_SetItem +PyDict_SetItem.argtypes = [py_object, py_object, py_object] +PyDict_SetItem.restype = c_int + + +class CPyObjSpace: + W_Object = py_object + + def __init__(self): + self.w_int = py_object(int) + self.w_None = py_object(None) + self.w_False = py_object(False) + self.w_True = py_object(True) + + def getbuiltinmodule(self, name): + return PyImport_ImportModule(name) + + def wrap(self, x): + if x is None: + return self.w_None + if isinstance(x, int): + return PyInt_FromLong(x) + if isinstance(x, str): + return PyString_FromStringAndSize(x, len(x)) + raise TypeError("wrap(%r)" % (x,)) + wrap._annspecialcase_ = "specialize:wrap" + + def getattr(self, w_obj, w_attr): + return PyObject_GetAttr(w_obj, w_attr) + + def call_function(self, w_callable): + return PyObject_Call(w_callable, PyTuple_New(0), PyDict_New()) + + def _freeze_(self): + return True + + def new_interned_str(self, s): + w_s = self.wrap(s) + PyString_InternInPlace(byref(w_s)) + return w_s + + def newdict(self, items_w): + w_dict = PyDict_New() + for w_key, w_value in items_w: + PyDict_SetItem(w_dict, w_key, w_value) + return w_dict Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Fri Apr 21 15:43:13 2006 @@ -1,63 +1,7 @@ from pypy.module._demo import demo from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy - -from ctypes import * - -Py_ssize_t = c_int # XXX changes in Python 2.5 - -PyObject_GetAttr = pythonapi.PyObject_GetAttr -PyObject_GetAttr.argtypes = [py_object, py_object] -PyObject_GetAttr.restype = py_object - -PyImport_ImportModule = pythonapi.PyImport_ImportModule -PyImport_ImportModule.argtypes = [c_char_p] -PyImport_ImportModule.restype = py_object - -PyInt_FromLong = pythonapi.PyInt_FromLong -PyInt_FromLong.argtypes = [c_long] -PyInt_FromLong.restype = py_object - -PyString_FromStringAndSize = pythonapi.PyString_FromStringAndSize -PyString_FromStringAndSize.argtypes = [c_char_p, Py_ssize_t] -PyString_FromStringAndSize.restype = py_object - -PyObject_Call = pythonapi.PyObject_Call -PyObject_Call.argtypes = [py_object, py_object, py_object] -PyObject_Call.restype = py_object - -PyTuple_New = pythonapi.PyTuple_New -PyTuple_New.argtypes = [Py_ssize_t] -PyTuple_New.restype = py_object - -PyDict_New = pythonapi.PyDict_New -PyDict_New.argtypes = [] -PyDict_New.restype = py_object - - -class CPyObjSpace: - - def __init__(self): - self.w_int = py_object(int) - self.w_None = py_object(None) - - def getbuiltinmodule(self, name): - return PyImport_ImportModule(name) - - def wrap(self, x): - if x is None: - return self.w_None - if isinstance(x, int): - return PyInt_FromLong(x) - if isinstance(x, str): - return PyString_FromStringAndSize(x, len(x)) - raise TypeError("wrap(%r)" % (x,)) - wrap._annspecialcase_ = "specialize:wrap" - - def getattr(self, w_obj, w_attr): - return PyObject_GetAttr(w_obj, w_attr) - - def call_function(self, w_callable): - return PyObject_Call(w_callable, PyTuple_New(0), PyDict_New()) +from pypy.rpython.rctypes.tool.cpyobjspace import CPyObjSpace +import pypy.rpython.rctypes.implementation space = CPyObjSpace() @@ -68,7 +12,7 @@ # _____ Define and setup target ___ def target(*args): - return entry_point, [int, py_object], PyPyAnnotatorPolicy() + return entry_point, [int, CPyObjSpace.W_Object], PyPyAnnotatorPolicy() if __name__ == '__main__': From cfbolz at codespeak.net Fri Apr 21 16:29:32 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 21 Apr 2006 16:29:32 +0200 (CEST) Subject: [pypy-svn] r26104 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060421142932.2D17510086@code0.codespeak.net> Author: cfbolz Date: Fri Apr 21 16:29:31 2006 New Revision: 26104 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: the manual inlining of RPyExceptionOccured broke those two brittle test Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Fri Apr 21 16:29:31 2006 @@ -293,7 +293,7 @@ for gc in rgc, bgc: t, transformer = rtype_and_transform(f, [object], gc, check=False) ops = getops(graphof(t, f)) - assert len(ops.get('getfield', [])) == 1 + assert len(ops.get('getfield', [])) == 2 def DONOTtest_protect_unprotect_no_exception_block(): def p(): protect('this is an object') @@ -396,7 +396,7 @@ t, transformer = rtype_and_transform(f, [], gctransform.RefcountingGCTransformer, check=False) graph = graphof(t, f) ops = getops(graph) - assert len(ops['getfield']) == 2 + assert len(ops['getfield']) == 5 assert len(ops['setfield']) == 4 def test_arraybarrier(): From mwh at codespeak.net Fri Apr 21 16:44:32 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 21 Apr 2006 16:44:32 +0200 (CEST) Subject: [pypy-svn] r26107 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060421144432.BA53710086@code0.codespeak.net> Author: mwh Date: Fri Apr 21 16:44:30 2006 New Revision: 26107 Added: pypy/dist/pypy/translator/stackless/test/test_depth.py (contents, props changed) Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/transform.py Log: add another test for the stackless transform, copied from the tests for the c version, which required a small fix to the way i make function pointers but pleasingly required no deep changes to the transform itself. Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Fri Apr 21 16:44:30 2006 @@ -20,6 +20,7 @@ ('c', llmemory.Address)) def switch(c): + # this is untested so far! if not global_state.restartstate: u = UnwindException() s = lltype.malloc(SWITCH_STATE) @@ -35,18 +36,24 @@ global_state.top = s.c return top.f_back + + def stack_frames_depth(): - if not global_state.restartstate: + if not global_state.restart_substate: u = UnwindException() s = lltype.malloc(STATE_HEADER) s.restartstate = 1 - s.function = llmemory.cast_ptr_to_adr(count_stack_depth) - s.retval_type = RETVAL_VOID - add_frame_state(u, s.header) + # the next three lines are pure rtyper-pleasing hacks + f = stack_frames_depth + if global_state.restart_substate: + f = None + s.function = llmemory.cast_ptr_to_adr(f) + s.retval_type = RETVAL_LONG + add_frame_state(u, s) raise u else: cur = global_state.top - global_state.restartstate = 0 + global_state.restart_substate = 0 depth = 0 while cur: depth += 1 Added: pypy/dist/pypy/translator/stackless/test/test_depth.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/stackless/test/test_depth.py Fri Apr 21 16:44:30 2006 @@ -0,0 +1,30 @@ +from pypy.translator.stackless.test.test_transform import \ + llinterp_stackless_function, run_stackless_function +from pypy.translator.stackless import code +import os + +def test_simple(): + def g1(): + "just to check Void special cases around the code" + def g2(ignored): + pass + g1() + def f(n): + g1() + if n > 0: + res = f(n-1) + else: + res = code.stack_frames_depth() + g2(g1) + return res + + def fn(ignored): + count0 = f(0) + count10 = f(10) + return count10 - count0 + + res = llinterp_stackless_function(fn, fn, f, g2, g1) + assert res == 10 + + res = run_stackless_function(fn, fn, f, g2, g1) + assert res.strip() == "10" Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Fri Apr 21 16:44:30 2006 @@ -8,6 +8,7 @@ from pypy.rpython.annlowlevel import MixLevelHelperAnnotator from pypy.translator.stackless import code from pypy.rpython.rclass import getinstancerepr +from pypy.rpython.typesystem import getfunctionptr from pypy.translator.stackless.code import STATE_HEADER, null_state @@ -99,9 +100,8 @@ ADD_FRAME_STATE_TYPE = lltype.FuncType( [self.unwind_exception_type, lltype.Ptr(STATE_HEADER)], lltype.Void) - self.add_frame_state_ptr = model.Constant(lltype.functionptr( - ADD_FRAME_STATE_TYPE, "add_frame_state", - graph=add_frame_state_graph), + self.add_frame_state_ptr = model.Constant( + getfunctionptr(add_frame_state_graph), lltype.Ptr(ADD_FRAME_STATE_TYPE)) RESUME_STATE_TYPE = lltype.FuncType([], lltype.Signed) From cfbolz at codespeak.net Fri Apr 21 17:50:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 21 Apr 2006 17:50:49 +0200 (CEST) Subject: [pypy-svn] r26108 - pypy/dist/pypy/rpython/memory Message-ID: <20060421155049.0D3131009C@code0.codespeak.net> Author: cfbolz Date: Fri Apr 21 17:50:48 2006 New Revision: 26108 Modified: pypy/dist/pypy/rpython/memory/gc.py pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/gcwrapper.py Log: factor out some initialization from the __init__ methods of the GCs into a setup method. This is done so that the GC instance can be built at compile time. Modified: pypy/dist/pypy/rpython/memory/gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gc.py (original) +++ pypy/dist/pypy/rpython/memory/gc.py Fri Apr 21 17:50:48 2006 @@ -32,6 +32,7 @@ def get_dummy_annotate(gc_class, AddressLinkedList): def dummy_annotate(): gc = gc_class(AddressLinkedList) + gc.setup() gc.get_roots = dummy_get_roots1 #prevent the get_roots attribute to gc.get_roots = dummy_get_roots2 #be constants a = gc.malloc(1, 2) @@ -87,6 +88,9 @@ "NOT_RPYTHON" pass + def setup(self): + pass + class DummyGC(GCBase): _alloc_flavor_ = "raw" @@ -121,11 +125,14 @@ self.heap_size = start_heap_size #need to maintain a list of malloced objects, since we used the systems #allocator and can't walk the heap - self.malloced_objects = AddressLinkedList() + self.malloced_objects = None self.AddressLinkedList = AddressLinkedList #self.set_query_functions(None, None, None, None, None, None, None) self.get_roots = get_roots + def setup(self): + self.malloced_objects = self.AddressLinkedList() + def malloc(self, typeid, length=0): if self.bytes_malloced > self.heap_size: self.collect() @@ -250,13 +257,18 @@ get_roots=None): self.bytes_malloced = 0 self.space_size = space_size - self.tospace = raw_malloc(space_size) - self.top_of_space = self.tospace + space_size - self.fromspace = raw_malloc(space_size) - self.free = self.tospace - #self.set_query_functions(None, None, None, None, None, None, None) + self.tospace = NULL + self.top_of_space = NULL + self.fromspace = NULL + self.free = NULL self.get_roots = get_roots + def setup(self): + self.tospace = raw_malloc(self.space_size) + self.top_of_space = self.tospace + self.space_size + self.fromspace = raw_malloc(self.space_size) + self.free = self.tospace + def free_memory(self): "NOT_RPYTHON" raw_free(self.tospace) @@ -386,7 +398,7 @@ _alloc_flavor_ = "raw" def __init__(self, AddressLinkedList, max_refcount_zero=50, get_roots=None): - self.zero_ref_counts = AddressLinkedList() + self.zero_ref_counts = None self.AddressLinkedList = AddressLinkedList self.length_zero_ref_counts = 0 self.max_refcount_zero = max_refcount_zero @@ -394,6 +406,10 @@ self.get_roots = get_roots self.collecting = False + def setup(self): + self.zero_ref_counts = self.AddressLinkedList() + + def malloc(self, typeid, length=0): size = self.fixed_size(typeid) if self.is_varsize(typeid): Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 21 17:50:48 2006 @@ -720,6 +720,7 @@ gcdata.static_roots = lltype.malloc(lltype.Array(llmemory.Address), 0, immortal=True) gcdata.static_root_start = gcdata.static_root_end = llmemory.cast_ptr_to_adr(gcdata.static_roots) + gcdata.gc = GCData.GCClass(AddressLinkedList, GCData.startheapsize, StackRootIterator) self.gcdata = gcdata self.type_info_list = [] self.id_of_type = {} # {LLTYPE: type_id} @@ -755,7 +756,7 @@ stackbase = lladdress.raw_malloc(GCData.rootstacksize) gcdata.root_stack_top = stackbase gcdata.root_stack_base = stackbase - gcdata.gc = GCData.GCClass(AddressLinkedList, GCData.startheapsize, StackRootIterator) + gcdata.gc.setup() gcdata.gc.set_query_functions( q_is_varsize, q_offsets_to_gc_pointers, Modified: pypy/dist/pypy/rpython/memory/gcwrapper.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gcwrapper.py (original) +++ pypy/dist/pypy/rpython/memory/gcwrapper.py Fri Apr 21 17:50:48 2006 @@ -189,6 +189,7 @@ self.constantroots = fgcc.cvter.constantroots self.pseudo_root_pointers = NULL self.roots = [] + self.gc.setup() def get_arg_malloc(self, TYPE, size=0): @@ -276,6 +277,7 @@ def instantiate_gc(): gc = gc_class(AddressLinkedList) gc.set_query_functions(f1, f2, f3, f4, f5, f6, f7) + gc.setup() return gc func, dummy_get_roots1, dummy_get_roots2 = gc.get_dummy_annotate( self.gc.__class__, self.AddressLinkedList) From cfbolz at codespeak.net Fri Apr 21 18:08:13 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 21 Apr 2006 18:08:13 +0200 (CEST) Subject: [pypy-svn] r26109 - pypy/dist/pypy/rpython/memory Message-ID: <20060421160813.7527C100A0@code0.codespeak.net> Author: cfbolz Date: Fri Apr 21 18:08:11 2006 New Revision: 26109 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: create the GC instance at transformation time instead of in the initialization function. This safes one getfield per malloc. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Apr 21 18:08:11 2006 @@ -720,7 +720,6 @@ gcdata.static_roots = lltype.malloc(lltype.Array(llmemory.Address), 0, immortal=True) gcdata.static_root_start = gcdata.static_root_end = llmemory.cast_ptr_to_adr(gcdata.static_roots) - gcdata.gc = GCData.GCClass(AddressLinkedList, GCData.startheapsize, StackRootIterator) self.gcdata = gcdata self.type_info_list = [] self.id_of_type = {} # {LLTYPE: type_id} @@ -750,6 +749,7 @@ if self.stack_current.address[0] != NULL: return self.stack_current return NULL + gcdata.gc = GCData.GCClass(AddressLinkedList, GCData.startheapsize, StackRootIterator) def frameworkgc_setup(): # run-time initialization code @@ -1017,28 +1017,21 @@ c_size = rmodel.inputconst(lltype.Signed, info["fixedsize"]) # surely there's a better way of doing this? - s_gcdata = self.translator.annotator.bookkeeper.immutablevalue(self.gcdata) - r_gcdata = self.translator.rtyper.getrepr(s_gcdata) s_gc = self.translator.annotator.bookkeeper.valueoftype(self.gcdata.GCClass) r_gc = self.translator.rtyper.getrepr(s_gc) - - newop0 = SpaceOperation( - "getfield", - [rmodel.inputconst(r_gcdata, self.gcdata), Constant("inst_gc", lltype.Void)], - varoftype(r_gc.lowleveltype)) + const_gc = rmodel.inputconst(r_gc, self.gcdata.gc) if len(op.args) == 1: - args = [self.malloc_fixedsize_ptr, newop0.result, c_type_id, + args = [self.malloc_fixedsize_ptr, const_gc, c_type_id, c_size] else: v_length = op.args[1] c_ofstolength = rmodel.inputconst(lltype.Signed, info['ofstolength']) c_varitemsize = rmodel.inputconst(lltype.Signed, info['varitemsize']) - args = [self.malloc_varsize_ptr, newop0.result, c_type_id, + args = [self.malloc_varsize_ptr, const_gc, c_type_id, v_length, c_size, c_varitemsize, c_ofstolength] newop = SpaceOperation("direct_call", args, v) ops, index = self.protect_roots(newop, livevars, block, block.operations.index(op)) - ops.insert(0, newop0) ops.append(SpaceOperation("cast_adr_to_ptr", [v], op.result)) return ops From cfbolz at codespeak.net Fri Apr 21 18:48:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 21 Apr 2006 18:48:18 +0200 (CEST) Subject: [pypy-svn] r26110 - pypy/dist/pypy/rpython Message-ID: <20060421164818.A4536100A0@code0.codespeak.net> Author: cfbolz Date: Fri Apr 21 18:48:18 2006 New Revision: 26110 Modified: pypy/dist/pypy/rpython/objectmodel.py Log: remove unnecessary import Modified: pypy/dist/pypy/rpython/objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/objectmodel.py (original) +++ pypy/dist/pypy/rpython/objectmodel.py Fri Apr 21 18:48:18 2006 @@ -25,7 +25,6 @@ return lltype.Signed import new -import weakref def instantiate(cls): "Create an empty instance of 'cls'." From arigo at codespeak.net Fri Apr 21 19:30:27 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 19:30:27 +0200 (CEST) Subject: [pypy-svn] r26111 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem rpython/memory/test rpython/rctypes rpython/rctypes/test rpython/rctypes/tool rpython/test tool Message-ID: <20060421173027.CB8F5100A0@code0.codespeak.net> Author: arigo Date: Fri Apr 21 19:30:22 2006 New Revision: 26111 Added: pypy/dist/pypy/tool/instancemethod.py (contents, props changed) Modified: pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/extregistry.py pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/lltypesystem/rclass.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py pypy/dist/pypy/rpython/rctypes/aarray.py pypy/dist/pypy/rpython/rctypes/achar_p.py pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/apointer.py pypy/dist/pypy/rpython/rctypes/aprimitive.py pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/astringbuf.py pypy/dist/pypy/rpython/rctypes/astruct.py pypy/dist/pypy/rpython/rctypes/avoid_p.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/test/test_extregistry.py Log: Refactored the extregistry interface. See mail to pypy-dev. This is a massive diff, mostly because all rctypes/a*.py are entierely modified. Tedious, but the result looks quite nicer than before in my humble opinion. The original motivation was a problem with annotating built-in functions: the 'analyser' attribute cannot be a local function with a closure, at least as used in rctypes, because re-building another closure with exactly the same values in the closure makes a non-equal function object. Instead, in the new extregistry world, this 'analyser' is a bound method on an ExtRegistryEntry instance. Tests showing this problem: * rctypes.test.test_rfunc.test_annotate_reflow_bug * targetdemomodule.py with the change in the cpyobjspace I then remembered the hard way that CPython bound method instances don't have a sane __eq__ either, so I bite the bullet and wrote a custom InstanceMethod class to use as the 'analyser'... Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Fri Apr 21 19:30:22 2006 @@ -358,7 +358,7 @@ _module = getattr(x,"__module__","unknown") result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) elif extregistry.is_registered(x): - result = extregistry.lookup(x).get_annotation(tp, x) + result = extregistry.lookup(x).compute_annotation() ## elif hasattr(x, "compute_result_annotation"): ## result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) ## elif hasattr(tp, "compute_annotation"): @@ -522,7 +522,7 @@ ## elif hasattr(t, "compute_annotation"): ## return t.compute_annotation() elif extregistry.is_registered_type(t): - return extregistry.lookup_type(t).get_annotation(t) + return extregistry.lookup_type(t).compute_annotation() elif t.__module__ != '__builtin__' and t not in self.pbctypes: classdef = self.getuniqueclassdef(t) return SomeInstance(classdef) Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Fri Apr 21 19:30:22 2006 @@ -29,7 +29,7 @@ from types import BuiltinFunctionType, MethodType, FunctionType -import pypy +import pypy.tool.instancemethod from pypy.annotation.pairtype import pair, extendabletype from pypy.tool.tls import tlsobject from pypy.rpython.rarithmetic import r_uint, r_longlong, r_ulonglong @@ -388,6 +388,11 @@ immutable = True def __init__(self, analyser, s_self=None, methodname=None): + if isinstance(analyser, MethodType): + analyser = pypy.tool.instancemethod.InstanceMethod( + analyser.im_func, + analyser.im_self, + analyser.im_class) self.analyser = analyser self.s_self = s_self self.methodname = methodname Modified: pypy/dist/pypy/rpython/extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/extregistry.py (original) +++ pypy/dist/pypy/rpython/extregistry.py Fri Apr 21 19:30:22 2006 @@ -3,25 +3,69 @@ from pypy.tool.uid import Hashable -class ExtRegistryFunc(object): - def __init__(self, compute_result_annotation, specialize_call): - self.compute_result_annotation = compute_result_annotation - self.specialize_call = specialize_call - - def get_annotation(self, type, func=None): +class AutoRegisteringType(type): + + def __init__(selfcls, name, bases, dict): + super(AutoRegisteringType, selfcls).__init__(selfcls, + name, bases, dict) + if '_about_' in dict: + selfcls._register_value(dict['_about_']) + del selfcls._about_ # avoid keeping a ref + if '_type_' in dict: + selfcls._register_type(dict['_type_']) + del selfcls._type_ + if '_metatype_' in dict: + selfcls._register_metatype(dict['_metatype_']) + del selfcls._metatype_ + + def _register_value(selfcls, key): + assert key not in EXT_REGISTRY_BY_VALUE + EXT_REGISTRY_BY_VALUE[key] = selfcls + + def _register_type(selfcls, key): + assert key not in EXT_REGISTRY_BY_TYPE + EXT_REGISTRY_BY_TYPE[key] = selfcls + + def _register_metatype(selfcls, key): + assert key not in EXT_REGISTRY_BY_METATYPE + EXT_REGISTRY_BY_METATYPE[key] = selfcls + + +class ExtRegistryEntry(object): + __metaclass__ = AutoRegisteringType + + def __init__(self, type, instance=None): + self.type = type + self.instance = instance + + # structural equality, and trying hard to be hashable: Entry instances + # are used as keys to map annotations to Reprs in the rtyper. + # Warning, it's based on only 'type' and 'instance'. + def __eq__(self, other): + return (self.__class__ is other.__class__ and + self.type == other.type and + self.instance == other.instance) + + def __ne__(self, other): + return not self.__eq__(other) + + def __hash__(self): + return hash((self.__class__, self.type, Hashable(self.instance))) + + def compute_annotation(self): + # default implementation useful for built-in functions, + # can be overriden + func = self.instance assert func is not None from pypy.annotation import model as annmodel - return annmodel.SomeBuiltin(self.compute_result_annotation, - methodname=getattr(func, '__name__', None)) - -class ExtRegistryInstance(object): - def __init__(self, compute_annotation, specialize_call, get_repr): - self.compute_annotation = compute_annotation - self.specialize_call = specialize_call - self.get_repr = get_repr - - def get_annotation(self, type, instance=None): - return self.compute_annotation(type, instance) + analyser = self.compute_result_annotation + methodname = getattr(func, '__name__', None) + return annmodel.SomeBuiltin(analyser, methodname=methodname) + + def compute_result_annotation(self, *args_s, **kwds_s): + # default implementation for built-in functions with a constant + # result annotation, can be overriden + return self.s_result_annotation # ____________________________________________________________ @@ -68,64 +112,39 @@ EXT_REGISTRY_BY_TYPE = weakref.WeakKeyDictionary() EXT_REGISTRY_BY_METATYPE = weakref.WeakKeyDictionary() -def create_annotation_callable(annotation): - from pypy.annotation import model as annmodel - if isinstance(annotation, annmodel.SomeObject) or annotation is None: - s_result = annotation - def annotation(*args): - return s_result - - return annotation - -undefined = object() - -def create_entry(compute_result_annotation=undefined, compute_annotation=None, - specialize_call=None, get_repr=None): - if compute_result_annotation is not undefined: - compute_result_annotation = create_annotation_callable( - compute_result_annotation) - return ExtRegistryFunc(compute_result_annotation, specialize_call) - else: - return ExtRegistryInstance(compute_annotation, specialize_call, - get_repr) - -def register_value(value, **kwargs): - assert value not in EXT_REGISTRY_BY_VALUE - EXT_REGISTRY_BY_VALUE[value] = create_entry(**kwargs) - return EXT_REGISTRY_BY_VALUE[value] - -def register_type(t, **kwargs): - assert t not in EXT_REGISTRY_BY_TYPE - EXT_REGISTRY_BY_TYPE[t] = create_entry(**kwargs) - return EXT_REGISTRY_BY_TYPE[t] - -def register_metatype(t, **kwargs): - assert t not in EXT_REGISTRY_BY_METATYPE - EXT_REGISTRY_BY_METATYPE[t] = create_entry(**kwargs) - return EXT_REGISTRY_BY_METATYPE[t] +# ____________________________________________________________ +# Public interface to access the registry -def lookup_type(tp): +def _lookup_type_cls(tp): try: return EXT_REGISTRY_BY_TYPE[tp] except (KeyError, TypeError): return EXT_REGISTRY_BY_METATYPE[type(tp)] +def lookup_type(tp): + Entry = _lookup_type_cls(tp) + return Entry(tp) + def is_registered_type(tp): try: - lookup_type(tp) + _lookup_type_cls(tp) except KeyError: return False return True -def lookup(instance): +def _lookup_cls(instance): try: return EXT_REGISTRY_BY_VALUE[instance] except (KeyError, TypeError): - return lookup_type(type(instance)) - + return _lookup_type_cls(type(instance)) + +def lookup(instance): + Entry = _lookup_cls(instance) + return Entry(type(instance), instance) + def is_registered(instance): try: - lookup(instance) + _lookup_cls(instance) except KeyError: return False return True Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Fri Apr 21 19:30:22 2006 @@ -2,7 +2,7 @@ The table of all LL operations. """ -from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry class LLOp(object): @@ -28,11 +28,6 @@ # The operation manipulates PyObjects self.pyobj = pyobj - # XXX refactor extregistry to allow a single registration - extregistry.register_value(self, - compute_result_annotation = self.compute_result_annotation, - specialize_call = self.specialize) - # __________ make the LLOp instances callable from LL helpers __________ __name__ = property(lambda self: 'llop_'+self.opname) @@ -40,16 +35,6 @@ def __call__(self, RESULTTYPE, *args): raise TypeError, "llop is meant to be rtyped and not called direclty" - def compute_result_annotation(self, RESULTTYPE, *args): - from pypy.annotation.model import lltype_to_annotation - assert RESULTTYPE.is_constant() - return lltype_to_annotation(RESULTTYPE.const) - - def specialize(self, hop): - args_v = [hop.inputarg(r, i+1) for i, r in enumerate(hop.args_r[1:])] - hop.exception_is_here() - return hop.genop(self.opname, args_v, resulttype=hop.r_result.lowleveltype) - def enum_ops_without_sideeffects(raising_is_ok=False): """Enumerate operations that have no side-effects @@ -66,6 +51,22 @@ if not opdesc.canraise or raising_is_ok: yield opname + +class Entry(ExtRegistryEntry): + "Annotation and rtyping of LLOp instances, which are callable." + _type_ = LLOp + + def compute_result_annotation(self, RESULTTYPE, *args): + from pypy.annotation.model import lltype_to_annotation + assert RESULTTYPE.is_constant() + return lltype_to_annotation(RESULTTYPE.const) + + def specialize_call(self, hop): + op = self.instance # the LLOp object that was called + args_v = [hop.inputarg(r, i+1) for i, r in enumerate(hop.args_r[1:])] + hop.exception_is_here() + return hop.genop(op.opname, args_v, resulttype=hop.r_result.lowleveltype) + # ____________________________________________________________ # # This list corresponds to the operations implemented by the LLInterpreter. Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Fri Apr 21 19:30:22 2006 @@ -18,7 +18,7 @@ Array, Char, Void, attachRuntimeTypeInfo, \ FuncType, Bool, Signed, functionptr, FuncType, PyObject from pypy.rpython.robject import PyObjRepr, pyobj_repr -from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry from pypy.annotation import model as annmodel # @@ -602,18 +602,21 @@ def ll_call_destructor(thang, repr): return 42 # will be mapped -def rtype_destruct_object(hop): - v_inst, c_spec = hop.inputargs(*hop.args_r) - repr = c_spec.value - if repr.has_wrapper: - null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) - # XXX this is a hack! We need an operation to remove a broken PyObject - repr.setfield(v_inst, '_wrapper_', null, hop.llops, opname='bare_setfield') - hop.genop('gc_unprotect', [v_inst]) - -extregistry.register_value(ll_call_destructor, - compute_result_annotation=lambda *args: None, - specialize_call=rtype_destruct_object) +class Entry(ExtRegistryEntry): + _about_ = ll_call_destructor + s_result_annotation = None + + def specialize_call(self, hop): + v_inst, c_spec = hop.inputargs(*hop.args_r) + repr = c_spec.value + if repr.has_wrapper: + null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) + # XXX this is a hack! We need an operation to remove a broken + # PyObject + repr.setfield(v_inst, '_wrapper_', null, hop.llops, + opname='bare_setfield') + hop.genop('gc_unprotect', [v_inst]) + def create_pywrapper(thing, repr): return ll_create_pywrapper(thing, repr) @@ -635,24 +638,26 @@ llops.genop('gc_protect', [v_inst]) return v_inst -def rtype_wrap_object_create(hop): - v_inst, c_spec = hop.inputargs(*hop.args_r) - repr = c_spec.value - v_res = into_cobject(v_inst, repr, hop.llops) - v_cobj = v_res - c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) - c_0 = hop.inputconst(Signed, 0) - v_res = hop.llops.gencapicall('PyType_GenericAlloc', [c_cls, c_0], resulttype=pyobj_repr) - c_self = hop.inputconst(pyobj_repr, '__self__') - hop.genop('setattr', [v_res, c_self, v_cobj], resulttype=pyobj_repr) - if repr.has_wrapper: - repr.setfield(v_inst, '_wrapper_', v_res, hop.llops) - hop.genop('gc_unprotect', [v_res]) # yes a weak ref - return v_res - -extregistry.register_value(ll_create_pywrapper, - compute_result_annotation=annmodel.SomePtr(Ptr(PyObject)), - specialize_call=rtype_wrap_object_create) +class Entry(ExtRegistryEntry): + _about_ = ll_create_pywrapper + s_result_annotation = annmodel.SomePtr(Ptr(PyObject)) + + def specialize_call(self, hop): + v_inst, c_spec = hop.inputargs(*hop.args_r) + repr = c_spec.value + v_res = into_cobject(v_inst, repr, hop.llops) + v_cobj = v_res + c_cls = hop.inputconst(pyobj_repr, repr.classdef.classdesc.pyobj) + c_0 = hop.inputconst(Signed, 0) + v_res = hop.llops.gencapicall('PyType_GenericAlloc', [c_cls, c_0], + resulttype=pyobj_repr) + c_self = hop.inputconst(pyobj_repr, '__self__') + hop.genop('setattr', [v_res, c_self, v_cobj], resulttype=pyobj_repr) + if repr.has_wrapper: + repr.setfield(v_inst, '_wrapper_', v_res, hop.llops) + hop.genop('gc_unprotect', [v_res]) # yes a weak ref + return v_res + def fetch_pywrapper(thing, repr): return ll_fetch_pywrapper(thing, repr) @@ -660,19 +665,20 @@ def ll_fetch_pywrapper(thing, repr): return 42 -def rtype_wrap_object_fetch(hop): - v_inst, c_spec = hop.inputargs(*hop.args_r) - repr = c_spec.value - if repr.has_wrapper: - return repr.getfield(v_inst, '_wrapper_', hop.llops) - else: - null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) - return null +class Entry(ExtRegistryEntry): + _about_ = ll_fetch_pywrapper + s_result_annotation = annmodel.SomePtr(Ptr(PyObject)) + + def specialize_call(self, hop): + v_inst, c_spec = hop.inputargs(*hop.args_r) + repr = c_spec.value + if repr.has_wrapper: + return repr.getfield(v_inst, '_wrapper_', hop.llops) + else: + null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) + return null + -extregistry.register_value(ll_fetch_pywrapper, - compute_result_annotation=annmodel.SomePtr(Ptr(PyObject)), - specialize_call=rtype_wrap_object_fetch) - def ll_wrap_object(obj, repr): ret = fetch_pywrapper(obj, repr) if not ret: Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Fri Apr 21 19:30:22 2006 @@ -6,7 +6,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.objspace.flow.model import Variable from pypy.annotation import model as annmodel -from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry from pypy import conftest import py @@ -248,12 +248,18 @@ def protect(obj): RaiseNameError def unprotect(obj): RaiseNameError -def rtype_protect(hop): hop.genop('gc_protect', [hop.inputargs(hop.args_r[0])[0]]) -def rtype_unprotect(hop): hop.genop('gc_unprotect', [hop.inputargs(hop.args_r[0])[0]]) -extregistry.register_value(protect, - compute_result_annotation=lambda *args: None, specialize_call=rtype_protect) -extregistry.register_value(unprotect, - compute_result_annotation=lambda *args: None, specialize_call=rtype_unprotect) + +class Entry(ExtRegistryEntry): + _about_ = protect + s_result_annotation = None + def specialize_call(self, hop): + hop.genop('gc_protect', hop.inputargs(hop.args_r[0])) + +class Entry(ExtRegistryEntry): + _about_ = unprotect + s_result_annotation = None + def specialize_call(self, hop): + hop.genop('gc_unprotect', hop.inputargs(hop.args_r[0])) def test_protect_unprotect(): def p(): protect('this is an object') @@ -270,14 +276,15 @@ assert len(ops.get('direct_call', [])) == ex def generic_op(*args): RaiseNameError -def rtype_generic_op(hop): - args = hop.inputargs(*hop.args_r) - args.pop(0) - op = hop.args_s[0].const - hop.genop(op, args) - -extregistry.register_value(generic_op, - compute_result_annotation=lambda *args: None, specialize_call=rtype_generic_op) + +class Entry(ExtRegistryEntry): + _about_ = generic_op + s_result_annotation = None + def specialize_call(self, hop): + args = hop.inputargs(*hop.args_r) + args.pop(0) + op = hop.args_s[0].const + hop.genop(op, args) def test_bare_setfield(): class A: Modified: pypy/dist/pypy/rpython/rctypes/aarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/aarray.py Fri Apr 21 19:30:22 2006 @@ -1,46 +1,40 @@ from ctypes import ARRAY, c_int, c_char -from pypy.annotation.model import SomeCTypesObject, SomeBuiltin, SomeString -from pypy.rpython import extregistry +from pypy.annotation.model import SomeCTypesObject, SomeString +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.rpython.lltypesystem import lltype ArrayType = type(ARRAY(c_int, 10)) -def arraytype_specialize_call(hop): - from pypy.rpython.error import TyperError - from pypy.rpython.rmodel import inputconst - r_array = hop.r_result - v_result = r_array.allocate_instance(hop.llops) - if hop.nb_args > r_array.length: - raise TyperError("too many arguments for an array of length %d" % ( - r_array.length,)) - for i in range(hop.nb_args): - v_item = hop.inputarg(r_array.r_item, arg=i) - c_index = inputconst(lltype.Signed, i) - r_array.setitem(hop.llops, v_result, c_index, v_item) - return v_result - -def arraytype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - return SomeBuiltin(compute_result_annotation, methodname=type.__name__) - -extregistry.register_type(ArrayType, - compute_annotation=arraytype_compute_annotation, - specialize_call=arraytype_specialize_call) - -def array_instance_compute_annotation(type, instance): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - -def arraytype_get_repr(rtyper, s_array): - from pypy.rpython.rctypes.rarray import ArrayRepr - return ArrayRepr(rtyper, s_array) - -entry = extregistry.register_metatype(ArrayType, - compute_annotation=array_instance_compute_annotation, - get_repr=arraytype_get_repr) -def char_array_get_field_annotation(s_array, fieldname): - assert fieldname == 'value' - if s_array.knowntype._type_ != c_char: - raise Exception("only arrays of chars have a .value attribute") - return SomeString() # can_be_None = False -entry.get_field_annotation = char_array_get_field_annotation + +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to array types." + _type_ = ArrayType + + def specialize_call(self, hop): + from pypy.rpython.error import TyperError + from pypy.rpython.rmodel import inputconst + r_array = hop.r_result + v_result = r_array.allocate_instance(hop.llops) + if hop.nb_args > r_array.length: + raise TyperError("too many arguments for an array of length %d" % ( + r_array.length,)) + for i in range(hop.nb_args): + v_item = hop.inputarg(r_array.r_item, arg=i) + c_index = inputconst(lltype.Signed, i) + r_array.setitem(hop.llops, v_result, c_index, v_item) + return v_result + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of array instances." + _metatype_ = ArrayType + + def get_field_annotation(self, s_array, fieldname): + assert fieldname == 'value' + if self.type._type_ != c_char: + raise Exception("only arrays of chars have a .value attribute") + return SomeString() # can_be_None = False + + def get_repr(self, rtyper, s_array): + from pypy.rpython.rctypes.rarray import ArrayRepr + return ArrayRepr(rtyper, s_array) Modified: pypy/dist/pypy/rpython/rctypes/achar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/achar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/achar_p.py Fri Apr 21 19:30:22 2006 @@ -1,42 +1,33 @@ -from pypy.rpython import extregistry -from pypy.annotation import model as annmodel +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry +from pypy.annotation.model import SomeString from ctypes import c_char_p -def c_char_p_compute_result_annotation(s_arg=None): - return annmodel.SomeCTypesObject(c_char_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_char_p_specialize_call(hop): - from pypy.rpython.rstr import string_repr - r_char_p = hop.r_result - v_result = r_char_p.allocate_instance(hop.llops) - if len(hop.args_s): - v_value, = hop.inputargs(string_repr) - r_char_p.setstring(hop.llops, v_result, v_value) - return v_result - -extregistry.register_value(c_char_p, - compute_result_annotation=c_char_p_compute_result_annotation, - specialize_call=c_char_p_specialize_call - ) - -def c_char_compute_annotation(the_type, instance): - return annmodel.SomeCTypesObject(c_char_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_char_p_get_repr(rtyper, s_char_p): - from pypy.rpython.rctypes import rchar_p - return rchar_p.CCharPRepr(rtyper, s_char_p, rchar_p.CCHARP) - -entry = extregistry.register_type(c_char_p, - compute_annotation = c_char_compute_annotation, - get_repr = c_char_p_get_repr, - ) -s_value_annotation = annmodel.SomeString(can_be_None=True) -def c_char_p_get_field_annotation(s_char_p, fieldname): - assert fieldname == 'value' - return s_value_annotation -entry.get_field_annotation = c_char_p_get_field_annotation -entry.s_return_trick = s_value_annotation +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to c_char_p." + _about_ = c_char_p + + def specialize_call(self, hop): + from pypy.rpython.rstr import string_repr + r_char_p = hop.r_result + v_result = r_char_p.allocate_instance(hop.llops) + if len(hop.args_s): + v_value, = hop.inputargs(string_repr) + r_char_p.setstring(hop.llops, v_result, v_value) + return v_result + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of c_char_p instances." + _type_ = c_char_p + + s_return_trick = SomeString(can_be_None=True) + + def get_field_annotation(self, s_char_p, fieldname): + assert fieldname == 'value' + return self.s_return_trick + + def get_repr(self, rtyper, s_char_p): + from pypy.rpython.rctypes import rchar_p + return rchar_p.CCharPRepr(rtyper, s_char_p, rchar_p.CCHARP) Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Fri Apr 21 19:30:22 2006 @@ -1,5 +1,5 @@ -from pypy.annotation import model as annmodel -from pypy.rpython import extregistry +from pypy.annotation.model import SomeCTypesObject +from pypy.rpython.extregistry import ExtRegistryEntry from pypy.rpython.lltypesystem import lltype import ctypes @@ -7,86 +7,78 @@ CFuncPtrType = type(ctypes.CFUNCTYPE(None)) -def cfuncptrtype_compute_annotation(type, instance): - def compute_result_annotation(*args_s): +class CallEntry(ExtRegistryEntry): + """Annotation and rtyping of calls to external functions + declared with ctypes. + """ + _metatype_ = CFuncPtrType + + def compute_result_annotation(self, *args_s): """ Answer the annotation of the external function's result """ - result_ctype = instance.restype + result_ctype = self.instance.restype if result_ctype is None: return None - s_result = annmodel.SomeCTypesObject(result_ctype, - annmodel.SomeCTypesObject.OWNSMEMORY) + s_result = SomeCTypesObject(result_ctype, SomeCTypesObject.OWNSMEMORY) return s_result.return_annotation() - return annmodel.SomeBuiltin(compute_result_annotation, - methodname=getattr(instance, '__name__', None)) + def specialize_call(self, hop): + from pypy.rpython.rctypes.rmodel import CTypesValueRepr + cfuncptr = self.instance + fnname = cfuncptr.__name__ + + args_r = [] + for ctype in cfuncptr.argtypes: + s_arg = SomeCTypesObject(ctype, SomeCTypesObject.MEMORYALIAS) + r_arg = hop.rtyper.getrepr(s_arg) + args_r.append(r_arg) + + vlist = hop.inputargs(*args_r) + unwrapped_args_v = [] + ARGTYPES = [] + for r_arg, v in zip(args_r, vlist): + if isinstance(r_arg, CTypesValueRepr): + # ValueRepr case + unwrapped_args_v.append(r_arg.getvalue(hop.llops, v)) + ARGTYPES.append(r_arg.ll_type) + else: + # RefRepr case -- i.e. the function argument that we pass by + # value is e.g. a complete struct; we pass a pointer to it + # in the low-level graphs and it's up to the back-end to + # generate the correct dereferencing + unwrapped_args_v.append(r_arg.get_c_data(hop.llops, v)) + ARGTYPES.append(r_arg.c_data_type) + if cfuncptr.restype is not None: + s_res = SomeCTypesObject(cfuncptr.restype, + SomeCTypesObject.OWNSMEMORY) + r_res = hop.rtyper.getrepr(s_res) + RESTYPE = r_res.ll_type + else: + RESTYPE = lltype.Void -def cfuncptrtype_specialize_call(hop): - from pypy.rpython.rctypes.rmodel import CTypesValueRepr + kwds = {} + if hasattr(cfuncptr, 'llinterp_friendly_version'): + kwds['_callable'] = cfuncptr.llinterp_friendly_version + if (cfuncptr._flags_ & ctypes._FUNCFLAG_PYTHONAPI) == 0: + kwds['includes'] = getattr(cfuncptr, 'includes', ()) + #else: + # no 'includes': hack to trigger in GenC a PyErr_Occurred() check + + v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, + resulttype = RESTYPE, + **kwds) + # XXX hack! hack! temporary! I promize! + FUNCTYPE = lltype.FuncType(ARGTYPES, RESTYPE) + last_op = hop.llops[-1] + assert last_op.opname == 'direct_call' + last_op.args[0].concretetype = lltype.Ptr(FUNCTYPE) + last_op.args[0].value._set_TYPE(last_op.args[0].concretetype) + last_op.args[0].value._set_T(FUNCTYPE) + last_op.args[0].value._obj._TYPE = FUNCTYPE - # this is necessary to get the original function pointer when specializing - # the metatype - assert hop.spaceop.opname == "simple_call" - cfuncptr = hop.spaceop.args[0].value - fnname = cfuncptr.__name__ - - args_r = [] - for ctype in cfuncptr.argtypes: - s_arg = annmodel.SomeCTypesObject(ctype, - annmodel.SomeCTypesObject.MEMORYALIAS) - r_arg = hop.rtyper.getrepr(s_arg) - args_r.append(r_arg) - - vlist = hop.inputargs(*args_r) - unwrapped_args_v = [] - ARGTYPES = [] - for r_arg, v in zip(args_r, vlist): - if isinstance(r_arg, CTypesValueRepr): - # ValueRepr case - unwrapped_args_v.append(r_arg.getvalue(hop.llops, v)) - ARGTYPES.append(r_arg.ll_type) + if RESTYPE is lltype.Void: + return None else: - # RefRepr case -- i.e. the function argument that we pass by - # value is e.g. a complete struct; we pass a pointer to it - # in the low-level graphs and it's up to the back-end to - # generate the correct dereferencing - unwrapped_args_v.append(r_arg.get_c_data(hop.llops, v)) - ARGTYPES.append(r_arg.c_data_type) - if cfuncptr.restype is not None: - s_res = annmodel.SomeCTypesObject(cfuncptr.restype, - annmodel.SomeCTypesObject.OWNSMEMORY) - r_res = hop.rtyper.getrepr(s_res) - RESTYPE = r_res.ll_type - else: - RESTYPE = lltype.Void - - kwds = {} - if hasattr(cfuncptr, 'llinterp_friendly_version'): - kwds['_callable'] = cfuncptr.llinterp_friendly_version - if (cfuncptr._flags_ & ctypes._FUNCFLAG_PYTHONAPI) == 0: - kwds['includes'] = getattr(cfuncptr, 'includes', ()) - #else: - # no 'includes': hack to trigger in GenC a PyErr_Occurred() check - - v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, - resulttype = RESTYPE, - **kwds) - # XXX hack! hack! temporary! I promize! - FUNCTYPE = lltype.FuncType(ARGTYPES, RESTYPE) - last_op = hop.llops[-1] - assert last_op.opname == 'direct_call' - last_op.args[0].concretetype = lltype.Ptr(FUNCTYPE) - last_op.args[0].value._set_TYPE(last_op.args[0].concretetype) - last_op.args[0].value._set_T(FUNCTYPE) - last_op.args[0].value._obj._TYPE = FUNCTYPE - - if RESTYPE is lltype.Void: - return None - else: - return r_res.return_value(hop.llops, v_result) - -extregistry.register_metatype(CFuncPtrType, - compute_annotation=cfuncptrtype_compute_annotation, - specialize_call=cfuncptrtype_specialize_call) + return r_res.return_value(hop.llops, v_result) Modified: pypy/dist/pypy/rpython/rctypes/apointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/apointer.py Fri Apr 21 19:30:22 2006 @@ -1,81 +1,73 @@ -from pypy.rpython import extregistry -from pypy.annotation import model as annmodel -from pypy.annotation.pairtype import pairtype +from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry +from pypy.annotation.model import SomeCTypesObject from pypy.rpython.lltypesystem import lltype from ctypes import pointer, POINTER, byref, c_int -def pointertype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s): - return annmodel.SomeCTypesObject(type, - annmodel.SomeCTypesObject.OWNSMEMORY) - return annmodel.SomeBuiltin(compute_result_annotation, - methodname=type.__name__) - -def pointertype_specialize_call(hop): - r_ptr = hop.r_result - v_result = r_ptr.allocate_instance(hop.llops) - if len(hop.args_s): - v_contentsbox, = hop.inputargs(r_ptr.r_contents) - r_ptr.setcontents(hop.llops, v_result, v_contentsbox) - return v_result - -def pointerinstance_compute_annotation(type, instance): - return annmodel.SomeCTypesObject(type, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def pointerinstance_field_annotation(s_pointer, fieldname): - assert fieldname == "contents" - ptrtype = s_pointer.knowntype - return annmodel.SomeCTypesObject(ptrtype._type_, - annmodel.SomeCTypesObject.MEMORYALIAS) - -def pointerinstance_get_repr(rtyper, s_pointer): - from pypy.rpython.rctypes.rpointer import PointerRepr - return PointerRepr(rtyper, s_pointer) - PointerType = type(POINTER(c_int)) -extregistry.register_type(PointerType, - compute_annotation=pointertype_compute_annotation, - specialize_call=pointertype_specialize_call) - -entry = extregistry.register_metatype(PointerType, - compute_annotation=pointerinstance_compute_annotation, - get_repr=pointerinstance_get_repr) -entry.get_field_annotation = pointerinstance_field_annotation - -def pointerfn_compute_annotation(s_arg): - assert isinstance(s_arg, annmodel.SomeCTypesObject) - ctype = s_arg.knowntype - result_ctype = POINTER(ctype) - return annmodel.SomeCTypesObject(result_ctype, - annmodel.SomeCTypesObject.OWNSMEMORY) - -extregistry.register_value(pointer, - compute_result_annotation=pointerfn_compute_annotation, - # same rtyping for calling pointer() or calling a specific instance - # of PointerType: - specialize_call=pointertype_specialize_call) + +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to POINTER types." + _type_ = PointerType + + def specialize_call(self, hop): + # delegate calls to the logic for calls to ctypes.pointer() + return PointerFnEntry.specialize_call(hop) + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of pointer instances." + _metatype_ = PointerType + + def get_field_annotation(self, s_pointer, fieldname): + assert fieldname == "contents" + ptrtype = self.type + assert s_pointer.knowntype == ptrtype + return SomeCTypesObject(ptrtype._type_, + SomeCTypesObject.MEMORYALIAS) + + def get_repr(self, rtyper, s_pointer): + from pypy.rpython.rctypes.rpointer import PointerRepr + return PointerRepr(rtyper, s_pointer) + + +class PointerFnEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to ctypes.pointer()." + _about_ = pointer + + def compute_result_annotation(self, s_arg): + assert isinstance(s_arg, SomeCTypesObject) + ctype = s_arg.knowntype + result_ctype = POINTER(ctype) + return SomeCTypesObject(result_ctype, SomeCTypesObject.OWNSMEMORY) + + def specialize_call(hop): + r_ptr = hop.r_result + v_result = r_ptr.allocate_instance(hop.llops) + if len(hop.args_s): + v_contentsbox, = hop.inputargs(r_ptr.r_contents) + r_ptr.setcontents(hop.llops, v_result, v_contentsbox) + return v_result + specialize_call = staticmethod(specialize_call) # byref() is equivalent to pointer() -- the difference is only an # optimization that is useful in ctypes but not in rctypes. -extregistry.register_value(byref, - compute_result_annotation=pointerfn_compute_annotation, - specialize_call=pointertype_specialize_call) - -# constant-fold POINTER(CONSTANT_CTYPE) calls -def POINTER_compute_annotation(s_arg): - from pypy.annotation.bookkeeper import getbookkeeper - assert s_arg.is_constant(), "POINTER(%r): argument must be constant" % ( - s_arg,) - RESTYPE = POINTER(s_arg.const) - return getbookkeeper().immutablevalue(RESTYPE) - -def POINTER_specialize_call(hop): - assert hop.s_result.is_constant() - return hop.inputconst(lltype.Void, hop.s_result.const) - -extregistry.register_value(POINTER, - compute_result_annotation=POINTER_compute_annotation, - specialize_call=POINTER_specialize_call) +PointerFnEntry._register_value(byref) + + +class POINTERFnEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to ctypes.POINTER(): constant-folded." + _about_ = POINTER + + def compute_result_annotation(self, s_arg): + from pypy.annotation.bookkeeper import getbookkeeper + assert s_arg.is_constant(), ( + "POINTER(%r): argument must be constant" % (s_arg,)) + RESTYPE = POINTER(s_arg.const) + return getbookkeeper().immutablevalue(RESTYPE) + + def specialize_call(self, hop): + assert hop.s_result.is_constant() + return hop.inputconst(lltype.Void, hop.s_result.const) Modified: pypy/dist/pypy/rpython/rctypes/aprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/aprimitive.py Fri Apr 21 19:30:22 2006 @@ -2,7 +2,7 @@ from ctypes import c_long, c_ulong, c_longlong, c_ulonglong, c_float from ctypes import c_double, c_wchar, c_char_p from pypy.annotation import model as annmodel -from pypy.rpython import extregistry +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.rpython.lltypesystem import lltype ctypes_annotation_list = { @@ -20,45 +20,40 @@ c_ulonglong: lltype.UnsignedLongLong, c_float: lltype.Float, c_double: lltype.Float, -}.items() # nb. platform-dependent duplicate ctypes are removed +} # nb. platform-dependent duplicate ctypes are removed -def primitive_specialize_call(hop): - r_primitive = hop.r_result - v_result = r_primitive.allocate_instance(hop.llops) - if len(hop.args_s): - v_value, = hop.inputargs(r_primitive.ll_type) - r_primitive.setvalue(hop.llops, v_result, v_value) - return v_result - -def do_register(the_type, ll_type): - def compute_result_annotation_function(s_arg=None): - return annmodel.SomeCTypesObject(the_type, - annmodel.SomeCTypesObject.OWNSMEMORY) - - extregistry.register_value(the_type, - compute_result_annotation=compute_result_annotation_function, - specialize_call=primitive_specialize_call - ) - - def compute_prebuilt_instance_annotation(the_type, instance): - return annmodel.SomeCTypesObject(the_type, - annmodel.SomeCTypesObject.OWNSMEMORY) +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to primitive c_xxx types." - def primitive_get_repr(rtyper, s_primitive): + def specialize_call(self, hop): + r_primitive = hop.r_result + v_result = r_primitive.allocate_instance(hop.llops) + if len(hop.args_s): + v_value, = hop.inputargs(r_primitive.ll_type) + r_primitive.setvalue(hop.llops, v_result, v_value) + return v_result + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of instances of the primitive c_xxx type." + + def get_repr(self, rtyper, s_primitive): from pypy.rpython.rctypes.rprimitive import PrimitiveRepr + ll_type = ctypes_annotation_list[self.type] return PrimitiveRepr(rtyper, s_primitive, ll_type) - entry = extregistry.register_type(the_type, - compute_annotation=compute_prebuilt_instance_annotation, - get_repr=primitive_get_repr, - ) - s_value_annotation = annmodel.lltype_to_annotation(ll_type) - def primitive_get_field_annotation(s_primitive, fieldname): + def get_field_annotation(self, s_primitive, fieldname): assert fieldname == 'value' - return s_value_annotation - entry.get_field_annotation = primitive_get_field_annotation - entry.s_return_trick = s_value_annotation + return self.get_s_value() + + def get_s_value(self): + ll_type = ctypes_annotation_list[self.type] + return annmodel.lltype_to_annotation(ll_type) + + s_return_trick = property(get_s_value) + -for the_type, ll_type in ctypes_annotation_list: - do_register(the_type, ll_type) +for _ctype in ctypes_annotation_list: + CallEntry._register_value(_ctype) + ObjEntry._register_type(_ctype) Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Fri Apr 21 19:30:22 2006 @@ -1,37 +1,28 @@ from ctypes import py_object from pypy.annotation.model import SomeCTypesObject -from pypy.rpython import extregistry +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.rpython.lltypesystem import lltype -# __________ py_object() calls __________ - -def py_object_compute_result_annotation(s_obj=None): - return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) - -def py_object_specialize_call(hop): - from pypy.rpython.robject import pyobj_repr - r_pyobject = hop.r_result - v_result = r_pyobject.allocate_instance(hop.llops) - if len(hop.args_s): - [v_input] = hop.inputargs(pyobj_repr) - r_pyobject.setvalue(hop.llops, v_result, v_input) - return v_result - -extregistry.register_value(py_object, - compute_result_annotation=py_object_compute_result_annotation, - specialize_call=py_object_specialize_call) - -# __________ prebuilt py_object instances __________ - -def py_object_instance_compute_annotation(type, instance): - return SomeCTypesObject(py_object, SomeCTypesObject.OWNSMEMORY) - -def py_object_instance_get_repr(rtyper, s_pyobject): - from pypy.rpython.rctypes.rpyobject import CTypesPyObjRepr - lowleveltype = lltype.Ptr(lltype.PyObject) - return CTypesPyObjRepr(rtyper, s_pyobject, lowleveltype) - -extregistry.register_type(py_object, - compute_annotation=py_object_instance_compute_annotation, - get_repr=py_object_instance_get_repr) +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to py_object." + _about_ = py_object + + def specialize_call(self, hop): + from pypy.rpython.robject import pyobj_repr + r_pyobject = hop.r_result + v_result = r_pyobject.allocate_instance(hop.llops) + if len(hop.args_s): + [v_input] = hop.inputargs(pyobj_repr) + r_pyobject.setvalue(hop.llops, v_result, v_input) + return v_result + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of py_object instances." + _type_ = py_object + + def get_repr(self, rtyper, s_pyobject): + from pypy.rpython.rctypes.rpyobject import CTypesPyObjRepr + lowleveltype = lltype.Ptr(lltype.PyObject) + return CTypesPyObjRepr(rtyper, s_pyobject, lowleveltype) Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/astringbuf.py Fri Apr 21 19:30:22 2006 @@ -1,5 +1,6 @@ -from pypy.rpython import extregistry -from pypy.annotation import model as annmodel +from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.rpython.rctypes.implementation import CTypesObjEntry +from pypy.annotation.model import SomeCTypesObject, SomeString from ctypes import create_string_buffer, c_char @@ -13,34 +14,35 @@ #_length_ = unspecified -def stringbuf_compute_result_annotation(s_length): - if s_length.knowntype != int: - raise Exception("rctypes only supports create_string_buffer(length)") - return annmodel.SomeCTypesObject(StringBufferType, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def stringbuf_specialize_call(hop): - from pypy.rpython.lltypesystem import lltype - [v_length] = hop.inputargs(lltype.Signed) - r_stringbuf = hop.r_result - return hop.genop("malloc_varsize", [ - hop.inputconst(lltype.Void, r_stringbuf.lowleveltype.TO), - v_length, - ], resulttype=r_stringbuf.lowleveltype, - ) - -extregistry.register_value(create_string_buffer, - compute_result_annotation=stringbuf_compute_result_annotation, - specialize_call=stringbuf_specialize_call, - ) - -def stringbuf_get_repr(rtyper, s_stringbuf): - from pypy.rpython.rctypes import rstringbuf - return rstringbuf.StringBufRepr(rtyper, s_stringbuf, rstringbuf.STRBUFTYPE) - -entry = extregistry.register_type(StringBufferType, - get_repr=stringbuf_get_repr) -def stringbuf_get_field_annotation(s_array, fieldname): - assert fieldname == 'value' - return annmodel.SomeString() # can_be_None = False -entry.get_field_annotation = stringbuf_get_field_annotation +class CreateStringBufferFnEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to ctypes.create_string_buffer()" + _about_ = create_string_buffer + + def compute_result_annotation(self, s_length): + if s_length.knowntype != int: + raise Exception("only supports create_string_buffer(length)") + return SomeCTypesObject(StringBufferType, SomeCTypesObject.OWNSMEMORY) + + def specialize_call(self, hop): + from pypy.rpython.lltypesystem import lltype + [v_length] = hop.inputargs(lltype.Signed) + r_stringbuf = hop.r_result + return hop.genop("malloc_varsize", [ + hop.inputconst(lltype.Void, r_stringbuf.lowleveltype.TO), + v_length, + ], resulttype=r_stringbuf.lowleveltype, + ) + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of instances of the pseudo-ctype StringBufferType" + _type_ = StringBufferType + + def get_field_annotation(self, s_array, fieldname): + assert fieldname == 'value' + return SomeString() # can_be_None = False + + def get_repr(self, rtyper, s_stringbuf): + from pypy.rpython.rctypes import rstringbuf + return rstringbuf.StringBufRepr(rtyper, s_stringbuf, + rstringbuf.STRBUFTYPE) Modified: pypy/dist/pypy/rpython/rctypes/astruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/astruct.py Fri Apr 21 19:30:22 2006 @@ -1,69 +1,62 @@ from ctypes import Structure -from pypy.annotation.model import SomeCTypesObject, SomeBuiltin -from pypy.rpython import extregistry +from pypy.annotation.model import SomeCTypesObject +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.rpython.lltypesystem import lltype StructType = type(Structure) -def structtype_specialize_call(hop, **kwds_i): - from pypy.rpython.error import TyperError - r_struct = hop.r_result - v_result = r_struct.allocate_instance(hop.llops) - index_by_name = {} - name_by_index = {} - # collect the keyword arguments - for key, index in kwds_i.items(): - assert key.startswith('i_') - name = key[2:] - assert index not in name_by_index - index_by_name[name] = index - name_by_index[index] = name - # add the positional arguments - fieldsiter = iter(r_struct.c_data_type._names) - for i in range(hop.nb_args): - if i not in name_by_index: - try: - name = fieldsiter.next() - except StopIteration: - raise TyperError("too many arguments in struct construction") +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to structure types." + _type_ = StructType + + def specialize_call(self, hop, **kwds_i): + from pypy.rpython.error import TyperError + r_struct = hop.r_result + v_result = r_struct.allocate_instance(hop.llops) + index_by_name = {} + name_by_index = {} + # collect the keyword arguments + for key, index in kwds_i.items(): + assert key.startswith('i_') + name = key[2:] + assert index not in name_by_index + index_by_name[name] = index + name_by_index[index] = name + # add the positional arguments + ctype = self.instance + fieldsiter = iter(ctype._fields_) + for i in range(hop.nb_args): + if i not in name_by_index: + try: + name, _ = fieldsiter.next() + except StopIteration: + raise TyperError("too many arguments in struct construction") + if name in index_by_name: + raise TyperError("multiple values for field %r" % (name,)) + index_by_name[name] = i + name_by_index[i] = name + # initialize the fields from the arguments, as far as they are present + for name, _ in ctype._fields_: if name in index_by_name: - raise TyperError("multiple values for field %r" % (name,)) - index_by_name[name] = i - name_by_index[i] = name - # initialize the fields from the arguments, as far as they are present - for name in r_struct.c_data_type._names: - if name in index_by_name: - index = index_by_name[name] - v_valuebox = hop.inputarg(r_struct.r_fields[name], arg=index) - r_struct.setfield(hop.llops, v_result, name, v_valuebox) - return v_result - -def structtype_compute_annotation(metatype, type): - def compute_result_annotation(*arg_s, **kwds_s): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - return SomeBuiltin(compute_result_annotation, methodname=type.__name__) - -extregistry.register_type(StructType, - compute_annotation=structtype_compute_annotation, - specialize_call=structtype_specialize_call) - -def struct_instance_compute_annotation(type, instance): - return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) - -def struct_instance_field_annotation(s_struct, fieldname): - structtype = s_struct.knowntype - for name, ctype in structtype._fields_: - if name == fieldname: - s_result = SomeCTypesObject(ctype, SomeCTypesObject.MEMORYALIAS) - return s_result.return_annotation() - raise AttributeError('%r has no field %r' % (structtype, fieldname)) - -def structtype_get_repr(rtyper, s_struct): - from pypy.rpython.rctypes.rstruct import StructRepr - return StructRepr(rtyper, s_struct) - -entry = extregistry.register_metatype(StructType, - compute_annotation=struct_instance_compute_annotation, - get_repr=structtype_get_repr) -entry.get_field_annotation = struct_instance_field_annotation + index = index_by_name[name] + v_valuebox = hop.inputarg(r_struct.r_fields[name], arg=index) + r_struct.setfield(hop.llops, v_result, name, v_valuebox) + return v_result + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of structure instances." + _metatype_ = StructType + + def get_field_annotation(self, s_struct, fieldname): + for name, ctype in self.type._fields_: + if name == fieldname: + s_result = SomeCTypesObject(ctype, + SomeCTypesObject.MEMORYALIAS) + return s_result.return_annotation() + raise AttributeError('%r has no field %r' % (self.type, fieldname)) + + def get_repr(self, rtyper, s_struct): + from pypy.rpython.rctypes.rstruct import StructRepr + return StructRepr(rtyper, s_struct) Modified: pypy/dist/pypy/rpython/rctypes/avoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/avoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/avoid_p.py Fri Apr 21 19:30:22 2006 @@ -1,77 +1,65 @@ -from pypy.rpython import extregistry -from pypy.annotation import model as annmodel +from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry +from pypy.annotation.model import SomeCTypesObject from ctypes import c_void_p, c_int, POINTER, cast PointerType = type(POINTER(c_int)) -# c_void_p() as a function -def c_void_p_compute_result_annotation(): - return annmodel.SomeCTypesObject(c_void_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_void_specialize_call(hop): - r_void_p = hop.r_result - v_result = r_void_p.allocate_instance(hop.llops) - return v_result - -extregistry.register_value(c_void_p, - compute_result_annotation=c_void_p_compute_result_annotation, - specialize_call=c_void_specialize_call, - ) - -# c_void_p instances -def c_void_compute_annotation(the_type, instance): - return annmodel.SomeCTypesObject(c_void_p, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def c_void_p_get_repr(rtyper, s_void_p): - from pypy.rpython.rctypes.rvoid_p import CVoidPRepr - from pypy.rpython.lltypesystem import llmemory - return CVoidPRepr(rtyper, s_void_p, llmemory.Address) - -extregistry.register_type(c_void_p, - compute_annotation = c_void_compute_annotation, - get_repr = c_void_p_get_repr, - ) - -# cast() support -def cast_compute_result_annotation(s_arg, s_type): - assert s_type.is_constant(), "cast(p, %r): argument 2 must be constant" % ( - s_type,) - type = s_type.const +class CallEntry(CTypesCallEntry): + "Annotation and rtyping of calls to c_void_p." + _about_ = c_void_p - def checkptr(ctype): + def specialize_call(self, hop): + r_void_p = hop.r_result + v_result = r_void_p.allocate_instance(hop.llops) + return v_result + + +class ObjEntry(CTypesObjEntry): + "Annotation and rtyping of c_void_p instances." + _type_ = c_void_p + + def get_repr(self, rtyper, s_void_p): + from pypy.rpython.rctypes.rvoid_p import CVoidPRepr + from pypy.rpython.lltypesystem import llmemory + return CVoidPRepr(rtyper, s_void_p, llmemory.Address) + + +class CastFnEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to ctypes.cast()" + _about_ = cast + + def checkptr(self, ctype): assert isinstance(ctype, PointerType) or ctype == c_void_p, ( "cast(): can only cast between pointers so far, not %r" % (ctype,)) - checkptr(s_arg.knowntype) - checkptr(type) - return annmodel.SomeCTypesObject(type, - annmodel.SomeCTypesObject.OWNSMEMORY) - -def cast_specialize_call(hop): - from pypy.rpython.rctypes.rpointer import PointerRepr - from pypy.rpython.rctypes.rvoid_p import CVoidPRepr - from pypy.rpython.lltypesystem import lltype, llmemory - assert isinstance(hop.args_r[0], (PointerRepr, CVoidPRepr)) - targetctype = hop.args_s[1].const - v_box, c_targetctype = hop.inputargs(hop.args_r[0], lltype.Void) - v_adr = hop.args_r[0].getvalue(hop.llops, v_box) - if v_adr.concretetype != llmemory.Address: - v_adr = hop.genop('cast_ptr_to_adr', [v_adr], - resulttype = llmemory.Address) - - if targetctype == c_void_p: - # cast to void - v_result = v_adr - else: - # cast to pointer - v_result = hop.genop('cast_adr_to_ptr', [v_adr], - resulttype = hop.r_result.ll_type) - return hop.r_result.return_value(hop.llops, v_result) - -extregistry.register_value(cast, - compute_result_annotation=cast_compute_result_annotation, - specialize_call=cast_specialize_call, - ) + + def compute_result_annotation(self, s_arg, s_type): + assert s_type.is_constant(), ( + "cast(p, %r): argument 2 must be constant" % (s_type,)) + type = s_type.const + self.checkptr(s_arg.knowntype) + self.checkptr(type) + return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) + + def specialize_call(self, hop): + from pypy.rpython.rctypes.rpointer import PointerRepr + from pypy.rpython.rctypes.rvoid_p import CVoidPRepr + from pypy.rpython.lltypesystem import lltype, llmemory + assert isinstance(hop.args_r[0], (PointerRepr, CVoidPRepr)) + targetctype = hop.args_s[1].const + v_box, c_targetctype = hop.inputargs(hop.args_r[0], lltype.Void) + v_adr = hop.args_r[0].getvalue(hop.llops, v_box) + if v_adr.concretetype != llmemory.Address: + v_adr = hop.genop('cast_ptr_to_adr', [v_adr], + resulttype = llmemory.Address) + + if targetctype == c_void_p: + # cast to void + v_result = v_adr + else: + # cast to pointer + v_result = hop.genop('cast_adr_to_ptr', [v_adr], + resulttype = hop.r_result.ll_type) + return hop.r_result.return_value(hop.llops, v_result) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Fri Apr 21 19:30:22 2006 @@ -1,3 +1,25 @@ +# Base classes describing annotation and rtyping +from pypy.annotation.model import SomeCTypesObject +from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry + + +class CTypesCallEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to ctypes types." + + def compute_result_annotation(self, *args_s, **kwds_s): + ctype = self.instance # the ctype is the called object + return SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) + + +class CTypesObjEntry(ExtRegistryEntry): + "Annotation and rtyping of ctypes instances." + + def compute_annotation(self): + ctype = self.type + return SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) + + # Importing for side effect of registering types with extregistry import pypy.rpython.rctypes.aprimitive import pypy.rpython.rctypes.apointer @@ -13,9 +35,6 @@ # Register the correspondance between SomeCTypesObject and the get_repr() # functions attached to the extregistry to create CTypesReprs -from pypy.annotation.model import SomeCTypesObject -from pypy.rpython import extregistry - class __extend__( SomeCTypesObject ): def rtyper_makerepr( self, rtyper ): entry = extregistry.lookup_type(self.knowntype) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Fri Apr 21 19:30:22 2006 @@ -111,6 +111,24 @@ if conftest.option.view: a.translator.view() + def test_annotate_reflow_bug(self): + class Space: + meth = staticmethod(labs) + def _freeze_(self): + return True + def g(x): + return x + def fn(space): + space.meth(g(0)) + g(-1) + space = Space() + def ep(): + return fn(space) + a = RPythonAnnotator() + a.build_types(ep, []) + if conftest.option.view: + a.translator.view() + class Test_specialization: def test_specialize_labs(self): res = interpret(test_labs, [-11]) Modified: pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py Fri Apr 21 19:30:22 2006 @@ -67,8 +67,7 @@ raise TypeError("wrap(%r)" % (x,)) wrap._annspecialcase_ = "specialize:wrap" - def getattr(self, w_obj, w_attr): - return PyObject_GetAttr(w_obj, w_attr) + getattr = staticmethod(PyObject_GetAttr) def call_function(self, w_callable): return PyObject_Call(w_callable, PyTuple_New(0), PyDict_New()) Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Fri Apr 21 19:30:22 2006 @@ -10,7 +10,7 @@ from pypy.rpython.module.support import init_opaque_object from pypy.rpython.module.support import to_opaque_object, from_opaque_object from pypy.rpython.module.support import from_rstr -from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry # for debugging, sanity checks in non-RPython code @@ -256,24 +256,31 @@ # helpers def setannotation(func, annotation, specialize_as_constant=False): - if specialize_as_constant: - def specialize(hop): - llvalue = func(hop.args_s[0].const) - return hop.inputconst(lltype.typeOf(llvalue), llvalue) - else: - # specialize as direct_call - def specialize(hop): - FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], - hop.r_result.lowleveltype) - args_v = hop.inputargs(*hop.args_r) - funcptr = lltype.functionptr(FUNCTYPE, func.__name__, - _callable=func) - cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) - return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) - - extregistry.register_value(func, - compute_result_annotation = annotation, - specialize_call = specialize) + + class Entry(ExtRegistryEntry): + "Annotation and specialization for calls to 'func'." + _about_ = func + + if annotation is None or isinstance(annotation, annmodel.SomeObject): + s_result_annotation = annotation + else: + def compute_result_annotation(self, *args_s): + return annotation(*args_s) + + if specialize_as_constant: + def specialize_call(self, hop): + llvalue = func(hop.args_s[0].const) + return hop.inputconst(lltype.typeOf(llvalue), llvalue) + else: + # specialize as direct_call + def specialize_call(self, hop): + FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], + hop.r_result.lowleveltype) + args_v = hop.inputargs(*hop.args_r) + funcptr = lltype.functionptr(FUNCTYPE, func.__name__, + _callable=func) + cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) + return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) # annotations from pypy.annotation import model as annmodel Modified: pypy/dist/pypy/rpython/test/test_extregistry.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_extregistry.py (original) +++ pypy/dist/pypy/rpython/test/test_extregistry.py Fri Apr 21 19:30:22 2006 @@ -3,6 +3,7 @@ ##py.test.skip('In progress at PyCon') from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry from pypy.annotation import model as annmodel from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext @@ -13,8 +14,9 @@ def dummy(): raiseNameError -extregistry.register_value(dummy, - compute_result_annotation=annmodel.SomeInteger()) +class Entry(ExtRegistryEntry): + _about_ = dummy + s_result_annotation = annmodel.SomeInteger() def test_call_dummy(): def func(): @@ -28,13 +30,13 @@ def test_callable_annotation(): def dummy2(): raiseNameError - - def return_annotation(): - return annmodel.SomeInteger() - - extregistry.register_value(dummy2, - compute_result_annotation=return_annotation) - + + class Entry(ExtRegistryEntry): + _about_ = dummy2 + + def compute_result_annotation(self): + return annmodel.SomeInteger() + def func(): x = dummy2() return x @@ -52,12 +54,12 @@ def func(): return dummy_type - def get_annotation(type, instance): - assert instance is dummy_type - return annmodel.SomeInteger() - - extregistry.register_type(DummyType, compute_annotation=get_annotation) - + class Entry(ExtRegistryEntry): + _type_ = DummyType + def compute_annotation(self): + assert self.instance is dummy_type + return annmodel.SomeInteger() + a = RPythonAnnotator() s = a.build_types(func, []) assert isinstance(s, annmodel.SomeInteger) @@ -74,13 +76,13 @@ def func(): return real_class - def get_annotation(t, x=None): - assert t is RealClass - assert x is real_class - return annmodel.SomeInteger() - - extregistry.register_metatype(MetaType, compute_annotation=get_annotation) - + class Entry(ExtRegistryEntry): + _metatype_ = MetaType + def compute_annotation(self): + assert self.type is RealClass + assert self.instance is real_class + return annmodel.SomeInteger() + a = RPythonAnnotator() s = a.build_types(func, []) assert isinstance(s, annmodel.SomeInteger) @@ -95,13 +97,13 @@ def func(real_class): return real_class - def get_annotation(t, x=None): - assert t is RealClass - assert x is None - return annmodel.SomeInteger() - - extregistry.register_metatype(MetaType, compute_annotation=get_annotation) - + class Entry(ExtRegistryEntry): + _metatype_ = MetaType + def compute_annotation(self): + assert self.type is RealClass + assert self.instance is None + return annmodel.SomeInteger() + a = RPythonAnnotator() s = a.build_types(func, [RealClass]) assert isinstance(s, annmodel.SomeInteger) @@ -110,13 +112,12 @@ def dummy_func(): raiseNameError - def dummy_specialize(hop): - return hop.inputconst(lltype.Signed, 42) - - extregistry.register_value(dummy_func, - compute_result_annotation=annmodel.SomeInteger(), - specialize_call=dummy_specialize) - + class Entry(ExtRegistryEntry): + _about_ = dummy_func + s_result_annotation = annmodel.SomeInteger() + def specialize_call(self, hop): + return hop.inputconst(lltype.Signed, 42) + def func(): return dummy_func() @@ -135,25 +136,25 @@ def rtyper_makekey( self ): return self.__class__, self.knowntype - - def get_annotation(type, instance=None): - assert type is DummyClass - dummy_object = SomeDummyObject() - dummy_object.knowntype = DummyClass - return dummy_object - + class DummyRepr(Repr): lowleveltype = lltype.Signed def convert_const(self, value): return 42 - - def get_repr(rtyper, s_instance): - return DummyRepr() - - extregistry.register_type(DummyClass, compute_annotation=get_annotation, - get_repr=get_repr) - + + class Entry(ExtRegistryEntry): + _type_ = DummyClass + + def compute_annotation(self): + assert self.type is DummyClass + dummy_object = SomeDummyObject() + dummy_object.knowntype = DummyClass + return dummy_object + + def get_repr(self, rtyper, s_instance): + return DummyRepr() + dummy_class = DummyClass() def func(): @@ -166,13 +167,15 @@ def test_register_unhashable(): lst1 = [5, 6] lst2 = [5, 6] - entry = extregistry.register_value(lst1) - assert extregistry.lookup(lst1) is entry + class Entry(ExtRegistryEntry): + _about_ = lst1 + assert isinstance(extregistry.lookup(lst1), Entry) py.test.raises(KeyError, "extregistry.lookup(lst2)") def test_register_non_weakly_refable(n=6): tup1 = (5, 6) tup2 = (5, n) - entry = extregistry.register_value(tup1) - assert extregistry.lookup(tup1) is entry - assert extregistry.lookup(tup2) is entry + class Entry(ExtRegistryEntry): + _about_ = tup1 + assert isinstance(extregistry.lookup(tup1), Entry) + assert isinstance(extregistry.lookup(tup2), Entry) Added: pypy/dist/pypy/tool/instancemethod.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/tool/instancemethod.py Fri Apr 21 19:30:22 2006 @@ -0,0 +1,30 @@ + +class InstanceMethod(object): + "Like types.InstanceMethod, but with a reasonable (structural) equality." + + def __init__(self, im_func, im_self, im_class): + self.im_func = im_func + self.im_self = im_self + self.im_class = im_class + + def __call__(__self, *args, **kwds): + firstarg = __self.im_self + if firstarg is None: + if not args or not isinstance(args[0], __self.im_class): + raise TypeError( + "must be called with %r instance as first argument" % ( + __self.im_class,)) + firstarg = args[0] + args = args[1:] + return __self.im_func(firstarg, *args, **kwds) + + def __eq__(self, other): + return isinstance(other, InstanceMethod) and ( + self.im_func == other.im_func and + self.im_self == other.im_self) + + def __ne__(self, other): + return not self.__eq__(other) + + def __hash__(self): + return hash((self.im_func, self.im_self)) From arigo at codespeak.net Fri Apr 21 21:16:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 21:16:05 +0200 (CEST) Subject: [pypy-svn] r26112 - pypy/dist/pypy/translator/c/test Message-ID: <20060421191605.0CCF01007D@code0.codespeak.net> Author: arigo Date: Fri Apr 21 21:16:05 2006 New Revision: 26112 Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py Log: Forgot this usage of extregistry. One of the tests is still failing, but it already did before the extregistry refactoring. Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Fri Apr 21 21:16:05 2006 @@ -1,7 +1,7 @@ from pypy.translator.translator import TranslationContext from pypy import conftest from py.test import raises -from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem import lltype from pypy.rpython import robject, rclass @@ -93,36 +93,37 @@ unwrap._annspecialcase_ = 'specialize:arg(1)' -def rtype_unwrap(hop): - v_obj = hop.inputarg(hop.args_r[0], 0) - return hop.llops.convertvar(v_obj, hop.args_r[0], hop.r_result) - -def rtype_wrap(hop): - assert len(hop.args_r) == 1, 'wrap() takes exactly one argument' - v_obj, = hop.inputargs(*hop.args_r) - return hop.llops.convertvar(v_obj, hop.args_r[0], robject.pyobj_repr) - -def compute_annotation_unwrap(s_wrapped, s_spec): - # this will go away, much better way found! - assert hasattr(s_spec, 'descriptions'), 'need a class in unwrap 2nd arg' - descs = s_spec.descriptions - assert len(descs) == 1, 'missing specialisation, classdesc not unique!' - for desc in descs.keys(): - classdef = desc.getuniqueclassdef() - return annmodel.SomeInstance(classdef) - # XXX # wrapping/unwrapping should be annotatable. # Idea: create tunnel objects which share # annotation across SomeObjectness, sharing a key! -extregistry.register_value(wrap, - compute_result_annotation=annmodel.SomeObject(), - specialize_call=rtype_wrap) - -extregistry.register_value(unwrap, - compute_result_annotation=compute_annotation_unwrap, - specialize_call=rtype_unwrap) +class Entry(ExtRegistryEntry): + _about_ = unwrap + + def compute_result_annotation(self, s_wrapped, s_spec): + # this will go away, much better way found! + assert hasattr(s_spec, 'descriptions'), 'need a class in unwrap 2nd arg' + descs = s_spec.descriptions + assert len(descs) == 1, 'missing specialisation, classdesc not unique!' + for desc in descs.keys(): + classdef = desc.getuniqueclassdef() + return annmodel.SomeInstance(classdef) + + def specialize_call(self, hop): + v_obj = hop.inputarg(hop.args_r[0], 0) + return hop.llops.convertvar(v_obj, hop.args_r[0], hop.r_result) + + +class Entry(ExtRegistryEntry): + _about_ = wrap + + s_result_annotation = annmodel.SomeObject() + + def specialize_call(self, hop): + assert len(hop.args_r) == 1, 'wrap() takes exactly one argument' + v_obj, = hop.inputargs(*hop.args_r) + return hop.llops.convertvar(v_obj, hop.args_r[0], robject.pyobj_repr) # _______________________________________________- # the actual tests From arigo at codespeak.net Fri Apr 21 21:24:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 21:24:16 +0200 (CEST) Subject: [pypy-svn] r26113 - pypy/dist/pypy/jit/codegen/llvm Message-ID: <20060421192416.8E34B1009C@code0.codespeak.net> Author: arigo Date: Fri Apr 21 21:24:16 2006 New Revision: 26113 Modified: pypy/dist/pypy/jit/codegen/llvm/rgenop.py Log: Another forgotten usage of extregistry. (Please try to merge this with the original rgenop.py at some point...) Modified: pypy/dist/pypy/jit/codegen/llvm/rgenop.py ============================================================================== --- pypy/dist/pypy/jit/codegen/llvm/rgenop.py (original) +++ pypy/dist/pypy/jit/codegen/llvm/rgenop.py Fri Apr 21 21:24:16 2006 @@ -10,7 +10,7 @@ from pypy.rpython.module.support import init_opaque_object from pypy.rpython.module.support import to_opaque_object, from_opaque_object from pypy.rpython.module.support import from_rstr -from pypy.rpython import extregistry +from pypy.rpython.extregistry import ExtRegistryEntry from pypy.jit.codegen.llvm.jitcode import JITcode @@ -257,24 +257,31 @@ # helpers def setannotation(func, annotation, specialize_as_constant=False): - if specialize_as_constant: - def specialize(hop): - llvalue = func(hop.args_s[0].const) - return hop.inputconst(lltype.typeOf(llvalue), llvalue) - else: - # specialize as direct_call - def specialize(hop): - FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], - hop.r_result.lowleveltype) - args_v = hop.inputargs(*hop.args_r) - funcptr = lltype.functionptr(FUNCTYPE, func.__name__, - _callable=func) - cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) - return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) - - extregistry.register_value(func, - compute_result_annotation = annotation, - specialize_call = specialize) + + class Entry(ExtRegistryEntry): + "Annotation and specialization for calls to 'func'." + _about_ = func + + if annotation is None or isinstance(annotation, annmodel.SomeObject): + s_result_annotation = annotation + else: + def compute_result_annotation(self, *args_s): + return annotation(*args_s) + + if specialize_as_constant: + def specialize_call(self, hop): + llvalue = func(hop.args_s[0].const) + return hop.inputconst(lltype.typeOf(llvalue), llvalue) + else: + # specialize as direct_call + def specialize_call(self, hop): + FUNCTYPE = lltype.FuncType([r.lowleveltype for r in hop.args_r], + hop.r_result.lowleveltype) + args_v = hop.inputargs(*hop.args_r) + funcptr = lltype.functionptr(FUNCTYPE, func.__name__, + _callable=func) + cfunc = hop.inputconst(lltype.Ptr(FUNCTYPE), funcptr) + return hop.genop('direct_call', [cfunc] + args_v, hop.r_result) # annotations from pypy.annotation import model as annmodel From arigo at codespeak.net Fri Apr 21 21:32:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 21:32:10 +0200 (CEST) Subject: [pypy-svn] r26114 - in pypy/dist/pypy/objspace: . test Message-ID: <20060421193210.B8A0D100A6@code0.codespeak.net> Author: arigo Date: Fri Apr 21 21:32:09 2006 New Revision: 26114 Removed: pypy/dist/pypy/objspace/test/test_trivialobjspace.py pypy/dist/pypy/objspace/trivial.py Log: Remove the trivial objspace again. It's not giving useful parallels for the CPyObjSpace. From arigo at codespeak.net Fri Apr 21 21:41:43 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 21 Apr 2006 21:41:43 +0200 (CEST) Subject: [pypy-svn] r26115 - in pypy/dist/pypy/rpython/rctypes: . test tool Message-ID: <20060421194143.57327100A2@code0.codespeak.net> Author: arigo Date: Fri Apr 21 21:41:41 2006 New Revision: 26115 Modified: pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py Log: Use a ctypes trick: use a subclass W_Object of py_object. This prevents automatic unwrapping: functions with res_type == W_Object will really return a W_Object instance, which is not the case if res_type == py_object. Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Fri Apr 21 21:41:41 2006 @@ -21,6 +21,13 @@ result_ctype = self.instance.restype if result_ctype is None: return None + if result_ctype is ctypes.py_object: + raise Exception("ctypes functions cannot have restype=py_object; " + "set their restype to a subclass of py_object " + "and call apyobject.register_py_object_subclass") + #... because then in ctypes you don't get automatic unwrapping. + # That would not be annotatable, for the same reason that + # reading the .value attribute of py_object is not annotatable s_result = SomeCTypesObject(result_ctype, SomeCTypesObject.OWNSMEMORY) return s_result.return_annotation() Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Fri Apr 21 21:41:41 2006 @@ -26,3 +26,9 @@ from pypy.rpython.rctypes.rpyobject import CTypesPyObjRepr lowleveltype = lltype.Ptr(lltype.PyObject) return CTypesPyObjRepr(rtyper, s_pyobject, lowleveltype) + + +def register_py_object_subclass(subcls): + assert issubclass(subcls, py_object) + CallEntry._register_value(subcls) + ObjEntry._register_type(subcls) Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Fri Apr 21 21:41:41 2006 @@ -178,3 +178,16 @@ pythonapi.PyInt_AsLong.restype = c_long assert pythonapi.PyInt_AsLong(py_object(17L)) == 17 py.test.raises(TypeError, "pythonapi.PyInt_AsLong(py_object('hello'))") + +def test_py_object_subclass(): + # automatic unwrapping of the py_object result + pythonapi.PyInt_FromLong.argtypes = [c_long] + pythonapi.PyInt_FromLong.restype = py_object + assert isinstance(pythonapi.PyInt_FromLong(17), int) + + # but not if we subclass it... + class W_Object(py_object): + pass + pythonapi.PyInt_FromLong.argtypes = [c_long] + pythonapi.PyInt_FromLong.restype = W_Object + assert isinstance(pythonapi.PyInt_FromLong(17), W_Object) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Fri Apr 21 21:41:41 2006 @@ -183,24 +183,28 @@ assert int(t1) <= t2 <= int(t3 + 1.0) def test_compile_pythonapi(self): + from pypy.rpython.rctypes import apyobject + class W_Object(py_object): + pass + apyobject.register_py_object_subclass(W_Object) PyInt_AsLong = pythonapi.PyInt_AsLong - PyInt_AsLong.argtypes = [py_object] + PyInt_AsLong.argtypes = [W_Object] PyInt_AsLong.restype = c_long assert PyInt_AsLong._flags_ & _FUNCFLAG_PYTHONAPI PyNumber_Add = pythonapi.PyNumber_Add - PyNumber_Add.argtypes = [py_object, py_object] - PyNumber_Add.restype = py_object + PyNumber_Add.argtypes = [W_Object, W_Object] + PyNumber_Add.restype = W_Object def fn1(x, crash): - pyobj = py_object(x) + pyobj = W_Object(x) pyobj = PyNumber_Add(pyobj, pyobj) x = PyInt_AsLong(pyobj) if crash: # fn(sys.maxint, 1) should crash on PyInt_AsLong before # it arrives here. If by mistake it arrives here then # we get a TypeError instead of the OverflowError - PyNumber_Add(py_object(5), py_object("x")) + PyNumber_Add(W_Object(5), W_Object("x")) return x fn = compile(fn1, [int, int]) Modified: pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py Fri Apr 21 21:41:41 2006 @@ -1,58 +1,65 @@ import sys from ctypes import * +class W_Object(py_object): + "A py_object subclass, representing wrapped objects for the CPyObjSpace." + +from pypy.rpython.rctypes import apyobject +apyobject.register_py_object_subclass(W_Object) + + assert sys.version < (2, 5), "XXX fix Py_ssize_t for Python 2.5" Py_ssize_t = c_int PyObject_GetAttr = pythonapi.PyObject_GetAttr -PyObject_GetAttr.argtypes = [py_object, py_object] -PyObject_GetAttr.restype = py_object +PyObject_GetAttr.argtypes = [W_Object, W_Object] +PyObject_GetAttr.restype = W_Object PyImport_ImportModule = pythonapi.PyImport_ImportModule PyImport_ImportModule.argtypes = [c_char_p] -PyImport_ImportModule.restype = py_object +PyImport_ImportModule.restype = W_Object PyInt_FromLong = pythonapi.PyInt_FromLong PyInt_FromLong.argtypes = [c_long] -PyInt_FromLong.restype = py_object +PyInt_FromLong.restype = W_Object PyString_FromStringAndSize = pythonapi.PyString_FromStringAndSize PyString_FromStringAndSize.argtypes = [c_char_p, Py_ssize_t] -PyString_FromStringAndSize.restype = py_object +PyString_FromStringAndSize.restype = W_Object PyString_InternInPlace = pythonapi.PyString_InternInPlace -PyString_InternInPlace.argtypes = [POINTER(py_object)] +PyString_InternInPlace.argtypes = [POINTER(W_Object)] PyString_InternInPlace.restype = None PyObject_SetItem = pythonapi.PyObject_SetItem -PyObject_SetItem.argtypes = [py_object, py_object, py_object] +PyObject_SetItem.argtypes = [W_Object, W_Object, W_Object] PyObject_SetItem.restype = c_int PyObject_Call = pythonapi.PyObject_Call -PyObject_Call.argtypes = [py_object, py_object, py_object] -PyObject_Call.restype = py_object +PyObject_Call.argtypes = [W_Object, W_Object, W_Object] +PyObject_Call.restype = W_Object PyTuple_New = pythonapi.PyTuple_New PyTuple_New.argtypes = [Py_ssize_t] -PyTuple_New.restype = py_object +PyTuple_New.restype = W_Object PyDict_New = pythonapi.PyDict_New PyDict_New.argtypes = [] -PyDict_New.restype = py_object +PyDict_New.restype = W_Object PyDict_SetItem = pythonapi.PyDict_SetItem -PyDict_SetItem.argtypes = [py_object, py_object, py_object] +PyDict_SetItem.argtypes = [W_Object, W_Object, W_Object] PyDict_SetItem.restype = c_int class CPyObjSpace: - W_Object = py_object + W_Object = W_Object def __init__(self): - self.w_int = py_object(int) - self.w_None = py_object(None) - self.w_False = py_object(False) - self.w_True = py_object(True) + self.w_int = W_Object(int) + self.w_None = W_Object(None) + self.w_False = W_Object(False) + self.w_True = W_Object(True) def getbuiltinmodule(self, name): return PyImport_ImportModule(name) From arigo at codespeak.net Sat Apr 22 11:01:57 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 22 Apr 2006 11:01:57 +0200 (CEST) Subject: [pypy-svn] r26121 - in pypy/dist/pypy/tool: . test Message-ID: <20060422090157.E71791008D@code0.codespeak.net> Author: arigo Date: Sat Apr 22 11:01:56 2006 New Revision: 26121 Added: pypy/dist/pypy/tool/test/test_instancemethod.py (contents, props changed) Modified: pypy/dist/pypy/tool/instancemethod.py Log: InstanceMethod: remove the pointless __self trick (see #pypy discussion). Added test. Modified: pypy/dist/pypy/tool/instancemethod.py ============================================================================== --- pypy/dist/pypy/tool/instancemethod.py (original) +++ pypy/dist/pypy/tool/instancemethod.py Sat Apr 22 11:01:56 2006 @@ -7,16 +7,16 @@ self.im_self = im_self self.im_class = im_class - def __call__(__self, *args, **kwds): - firstarg = __self.im_self + def __call__(self, *args, **kwds): + firstarg = self.im_self if firstarg is None: - if not args or not isinstance(args[0], __self.im_class): + if not args or not isinstance(args[0], self.im_class): raise TypeError( "must be called with %r instance as first argument" % ( - __self.im_class,)) + self.im_class,)) firstarg = args[0] args = args[1:] - return __self.im_func(firstarg, *args, **kwds) + return self.im_func(firstarg, *args, **kwds) def __eq__(self, other): return isinstance(other, InstanceMethod) and ( Added: pypy/dist/pypy/tool/test/test_instancemethod.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/tool/test/test_instancemethod.py Sat Apr 22 11:01:56 2006 @@ -0,0 +1,29 @@ +from pypy.tool.instancemethod import InstanceMethod + +class X(object): + def f(self, *args, **kwds): + return args, kwds + +def test_bound(): + obj = X() + obj.x = 12 + meth = InstanceMethod(X.f.im_func, obj, X) + assert meth(1, z=2) == ((1,), {'z': 2}) + +def test_unbound(): + obj = X() + obj.x = 12 + meth = InstanceMethod(X.f.im_func, None, X) + assert meth(obj, 1, z=2) == ((1,), {'z': 2}) + +def test_eq_hash(): + obj1 = X() + obj1.x = 12 + meth1 = InstanceMethod(X.f.im_func, obj1, X) + meth1bis = InstanceMethod(X.f.im_func, obj1, X) + obj2 = X() + obj2.x = 12 + meth2 = InstanceMethod(X.f.im_func, obj2, X) + d = {meth1: 123, meth2: 456} + assert len(d) == 2 + assert d[meth1bis] == 123 From hpk at codespeak.net Sat Apr 22 11:40:20 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 22 Apr 2006 11:40:20 +0200 (CEST) Subject: [pypy-svn] r26124 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060422094020.A09CE1009B@code0.codespeak.net> Author: hpk Date: Sat Apr 22 11:40:19 2006 New Revision: 26124 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: adding lawrence info (participance pending) Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Sat Apr 22 11:40:19 2006 @@ -109,3 +109,16 @@ organisation: Merlinux GmbH comments: no flight departure before 12AM because of travel time to the airport; + +Lawrence Oluyede +------------------------ + +preferred arrival date: 20.05.06 but 19th or 21th it's fine to me. +preferred departure date: 28th or 29th it's the same for me. +start city / airport: Milano / Milano Malpensa (MXP) +mail: l.oluyede at gmail.com +birthdate: 3rd June 1983 +address: Via Misano 16, 24047 Treviglio (BG), Italy +organisation: +comments: every departure date it's fine to me. + From antocuni at codespeak.net Sat Apr 22 11:51:58 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 22 Apr 2006 11:51:58 +0200 (CEST) Subject: [pypy-svn] r26128 - pypy/dist/pypy/annotation Message-ID: <20060422095158.5AFF0100A2@code0.codespeak.net> Author: antocuni Date: Sat Apr 22 11:51:51 2006 New Revision: 26128 Modified: pypy/dist/pypy/annotation/model.py Log: Added support for all subclasses of ootype.BuiltinType, not only List Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Sat Apr 22 11:51:51 2006 @@ -559,7 +559,7 @@ except TypeError: s = None # unhashable T, e.g. a Ptr(GcForwardReference()) if s is None: - if isinstance(T, (ootype.Instance, ootype.List)): + if isinstance(T, (ootype.Instance, ootype.BuiltinType)): return SomeOOInstance(T) elif isinstance(T, ootype.StaticMethod): return SomeOOStaticMeth(T) From arigo at codespeak.net Sat Apr 22 12:31:54 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 22 Apr 2006 12:31:54 +0200 (CEST) Subject: [pypy-svn] r26132 - in pypy/dist/pypy: objspace/cpy objspace/cpy/test rpython/rctypes/tool translator/goal Message-ID: <20060422103154.9CE3E100BB@code0.codespeak.net> Author: arigo Date: Sat Apr 22 12:31:51 2006 New Revision: 26132 Added: pypy/dist/pypy/objspace/cpy/ (props changed) pypy/dist/pypy/objspace/cpy/__init__.py (contents, props changed) pypy/dist/pypy/objspace/cpy/capi.py - copied, changed from r26117, pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py pypy/dist/pypy/objspace/cpy/objspace.py - copied, changed from r26117, pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py pypy/dist/pypy/objspace/cpy/test/ (props changed) pypy/dist/pypy/objspace/cpy/test/__init__.py - copied unchanged from r26117, pypy/dist/pypy/rpython/rctypes/tool/test/__init__.py pypy/dist/pypy/objspace/cpy/test/test_objspace.py (contents, props changed) Removed: pypy/dist/pypy/rpython/rctypes/tool/cpyobjspace.py Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Split the cpyobjspace.py module into a package in objspace/. Adding some more C API functions. Added: pypy/dist/pypy/objspace/cpy/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/__init__.py Sat Apr 22 12:31:51 2006 @@ -0,0 +1,2 @@ +from objspace import CPyObjSpace +Space = CPyObjSpace Added: pypy/dist/pypy/objspace/cpy/test/test_objspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/test/test_objspace.py Sat Apr 22 12:31:51 2006 @@ -0,0 +1,10 @@ +from pypy.objspace.cpy.objspace import CPyObjSpace + +def test_simple(): + space = CPyObjSpace() + wk1 = space.wrap('key') + wone = space.wrap(1) + d = space.newdict([(space.wrap('zero'),space.wrap(0))]) + space.setitem(d,wk1,wone) + wback = space.getitem(d,wk1) + assert space.eq_w(wback,wone) Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Sat Apr 22 12:31:51 2006 @@ -1,6 +1,6 @@ from pypy.module._demo import demo from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy -from pypy.rpython.rctypes.tool.cpyobjspace import CPyObjSpace +from pypy.objspace.cpy.objspace import CPyObjSpace import pypy.rpython.rctypes.implementation From arigo at codespeak.net Sat Apr 22 13:31:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 22 Apr 2006 13:31:36 +0200 (CEST) Subject: [pypy-svn] r26136 - in pypy/dist/pypy/rpython/rctypes/tool: . test Message-ID: <20060422113136.A62711007E@code0.codespeak.net> Author: arigo Date: Sat Apr 22 13:31:35 2006 New Revision: 26136 Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py Log: An easy refactoring of ctypes_platform.py to allow many queries -- a whole configuration -- to be performed in a single pass through the C compiler. This also gives a natural place to put some new general settings, like _include_dirs_ for a list of include dirs to pass to the compiler. Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Sat Apr 22 13:31:35 2006 @@ -5,6 +5,241 @@ from pypy.translator.tool.cbuild import build_executable from pypy.tool.udir import udir +# ____________________________________________________________ +# +# Helpers for simple cases + +def getstruct(name, c_header_source, interesting_fields): + class CConfig: + _header_ = c_header_source + STRUCT = Struct(name, interesting_fields) + return configure(CConfig)['STRUCT'] + +def getsimpletype(name, c_header_source, ctype_hint=ctypes.c_int): + class CConfig: + _header_ = c_header_source + TYPE = SimpleType(name, ctype_hint) + return configure(CConfig)['TYPE'] + +def getconstantinteger(name, c_header_source): + class CConfig: + _header_ = c_header_source + CONST = ConstantInteger(name) + return configure(CConfig)['CONST'] + +def getdefined(macro, c_header_source): + class CConfig: + _header_ = c_header_source + DEFINED = Defined(macro) + return configure(CConfig)['DEFINED'] + +# ____________________________________________________________ +# +# General interface + +def configure(CConfig): + """Examine the local system by running the C compiler. + The CConfig class contains CConfigEntry attribues that describe + what should be inspected; configure() returns a dict mapping + names to the results. + """ + entries = [] + for key in dir(CConfig): + value = getattr(CConfig, key) + if isinstance(value, CConfigEntry): + entries.append((key, value)) + + filepath = uniquefilepath() + f = filepath.open('w') + print >> f, C_HEADER + print >> f + print >> f, CConfig._header_ # mandatory + print >> f + for key, entry in entries: + print >> f, 'void dump_section_%s(void) {' % (key,) + print >> f, '\tprintf("-+- %s\\n");' % (key,) + for line in entry.prepare_code(): + if line and line[0] != '#': + line = '\t' + line + print >> f, line + print >> f, '\tprintf("---\\n");' + print >> f, '}' + print >> f + + print >> f, 'int main(void) {' + for key, entry in entries: + print >> f, '\tdump_section_%s();' % (key,) + print >> f, '\treturn 0;' + print >> f, '}' + f.close() + + include_dirs = getattr(CConfig, '_include_dirs_', []) + infolist = list(run_example_code(filepath, include_dirs)) + assert len(infolist) == len(entries) + + result = {} + for info, (key, entry) in zip(infolist, entries): + result[key] = entry.build_result(info) + return result + +# ____________________________________________________________ + + +class CConfigEntry(object): + "Abstract base class." + + +class Struct(CConfigEntry): + """An entry in a CConfig class that stands for an externally + defined structure. + """ + def __init__(self, name, interesting_fields): + self.name = name + self.interesting_fields = interesting_fields + + def prepare_code(self): + yield 'typedef %s ctypesplatcheck_t;' % (self.name,) + yield 'typedef struct {' + yield ' char c;' + yield ' ctypesplatcheck_t s;' + yield '} ctypesplatcheck2_t;' + yield '' + yield 'ctypesplatcheck_t s;' + yield 'dump("align", offsetof(ctypesplatcheck2_t, s));' + yield 'dump("size", sizeof(ctypesplatcheck_t));' + for fieldname, fieldtype in self.interesting_fields: + yield 'dump("fldofs %s", offsetof(ctypesplatcheck_t, %s));'%( + fieldname, fieldname) + yield 'dump("fldsize %s", sizeof(s.%s));' % ( + fieldname, fieldname) + if fieldtype in integer_class: + yield 's.%s = 0; s.%s = ~s.%s;' % (fieldname, + fieldname, + fieldname) + yield 'dump("fldunsigned %s", s.%s > 0);' % (fieldname, + fieldname) + + def build_result(self, info): + alignment = 1 + layout = [None] * info['size'] + for fieldname, fieldtype in self.interesting_fields: + offset = info['fldofs ' + fieldname] + size = info['fldsize ' + fieldname] + sign = info.get('fldunsigned ' + fieldname, False) + if (size, sign) != size_and_sign(fieldtype): + fieldtype = fixup_ctype(fieldtype, fieldname, (size, sign)) + layout_addfield(layout, offset, fieldtype, fieldname) + alignment = max(alignment, ctypes.alignment(fieldtype)) + + # try to enforce the same alignment as the one of the original + # structure + if alignment < info['align']: + choices = [ctype for ctype in alignment_types + if ctypes.alignment(ctype) == info['align']] + assert choices, "unsupported alignment %d" % (info['align'],) + choices = [(ctypes.sizeof(ctype), i, ctype) + for i, ctype in enumerate(choices)] + csize, _, ctype = min(choices) + for i in range(0, info['size'] - csize + 1, info['align']): + if layout[i:i+csize] == [None] * csize: + layout_addfield(layout, i, ctype, '_alignment') + break + else: + raise AssertionError("unenforceable alignment %d" % ( + info['align'],)) + + n = 0 + for i, cell in enumerate(layout): + if cell is not None: + continue + layout_addfield(layout, i, ctypes.c_char, '_pad%d' % (n,)) + n += 1 + + # build the ctypes Structure + seen = {} + fields = [] + for cell in layout: + if cell in seen: + continue + fields.append((cell.name, cell.ctype)) + seen[cell] = True + + class S(ctypes.Structure): + _fields_ = fields + name = self.name + if name.startswith('struct '): + name = name[7:] + S.__name__ = name + return S + + +class SimpleType(CConfigEntry): + """An entry in a CConfig class that stands for an externally + defined simple numeric type. + """ + def __init__(self, name, ctype_hint=ctypes.c_int): + self.name = name + self.ctype_hint = ctype_hint + + def prepare_code(self): + yield 'typedef %s ctypesplatcheck_t;' % (self.name,) + yield '' + yield 'ctypesplatcheck_t x;' + yield 'dump("size", sizeof(ctypesplatcheck_t));' + if self.ctype_hint in integer_class: + yield 'x = 0; x = ~x;' + yield 'dump("unsigned", x > 0);' + + def build_result(self, info): + size = info['size'] + sign = info.get('unsigned', False) + ctype = self.ctype_hint + if (size, sign) != size_and_sign(ctype): + ctype = fixup_ctype(ctype, self.name, (size, sign)) + return ctype + + +class ConstantInteger(CConfigEntry): + """An entry in a CConfig class that stands for an externally + defined integer constant. + """ + def __init__(self, name): + self.name = name + + def prepare_code(self): + yield ' if ((%s) < 0) {' % (self.name,) + yield ' long long x = (long long)(%s);' % (self.name,) + yield ' printf("value: %lld\\n", x);' + yield ' } else {' + yield ' unsigned long long x = (unsigned long long)(%s);' % ( + self.name,) + yield ' printf("value: %llu\\n", x);' + yield ' }' + + def build_result(self, info): + return info['value'] + + +class Defined(CConfigEntry): + """A boolean, corresponding to an #ifdef. + """ + def __init__(self, macro): + self.macro = macro + self.name = macro + + def prepare_code(self): + yield '#ifdef %s' % (self.macro,) + yield 'dump("defined", 1);' + yield '#else' + yield 'dump("defined", 0);' + yield '#endif' + + def build_result(self, info): + return bool(info['defined']) + +# ____________________________________________________________ +# +# internal helpers def uniquefilepath(LAST=[0]): i = LAST[0] @@ -75,157 +310,24 @@ } """ -def run_example_code(filepath): - executable = build_executable([filepath]) +def run_example_code(filepath, include_dirs=[]): + executable = build_executable([filepath], include_dirs=include_dirs) output = py.process.cmdexec(executable) - info = {} + section = None for line in output.splitlines(): - key, value = line.strip().split(': ') - info[key] = int(value) - return info - -def getstruct(name, c_header_source, interesting_fields): - filepath = uniquefilepath() - f = filepath.open('w') - print >> f, C_HEADER - print >> f - print >> f, c_header_source - print >> f - print >> f, 'typedef %s ctypesplatcheck_t;' % (name,) - print >> f, 'typedef struct {' - print >> f, ' char c;' - print >> f, ' ctypesplatcheck_t s;' - print >> f, '} ctypesplatcheck2_t;' - print >> f - print >> f, 'int main(void) {' - print >> f, ' ctypesplatcheck_t s;' - print >> f, ' dump("align", offsetof(ctypesplatcheck2_t, s));' - print >> f, ' dump("size", sizeof(ctypesplatcheck_t));' - for fieldname, fieldtype in interesting_fields: - print >> f, ' dump("fldofs %s", offsetof(ctypesplatcheck_t, %s));'%( - fieldname, fieldname) - print >> f, ' dump("fldsize %s", sizeof(s.%s));' % ( - fieldname, fieldname) - if fieldtype in integer_class: - print >> f, ' s.%s = 0; s.%s = ~s.%s;' % (fieldname, - fieldname, - fieldname) - print >> f, ' dump("fldunsigned %s", s.%s > 0);' % (fieldname, - fieldname) - print >> f, ' return 0;' - print >> f, '}' - f.close() - - info = run_example_code(filepath) - - alignment = 1 - layout = [None] * info['size'] - for fieldname, fieldtype in interesting_fields: - offset = info['fldofs ' + fieldname] - size = info['fldsize ' + fieldname] - sign = info.get('fldunsigned ' + fieldname, False) - if (size, sign) != size_and_sign(fieldtype): - fieldtype = fixup_ctype(fieldtype, fieldname, (size, sign)) - layout_addfield(layout, offset, fieldtype, fieldname) - alignment = max(alignment, ctypes.alignment(fieldtype)) - - # try to enforce the same alignment as the one of the original structure - if alignment < info['align']: - choices = [ctype for ctype in alignment_types - if ctypes.alignment(ctype) == info['align']] - assert choices, "unsupported alignment %d" % (info['align'],) - choices = [(ctypes.sizeof(ctype), i, ctype) - for i, ctype in enumerate(choices)] - csize, _, ctype = min(choices) - for i in range(0, info['size'] - csize + 1, info['align']): - if layout[i:i+csize] == [None] * csize: - layout_addfield(layout, i, ctype, '_alignment') - break - else: - raise AssertionError("unenforceable alignment %d" % ( - info['align'],)) - - n = 0 - for i, cell in enumerate(layout): - if cell is not None: - continue - layout_addfield(layout, i, ctypes.c_char, '_pad%d' % (n,)) - n += 1 - - # build the ctypes Structure - seen = {} - fields = [] - for cell in layout: - if cell in seen: - continue - fields.append((cell.name, cell.ctype)) - seen[cell] = True - - class S(ctypes.Structure): - _fields_ = fields - if name.startswith('struct '): - name = name[7:] - S.__name__ = name - return S - - -def getsimpletype(name, c_header_source, ctype_hint): - filepath = uniquefilepath() - f = filepath.open('w') - print >> f, C_HEADER - print >> f - print >> f, c_header_source - print >> f - print >> f, 'typedef %s ctypesplatcheck_t;' % (name,) - print >> f - print >> f, 'int main(void) {' - print >> f, ' ctypesplatcheck_t x;' - print >> f, ' dump("size", sizeof(ctypesplatcheck_t));' - if ctype_hint in integer_class: - print >> f, ' x = 0; x = ~x;' - print >> f, ' dump("unsigned", x > 0);' - print >> f, ' return 0;' - print >> f, '}' - f.close() - - info = run_example_code(filepath) - - size = info['size'] - sign = info.get('unsigned', False) - if (size, sign) != size_and_sign(ctype_hint): - ctype_hint = fixup_ctype(ctype_hint, name, (size, sign)) - return ctype_hint - - -def getconstantinteger(name, c_header_source): - filepath = uniquefilepath() - f = filepath.open('w') - print >> f, C_HEADER - print >> f - print >> f, c_header_source - print >> f - print >> f, 'int main(void) {' - print >> f, ' if ((%s) < 0) {' % (name,) - print >> f, ' long long x = (long long)(%s);' % (name,) - print >> f, ' printf("value: %lld\\n", x);' - print >> f, ' } else {' - print >> f, ' unsigned long long x = (unsigned long long)(%s);' % ( - name,) - print >> f, ' printf("value: %llu\\n", x);' - print >> f, ' }' - print >> f, ' return 0;' - print >> f, '}' - f.close() - - info = run_example_code(filepath) - - return info['value'] - - -def getdefined(name, c_header_source): - return getconstantinteger('\n#ifdef %s\n1\n#else\n0\n#endif\n' % (name), - c_header_source) + line = line.strip() + if line.startswith('-+- '): # start of a new section + section = {} + elif line == '---': # section end + assert section is not None + yield section + section = None + elif line: + assert section is not None + key, value = line.split(': ') + section[key] = int(value) +# ____________________________________________________________ if __name__ == '__main__': doc = """Example: Modified: pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py Sat Apr 22 13:31:35 2006 @@ -1,5 +1,6 @@ import py, sys, struct from pypy.rpython.rctypes.tool import ctypes_platform +from pypy.tool.udir import udir import ctypes @@ -102,3 +103,24 @@ res = ctypes_platform.getdefined('ALFKJLKJFLKJFKLEJDLKEWMECEE', '#define ALFKJLKJFLKJFKLEJDLKEWMECEE') assert res + +def test_configure(): + test_h = udir.join('test_ctypes_platform.h') + test_h.write('#define XYZZY 42\n') + + class CConfig: + _header_ = """ /* a C comment */ + #include + #include + """ + _include_dirs_ = [str(udir)] + + FILE = ctypes_platform.Struct('FILE', []) + ushort = ctypes_platform.SimpleType('unsigned short') + XYZZY = ctypes_platform.ConstantInteger('XYZZY') + + res = ctypes_platform.configure(CConfig) + assert issubclass(res['FILE'], ctypes.Structure) + assert res == {'FILE': res['FILE'], + 'ushort': ctypes.c_ushort, + 'XYZZY': 42} From antocuni at codespeak.net Sat Apr 22 13:36:52 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 22 Apr 2006 13:36:52 +0200 (CEST) Subject: [pypy-svn] r26138 - in pypy/dist/pypy: rpython/ootypesystem rpython/test translator/cli/test Message-ID: <20060422113652.315861007D@code0.codespeak.net> Author: antocuni Date: Sat Apr 22 13:36:32 2006 New Revision: 26138 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py pypy/dist/pypy/translator/cli/test/compile.py Log: More rdict support. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sat Apr 22 13:36:32 2006 @@ -283,8 +283,11 @@ self._GENERIC_METHODS = frozendict({ "ll_length": Meth([], Signed), - "ll_getitem": Meth([self.KEYTYPE_T], self.VALUETYPE_T), - "ll_setitem": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), + "ll_get": Meth([self.KEYTYPE_T, self.VALUETYPE_T], self.VALUETYPE_T), # ll_get(key, default) + "ll_set": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), + "ll_remove": Meth([self.KEYTYPE_T], Bool), # return False is key was not present + "ll_contains": Meth([self.KEYTYPE_T], Bool), + #"ll_keys": Meth([], List(self.KEYTYPE_T)), }) self._setup_methods(generic_types) @@ -643,15 +646,32 @@ # NOT_RPYTHON return len(self._dict) - def ll_getitem(self, key): + def ll_get(self, key, default): + # NOT_RPYTHON assert typeOf(key) == self._TYPE._KEYTYPE - return self._dict[key] + assert typeOf(key) == self._TYPE._VALUETYPE + return self._dict.get(key, default) - def ll_setitem(self, key, value): + def ll_set(self, key, value): + # NOT_RPYTHON assert typeOf(key) == self._TYPE._KEYTYPE assert typeOf(value) == self._TYPE._VALUETYPE self._dict[key] = value + def ll_remove(self, key): + # NOT_RPYTHON + assert typeOf(key) == self._TYPE._KEYTYPE + try: + del self._dict[key] + return True + except KeyError: + return False + + def ll_contains(self, key): + # NOT_RPYTHON + assert typeOf(key) == self._TYPE._KEYTYPE + return key in self._dict + class _null_dict(_null_mixin(_dict), _dict): def __init__(self, DICT): Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Sat Apr 22 13:36:32 2006 @@ -62,6 +62,29 @@ return hop.genop("oosend", [c_name] + v_args, resulttype=hop.r_result.lowleveltype) + def rtype_len(self, hop): + v_dict, = hop.inputargs(self) + return self.send_message(hop, 'll_length') + + def rtype_is_true(self, hop): + v_dict, = hop.inputargs(self) + return hop.gendirectcall(ll_dict_is_true, v_dict) + + def rtype_method_get(self, hop): + v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, + self.value_repr) + hop.exception_cannot_occur() + v_res = self.send_message(hop, 'll_get') + #v_res = hop.gendirectcall(ll_get, v_dict, v_key, v_default) + return self.recast_value(hop.llops, v_res) + + def rtype_method_setdefault(self, hop): + v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, + self.value_repr) + hop.exception_cannot_occur() + v_res = hop.gendirectcall(ll_dict_setdefault, v_dict, v_key, v_default) + return self.recast_value(hop.llops, v_res) + class __extend__(pairtype(DictRepr, rmodel.Repr)): @@ -70,15 +93,16 @@ if not r_dict.custom_eq_hash: # TODO: why only in this case? hop.has_implicit_exception(KeyError) # record that we know about it hop.exception_is_here() - v_res = r_dict.send_message(hop, 'll_getitem', can_raise=True) + c_dummy_default = hop.inputconst(r_dict.value_repr.lowleveltype, r_dict.value_repr.ll_dummy_value) + v_res = hop.gendirectcall(ll_dict_getitem, v_dict, v_key, c_dummy_default) return r_dict.recast_value(hop.llops, v_res) -## def rtype_delitem((r_dict, r_key), hop): -## v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) -## if not r_dict.custom_eq_hash: -## hop.has_implicit_exception(KeyError) # record that we know about it -## hop.exception_is_here() -## return hop.gendirectcall(ll_dict_delitem, v_dict, v_key) + def rtype_delitem((r_dict, r_key), hop): + v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + if not r_dict.custom_eq_hash: # TODO: why only in this case? + hop.has_implicit_exception(KeyError) # record that we know about it + hop.exception_is_here() + return hop.gendirectcall(ll_dict_delitem, v_dict, v_key) def rtype_setitem((r_dict, r_key), hop): v_dict, v_key, v_value = hop.inputargs(r_dict, r_dict.key_repr, r_dict.value_repr) @@ -87,8 +111,35 @@ ## else: ## hop.exception_cannot_occur() hop.exception_is_here() - return r_dict.send_message(hop, 'll_setitem', can_raise=True) + return r_dict.send_message(hop, 'll_set', can_raise=True) + + def rtype_contains((r_dict, r_key), hop): + v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + return r_dict.send_message(hop, 'll_contains') + def ll_newdict(DICT): return ootype.new(DICT) + +def ll_dict_is_true(d): + # check if a dict is True, allowing for None + return bool(d) and d.ll_length() != 0 + +def ll_dict_getitem(d, key, dummy_default): + # TODO: this is inefficient because it does two lookups + if d.ll_contains(key): + return d.ll_get(key, dummy_default) # dummy_default is never returned + else: + raise KeyError + +def ll_dict_delitem(d, key): + if not d.ll_remove(key): + raise KeyError + +def ll_dict_setdefault(d, key, default): + try: + return ll_dict_getitem(d, key, default) + except KeyError: + d.ll_set(key, default) + return default Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Sat Apr 22 13:36:32 2006 @@ -1,6 +1,6 @@ from pypy.translator.translator import TranslationContext from pypy.rpython.lltypesystem import lltype -from pypy.rpython.test.test_llinterp import interpret +from pypy.rpython.test.test_llinterp import interpret, interpret_raises from pypy.rpython import rstr, rint from pypy.rpython.lltypesystem import rdict @@ -15,13 +15,6 @@ return interpret_raises(exc, fn, args, type_system=self.ts) - def test_dict_creation(self): - def createdict(i): - d = {i: i+1} - return d[i] - res = self.interpret(createdict, [42]) - assert res == 43 - # XXX: most tests doesn't works because ootypesystem doesn't support strings, yet def test_dict_creation(): def createdict(i): @@ -732,3 +725,68 @@ class TestOotypeRtyping(BaseTestDictRtyping): ts = "ootype" + + # these tests are similar to those above, but they don't use strings + def test_dict_creation(self): + def createdict(i): + d = {i: i+1} + return d[i] + res = self.interpret(createdict, [42]) + assert res == 43 + + def test_dict_getitem_setitem(self): + def func(i): + d = {i: i+1} + d[i] = i+2 + return d[i] + res = self.interpret(func, [42]) + + def test_dict_is_true(self): + def func(i): + if i: + d = {} + else: + d = {i: i+1} + if d: + return i + else: + return i+1 + assert self.interpret(func, [42]) == 43 + assert self.interpret(func, [0]) == 0 + + def test_contains(self): + def func(x, y): + d = {x: x+1} + return y in d + assert self.interpret(func, [42, 0]) == False + assert self.interpret(func, [42, 42]) == True + + def test_delitem(self): + def func(x, y): + d = {x: x+1} + del d[y] + self.interpret(func, [42, 42]) # don't raise anything + self.interpret_raises(KeyError, func, [42, 0]) + + def test_length(self): + def func(num): + d = {} + for i in range(num): + d[i] = i+1 + return len(d) + assert self.interpret(func, [0]) == 0 + assert self.interpret(func, [2]) == 2 + + def test_get(self): + def func(x, y): + d = {x: x+1} + return d.get(x, y) + d.get(x+1, y) + assert self.interpret(func, [42, 13]) == 56 + + def test_setdefault(self): + def func(x, y): + d = {} + d.setdefault(x, y) + return d[x] + assert self.interpret(func, [42, 13]) == 13 + Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Sat Apr 22 13:36:32 2006 @@ -26,10 +26,8 @@ def bar(x, y): - lst = [1,2] - lst.append(x) - return lst[2] - + d = {x: y} + return d[x] f = compile_function(bar, [int, int]) From arigo at codespeak.net Sat Apr 22 13:41:21 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 22 Apr 2006 13:41:21 +0200 (CEST) Subject: [pypy-svn] r26139 - in pypy/dist/pypy: objspace/cpy rpython/rctypes/tool Message-ID: <20060422114121.68EAD10080@code0.codespeak.net> Author: arigo Date: Sat Apr 22 13:41:20 2006 New Revision: 26139 Modified: pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Log: ANSI-C-ification. Use ctypes_platform.configure() in the cpyobjspace. Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Sat Apr 22 13:41:20 2006 @@ -14,23 +14,26 @@ ############################################################### # ____________________ Types and constants ____________________ -##Py_ssize_t = ctypes_platform.getsimpletype('Py_ssize_t', -## """ #include -## #if PY_VERSION_HEX < 0x02050000 /* < 2.5 */ -## typedef int Py_ssize_t; -## #endif -## """, c_int) - -##Py_LT = ctypes_platform.getconstantinteger('Py_LT', "#include ") -##Py_LE = ctypes_platform.getconstantinteger('Py_LE', "#include ") -##Py_EQ = ctypes_platform.getconstantinteger('Py_EQ', "#include ") -##Py_NE = ctypes_platform.getconstantinteger('Py_NE', "#include ") -##Py_GT = ctypes_platform.getconstantinteger('Py_GT', "#include ") -##Py_GE = ctypes_platform.getconstantinteger('Py_GE', "#include ") - -# XXX ctypes_platform needs to be enhanced... -Py_ssize_t = c_int -Py_LT, Py_LE, Py_EQ, Py_NE, Py_GT, Py_GE = range(6) +class CConfig: + _header_ = """ +#include +#if PY_VERSION_HEX < 0x02050000 /* < 2.5 */ +typedef int Py_ssize_t; +#endif + """ + _include_dirs_ = [ctypes_platform.get_python_include_dir()] + + Py_ssize_t = ctypes_platform.SimpleType('Py_ssize_t') + + Py_LT = ctypes_platform.ConstantInteger('Py_LT') + Py_LE = ctypes_platform.ConstantInteger('Py_LE') + Py_EQ = ctypes_platform.ConstantInteger('Py_EQ') + Py_NE = ctypes_platform.ConstantInteger('Py_NE') + Py_GT = ctypes_platform.ConstantInteger('Py_GT') + Py_GE = ctypes_platform.ConstantInteger('Py_GE') + +globals().update(ctypes_platform.configure(CConfig)) +del CConfig ########################################################### Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Sat Apr 22 13:41:20 2006 @@ -57,18 +57,18 @@ print >> f for key, entry in entries: print >> f, 'void dump_section_%s(void) {' % (key,) - print >> f, '\tprintf("-+- %s\\n");' % (key,) for line in entry.prepare_code(): if line and line[0] != '#': line = '\t' + line print >> f, line - print >> f, '\tprintf("---\\n");' print >> f, '}' print >> f print >> f, 'int main(void) {' for key, entry in entries: + print >> f, '\tprintf("-+- %s\\n");' % (key,) print >> f, '\tdump_section_%s();' % (key,) + print >> f, '\tprintf("---\\n");' print >> f, '\treturn 0;' print >> f, '}' f.close() @@ -207,14 +207,14 @@ self.name = name def prepare_code(self): - yield ' if ((%s) < 0) {' % (self.name,) - yield ' long long x = (long long)(%s);' % (self.name,) - yield ' printf("value: %lld\\n", x);' - yield ' } else {' - yield ' unsigned long long x = (unsigned long long)(%s);' % ( - self.name,) - yield ' printf("value: %llu\\n", x);' - yield ' }' + yield 'if ((%s) < 0) {' % (self.name,) + yield ' long long x = (long long)(%s);' % (self.name,) + yield ' printf("value: %lld\\n", x);' + yield '} else {' + yield ' unsigned long long x = (unsigned long long)(%s);' % ( + self.name,) + yield ' printf("value: %llu\\n", x);' + yield '}' def build_result(self, info): return info['value'] @@ -329,6 +329,11 @@ # ____________________________________________________________ +def get_python_include_dir(): + from distutils import sysconfig + gcv = sysconfig.get_config_vars() + return gcv['INCLUDEPY'] + if __name__ == '__main__': doc = """Example: From antocuni at codespeak.net Sat Apr 22 14:23:41 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 22 Apr 2006 14:23:41 +0200 (CEST) Subject: [pypy-svn] r26141 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060422122341.8AECC1007A@code0.codespeak.net> Author: antocuni Date: Sat Apr 22 14:23:30 2006 New Revision: 26141 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: Added support for recursive generic types in BuiltinType._GENERIC_METHODS. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sat Apr 22 14:23:30 2006 @@ -186,11 +186,22 @@ def _setup_methods(self, generic_types): methods = {} for name, meth in self._GENERIC_METHODS.iteritems(): - args = [generic_types.get(arg, arg) for arg in meth.ARGS] - result = generic_types.get(meth.RESULT, meth.RESULT) + #args = [generic_types.get(arg, arg) for arg in meth.ARGS] + #result = generic_types.get(meth.RESULT, meth.RESULT) + args = [self._specialize_type(arg, generic_types) for arg in meth.ARGS] + result = self._specialize_type(meth.RESULT, generic_types) methods[name] = Meth(args, result) self._METHODS = frozendict(methods) + def _specialize_type(self, type_, generic_types): + if isinstance(type_, BuiltinType): + return type_._specialize(generic_types) + else: + return generic_types.get(type_, type_) + + def _specialize(self, generic_types): + raise NotImplementedError + def _lookup(self, meth_name): METH = self._METHODS.get(meth_name) meth = None @@ -263,6 +274,10 @@ def _get_interp_class(self): return _list + def _specialize(self, generic_types): + ITEMTYPE = self._specialize_type(self._ITEMTYPE, generic_types) + return self.__class__(ITEMTYPE) + class Dict(BuiltinType): # placeholders for types @@ -306,6 +321,10 @@ def _get_interp_class(self): return _dict + def _specialize(self, generic_types): + KEYTYPE = self._specialize_type(self._KEYTYPE, generic_types) + VALUETYPE = self._specialize_type(self._VALUETYPE, generic_types) + return self.__class__(KEYTYPE, VALUETYPE) class ForwardReference(OOType): def become(self, real_instance): @@ -757,4 +776,3 @@ ROOT = Instance('Root', None, _is_root=True) - From antocuni at codespeak.net Sat Apr 22 20:26:10 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 22 Apr 2006 20:26:10 +0200 (CEST) Subject: [pypy-svn] r26145 - in pypy/dist/pypy/rpython: ootypesystem test Message-ID: <20060422182610.7F03D1007A@code0.codespeak.net> Author: antocuni Date: Sat Apr 22 20:26:00 2006 New Revision: 26145 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: Added method keys() to rdict. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sat Apr 22 20:26:00 2006 @@ -302,7 +302,7 @@ "ll_set": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), "ll_remove": Meth([self.KEYTYPE_T], Bool), # return False is key was not present "ll_contains": Meth([self.KEYTYPE_T], Bool), - #"ll_keys": Meth([], List(self.KEYTYPE_T)), + "ll_keys": Meth([], List(self.KEYTYPE_T)), }) self._setup_methods(generic_types) @@ -691,6 +691,12 @@ assert typeOf(key) == self._TYPE._KEYTYPE return key in self._dict + def ll_keys(self): + # NOT_RPYTHON + keys = _list(List(self._TYPE._KEYTYPE)) + keys._list = self._dict.keys() + return keys + class _null_dict(_null_mixin(_dict), _dict): def __init__(self, DICT): Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Sat Apr 22 20:26:00 2006 @@ -85,6 +85,10 @@ v_res = hop.gendirectcall(ll_dict_setdefault, v_dict, v_key, v_default) return self.recast_value(hop.llops, v_res) + def rtype_method_keys(self, hop): + hop.exception_cannot_occur() + return self.send_message(hop, 'll_keys') + class __extend__(pairtype(DictRepr, rmodel.Repr)): Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Sat Apr 22 20:26:00 2006 @@ -726,6 +726,9 @@ class TestOotypeRtyping(BaseTestDictRtyping): ts = "ootype" + def ll_to_list(self, l): + return l._list[:] + # these tests are similar to those above, but they don't use strings def test_dict_creation(self): def createdict(i): @@ -789,4 +792,10 @@ d.setdefault(x, y) return d[x] assert self.interpret(func, [42, 13]) == 13 - + + def test_keys(self): + def func(x, y): + d = {x: x+1, y: y+1} + return d.keys() + res = self.ll_to_list(self.interpret(func, [42, 13])) + assert res == [42, 13] or res == [13, 42] From tismer at codespeak.net Sun Apr 23 03:46:19 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 23 Apr 2006 03:46:19 +0200 (CEST) Subject: [pypy-svn] r26152 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060423014619.66FDA10079@code0.codespeak.net> Author: tismer Date: Sun Apr 23 03:46:15 2006 New Revision: 26152 Modified: pypy/dist/pypy/rpython/lltypesystem/rtuple.py Log: external_items_r is needed when wrapping instances in tuples. It is not yet clear how to go for the opposite, so this case is not handled, yet. Modified: pypy/dist/pypy/rpython/lltypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rtuple.py Sun Apr 23 03:46:15 2006 @@ -77,7 +77,7 @@ v_item = llops.gencapicall('PyTuple_GetItem_WithIncref', [v, ci], resulttype = pyobj_repr) v_converted = llops.convertvar(v_item, pyobj_repr, - r_to.external_items_r[i]) + r_to.items_r[i]) vlist.append(v_converted) return r_to.newtuple(llops, r_to, vlist) From tismer at codespeak.net Sun Apr 23 03:47:45 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 23 Apr 2006 03:47:45 +0200 (CEST) Subject: [pypy-svn] r26153 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060423014745.8BAB710079@code0.codespeak.net> Author: tismer Date: Sun Apr 23 03:47:43 2006 New Revision: 26153 Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py Log: comment on bare_setfield. What actually would be needed is an operation that writes into an uninitialized field. Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Sun Apr 23 03:47:43 2006 @@ -611,8 +611,9 @@ repr = c_spec.value if repr.has_wrapper: null = hop.inputconst(Ptr(PyObject), nullptr(PyObject)) - # XXX this is a hack! We need an operation to remove a broken - # PyObject + # XXX this bare_setfield is needed because we cannot do refcount operations + # XXX on a dead object. Actually this is an abuse. Instead, + # XXX we should consider a different operation for 'uninitialized fields' repr.setfield(v_inst, '_wrapper_', null, hop.llops, opname='bare_setfield') hop.genop('gc_unprotect', [v_inst]) From tismer at codespeak.net Sun Apr 23 03:50:29 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 23 Apr 2006 03:50:29 +0200 (CEST) Subject: [pypy-svn] r26154 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060423015029.C0F1410079@code0.codespeak.net> Author: tismer Date: Sun Apr 23 03:50:26 2006 New Revision: 26154 Modified: pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/wrapper.py Log: creating instances as a side effect of __init__ was much much harder than expected. __init__ needs to be cloned for every subclass that doesn't have its own. Suppress wrapping of __del__ because it is supported by RPython already. Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Sun Apr 23 03:50:26 2006 @@ -2,10 +2,10 @@ import autopath, os, sys, __builtin__, marshal, zlib from types import FunctionType, CodeType, InstanceType, ClassType -from pypy.objspace.flow.model import Variable, Constant +from pypy.objspace.flow.model import Variable, Constant, FunctionGraph from pypy.translator.gensupp import builtin_base, builtin_type_base from pypy.translator.c.support import log -from pypy.translator.c.wrapper import gen_wrapper +from pypy.translator.c.wrapper import gen_wrapper, new_method_graph from pypy.rpython.rarithmetic import r_int, r_uint from pypy.rpython.lltypesystem.lltype import pyobjectptr, LowLevelType @@ -55,6 +55,8 @@ def computenameof(self, obj): obj_builtin_base = builtin_base(obj) if obj_builtin_base in (object, int, long) and type(obj) is not obj_builtin_base: + if isinstance(obj, FunctionGraph): + return self.nameof_graph(obj) # assume it's a user defined thingy return self.nameof_instance(obj) else: @@ -209,7 +211,8 @@ if self.shouldskipfunc(func): return self.skipped_function(func) - fwrapper = gen_wrapper(func, self.translator, self.name_for_meth.get(func, func.__name__)) + fwrapper = gen_wrapper(func, self.translator, + newname=self.name_for_meth.get(func, func.__name__)) pycfunctionobj = self.uniquename('gfunc_' + func.__name__) self.wrappers[pycfunctionobj] = func.__name__, self.getvalue(fwrapper), func.__doc__ return pycfunctionobj @@ -371,7 +374,8 @@ ignore = getattr(cls, 'NOT_RPYTHON_ATTRIBUTES', []) for key, value in content: if key.startswith('__'): - if key in ['__module__', '__doc__', '__dict__', + # we do not expose __del__, because it would be called twice + if key in ['__module__', '__doc__', '__dict__', '__del__', '__weakref__', '__repr__', '__metaclass__']: continue # XXX some __NAMES__ are important... nicer solution sought @@ -534,20 +538,46 @@ metaclass = "type" name = self.uniquename('gwcls_' + cls.__name__) basenames = [self.nameof(base) for base in cls.__bases__] + # we merge the class dicts for more speed + def merge_classdicts(cls): + dic = {} + for cls in cls.mro()[:-1]: + for key, value in cls.__dict__.items(): + if key not in dic: + dic[key] = value + return dic def initclassobj(): - content = cls.__dict__.items() + content = merge_classdicts(cls).items() content.sort() + init_seen = False for key, value in content: if key.startswith('__'): - if key in ['__module__', '__dict__', '__doc__', + # we do not expose __del__, because it would be called twice + if key in ['__module__', '__dict__', '__doc__', '__del__', '__weakref__', '__repr__', '__metaclass__']: continue if self.shouldskipfunc(value): log.WARNING("skipped class function: %r" % value) continue - if callable(value): - self.name_for_meth[value] = '%s.%s' % (cls.__name__, value.__name__) + if isinstance(value, FunctionType): + func = value + fname = '%s.%s' % (cls.__name__, func.__name__) + if func.__name__ == '__init__': + init_seen = True + # there is the problem with exposed classes inheriting from + # classes which are internal. We need to create a new wrapper + # for every class which uses an inherited __init__, because + # this is the context where we create the instance. + ann = self.translator.annotator + clsdef = ann.bookkeeper.getuniqueclassdef(cls) + graph = ann.bookkeeper.getdesc(func).cachedgraph(None) + if ann.binding(graph.getargs()[0]).classdef is not clsdef: + value = new_method_graph(graph, clsdef, fname, self.translator) + self.name_for_meth[value] = fname yield '%s.%s = %s' % (name, key, self.nameof(value)) + if not init_seen: + log.WARNING('No __init__ found for %s - you cannot build instances' % + cls.__name__) baseargs = ", ".join(basenames) if baseargs: @@ -561,3 +591,10 @@ a(' __slots__ = ["__self__"] # for PyCObject') self.later(initclassobj()) return name + + def nameof_graph(self, g): + newname=self.name_for_meth.get(g, g.func.__name__) + fwrapper = gen_wrapper(g, self.translator, newname=newname) + pycfunctionobj = self.uniquename('gfunc_' + newname) + self.wrappers[pycfunctionobj] = g.func.__name__, self.getvalue(fwrapper), g.func.__doc__ + return pycfunctionobj Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Sun Apr 23 03:50:26 2006 @@ -7,7 +7,7 @@ from pypy.rpython import robject, rclass from pypy.translator.tool.cbuild import enable_fast_compilation -import sys +import sys, types P = False # debug printing @@ -22,7 +22,7 @@ missing = [object] * (func.func_code.co_argcount - len(argstypelist)) return missing + argstypelist -def get_compiled_module(func, view=conftest.option.view, inline_threshold=0*1, +def get_compiled_module(func, view=conftest.option.view, inline_threshold=1, use_boehm=False, exports=None): from pypy.translator.translator import TranslationContext from pypy.translator.backendopt.all import backend_optimizations @@ -35,22 +35,20 @@ t.buildannotator() rtyper = t.buildrtyper() bk = rtyper.annotator.bookkeeper - t.annotator.build_types(func, get_annotation(func)) if not exports: exports = [] all = [obj.__name__ for obj in exports] exports = exports + [('__all__', all)] + + t.annotator.build_types(func, get_annotation(func)) + for obj in exports: if isinstance(obj, type): clsdef = bk.getuniqueclassdef(obj) rtyper.add_wrapper(clsdef) - # pull it all out - for name, value in obj.__dict__.items(): - continue # do we want that - if callable(value): - t.annotator.build_types(value, get_annotation(value)) - elif callable(obj): + elif isinstance(obj, types.FunctionType): t.annotator.build_types(obj, get_annotation(obj)) + if view: t.viewcg() rtyper.specialize() @@ -70,10 +68,10 @@ # explicit build of database db = cbuilder.build_database(exports=exports) cbuilder.generate_source(db) - cbuilder.compile() - if view: t.viewcg() + cbuilder.compile() + return cbuilder.import_module() def getcompiled(func, *args, **kwds): @@ -142,10 +140,6 @@ delmonitor = DelMonitor() class DemoBaseNotExposed(object): - pass - -# a trivial class to be exposed -class DemoClass(DemoBaseNotExposed): """this is the doc string""" def __init__(self, a, b): self.a = a @@ -157,6 +151,9 @@ if P:print 'demo' return self.a + self.b + +# a trivial class to be exposed +class DemoClass(DemoBaseNotExposed): def demonotcalled(self): return self.demo() + 42 @@ -283,17 +280,19 @@ assert res == DemoClass(2, 3).demo() def t(a=int, b=int, c=DemoClass): - DemoClass(a, b).demo() + x = DemoClass(a, b) + x.demo() DemoSubclass(a, a, b).demo() DemoSubclass(a, a, b).demo(6) - DemoSubclass(a, a, b).demo(6, 'hu') + y = DemoSubclass(a, a, b).demo(6, 'hu') if isinstance(c, DemoSubclass): print 42 - + return DemoBaseNotExposed(17, 4) # see if it works without wrapper + # exposing and using classes from a generasted extension module def test_asd(): m = get_compiled_module(t, use_boehm=not True, exports=[ - DemoClass, DemoSubclass, DemoNotAnnotated, setup_new_module]) + DemoClass, DemoSubclass, DemoNotAnnotated]) obj = m.DemoClass(2, 3) res = obj.demo() assert res == DemoClass(2, 3).demo() Modified: pypy/dist/pypy/translator/c/wrapper.py ============================================================================== --- pypy/dist/pypy/translator/c/wrapper.py (original) +++ pypy/dist/pypy/translator/c/wrapper.py Sun Apr 23 03:50:26 2006 @@ -1,14 +1,17 @@ from pypy.objspace.flow.model import Variable, Constant from pypy.objspace.flow.model import Block, Link, FunctionGraph, checkgraph from pypy.rpython.lltypesystem.lltype import \ - Ptr, PyObject, typeOf, Signed, FuncType, functionptr + Ptr, PyObject, typeOf, Signed, FuncType, functionptr, nullptr, Void from pypy.rpython.rtyper import LowLevelOpList from pypy.rpython.rmodel import inputconst, PyObjPtr from pypy.rpython.robject import pyobj_repr from pypy.interpreter.pycode import CO_VARARGS from pypy.rpython.typesystem import getfunctionptr +from pypy.annotation.model import s_None, SomeInstance +from pypy.translator.backendopt.inline import simple_inline_function +ALWAYS_INLINE = False def gen_wrapper(func, translator, newname=None): """generate a wrapper function for 'func' that can be put in a @@ -21,20 +24,28 @@ # have been decoded. # get the fully typed low-level pointer to the function, if available - nb_positional_args = func.func_code.co_argcount - vararg = bool(func.func_code.co_flags & CO_VARARGS) + do_inline = ALWAYS_INLINE if translator.annotator is None: # get the graph from the translator, "push it back" so that it's # still available for further buildflowgraph() calls graph = translator.buildflowgraph(func) translator._prebuilt_graphs[func] = graph else: - bk = translator.annotator.bookkeeper - graph = bk.getdesc(func).cachedgraph(None) + if isinstance(func, FunctionGraph): + graph = func + func = graph.func + # in this case we want to inline for sure, because we + # created this extra graph with a single call-site. + do_inline = True + else: + bk = translator.annotator.bookkeeper + graph = bk.getdesc(func).cachedgraph(None) f = getfunctionptr(graph) FUNCTYPE = typeOf(f).TO + nb_positional_args = func.func_code.co_argcount + vararg = bool(func.func_code.co_flags & CO_VARARGS) assert len(FUNCTYPE.ARGS) == nb_positional_args + vararg newops = LowLevelOpList(translator.rtyper) @@ -133,6 +144,8 @@ # that need to be specialized now translator.rtyper.specialize_more_blocks() + if do_inline: + simple_inline_function(translator, graph, wgraph) return functionptr(FuncType([PyObjPtr, PyObjPtr, PyObjPtr], @@ -140,3 +153,46 @@ wgraph.name, graph = wgraph, exception_policy = "CPython") + +def new_method_graph(graph, clsdef, newname, translator): + ann = translator.annotator + rtyper = translator.rtyper + + f = getfunctionptr(graph) + FUNCTYPE = typeOf(f).TO + + newops = LowLevelOpList(translator.rtyper) + + callargs = graph.getargs()[:] + v_self_old = callargs.pop(0) + v_self = Variable(v_self_old.name) + binding = SomeInstance(clsdef) + v_self.concretetype = rtyper.getrepr(binding).lowleveltype + ann.setbinding(v_self, binding) + v_self_call = newops.convertvar(v_self, + r_from = rtyper.bindingrepr(v_self), + r_to = rtyper.bindingrepr(v_self_old)) + + vlist = [inputconst(typeOf(f), f)] + [v_self_call] + callargs + newops.genop('direct_call', vlist, resulttype=Void) + + # "return result" + funcargs = [v_self] + callargs + block = Block(funcargs) + newgraph = FunctionGraph(newname, block) + translator.update_call_graph(newgraph, graph, object()) + translator.graphs.append(newgraph) + block.operations[:] = newops + block.closeblock(Link([inputconst(Void, None)], newgraph.returnblock)) + + vres = newgraph.getreturnvar() + ann.setbinding(vres, s_None) + checkgraph(newgraph) + # pretend to be the same function, as we actually + # will become inlined. + newgraph.func = graph.func + translator.rtyper.specialize_more_blocks() + # not sure if we want this all the time? + if ALWAYS_INLINE: + simple_inline_function(translator, graph, newgraph) + return newgraph From nik at codespeak.net Sun Apr 23 05:16:38 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sun, 23 Apr 2006 05:16:38 +0200 (CEST) Subject: [pypy-svn] r26155 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060423031638.4110510074@code0.codespeak.net> Author: nik Date: Sun Apr 23 05:16:34 2006 New Revision: 26155 Added: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (contents, props changed) Log: planning for today Added: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Sun Apr 23 05:16:34 2006 @@ -0,0 +1,48 @@ +Tokyo Sprint Planning +--------------------- + +Sprint Times: 10 am to 7 pm + +Break day: Wednesday + + +Sunday, 23. April 2006: +*********************** + +Present: Samuele, Arre, Valentino, Eric, Yutaka, Sanghyeon, Yusei, Bea, Nik + +General Interests: + + - gensqueak/ootypesystem (Nik) + + - Implementing Python 2.5 features in PyPy. (Yusei) + + - Progress further on an 'rctypes' module aiming at letting us use a ctypes + implementation of an extension module from the compiled pypy-c. + - check status + - ctypes time and socket (and select ;)) + (Arre, Valentino, Eric) + + - rctypes support for LLVM (Eric) + + - convert JS backend to ootypesystem (Eric, Nik, Yusei) + + - genjs test failure fixes (Eric) + + - raisingops transformation (Eric) + + - PyPy debian packaging (Yutaka, Holger?) + + - Fun with the Common Lisp backend (Yutaka, Valentino) + + - Possibly experiment with writing modules translatable for use both + in PyPy and CPython. (Sanghyeon, Armin?) + +Pairings today: + + - ctypes time and socket (and select ;)) (Arre, Valentino, Eric, Sanghyeon) + + - ootypesystem introduction, gensqueak/gencl (Nik, Yutaka, Yusei) + + - general wizardry (Samuele) + From tismer at codespeak.net Sun Apr 23 05:25:47 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 23 Apr 2006 05:25:47 +0200 (CEST) Subject: [pypy-svn] r26156 - in pypy/dist/pypy/translator/c: test winproj/extension Message-ID: <20060423032547.4C8EF10079@code0.codespeak.net> Author: tismer Date: Sun Apr 23 05:25:43 2006 New Revision: 26156 Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Log: passing tuples containing instances in and out works. methods like __add__ work nicely from outside, although they need to be spelled directly from inside [ inst1.__add__(inst2) ] Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Sun Apr 23 05:25:43 2006 @@ -47,7 +47,7 @@ clsdef = bk.getuniqueclassdef(obj) rtyper.add_wrapper(clsdef) elif isinstance(obj, types.FunctionType): - t.annotator.build_types(obj, get_annotation(obj)) + pass#t.annotator.build_types(obj, get_annotation(obj)) if view: t.viewcg() @@ -161,6 +161,11 @@ delmonitor.notify() if P:print 'del' + def __add__(self, other): + # XXX I would like to use type(), but its support is very limited + #return type(self)(self.a + other.a, self.b + other.b) + return DemoClass(self.a + other.a, self.b + other.b) + # see if we get things exported with subclassing class DemoSubclass(DemoClass): def __init__(self, a, b, c): @@ -279,20 +284,29 @@ res = obj.demo() assert res == DemoClass(2, 3).demo() +def extfunc(inst): + return inst.demo() + +def extfunc2(tup): + inst1, inst2 = tup + return inst1.__add__(inst2) + def t(a=int, b=int, c=DemoClass): x = DemoClass(a, b) x.demo() DemoSubclass(a, a, b).demo() DemoSubclass(a, a, b).demo(6) y = DemoSubclass(a, a, b).demo(6, 'hu') + extfunc(x) + extfunc2( (x, x) ) if isinstance(c, DemoSubclass): print 42 - return DemoBaseNotExposed(17, 4) # see if it works without wrapper + return x.__add__(x), DemoBaseNotExposed(17, 4) # see if it works without wrapper # exposing and using classes from a generasted extension module def test_asd(): m = get_compiled_module(t, use_boehm=not True, exports=[ - DemoClass, DemoSubclass, DemoNotAnnotated]) + DemoClass, DemoSubclass, DemoNotAnnotated, extfunc, extfunc2]) obj = m.DemoClass(2, 3) res = obj.demo() assert res == DemoClass(2, 3).demo() Modified: pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj ============================================================================== --- pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj (original) +++ pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Sun Apr 23 05:25:43 2006 @@ -208,7 +208,7 @@ + RelativePath="..\..\..\..\..\..\..\Documents and Settings\ctismer\Local Settings\Temp\usession-2331\testing_1\testing_1.c"> From pedronis at codespeak.net Sun Apr 23 06:51:10 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 23 Apr 2006 06:51:10 +0200 (CEST) Subject: [pypy-svn] r26157 - pypy/dist/pypy/doc/image Message-ID: <20060423045110.E1DB810079@code0.codespeak.net> Author: pedronis Date: Sun Apr 23 06:51:06 2006 New Revision: 26157 Modified: pypy/dist/pypy/doc/image/arch-translation.graffle pypy/dist/pypy/doc/image/arch-translation.pdf Log: typo fixing, use object oriented typing terminology instead of confusing high-level. Modified: pypy/dist/pypy/doc/image/arch-translation.graffle ============================================================================== --- pypy/dist/pypy/doc/image/arch-translation.graffle (original) +++ pypy/dist/pypy/doc/image/arch-translation.graffle Sun Apr 23 06:51:06 2006 @@ -25,7 +25,7 @@ FitText YES ID - 251 + 18 Shape Rectangle Style @@ -61,7 +61,7 @@ Class LineGraphic ID - 250 + 17 Points {221.362, 359.809} @@ -84,7 +84,7 @@ Tail ID - 241 + 8 @@ -93,14 +93,14 @@ Head ID - 244 + 11 ID - 249 + 16 Points - {213.769, 321.845} - {159.43, 304.457} + {213.634, 321.17} + {159.738, 304.246} Style @@ -119,14 +119,14 @@ Tail ID - 241 + 8 Class LineGraphic ID - 248 + 15 Points {280.421, 370.895} @@ -149,7 +149,7 @@ Tail ID - 241 + 8 @@ -158,14 +158,14 @@ Head ID - 241 + 8 ID - 247 + 14 Points - {359.656, 369.082} - {285.738, 344.308} + {359.872, 368.983} + {285.748, 344.258} Style @@ -184,7 +184,7 @@ Tail ID - 239 + 6 @@ -193,10 +193,10 @@ Head ID - 242 + 9 ID - 246 + 13 Labels @@ -263,7 +263,7 @@ Tail ID - 244 + 11 @@ -272,15 +272,15 @@ Head ID - 244 + 11 ID - 245 + 12 Labels FixedWidth - 1.138190e+02 + 1.276204e+02 Label Align @@ -291,7 +291,7 @@ {\colortbl;\red255\green255\blue255;} \pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\ql\qnatural -\f0\fs24 \cf0 high level typing} +\f0\fs24 \cf0 object oriented typing} LabelVisible YES @@ -325,7 +325,7 @@ Points - {233.526, 192.904} + {233.527, 192.904} {135.838, 266.911} Style @@ -343,7 +343,7 @@ Tail ID - 237 + 4 @@ -352,7 +352,7 @@ Class ShapedGraphic ID - 244 + 11 Shape Cloud Style @@ -371,7 +371,7 @@ {\colortbl;\red255\green255\blue255;} \pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\qc -\f0\fs24 \cf0 higher level flow graphs} +\f0\fs24 \cf0 oo-typed flow graphs} @@ -380,10 +380,10 @@ Head ID - 242 + 9 ID - 243 + 10 Labels @@ -432,8 +432,8 @@ Points - {391.292, 408.13} - {282.587, 511.554} + {391.834, 408.009} + {282.638, 511.62} Style @@ -450,7 +450,7 @@ Tail ID - 239 + 6 @@ -459,7 +459,7 @@ Class ShapedGraphic ID - 242 + 9 Shape Hexagon Style @@ -494,7 +494,7 @@ Class ShapedGraphic ID - 241 + 8 Shape Trapazoid Text @@ -507,7 +507,7 @@ \f0\fs24 \cf0 \ Translation Aspects\ -theading\ +threading\ GC\ ...\ } @@ -519,10 +519,10 @@ Head ID - 239 + 6 ID - 240 + 7 Labels @@ -570,8 +570,8 @@ Points - {271.12, 195.121} - {396.878, 364.007} + {271.112, 195.122} + {396.877, 364.007} Style @@ -588,7 +588,7 @@ Tail ID - 237 + 4 @@ -597,7 +597,7 @@ Class ShapedGraphic ID - 239 + 6 Shape Cloud Style @@ -631,10 +631,10 @@ Head ID - 237 + 4 ID - 238 + 5 Labels @@ -682,7 +682,7 @@ Points - {382.146, 95.5429} + {382.147, 95.5427} {284.004, 157.997} Style @@ -700,7 +700,7 @@ Tail ID - 235 + 2 @@ -709,7 +709,7 @@ Class ShapedGraphic ID - 237 + 4 Shape Cloud Text @@ -729,10 +729,10 @@ Head ID - 235 + 2 ID - 236 + 3 Labels @@ -782,8 +782,8 @@ Points - {171, 77.5853} - {352.781, 80.238} + {171, 77.4621} + {352.742, 80.2099} Style @@ -800,7 +800,7 @@ Tail ID - 234 + 1 @@ -809,7 +809,7 @@ Class ShapedGraphic ID - 235 + 2 Shape Cloud Text @@ -829,7 +829,7 @@ Class ShapedGraphic ID - 234 + 1 Shape Rectangle Style @@ -982,7 +982,7 @@ bD4KCQkJCQk8L2FycmF5PgoJCQkJCTxrZXk+Y29tLmFwcGxlLnByaW50LnRpY2tldC5j bGllbnQ8L2tleT4KCQkJCQk8c3RyaW5nPmNvbS5hcHBsZS5wcmludGluZ21hbmFnZXI8 L3N0cmluZz4KCQkJCQk8a2V5PmNvbS5hcHBsZS5wcmludC50aWNrZXQubW9kRGF0ZTwv - a2V5PgoJCQkJCTxkYXRlPjIwMDYtMDItMjVUMTA6Mjk6NTJaPC9kYXRlPgoJCQkJCTxr + a2V5PgoJCQkJCTxkYXRlPjIwMDYtMDQtMjNUMDQ6NDE6NTdaPC9kYXRlPgoJCQkJCTxr ZXk+Y29tLmFwcGxlLnByaW50LnRpY2tldC5zdGF0ZUZsYWc8L2tleT4KCQkJCQk8aW50 ZWdlcj4wPC9pbnRlZ2VyPgoJCQkJPC9kaWN0PgoJCQk8L2FycmF5PgoJCTwvZGljdD4K CQk8a2V5PmNvbS5hcHBsZS5wcmludC5QYWdlRm9ybWF0LlBNQWRqdXN0ZWRQYXBlclJl @@ -997,7 +997,7 @@ ZSswMjwvcmVhbD4KCQkJCQk8L2FycmF5PgoJCQkJCTxrZXk+Y29tLmFwcGxlLnByaW50 LnRpY2tldC5jbGllbnQ8L2tleT4KCQkJCQk8c3RyaW5nPmNvbS5hcHBsZS5wcmludGlu Z21hbmFnZXI8L3N0cmluZz4KCQkJCQk8a2V5PmNvbS5hcHBsZS5wcmludC50aWNrZXQu - bW9kRGF0ZTwva2V5PgoJCQkJCTxkYXRlPjIwMDYtMDItMjVUMTA6Mjk6NTJaPC9kYXRl + bW9kRGF0ZTwva2V5PgoJCQkJCTxkYXRlPjIwMDYtMDQtMjNUMDQ6NDE6NTdaPC9kYXRl PgoJCQkJCTxrZXk+Y29tLmFwcGxlLnByaW50LnRpY2tldC5zdGF0ZUZsYWc8L2tleT4K CQkJCQk8aW50ZWdlcj4wPC9pbnRlZ2VyPgoJCQkJPC9kaWN0PgoJCQk8L2FycmF5PgoJ CTwvZGljdD4KCQk8a2V5PmNvbS5hcHBsZS5wcmludC5QYXBlckluZm8uUE1QYXBlck5h @@ -1064,14 +1064,14 @@ cml2YXRlTG9jazwva2V5PgoJPGZhbHNlLz4KCTxrZXk+Y29tLmFwcGxlLnByaW50LnRp Y2tldC50eXBlPC9rZXk+Cgk8c3RyaW5nPmNvbS5hcHBsZS5wcmludC5QYWdlRm9ybWF0 VGlja2V0PC9zdHJpbmc+CjwvZGljdD4KPC9wbGlzdD4KhpKEmZkPTlNQcmludEFsbFBh - Z2VzhpKgkoSZmQtOU1BhcGVyTmFtZYaShJmZCW5hLWxldHRlcoaShJmZFU5TSG9yaXpv - bmFsUGFnaW5hdGlvboaShKGbop4AhpKEmZkWTlNIb3Jpem9udGFsbHlDZW50ZXJlZIaS - ppKEmZkJTlNQcmludGVyhpKEhIQJTlNQcmludGVyAJSShJmZASCGhpKEmZkITlNDb3Bp - ZXOGkoShm4SEAVOiAYaShJmZD05TU2NhbGluZ0ZhY3RvcoaShKGbhIQBZqMBhpKEmZkN - TlNSaWdodE1hcmdpboaShKGbuaMkhpKEmZkOTlNCb3R0b21NYXJnaW6GkoShm7mjJIaS - hJmZDE5TTGVmdE1hcmdpboaShKGbuaMkhpKEmZkLTlNUb3BNYXJnaW6GkoShm7mjJIaS - hJmZCk5TTGFzdFBhZ2WGkoShm4SXl4J/////hpKEmZkLTlNGaXJzdFBhZ2WGkoShm7ai - AYaShJmZDU5TT3JpZW50YXRpb26GkoShm6KeAIaGhg== + Z2VzhpKgkoSZmQtOU1BhcGVyTmFtZYaShJmZBkxldHRlcoaShJmZFU5TSG9yaXpvbmFs + UGFnaW5hdGlvboaShKGbop4AhpKEmZkWTlNIb3Jpem9udGFsbHlDZW50ZXJlZIaSppKE + mZkJTlNQcmludGVyhpKEhIQJTlNQcmludGVyAJSShJmZEEhQIExhc2VySmV0IDEyMDCG + hpKEmZkITlNDb3BpZXOGkoShm4SEAVOiAYaShJmZD05TU2NhbGluZ0ZhY3RvcoaShKGb + hIQBZqMBhpKEmZkNTlNSaWdodE1hcmdpboaShKGbuaMkhpKEmZkOTlNCb3R0b21NYXJn + aW6GkoShm7mjJIaShJmZDE5TTGVmdE1hcmdpboaShKGbuaMkhpKEmZkLTlNUb3BNYXJn + aW6GkoShm7mjJIaShJmZCk5TTGFzdFBhZ2WGkoShm4SXl4J/////hpKEmZkLTlNGaXJz + dFBhZ2WGkoShm7aiAYaShJmZDU5TT3JpZW50YXRpb26GkoShm6KeAIaGhg== RowAlign 0 @@ -1082,7 +1082,7 @@ WindowInfo Frame - {{123, 265}, {555, 578}} + {{123, 254}, {555, 578}} VisibleRegion {{0, 0}, {540, 501}} Zoom Modified: pypy/dist/pypy/doc/image/arch-translation.pdf ============================================================================== Binary files. No diff available. From pedronis at codespeak.net Sun Apr 23 06:54:45 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 23 Apr 2006 06:54:45 +0200 (CEST) Subject: [pypy-svn] r26158 - in pypy/extradoc/talk/louvain-la-neuve-2006: . ui ui/default Message-ID: <20060423045445.46B5710079@code0.codespeak.net> Author: pedronis Date: Sun Apr 23 06:54:36 2006 New Revision: 26158 Added: pypy/extradoc/talk/louvain-la-neuve-2006/ pypy/extradoc/talk/louvain-la-neuve-2006/pypy-intro.txt - copied, changed from r24037, pypy/eu-tracking/talk/review1/wp4-results.txt pypy/extradoc/talk/louvain-la-neuve-2006/run_example1.py - copied unchanged from r24037, pypy/extradoc/talk/mallorca-talk-2006/run_example1.py pypy/extradoc/talk/louvain-la-neuve-2006/ui/ - copied from r24037, pypy/eu-tracking/talk/review1/ui/ pypy/extradoc/talk/louvain-la-neuve-2006/ui/default/ - copied from r26156, pypy/eu-tracking/talk/review1/ui/default/ pypy/extradoc/talk/louvain-la-neuve-2006/ui/py-web.png - copied unchanged from r26156, pypy/eu-tracking/talk/review1/ui/py-web.png pypy/extradoc/talk/louvain-la-neuve-2006/ui/py.css - copied unchanged from r26156, pypy/eu-tracking/talk/review1/ui/py.css pypy/extradoc/talk/louvain-la-neuve-2006/wp5_example.py - copied unchanged from r24037, pypy/extradoc/talk/mallorca-talk-2006/wp5_example.py Log: oops, I never checked in the talk I gave in Louvain La Neuve, to the oz people From pedronis at codespeak.net Sun Apr 23 06:56:34 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 23 Apr 2006 06:56:34 +0200 (CEST) Subject: [pypy-svn] r26159 - in pypy/extradoc/talk/tokyo: . ui Message-ID: <20060423045634.8D1E41007A@code0.codespeak.net> Author: pedronis Date: Sun Apr 23 06:56:27 2006 New Revision: 26159 Added: pypy/extradoc/talk/tokyo/ - copied from r26151, pypy/extradoc/talk/mallorca-talk-2006/ pypy/extradoc/talk/tokyo/run_example1.py - copied unchanged from r26156, pypy/extradoc/talk/mallorca-talk-2006/run_example1.py pypy/extradoc/talk/tokyo/talk.txt - copied, changed from r26156, pypy/extradoc/talk/mallorca-talk-2006/talk.txt pypy/extradoc/talk/tokyo/ui/ - copied from r26156, pypy/extradoc/talk/mallorca-talk-2006/ui/ pypy/extradoc/talk/tokyo/wp5_example.py - copied unchanged from r26156, pypy/extradoc/talk/mallorca-talk-2006/wp5_example.py Removed: pypy/extradoc/talk/tokyo/interpreter-overview.png pypy/extradoc/talk/tokyo/translation-overview.png Log: talk given in Tokyo (a derivative of the Mallorca one) From nik at codespeak.net Sun Apr 23 07:07:33 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sun, 23 Apr 2006 07:07:33 +0200 (CEST) Subject: [pypy-svn] r26160 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060423050733.8AD341007A@code0.codespeak.net> Author: nik Date: Sun Apr 23 07:07:29 2006 New Revision: 26160 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: update Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Sun Apr 23 07:07:29 2006 @@ -9,7 +9,7 @@ Sunday, 23. April 2006: *********************** -Present: Samuele, Arre, Valentino, Eric, Yutaka, Sanghyeon, Yusei, Bea, Nik +Present: Samuele, Arre, Valentino, Eric, Yutaka, Sanghyeon, Yusei, Bea, Nik, George General Interests: @@ -42,7 +42,7 @@ - ctypes time and socket (and select ;)) (Arre, Valentino, Eric, Sanghyeon) - - ootypesystem introduction, gensqueak/gencl (Nik, Yutaka, Yusei) + - ootypesystem introduction, gensqueak/gencl (Nik, Yutaka, Yusei, George) - general wizardry (Samuele) From tismer at codespeak.net Sun Apr 23 07:38:12 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 23 Apr 2006 07:38:12 +0200 (CEST) Subject: [pypy-svn] r26161 - pypy/dist/pypy/translator/c/test Message-ID: <20060423053812.4738B1007C@code0.codespeak.net> Author: tismer Date: Sun Apr 23 07:38:10 2006 New Revision: 26161 Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py Log: baby steps. can finish quite soon Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Sun Apr 23 07:38:10 2006 @@ -9,7 +9,7 @@ import sys, types -P = False # debug printing +P = not False # debug printing def get_annotation(func): argstypelist = [] @@ -32,7 +32,7 @@ global _t # allow us to view later _t = t = TranslationContext(do_imports_immediately=False) - t.buildannotator() + ann = t.buildannotator() rtyper = t.buildrtyper() bk = rtyper.annotator.bookkeeper if not exports: @@ -40,14 +40,16 @@ all = [obj.__name__ for obj in exports] exports = exports + [('__all__', all)] - t.annotator.build_types(func, get_annotation(func)) + ann.build_types(func, get_annotation(func)) for obj in exports: if isinstance(obj, type): clsdef = bk.getuniqueclassdef(obj) rtyper.add_wrapper(clsdef) elif isinstance(obj, types.FunctionType): - pass#t.annotator.build_types(obj, get_annotation(obj)) + if not ann.bookkeeper.getdesc(obj).querycallfamily(): + # not annotated, so enforce it + ann.build_types(obj, get_annotation(obj)) if view: t.viewcg() @@ -246,7 +248,7 @@ #bltn = BuiltinHelper() # less code: import __builtin__ as bltn - print bltn.list('hallo') + if P:print bltn.list('hallo') #from twisted.internet import reactor #print dir(reactor) #whow this works @@ -254,17 +256,17 @@ # above is possible, this is probably a better compromise: isinstance = bltn.isinstance for obj in allobjs: - if isinstance(obj, types.FunctionType): - funcs.append( (obj.func_name, obj) ) - print 'funcs=', funcs - print funcs[3:] + if isinstance(obj, types.BuiltinFunctionType): + funcs.append( (obj.__name__, obj) ) + if P:print 'funcs=', funcs + if P:print funcs[3:] #funcs += [2, 3, 5] # not yet stuff = bltn.range(10) - print stuff[3:] - print stuff[:3] - print stuff[3:7] - print stuff[:-1] + if P:print stuff[3:] + if P:print stuff[:3] + if P:print stuff[3:7] + if P:print stuff[:-1] funcs.sort() return m From sanxiyn at codespeak.net Sun Apr 23 08:00:12 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 23 Apr 2006 08:00:12 +0200 (CEST) Subject: [pypy-svn] r26162 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060423060012.8A41E10079@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 23 08:00:09 2006 New Revision: 26162 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (sanxiyn, dialtone, arre, ericvrp) Updated to new ctypes API to find native dynamic library Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sun Apr 23 08:00:09 2006 @@ -1,5 +1,6 @@ import os from pypy.rpython.rctypes.tool import ctypes_platform +import ctypes.util from ctypes import * includes = ('sys/types.h', @@ -52,7 +53,8 @@ SetPointerType(addrinfo_ptr, addrinfo) # functions -socketdll = cdll.load('libc.so.6') +dllname = ctypes.util.find_library('c') +socketdll = ctypes.cdll.LoadLibrary(dllname) socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] From sanxiyn at codespeak.net Sun Apr 23 08:04:04 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 23 Apr 2006 08:04:04 +0200 (CEST) Subject: [pypy-svn] r26163 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060423060404.D176510079@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 23 08:04:00 2006 New Revision: 26163 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test/ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_addr.py - copied unchanged from r26157, pypy/dist/pypy/rpython/rctypes/socketmodule/_test_addr.py Removed: pypy/dist/pypy/rpython/rctypes/socketmodule/_test_addr.py Log: (sanxiyn, dialtone, arre, ericvrp) Note for r26162: You need ctypes 0.9.9.6 or later! Create test subdirectory and move a test there From nik at codespeak.net Sun Apr 23 08:17:03 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sun, 23 Apr 2006 08:17:03 +0200 (CEST) Subject: [pypy-svn] r26164 - in pypy/dist/pypy: rpython/ootypesystem/test translator translator/test translator/tool Message-ID: <20060423061703.B36261007A@code0.codespeak.net> Author: nik Date: Sun Apr 23 08:16:55 2006 New Revision: 26164 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py pypy/dist/pypy/translator/gencl.py pypy/dist/pypy/translator/test/test_cltrans.py pypy/dist/pypy/translator/tool/buildcl.py Log: (george, yusei, niibe, nik) use the rtyper in the CL backend. make a very simple test work. skip the other tests for now. Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Sun Apr 23 08:16:55 2006 @@ -168,3 +168,17 @@ lst._ll_resize(1) res = interpret(oof, [lst], type_system='ootype') assert res == 1 + +def test_ootypeintro(): + + class A: + def method(self, number): + return number + 2 + + def oof(): + a = A() + return a.method(3) + + res = interpret(oof, [], type_system='ootype') + + Modified: pypy/dist/pypy/translator/gencl.py ============================================================================== --- pypy/dist/pypy/translator/gencl.py (original) +++ pypy/dist/pypy/translator/gencl.py Sun Apr 23 08:16:55 2006 @@ -29,6 +29,7 @@ binary_ops = { #"add": "+", + "int_add": "+", "sub": "-", "inplace_add": "+", # weird, but it works "inplace_lshift": "ash", @@ -174,9 +175,6 @@ simplify_graph(fun) self.fun = fun self.blockref = {} - self.annotate(input_arg_types) - transform_graph(self.ann, extra_passes=default_extra_passes - +[transform_slice]) def annotate(self, input_arg_types): ann = RPythonAnnotator() @@ -188,7 +186,7 @@ self.ann = annotator def get_type(self, var): - return self.ann.gettype(var) + return var.concretetype def repr_unknown(self, obj): return '#<%r>' % (obj,) Modified: pypy/dist/pypy/translator/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/test/test_cltrans.py Sun Apr 23 08:16:55 2006 @@ -45,6 +45,7 @@ py.test.skip("Common Lisp neither configured nor detected.") def test_if(self): + py.test.skip("temporarily disabled") cl_if = make_cl_func(t.if_then_else, [object, object, object]) assert cl_if(True, 50, 100) == 50 assert cl_if(False, 50, 100) == 100 @@ -54,21 +55,25 @@ assert cl_if([[]], 50, 100) == 50 def test_gcd(self): + py.test.skip("temporarily disabled") cl_gcd = make_cl_func(t.my_gcd, [int, int]) assert cl_gcd(96, 64) == 32 def test_is_perfect(self): # pun intended + py.test.skip("temporarily disabled") cl_perfect = make_cl_func(t.is_perfect_number, [int]) assert cl_perfect(24) == False assert cl_perfect(28) == True def test_bool(self): + py.test.skip("temporarily disabled") cl_bool = make_cl_func(t.my_bool, [object]) assert cl_bool(0) == False assert cl_bool(42) == True assert cl_bool(True) == True def test_contains(self): + py.test.skip("temporarily disabled") my_contains = make_cl_func(t.my_contains, [list, int]) assert my_contains([1, 2, 3], 1) assert not my_contains([1, 2, 3], 0) @@ -77,14 +82,17 @@ assert not is_one_or_two(3) def test_array(self): + py.test.skip("temporarily disabled") cl_four = make_cl_func(t.two_plus_two) assert cl_four() == 4 def test_sieve(self): + py.test.skip("temporarily disabled") cl_sieve = make_cl_func(t.sieve_of_eratosthenes) assert cl_sieve() == 1028 def test_easy(self): + py.test.skip("temporarily disabled") # These are the Pyrex tests which were easy to adopt. f1 = make_cl_func(t.simple_func, [int]) assert f1(1) == 2 @@ -101,6 +109,7 @@ assert f6(30) == 3657 def test_string(self): + py.test.skip("temporarily disabled") cl_greet = make_cl_func(t.greet, [str]) assert cl_greet("world") == "helloworld" cl_stringmaker = make_cl_func(t.nested_whiles, [int, int]) @@ -108,27 +117,38 @@ "...!...!...!...!...!") def test_for(self): + py.test.skip("temporarily disabled") cl_python = make_cl_func(t.choose_last) assert cl_python() == "python" def test_builtin(self): + py.test.skip("temporarily disabled") cl_builtinusage = make_cl_func(t.builtinusage) assert cl_builtinusage() == 4 def test_slice(self): + py.test.skip("temporarily disabled") cl_half = make_cl_func(t.half_of_n, [int]) assert cl_half(10) == 5 def test_powerset(self): + py.test.skip("temporarily disabled") cl_powerset = make_cl_func(t.powerset, [int]) result = cl_powerset(3) assert result.__class__ == Literal assert result.val == ( '#(#() #(0) #(1) #(0 1) #(2) #(0 2) #(1 2) #(0 1 2))') def test_yast(self): + py.test.skip("temporarily disabled") cl_sum = make_cl_func(t.yast, [list]) # yet another sum test assert cl_sum(range(12)) == 66 + def test_int_add(self): + def f(number): + return number + 2 + cl_add = make_cl_func(f, [int]) + assert cl_add(5) == 7 + # TODO # poor_man_range Modified: pypy/dist/pypy/translator/tool/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/tool/buildcl.py (original) +++ pypy/dist/pypy/translator/tool/buildcl.py Sun Apr 23 08:16:55 2006 @@ -1,6 +1,7 @@ import autopath from pypy.objspace.flow import FlowObjSpace +from pypy.translator.translator import TranslationContext from pypy.translator.gencl import GenCL from py.process import cmdexec @@ -37,15 +38,19 @@ return content def _make_cl_func(func, cl, path, argtypes=[]): - fun = FlowObjSpace().build_flow(func) - gen = GenCL(fun, argtypes) + t = TranslationContext() + t.buildannotator().build_types(func, argtypes) + t.buildrtyper(type_system="ootype").specialize() + graph = t.graphs[0] + + gen = GenCL(graph, argtypes) out = gen.globaldeclarations() + '\n' + gen.emitcode() i = 1 - fpath = path.join("%s.lisp" % fun.name) + fpath = path.join("%s.lisp" % graph.name) def _(*args): fpath.write(out) fp = file(str(fpath), "a") - print >>fp, "(write (", fun.name, + print >>fp, "(write (", graph.name, for arg in args: print >>fp, writelisp(gen, arg), print >>fp, "))" From ericvrp at codespeak.net Sun Apr 23 08:39:10 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sun, 23 Apr 2006 08:39:10 +0200 (CEST) Subject: [pypy-svn] r26165 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060423063910.BC39410079@code0.codespeak.net> Author: ericvrp Date: Sun Apr 23 08:39:04 2006 New Revision: 26165 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/__init__.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_addr.py Log: (sanxiyn, dialtone, arre, ericvrp) Make sure the test uses pypy's _socket Added: pypy/dist/pypy/rpython/rctypes/socketmodule/__init__.py ============================================================================== Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sun Apr 23 08:39:04 2006 @@ -54,6 +54,7 @@ # functions dllname = ctypes.util.find_library('c') +assert dllname is not None socketdll = ctypes.cdll.LoadLibrary(dllname) socket = socketdll.socket Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_addr.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_addr.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_addr.py Sun Apr 23 08:39:04 2006 @@ -1,4 +1,5 @@ -from _socket import * +import py +from pypy.rpython.rctypes.socketmodule._socket import * def test_getaddrinfo(): lst = getaddrinfo('snake.cs.uni-duesseldorf.de', None) From ericvrp at codespeak.net Sun Apr 23 08:53:21 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sun, 23 Apr 2006 08:53:21 +0200 (CEST) Subject: [pypy-svn] r26166 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060423065321.EC35310079@code0.codespeak.net> Author: ericvrp Date: Sun Apr 23 08:53:18 2006 New Revision: 26166 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: (sanxiyn, dialtone, arre, ericvrp) Updated rctypes tests to ctypes 0.9.9.6 (LoadLibary was deprecated but then resurrected in 0.9.9.6) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Sun Apr 23 08:53:18 2006 @@ -14,25 +14,16 @@ from ctypes import cdll, pythonapi, _FUNCFLAG_PYTHONAPI from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer -from ctypes import POINTER, py_object +from ctypes import POINTER, py_object, util # __________ the standard C library __________ -# LoadLibrary is deprecated in ctypes, this should be removed at some point -if "load" in dir(cdll): - cdll_load = cdll.load -else: - cdll_load = cdll.LoadLibrary - if sys.platform == 'win32': - mylib = cdll_load('msvcrt.dll') -elif sys.platform == 'linux2': - mylib = cdll_load('libc.so.6') -elif sys.platform == 'darwin': - mylib = cdll.c + mylib = cdll.LoadLibrary('msvcrt.dll') else: - py.test.skip("don't know how to load the c lib for %s" % - sys.platform) + clib = util.find_library('c') + mylib = cdll.LoadLibrary(clib) + # ____________________________________________ labs = mylib.labs From yusei at codespeak.net Sun Apr 23 09:18:02 2006 From: yusei at codespeak.net (yusei at codespeak.net) Date: Sun, 23 Apr 2006 09:18:02 +0200 (CEST) Subject: [pypy-svn] r26167 - in pypy/dist/pypy/translator: . test tool Message-ID: <20060423071802.5843D10079@code0.codespeak.net> Author: yusei Date: Sun Apr 23 09:17:16 2006 New Revision: 26167 Modified: pypy/dist/pypy/translator/gencl.py pypy/dist/pypy/translator/test/test_cltrans.py pypy/dist/pypy/translator/tool/buildcl.py Log: (nik, yusei, george, niibe) Enable the gcd test to pass. Modified: pypy/dist/pypy/translator/gencl.py ============================================================================== --- pypy/dist/pypy/translator/gencl.py (original) +++ pypy/dist/pypy/translator/gencl.py Sun Apr 23 09:17:16 2006 @@ -34,6 +34,7 @@ "inplace_add": "+", # weird, but it works "inplace_lshift": "ash", "mod": "mod", + "int_mod": "mod", "lt": "<", "le": "<=", "eq": "=", @@ -81,18 +82,19 @@ self.gen.emit_typecase(table, arg1) print "))" - def op_is_true(self): - s = self.str - result, (arg1,) = self.result, self.args - print "(setq", s(result) + def op_is_true(self, arg): + print "(setq", self.str(self.result) table = { (bool,): "%s", (int,): "(not (zerop %s))", (long,): "(not (zerop %s))", (list,): "(not (zerop (length %s)))", } - self.gen.emit_typecase(table, arg1) + self.gen.emit_typecase(table, arg) print ")" + + def op_int_is_true(self): + self.op_is_true(self.args[0]) def op_newtuple(self): s = self.str Modified: pypy/dist/pypy/translator/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/test/test_cltrans.py Sun Apr 23 09:17:16 2006 @@ -55,7 +55,6 @@ assert cl_if([[]], 50, 100) == 50 def test_gcd(self): - py.test.skip("temporarily disabled") cl_gcd = make_cl_func(t.my_gcd, [int, int]) assert cl_gcd(96, 64) == 32 Modified: pypy/dist/pypy/translator/tool/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/tool/buildcl.py (original) +++ pypy/dist/pypy/translator/tool/buildcl.py Sun Apr 23 09:17:16 2006 @@ -4,6 +4,7 @@ from pypy.translator.translator import TranslationContext from pypy.translator.gencl import GenCL from py.process import cmdexec +from pypy import conftest class Literal: def __init__(self, val): @@ -41,6 +42,10 @@ t = TranslationContext() t.buildannotator().build_types(func, argtypes) t.buildrtyper(type_system="ootype").specialize() + + if conftest.option.view: + t.view() + graph = t.graphs[0] gen = GenCL(graph, argtypes) From arigo at codespeak.net Sun Apr 23 10:40:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 10:40:36 +0200 (CEST) Subject: [pypy-svn] r26168 - in pypy/dist/pypy/rpython/rctypes: socketmodule socketmodule/test tool Message-ID: <20060423084036.6018810079@code0.codespeak.net> Author: arigo Date: Sun Apr 23 10:40:33 2006 New Revision: 26168 Added: pypy/dist/pypy/rpython/rctypes/tool/util.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/ (props changed) Log: Copy the module ctypes.util from 0.9.9.6. For now, Python 2.5 only contains ctypes 0.9.9.4. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sun Apr 23 10:40:33 2006 @@ -1,6 +1,6 @@ import os from pypy.rpython.rctypes.tool import ctypes_platform -import ctypes.util +from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 from ctypes import * includes = ('sys/types.h', @@ -53,9 +53,9 @@ SetPointerType(addrinfo_ptr, addrinfo) # functions -dllname = ctypes.util.find_library('c') +dllname = util.find_library('c') assert dllname is not None -socketdll = ctypes.cdll.LoadLibrary(dllname) +socketdll = cdll.LoadLibrary(dllname) socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] Added: pypy/dist/pypy/rpython/rctypes/tool/util.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/tool/util.py Sun Apr 23 10:40:33 2006 @@ -0,0 +1,125 @@ +""" +This is the module 'ctypes.util', copied from ctypes 0.9.9.6. +""" +import sys, os +import ctypes + +# find_library(name) returns the pathname of a library, or None. +if os.name == "nt": + def find_library(name): + # See MSDN for the REAL search order. + for directory in os.environ['PATH'].split(os.pathsep): + fname = os.path.join(directory, name) + if os.path.exists(fname): + return fname + if fname.lower().endswith(".dll"): + continue + fname = fname + ".dll" + if os.path.exists(fname): + return fname + return None + +if os.name == "ce": + # search path according to MSDN: + # - absolute path specified by filename + # - The .exe launch directory + # - the Windows directory + # - ROM dll files (where are they?) + # - OEM specified search path: HKLM\Loader\SystemPath + def find_library(name): + return name + +if os.name == "posix" and sys.platform == "darwin": + from ctypes.macholib.dyld import dyld_find as _dyld_find + def find_library(name): + possible = ['lib%s.dylib' % name, + '%s.dylib' % name, + '%s.framework/%s' % (name, name)] + for name in possible: + try: + return _dyld_find(name) + except ValueError: + continue + return None + +elif os.name == "posix": + # Andreas Degert's find functions, using gcc, /sbin/ldconfig, objdump + import re, tempfile + + def _findLib_gcc(name): + expr = '[^\(\)\s]*lib%s\.[^\(\)\s]*' % name + cmd = 'if type gcc &>/dev/null; then CC=gcc; else CC=cc; fi;' \ + '$CC -Wl,-t -o /dev/null 2>&1 -l' + name + try: + fdout, outfile = tempfile.mkstemp() + fd = os.popen(cmd) + trace = fd.read() + err = fd.close() + finally: + try: + os.unlink(outfile) + except OSError, e: + if e.errno != errno.ENOENT: + raise + res = re.search(expr, trace) + if not res: + return None + return res.group(0) + + def _findLib_ld(name): + expr = '/[^\(\)\s]*lib%s\.[^\(\)\s]*' % name + res = re.search(expr, os.popen('/sbin/ldconfig -p 2>/dev/null').read()) + if not res: + # Hm, this works only for libs needed by the python executable. + cmd = 'ldd %s 2>/dev/null' % sys.executable + res = re.search(expr, os.popen(cmd).read()) + if not res: + return None + return res.group(0) + + def _get_soname(f): + cmd = "objdump -p -j .dynamic 2>/dev/null " + f + res = re.search(r'\sSONAME\s+([^\s]+)', os.popen(cmd).read()) + if not res: + return None + return res.group(1) + + def find_library(name): + lib = _findLib_ld(name) or _findLib_gcc(name) + if not lib: + return None + return _get_soname(lib) + +################################################################ +# test code + +def test(): + from ctypes import cdll + if os.name == "nt": + print cdll.msvcrt + print cdll.load("msvcrt") + print find_library("msvcrt") + + if os.name == "posix": + # find and load_version + print find_library("m") + print find_library("c") + print find_library("bz2") + + # getattr +## print cdll.m +## print cdll.bz2 + + # load + if sys.platform == "darwin": + print cdll.LoadLibrary("libm.dylib") + print cdll.LoadLibrary("libcrypto.dylib") + print cdll.LoadLibrary("libSystem.dylib") + print cdll.LoadLibrary("System.framework/System") + else: + print cdll.LoadLibrary("libm.so") + print cdll.LoadLibrary("libcrypt.so") + print find_library("crypt") + +if __name__ == "__main__": + test() From arigo at codespeak.net Sun Apr 23 10:40:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 10:40:52 +0200 (CEST) Subject: [pypy-svn] r26169 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060423084052.866041007C@code0.codespeak.net> Author: arigo Date: Sun Apr 23 10:40:51 2006 New Revision: 26169 Modified: pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: ctypes rules for calling functions with no 'argtypes' specification. Needed for calling functions with a variable number of arguments. Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Sun Apr 23 10:40:51 2006 @@ -1,4 +1,6 @@ from pypy.annotation.model import SomeCTypesObject +from pypy.annotation import model as annmodel +from pypy.rpython.error import TyperError from pypy.rpython.extregistry import ExtRegistryEntry from pypy.rpython.lltypesystem import lltype @@ -36,11 +38,28 @@ cfuncptr = self.instance fnname = cfuncptr.__name__ + def repr_for_ctype(ctype): + s = SomeCTypesObject(ctype, SomeCTypesObject.MEMORYALIAS) + return hop.rtyper.getrepr(s) + args_r = [] - for ctype in cfuncptr.argtypes: - s_arg = SomeCTypesObject(ctype, SomeCTypesObject.MEMORYALIAS) - r_arg = hop.rtyper.getrepr(s_arg) - args_r.append(r_arg) + if getattr(cfuncptr, 'argtypes', None) is not None: + for ctype in cfuncptr.argtypes: + args_r.append(repr_for_ctype(ctype)) + else: + # unspecified argtypes: use ctypes rules for arguments + for s_arg, r_arg in zip(hop.args_s, hop.args_r): + if not isinstance(s_arg, SomeCTypesObject): + # accept integers, strings, or None + if isinstance(s_arg, annmodel.SomeInteger): + r_arg = repr_for_ctype(c_long) + elif (isinstance(s_arg, annmodel.SomeString) + or s_arg == annmodel.s_None): + r_arg = repr_for_ctype(c_char_p) + else: + raise TyperError("call with no argtypes: don't know " + "how to convert argument %r"%(s_arg,)) + args_r.append(r_arg) vlist = hop.inputargs(*args_r) unwrapped_args_v = [] Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Sun Apr 23 10:40:51 2006 @@ -14,7 +14,8 @@ from ctypes import cdll, pythonapi, _FUNCFLAG_PYTHONAPI from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer -from ctypes import POINTER, py_object, util +from ctypes import POINTER, py_object, byref +from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 # __________ the standard C library __________ @@ -49,6 +50,10 @@ time_.restype = c_long # should rather use ctypes_platform.getsimpletype() time_.argtypes = [POINTER(c_long)] +ctime = mylib.ctime +ctime.restype = c_char_p +#ctimes.argtypes: omitted for this test + def test_labs(n=6): assert labs(n) == abs(n) @@ -85,6 +90,13 @@ t3 = time.time() assert int(t1) <= t2 <= int(t3 + 1.0) +def test_ctime(): + import time + N = 99999999 + s1 = time.ctime(N) + s2 = ctime(byref(c_long(N))) + assert s1.strip() == s2.strip() + class Test_annotation: def test_annotate_labs(self): a = RPythonAnnotator() @@ -202,3 +214,14 @@ res = fn(17, 0) assert res == 34 py.test.raises(OverflowError, 'fn(sys.maxint, 1)') + + def test_compile_ctime(self): + import time + N = 123456789 + def func(n): + return ctime(byref(c_long(n))) + + fn = compile(func, [int]) + s1 = time.ctime(N) + s2 = fn(N) + assert s1.strip() == s2.strip() From sanxiyn at codespeak.net Sun Apr 23 11:24:12 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 23 Apr 2006 11:24:12 +0200 (CEST) Subject: [pypy-svn] r26170 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060423092412.8A6371007E@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 23 11:24:07 2006 New Revision: 26170 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_errno.py Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (sanxiyn, arre) Access errno from ctypes with in_dll() method and use that to raise exception. To test that we added connect(), and make sure ECONNREFUSED is raised. To make *that* work do some host-to-network conversions. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sun Apr 23 11:24:07 2006 @@ -1,10 +1,19 @@ -from ctypes import c_char_p, POINTER, byref, cast, create_string_buffer +from ctypes import c_char_p, POINTER, pointer, byref, cast, create_string_buffer, sizeof import ctypes_socket as _c globals().update(_c.constants) +class error(Exception): + pass + +def _ip_to_number(ip): + p1, p2, p3, p4 = [ int(part) for part in ip.split('.') ] + num = ((p4 * 256 + p3) * 256 + p2) * 256 + p1 + return num + + class socket(object): def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0): @@ -13,7 +22,7 @@ self.proto = proto self._fd = _c.socket(family, type, proto) if self._fd == -1: - XXX + raise error(_c.errno.value) def __del__(self): if self._fd != -1: @@ -29,17 +38,26 @@ caddr, caddrlen = self._getsockaddr(addr) res = _c.bind(self._fd, caddr, caddrlen) if res < 0: - XXX + raise error(_c.errno.value) def _getsockaddr(self, addr): if self.family == AF_INET: (host, port) = addr - caddr = sockaddr_in() + ip = host # XXX + caddr = _c.sockaddr_in() caddr.sin_family = AF_INET - caddr.sin_port = port - caddr.sin_addr.s_addr = XXX(host) + caddr.sin_port = _c.htons(port) + caddr.sin_addr.s_addr = _ip_to_number(ip) + return caddr else: - XXX + raise NotImplementedError('sorry') # XXX + + def connect(self, addr): + caddr = self._getsockaddr(addr) + paddr = cast(pointer(caddr), _c.sockaddr_ptr) + result = _c.socketconnect(self._fd, paddr, sizeof(caddr)) + if result == -1: + raise error(_c.errno.value) def makeipaddr(caddr, caddrlen): Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sun Apr 23 11:24:07 2006 @@ -34,6 +34,7 @@ [('sa_family', c_int), # unknown and variable fields follow ]) +sockaddr_ptr = POINTER(sockaddr) in_addr = ctypes_platform.getstruct('struct in_addr', HEADER, [('s_addr', c_uint)]) sockaddr_in = ctypes_platform.getstruct('struct sockaddr_in', HEADER, @@ -57,12 +58,22 @@ assert dllname is not None socketdll = cdll.LoadLibrary(dllname) +errno = c_int.in_dll(socketdll, 'errno') + +htons = socketdll.htons +htons.argtypes = [uint16_t] +htons.restype = uint16_t + socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] socket.restype = c_int socketclose = os.close +socketconnect = socketdll.connect +socketconnect.argtypes = [c_int, POINTER(sockaddr), socklen_t] +socketconnect.restype = c_int + getaddrinfo = socketdll.getaddrinfo getaddrinfo.argtypes = [c_char_p, c_char_p, POINTER(addrinfo), POINTER(POINTER(addrinfo))] Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_errno.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_errno.py Sun Apr 23 11:24:07 2006 @@ -0,0 +1,9 @@ +import errno +import py.test +from pypy.rpython.rctypes.socketmodule._socket import * + +def test_connect_error(): + s = socket() + # This should be refused + e = py.test.raises(error, s.connect, ('127.0.0.1', 1000)) + assert e.value.args[0] == errno.ECONNREFUSED From sanxiyn at codespeak.net Sun Apr 23 11:26:03 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 23 Apr 2006 11:26:03 +0200 (CEST) Subject: [pypy-svn] r26171 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060423092603.E914D1007E@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 23 11:26:01 2006 New Revision: 26171 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Log: typo Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sun Apr 23 11:26:01 2006 @@ -26,7 +26,7 @@ def __del__(self): if self._fd != -1: - _c.closesocket(self._fd) + _c.socketclose(self._fd) def close(self): fd = self._fd From arigo at codespeak.net Sun Apr 23 11:33:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 11:33:49 +0200 (CEST) Subject: [pypy-svn] r26172 - in pypy/dist/pypy: interpreter objspace/cpy objspace/cpy/test Message-ID: <20060423093349.660121007A@code0.codespeak.net> Author: arigo Date: Sun Apr 23 11:33:47 2006 New Revision: 26172 Added: pypy/dist/pypy/objspace/cpy/test/test_wrappable.py (contents, props changed) pypy/dist/pypy/objspace/cpy/wrappable.py (contents, props changed) Modified: pypy/dist/pypy/interpreter/gateway.py pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/objspace/cpy/objspace.py Log: Started work on exposing built-in functions in the CPyObjSpace. Modified: pypy/dist/pypy/interpreter/gateway.py ============================================================================== --- pypy/dist/pypy/interpreter/gateway.py (original) +++ pypy/dist/pypy/interpreter/gateway.py Sun Apr 23 11:33:47 2006 @@ -374,6 +374,8 @@ frame_cls = type("BuiltinFrame_UwS_%s" % label, (BuiltinFrame,), d) class MyBuiltinFrameFactory(BuiltinFrameFactory): + # export 'unwrap_spec' for inspection from outside gateway.py + unwrap_spec = self.unwrap_spec def create(self, space, code, w_globals): newframe = frame_cls(space, code, w_globals) Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Sun Apr 23 11:33:47 2006 @@ -32,6 +32,13 @@ Py_GT = ctypes_platform.ConstantInteger('Py_GT') Py_GE = ctypes_platform.ConstantInteger('Py_GE') + PyMethodDef = ctypes_platform.Struct('PyMethodDef', + [('ml_name', c_char_p), + ('ml_meth', c_void_p), + ('ml_flags', c_int), + ('ml_doc', c_char_p)]) + METH_VARARGS = ctypes_platform.ConstantInteger('METH_VARARGS') + globals().update(ctypes_platform.configure(CConfig)) del CConfig @@ -83,6 +90,10 @@ PyInt_FromLong.argtypes = [c_long] PyInt_FromLong.restype = W_Object +PyInt_AsLong = pythonapi.PyInt_AsLong +PyInt_AsLong.argtypes = [W_Object] +PyInt_AsLong.restype = c_long + ################################################### # ____________________ Strings ____________________ @@ -134,3 +145,20 @@ PyImport_ImportModule = pythonapi.PyImport_ImportModule PyImport_ImportModule.argtypes = [c_char_p] PyImport_ImportModule.restype = W_Object + + +############################################################## +# ____________________ Built-in functions ____________________ + +PyArg_ParseTuple = pythonapi.PyArg_ParseTuple +PyArg_ParseTuple.restype = c_int +#PyArg_ParseTuple.argtypes = [W_Object, c_char_p, ...] + +PyArg_ParseTupleAndKeywords = pythonapi.PyArg_ParseTupleAndKeywords +PyArg_ParseTupleAndKeywords.restype = c_int +#PyArg_ParseTupleAndKeywords.argtypes = [W_Object, W_Object, +# c_char_p, POINTER(c_char_p), ...] + +PyCFunction_NewEx = pythonapi.PyCFunction_NewEx +PyCFunction_NewEx.argtypes = [POINTER(PyMethodDef), W_Object, W_Object] +PyCFunction_NewEx.restype = W_Object Modified: pypy/dist/pypy/objspace/cpy/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/objspace.py Sun Apr 23 11:33:47 2006 @@ -1,4 +1,5 @@ from pypy.objspace.cpy.capi import * +from pypy.annotation.pairtype import pair from pypy.interpreter import baseobjspace @@ -6,11 +7,12 @@ from pypy.objspace.cpy.capi import W_Object def __init__(self): - #self.fromcache = baseobjspace.InternalSpaceCache(self).getorbuild + self.fromcache = baseobjspace.InternalSpaceCache(self).getorbuild self.w_int = W_Object(int) self.w_None = W_Object(None) self.w_False = W_Object(False) self.w_True = W_Object(True) + self.wrap_cache = {} def enter_cache_building_mode(self): pass @@ -22,6 +24,16 @@ return PyImport_ImportModule(name) def wrap(self, x): + if isinstance(x, baseobjspace.Wrappable): + x = x.__spacebind__(self) + if isinstance(x, baseobjspace.Wrappable): + try: + return self.wrap_cache[x] + except KeyError: + import pypy.objspace.cpy.wrappable + result = pair(self, x).wrap() + self.wrap_cache[x] = result + return result if x is None: return self.w_None if isinstance(x, int): @@ -31,9 +43,14 @@ raise TypeError("wrap(%r)" % (x,)) wrap._annspecialcase_ = "specialize:wrap" + def unwrap(self, w_obj): + assert isinstance(w_obj, W_Object) + return w_obj.value + getattr = staticmethod(PyObject_GetAttr) getitem = staticmethod(PyObject_GetItem) setitem = staticmethod(PyObject_SetItem) + int_w = staticmethod(PyInt_AsLong) def call_function(self, w_callable, *args_w): w_args = self.newtuple(list(args_w)) # XXX not very efficient Added: pypy/dist/pypy/objspace/cpy/test/test_wrappable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/test/test_wrappable.py Sun Apr 23 11:33:47 2006 @@ -0,0 +1,19 @@ +from pypy.objspace.cpy.objspace import CPyObjSpace +from pypy.interpreter.function import BuiltinFunction +from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root + + +def test_builtin_function(): + def entrypoint(space, w_x): + x = space.int_w(w_x) + result = x * 7 + return space.wrap(result) + entrypoint.unwrap_spec = [ObjSpace, W_Root] + + space = CPyObjSpace() + func = interp2app(entrypoint).__spacebind__(space) + bltin = BuiltinFunction(func) + w_entrypoint = space.wrap(bltin) + w_result = space.call_function(w_entrypoint, space.wrap(-2)) + result = space.int_w(w_result) + assert result == -14 Added: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Sun Apr 23 11:33:47 2006 @@ -0,0 +1,32 @@ +""" +Support to turn interpreter objects (subclasses of Wrappable) +into CPython objects (subclasses of W_Object). +""" + +import py +from pypy.annotation.pairtype import pair, pairtype +from pypy.objspace.cpy.capi import * +from pypy.objspace.cpy.objspace import CPyObjSpace +from pypy.interpreter.function import BuiltinFunction +from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root + + +class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): + + def wrap((space, func)): + # make a built-in function + assert isinstance(func.code, BuiltinCode) + factory = func.code.framefactory + bltin = factory.behavior + unwrap_spec = factory.unwrap_spec + + assert unwrap_spec[0] == ObjSpace + for spec in unwrap_spec[1:]: + assert spec == W_Root # XXX + + def trampoline(*args): + args_w = [space.wrap(a) for a in args] + w_result = bltin(space, *args_w) + return space.unwrap(w_result) + + return W_Object(trampoline) From ericvrp at codespeak.net Sun Apr 23 11:41:40 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sun, 23 Apr 2006 11:41:40 +0200 (CEST) Subject: [pypy-svn] r26173 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060423094140.033441007C@code0.codespeak.net> Author: ericvrp Date: Sun Apr 23 11:41:36 2006 New Revision: 26173 Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Log: (ericvrp, dialtone) Added stubs for missing implemented methods to match _socket.socket interface Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sun Apr 23 11:41:36 2006 @@ -51,6 +51,69 @@ return caddr else: raise NotImplementedError('sorry') # XXX + + def listen(self, backlog): + if self._fd != -1: + fd = self._fd + res = _c.listen(fd, backlog) + if res == -1: + XXX + else: + XXX + + def accept(self): + pass + + def connect_ex(self): + pass + + def dup(self): + pass + + def fileno(self): + pass + + def getpeername(self): + pass + + def getsockname(self): + pass + + def getsockopt(self): + pass + + def gettimeout(self): + pass + + def makefile(self): + pass + + def recv(self): + pass + + def recvfrom(self): + pass + + def send(self): + pass + + def sendall(self): + pass + + def sendto(self): + pass + + def setblocking(self): + pass + + def setsockopt(self): + pass + + def settimeout(self): + pass + + def shutdown(self): + pass def connect(self, addr): caddr = self._getsockaddr(addr) @@ -59,7 +122,6 @@ if result == -1: raise error(_c.errno.value) - def makeipaddr(caddr, caddrlen): buf = create_string_buffer(NI_MAXHOST) error = _c.getnameinfo(caddr, caddrlen, buf, NI_MAXHOST, Added: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Sun Apr 23 11:41:36 2006 @@ -0,0 +1,15 @@ +import py +import _socket +from pypy.rpython.rctypes.socketmodule import _socket as _rsocket + +def interface_matcher(interface1, interface2): + members = [member for member in dir(interface1) if not member.startswith('_')] + verifying_set = dir(interface2) + for member in members: + assert member in verifying_set + +def test_interfaces(): + #interface_matcher(_socket, _rsocket) + #interface_matcher(_rsocket, _socket) + interface_matcher(_socket.socket, _rsocket.socket) + interface_matcher(_rsocket.socket, _socket.socket) From ac at codespeak.net Sun Apr 23 11:54:55 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sun, 23 Apr 2006 11:54:55 +0200 (CEST) Subject: [pypy-svn] r26174 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060423095455.9273D1007E@code0.codespeak.net> Author: ac Date: Sun Apr 23 11:54:55 2006 New Revision: 26174 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (arre, sanxiyn) Fix endianness problem. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sun Apr 23 11:54:55 2006 @@ -10,7 +10,7 @@ def _ip_to_number(ip): p1, p2, p3, p4 = [ int(part) for part in ip.split('.') ] - num = ((p4 * 256 + p3) * 256 + p2) * 256 + p1 + num = ((p1 * 256 + p2) * 256 + p3) * 256 + p4 return num @@ -44,10 +44,11 @@ if self.family == AF_INET: (host, port) = addr ip = host # XXX + num = _ip_to_number(ip) caddr = _c.sockaddr_in() caddr.sin_family = AF_INET caddr.sin_port = _c.htons(port) - caddr.sin_addr.s_addr = _ip_to_number(ip) + caddr.sin_addr.s_addr = _c.htonl(num) return caddr else: raise NotImplementedError('sorry') # XXX Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sun Apr 23 11:54:55 2006 @@ -60,10 +60,6 @@ errno = c_int.in_dll(socketdll, 'errno') -htons = socketdll.htons -htons.argtypes = [uint16_t] -htons.restype = uint16_t - socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] socket.restype = c_int From arigo at codespeak.net Sun Apr 23 12:10:12 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 12:10:12 +0200 (CEST) Subject: [pypy-svn] r26175 - in pypy/dist/pypy/rpython: . rctypes/test Message-ID: <20060423101012.864821007E@code0.codespeak.net> Author: arigo Date: Sun Apr 23 12:10:11 2006 New Revision: 26175 Modified: pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rctypes/test/test_rarray.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: Some hacks to allow *arg calls to rtyped built-in functions. Not very clean. Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Sun Apr 23 12:10:11 2006 @@ -86,7 +86,27 @@ from pypy.interpreter.argument import Arguments arguments = Arguments.fromshape(None, hop.args_s[1].const, # shape range(hop.nb_args-2)) - args_s, kwds = arguments.unpack() + if arguments.w_starstararg is not None: + raise TyperError("**kwds call not implemented") + if arguments.w_stararg is not None: + # expand the *arg in-place -- it must be a tuple + from pypy.rpython.rtuple import AbstractTupleRepr + if arguments.w_stararg != hop.nb_args - 3: + raise TyperError("call pattern too complex") + hop.nb_args -= 1 + v_tuple = hop.args_v.pop() + s_tuple = hop.args_s.pop() + r_tuple = hop.args_r.pop() + if not isinstance(r_tuple, AbstractTupleRepr): + raise TyperError("*arg must be a tuple") + for i in range(len(r_tuple.items_r)): + v_item = r_tuple.getitem(hop.llops, v_tuple, i) + hop.nb_args += 1 + hop.args_v.append(v_item) + hop.args_s.append(s_tuple.items[i]) + hop.args_r.append(r_tuple.items_r[i]) + + kwds = arguments.kwds_w or {} # prefix keyword arguments with 'i_' kwds_i = {} for key, index in kwds.items(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rarray.py Sun Apr 23 12:10:11 2006 @@ -268,6 +268,18 @@ assert res.c_data[3] == 0 assert res.c_data[4] == 0 + def test_specialize_constructor_stararg(self): + A = c_int * 5 + def func(x, y): + args = (x, y) + return A(*args) + res = interpret(func, [123, 456]) + assert res.c_data[0] == 123 + assert res.c_data[1] == 456 + assert res.c_data[2] == 0 + assert res.c_data[3] == 0 + assert res.c_data[4] == 0 + class Test_compilation: def test_compile_array_access(self): def access_array(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Sun Apr 23 12:10:11 2006 @@ -225,3 +225,15 @@ s1 = time.ctime(N) s2 = fn(N) assert s1.strip() == s2.strip() + + def test_compile_ctime_vararg(self): + import time + N = 101010101 + def func(n): + args = (byref(c_long(n)),) + return ctime(*args) + + fn = compile(func, [int]) + s1 = time.ctime(N) + s2 = fn(N) + assert s1.strip() == s2.strip() From arigo at codespeak.net Sun Apr 23 12:11:45 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 12:11:45 +0200 (CEST) Subject: [pypy-svn] r26176 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060423101145.CBF2C10081@code0.codespeak.net> Author: arigo Date: Sun Apr 23 12:11:44 2006 New Revision: 26176 Modified: pypy/dist/pypy/rpython/rctypes/afunc.py Log: Typos. Of course, it's untested code :-/ Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Sun Apr 23 12:11:44 2006 @@ -52,10 +52,10 @@ if not isinstance(s_arg, SomeCTypesObject): # accept integers, strings, or None if isinstance(s_arg, annmodel.SomeInteger): - r_arg = repr_for_ctype(c_long) + r_arg = repr_for_ctype(ctypes.c_long) elif (isinstance(s_arg, annmodel.SomeString) or s_arg == annmodel.s_None): - r_arg = repr_for_ctype(c_char_p) + r_arg = repr_for_ctype(ctypes.c_char_p) else: raise TyperError("call with no argtypes: don't know " "how to convert argument %r"%(s_arg,)) From arigo at codespeak.net Sun Apr 23 12:18:34 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 12:18:34 +0200 (CEST) Subject: [pypy-svn] r26177 - in pypy/dist/pypy: objspace/cpy objspace/cpy/test translator/c/test translator/goal Message-ID: <20060423101834.D07D71007E@code0.codespeak.net> Author: arigo Date: Sun Apr 23 12:18:32 2006 New Revision: 26177 Added: pypy/dist/pypy/objspace/cpy/test/test_compile.py (contents, props changed) Modified: pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/objspace/cpy/objspace.py pypy/dist/pypy/objspace/cpy/test/test_objspace.py pypy/dist/pypy/translator/c/test/test_genc.py pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Copied the current targetdemomodule as tests. A more efficient version of CPyObjSpace.call_function(). Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Sun Apr 23 12:18:32 2006 @@ -62,6 +62,10 @@ PyObject_Call.argtypes = [W_Object, W_Object, W_Object] PyObject_Call.restype = W_Object +PyObject_CallFunctionObjArgs = pythonapi.PyObject_CallFunctionObjArgs +PyObject_CallFunctionObjArgs.restype = W_Object +#PyObject_CallFunctionObjArgs.argtypes = [W_Object, ..., final NULL] + PyObject_RichCompare = pythonapi.PyObject_RichCompare PyObject_RichCompare.argtypes = [W_Object, W_Object, c_int] PyObject_RichCompare.restype = W_Object Modified: pypy/dist/pypy/objspace/cpy/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/objspace.py Sun Apr 23 12:18:32 2006 @@ -53,8 +53,8 @@ int_w = staticmethod(PyInt_AsLong) def call_function(self, w_callable, *args_w): - w_args = self.newtuple(list(args_w)) # XXX not very efficient - return PyObject_Call(w_callable, w_args, PyDict_New()) + args_w += (None,) + return PyObject_CallFunctionObjArgs(w_callable, *args_w) def _freeze_(self): return True Added: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Sun Apr 23 12:18:32 2006 @@ -0,0 +1,18 @@ +from pypy.translator.c.test.test_genc import compile +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy +from pypy.objspace.cpy.objspace import CPyObjSpace +import pypy.rpython.rctypes.implementation + + +def test_demo(): + from pypy.module._demo import demo + space = CPyObjSpace() + + def entry_point(n, w_callable): + return demo.measuretime(space, n, w_callable) + + fn = compile(entry_point, [int, CPyObjSpace.W_Object], + annotatorpolicy = PyPyAnnotatorPolicy()) + + res = fn(10, long) + assert isinstance(res, int) Modified: pypy/dist/pypy/objspace/cpy/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_objspace.py Sun Apr 23 12:18:32 2006 @@ -8,3 +8,10 @@ space.setitem(d,wk1,wone) wback = space.getitem(d,wk1) assert space.eq_w(wback,wone) + +def test_demo(): + from pypy.module._demo import demo + space = CPyObjSpace() + w_time = demo.measuretime(space, 10, CPyObjSpace.W_Object(int)) + assert isinstance(w_time, CPyObjSpace.W_Object) + assert isinstance(w_time.value, int) Modified: pypy/dist/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_genc.py (original) +++ pypy/dist/pypy/translator/c/test/test_genc.py Sun Apr 23 12:18:32 2006 @@ -30,9 +30,10 @@ libraries = db.gcpolicy.gc_libraries()) return m -def compile(fn, argtypes, view=False, gcpolicy=None, backendopt=True): +def compile(fn, argtypes, view=False, gcpolicy=None, backendopt=True, + annotatorpolicy=None): t = TranslationContext() - a = t.buildannotator() + a = t.buildannotator(policy=annotatorpolicy) a.build_types(fn, argtypes) t.buildrtyper().specialize() if backendopt: Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Sun Apr 23 12:18:32 2006 @@ -22,4 +22,4 @@ else: N = int(sys.argv[1]) print 'Timing for %d iterations...' % N - print entry_point(N, int), 'seconds' + print entry_point(N, space.W_Object(int)), 'seconds' From ac at codespeak.net Sun Apr 23 12:39:20 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sun, 23 Apr 2006 12:39:20 +0200 (CEST) Subject: [pypy-svn] r26178 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060423103920.CADBE10087@code0.codespeak.net> Author: ac Date: Sun Apr 23 12:39:20 2006 New Revision: 26178 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (arre, sanxiyn) Better ip-number parsing (hopefully fixing byte-order problems). Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sun Apr 23 12:39:20 2006 @@ -8,11 +8,6 @@ class error(Exception): pass -def _ip_to_number(ip): - p1, p2, p3, p4 = [ int(part) for part in ip.split('.') ] - num = ((p1 * 256 + p2) * 256 + p3) * 256 + p4 - return num - class socket(object): @@ -44,11 +39,10 @@ if self.family == AF_INET: (host, port) = addr ip = host # XXX - num = _ip_to_number(ip) caddr = _c.sockaddr_in() caddr.sin_family = AF_INET caddr.sin_port = _c.htons(port) - caddr.sin_addr.s_addr = _c.htonl(num) + _c.inet_aton(ip, pointer(caddr.sin_addr)) return caddr else: raise NotImplementedError('sorry') # XXX Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sun Apr 23 12:39:20 2006 @@ -100,3 +100,7 @@ ntohs = socketdll.htonl ntohs.argtypes = [uint16_t] ntohs.restype = uint16_t + +inet_aton = socketdll.inet_aton +inet_aton.argtypes = [c_char_p, POINTER(in_addr)] +inet_aton.restype = c_int From arigo at codespeak.net Sun Apr 23 13:41:22 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 23 Apr 2006 13:41:22 +0200 (CEST) Subject: [pypy-svn] r26179 - pypy/extradoc/minute Message-ID: <20060423114122.5E09E1009A@code0.codespeak.net> Author: arigo Date: Sun Apr 23 13:41:21 2006 New Revision: 26179 Modified: pypy/extradoc/minute/pypy-sync-2006-04-20.txt Log: Fix the minutes. Modified: pypy/extradoc/minute/pypy-sync-2006-04-20.txt ============================================================================== --- pypy/extradoc/minute/pypy-sync-2006-04-20.txt (original) +++ pypy/extradoc/minute/pypy-sync-2006-04-20.txt Sun Apr 23 13:41:21 2006 @@ -29,7 +29,7 @@ distribution of tasks until and after Iceland. This we agreed to do at a specific meeting for EU developers on - TUESDAY, 23rd April, 5pm (UTC+2) + TUESDAY, 25th April, 5pm (UTC+2) where Christian, Michael, Samuele, Holger, Carl, Aurelien, Anders L., and Gromit agreed to attend already. From stephan at codespeak.net Sun Apr 23 13:52:49 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Sun, 23 Apr 2006 13:52:49 +0200 (CEST) Subject: [pypy-svn] r26180 - in pypy/dist/pypy: objspace/std translator/goal Message-ID: <20060423115249.CD7781009A@code0.codespeak.net> Author: stephan Date: Sun Apr 23 13:52:47 2006 New Revision: 26180 Added: pypy/dist/pypy/translator/goal/targetrdicttest2.py Modified: pypy/dist/pypy/objspace/std/setobject.py pypy/dist/pypy/translator/goal/targetrdicttest.py Log: current set implementation (enable model.WITHSET if using). This version shows different behaviour for inplace_update in py.py and pypy-c. targetrdicttest2 shows that the problem seems to be NOT related to rdict.update behaviour. Modified: pypy/dist/pypy/objspace/std/setobject.py ============================================================================== --- pypy/dist/pypy/objspace/std/setobject.py (original) +++ pypy/dist/pypy/objspace/std/setobject.py Sun Apr 23 13:52:47 2006 @@ -454,7 +454,7 @@ return space.w_None def inplace_and__Set_Set(space, w_left, w_other): - set_intersection_update__Set_ANY(space, w_left, w_other) + set_intersection_update__Set_Set(space, w_left, w_other) return w_left inplace_and__Set_Frozenset = inplace_and__Set_Set Modified: pypy/dist/pypy/translator/goal/targetrdicttest.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetrdicttest.py (original) +++ pypy/dist/pypy/translator/goal/targetrdicttest.py Sun Apr 23 13:52:47 2006 @@ -28,30 +28,27 @@ os.write(1,'test 1: hashing an unhashable object\n') try: myhash(uhobj) - except KeyError: - os.write(1,'\tKeyError\n') - except TypeError: - os.write(1,'\tTypeError\n') + except KeyError,e: + os.write(1,'\tKeyError\t') + os.write(1,str(e) + '\n') + except TypeError,e: + os.write(1,'\tTypeError\t') + os.write(1,str(e) + '\n') else: os.write(1,'\tno exception\n') os.write(1,'test 2: getitem with unhashable key:\n') try: mydict[uhobj] - except KeyError: - os.write(1,'\tKeyError\n') - except TypeError: - os.write(1,'\tTypeError\n') + except KeyError,e: + os.write(1,'\tKeyError\t') + os.write(1,str(e) + '\n') + except TypeError,e: + os.write(1,'\tTypeError\t') + os.write(1,str(e) + '\n') else: os.write(1,'\tno exception\n') - os.write(1,'test 3: getitem with unhashable key: (and catching general Exception)\n') - try: - mydict[uhobj] - except Exception, e: - os.write(1,'\t' + str(e) + '\n') - else: - os.write(1,'\tno exception\n') # The Following can't be translated (for reasons I don't understand: # it has to do with the assignment. # Added: pypy/dist/pypy/translator/goal/targetrdicttest2.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetrdicttest2.py Sun Apr 23 13:52:47 2006 @@ -0,0 +1,47 @@ +from pypy.rpython.objectmodel import r_dict +import os, sys +import operator + +# __________ Entry point __________ + +def eq(arg1, arg2): + return arg1 == arg2 + +def myhash(arg): + return arg.__hash__() + +class number(object): + def __init__(self,arg): + self.num = arg + def __hash__(self): + return self.num + + def __eq__(self,other): + return self.num == other.num + +def entry_point(argv): + dict_one = r_dict(eq,myhash) + dict_two = r_dict(eq,myhash) + os.write(1,'test for rdict.update\n') + one = number(1) + two = number(2) + three = number(3) + four = number(4) + + dict_one[one] = None + dict_one[four] = None + + dict_two[two] = None + dict_two[three] = None + + dict_one.update(dict_two) + for key in dict_one.iterkeys(): + os.write(1,'found %s\n' % key.num) + + return 0 + +# _____ Define and setup target ___ + +def target(*args): + return entry_point, None + From antocuni at codespeak.net Sun Apr 23 20:38:50 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sun, 23 Apr 2006 20:38:50 +0200 (CEST) Subject: [pypy-svn] r26201 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060423183850.533A4100B7@code0.codespeak.net> Author: antocuni Date: Sun Apr 23 20:38:44 2006 New Revision: 26201 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py Log: Added ll_newlist to rlist.py Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sun Apr 23 20:38:44 2006 @@ -257,9 +257,8 @@ # this is the equivalent of the lltypesystem ll_newlist that is # marked as typeMethod. def ll_newlist(self, length): - lst = new(self) - lst._ll_resize_ge(length) - return lst + from pypy.rpython.ootypesystem import rlist + return rlist.ll_newlist(self, length) # NB: We are expecting Lists of the same ITEMTYPE to compare/hash # equal. We don't redefine __eq__/__hash__ since the implementations Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Sun Apr 23 20:38:44 2006 @@ -75,12 +75,6 @@ def newlist(llops, r_list, items_v): c_list = inputconst(ootype.Void, r_list.lowleveltype) v_result = llops.genop("new", [c_list], resulttype=r_list.lowleveltype) -## c_append = inputconst(ootype.Void, "append") -## # This is very inefficient for a large amount of initial items ... -## for v_item in items_v: -## llops.genop("oosend", [c_append, v_result, v_item], -## resulttype=ootype.Void) - c_resize = inputconst(ootype.Void, "_ll_resize") c_length = inputconst(ootype.Signed, len(items_v)) llops.genop("oosend", [c_resize, v_result, c_length], resulttype=ootype.Void) @@ -91,6 +85,10 @@ llops.genop("oosend", [c_setitem, v_result, ci, v_item], resulttype=ootype.Void) return v_result +def ll_newlist(LIST, length): + lst = ootype.new(LIST) + lst._ll_resize(length) + return lst # ____________________________________________________________ From sanxiyn at codespeak.net Mon Apr 24 02:02:43 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 02:02:43 +0200 (CEST) Subject: [pypy-svn] r26205 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424000243.B8CFE10081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 02:02:39 2006 New Revision: 26205 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Log: another typo Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 02:02:39 2006 @@ -27,7 +27,7 @@ fd = self._fd if fd != -1: self._fd = -1 - _c.closesocket(fd) + _c.socketclose(fd) def bind(self, addr): caddr, caddrlen = self._getsockaddr(addr) From ericvrp at codespeak.net Mon Apr 24 03:42:59 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Mon, 24 Apr 2006 03:42:59 +0200 (CEST) Subject: [pypy-svn] r26207 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060424014259.A952910087@code0.codespeak.net> Author: ericvrp Date: Mon Apr 24 03:42:50 2006 New Revision: 26207 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Result of todays status/planning meeting Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Mon Apr 24 03:42:50 2006 @@ -46,3 +46,41 @@ - general wizardry (Samuele) +Monday, 24th April 2006: +*********************** + +Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Bea, Nik, Anders, Daiki + +Status after day 1: + + - gensqueak/ootypesystem (Nik) + * introduction done + * steps towards making genclisp use current infrastructure + + - Implementing Python 2.5 features in PyPy. (Yusei) + + - Progress further on an 'rctypes' module aiming at letting us use a ctypes + implementation of an extension module from the compiled pypy-c. + - check status (DONE) + - ctypes time and socket (and select ;)) + * added errno functionality + * refactor to latest ctypes version + * wrote test for finding missing socket methods+added stubs + (Arre, Valentino, Eric) + +Pairings today: + + - ctypes time and socket (and select ;)) + (Arre, Valentino, Eric, Sanghyeon) + * Anders will join arre, Valentino and Eric + + - ootypesystem introduction, gensqueak/gencl + (Nik, Yutaka, Yusei, George) + * more steps towards making genclisp use current infrastructure + (Nik, Sanghyeon) + * during sprint some ootype issues will be resolved by + Samuele and Nik (tuples list...) + + - general wizardry (Samuele) + * bit running around and working with Nik in the afternoon + on ootype From dialtone at codespeak.net Mon Apr 24 03:53:01 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 03:53:01 +0200 (CEST) Subject: [pypy-svn] r26208 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424015301.A6A2D10081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 03:52:58 2006 New Revision: 26208 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (ericvrp, dialtone) added accept support to ctypes_socket.py Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 03:52:58 2006 @@ -104,3 +104,7 @@ inet_aton = socketdll.inet_aton inet_aton.argtypes = [c_char_p, POINTER(in_addr)] inet_aton.restype = c_int + +socketaccept = socketdll.accept +socketaccept.argtypes = [c_int, POINTER(sockaddr), socklen_t] +socketaccept.restype = c_int From dialtone at codespeak.net Mon Apr 24 03:56:14 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 03:56:14 +0200 (CEST) Subject: [pypy-svn] r26209 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424015614.C9EAD10081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 03:56:11 2006 New Revision: 26209 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (ericvrp, dialtone) socklen_t is actually a pointer Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 03:56:11 2006 @@ -106,5 +106,5 @@ inet_aton.restype = c_int socketaccept = socketdll.accept -socketaccept.argtypes = [c_int, POINTER(sockaddr), socklen_t] +socketaccept.argtypes = [c_int, POINTER(sockaddr), POINTER(socklen_t)] socketaccept.restype = c_int From sanxiyn at codespeak.net Mon Apr 24 04:35:14 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 04:35:14 +0200 (CEST) Subject: [pypy-svn] r26210 - in pypy/dist/pypy/translator: . cl cl/test test tool Message-ID: <20060424023514.618B310081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 04:35:02 2006 New Revision: 26210 Added: pypy/dist/pypy/translator/cl/ pypy/dist/pypy/translator/cl/__init__.py pypy/dist/pypy/translator/cl/buildcl.py - copied, changed from r26205, pypy/dist/pypy/translator/tool/buildcl.py pypy/dist/pypy/translator/cl/gencl.py - copied unchanged from r26205, pypy/dist/pypy/translator/gencl.py pypy/dist/pypy/translator/cl/test/ pypy/dist/pypy/translator/cl/test/autopath.py - copied unchanged from r26205, pypy/dist/pypy/translator/autopath.py pypy/dist/pypy/translator/cl/test/test_cltrans.py - copied, changed from r26205, pypy/dist/pypy/translator/test/test_cltrans.py Removed: pypy/dist/pypy/translator/gencl.py pypy/dist/pypy/translator/test/test_cltrans.py pypy/dist/pypy/translator/tool/buildcl.py Log: (sanxiyn, nik, yusei) Move Common Lisp backend to its own directory Added: pypy/dist/pypy/translator/cl/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/__init__.py Mon Apr 24 04:35:02 2006 @@ -0,0 +1 @@ +# empty From sanxiyn at codespeak.net Mon Apr 24 04:38:22 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 04:38:22 +0200 (CEST) Subject: [pypy-svn] r26211 - in pypy/dist/pypy/translator: cl tool Message-ID: <20060424023822.CA30E10084@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 04:38:16 2006 New Revision: 26211 Added: pypy/dist/pypy/translator/cl/cmuclinvoke.sh - copied unchanged from r26209, pypy/dist/pypy/translator/tool/cmuclinvoke.sh pypy/dist/pypy/translator/cl/sbclinvoke.sh - copied unchanged from r26209, pypy/dist/pypy/translator/tool/sbclinvoke.sh Removed: pypy/dist/pypy/translator/tool/cmuclinvoke.sh pypy/dist/pypy/translator/tool/sbclinvoke.sh Log: Move more From ac at codespeak.net Mon Apr 24 04:41:24 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 04:41:24 +0200 (CEST) Subject: [pypy-svn] r26212 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060424024124.D80C410081@code0.codespeak.net> Author: ac Date: Mon Apr 24 04:41:24 2006 New Revision: 26212 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Log: (aleale, arre) Implement accept. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 04:41:24 2006 @@ -11,14 +11,17 @@ class socket(object): - def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0): + def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _fd=None): self.family = family self.type = type self.proto = proto - self._fd = _c.socket(family, type, proto) - if self._fd == -1: - raise error(_c.errno.value) - + if _fd is None: + self._fd = _c.socket(family, type, proto) + if self._fd == -1: + raise error(_c.errno.value) + else: + self._fd = _fd + def __del__(self): if self._fd != -1: _c.socketclose(self._fd) @@ -45,20 +48,34 @@ _c.inet_aton(ip, pointer(caddr.sin_addr)) return caddr else: - raise NotImplementedError('sorry') # XXX - + raise NotImplementedError('Unsupported address family') # XXX + + def _convert_from_caddr(self, caddr): + family = caddr.sa_family + if family == AF_INET: + caddr = cast(pointer(caddr), POINTER(_c.sockaddr_in)).contents + return (_c.inet_ntoa(caddr.sin_addr), _c.ntohs(caddr.sin_port)) + else: + raise NotImplementedError('Unsupported address family') # XXX + def listen(self, backlog): if self._fd != -1: fd = self._fd res = _c.listen(fd, backlog) if res == -1: - XXX + raise error(_c.errno.value) else: XXX def accept(self): - pass - + peeraddr = _c.sockaddr() + + newfd = _c.socketaccept(self._fd, pointer(peeraddr), sizeof(peeraddr)) + if newfd < 0: + raise error(_c.errno.value) + newsocket = socket(self.family, self.type, self.proto, newfd) + return (newsocket, self._convert_from_caddr(peeraddr)) + def connect_ex(self): pass Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Mon Apr 24 04:41:24 2006 @@ -1,5 +1,5 @@ import py -import _socket +import _socket, errno from pypy.rpython.rctypes.socketmodule import _socket as _rsocket def interface_matcher(interface1, interface2): @@ -13,3 +13,10 @@ #interface_matcher(_rsocket, _socket) interface_matcher(_socket.socket, _rsocket.socket) interface_matcher(_rsocket.socket, _socket.socket) + +def test_accept(): + s = _rsocket.socket(_rsocket.AF_INET, _rsocket.SOCK_STREAM, + _socket.IPPROTO_TCP) + e = py.test.raises(_rsocket.error, s.accept).value + assert e.args[0] == errno.EINVAL + From dialtone at codespeak.net Mon Apr 24 04:44:06 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 04:44:06 +0200 (CEST) Subject: [pypy-svn] r26213 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424024406.CCDF910081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 04:44:03 2006 New Revision: 26213 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (ericvrp, dialtone) half way of completing ctypes_socket.py currently wrapped: accept, dup, fileno, getpeername, getsockname, getsockopt, setsockopt, recv, recvfrom Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 04:44:03 2006 @@ -6,6 +6,8 @@ includes = ('sys/types.h', 'sys/socket.h', 'netinet/in.h', + 'unistd.h', + 'stdio.h', 'netdb.h', 'arpa/inet.h', ) @@ -26,7 +28,7 @@ uint16_t = ctypes_platform.getsimpletype('uint16_t', HEADER, c_ushort) uint32_t = ctypes_platform.getsimpletype('uint32_t', HEADER, c_uint) size_t = ctypes_platform.getsimpletype('size_t', HEADER, c_int) -size_t = ctypes_platform.getsimpletype('size_t', HEADER, c_int) +ssize_t = ctypes_platform.getsimpletype('ssize_t', HEADER, c_int) socklen_t = ctypes_platform.getsimpletype('socklen_t', HEADER, c_int) # struct types @@ -53,6 +55,8 @@ ('ai_next', addrinfo_ptr)]) SetPointerType(addrinfo_ptr, addrinfo) +FILE_ptr = ctypes_platform.getstruct('FILE *', HEADER, + []) # functions dllname = util.find_library('c') assert dllname is not None @@ -67,20 +71,20 @@ socketclose = os.close socketconnect = socketdll.connect -socketconnect.argtypes = [c_int, POINTER(sockaddr), socklen_t] +socketconnect.argtypes = [c_int, sockaddr_ptr, socklen_t] socketconnect.restype = c_int getaddrinfo = socketdll.getaddrinfo -getaddrinfo.argtypes = [c_char_p, c_char_p, POINTER(addrinfo), - POINTER(POINTER(addrinfo))] +getaddrinfo.argtypes = [c_char_p, c_char_p, addrinfo_ptr, + POINTER(addrinfo_ptr)] getaddrinfo.restype = c_int freeaddrinfo = socketdll.freeaddrinfo -freeaddrinfo.argtypes = [POINTER(addrinfo)] +freeaddrinfo.argtypes = [addrinfo_ptr] freeaddrinfo.restype = None getnameinfo = socketdll.getnameinfo -getnameinfo.argtypes = [POINTER(sockaddr), socklen_t, +getnameinfo.argtypes = [sockaddr_ptr, socklen_t, c_char_p, size_t, c_char_p, size_t, c_int] getnameinfo.restype = c_int @@ -106,5 +110,41 @@ inet_aton.restype = c_int socketaccept = socketdll.accept -socketaccept.argtypes = [c_int, POINTER(sockaddr), POINTER(socklen_t)] +socketaccept.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] socketaccept.restype = c_int + +socketdup = socketdll.dup +socketdup.argtypes = [c_int] +socketdup.restype = c_int + +socketfileno = socketdll.fileno +socketfileno.argtypes = [FILE_ptr] +socketfileno.restype = c_int + +socketgetpeername = socketdll.getpeername +socketgetpeername.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] +socketgetpeername.restype = c_int + +socketgetsockname = socketdll.getsockname +socketgetsockname.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] +socketgetsockname.restype = c_int + +socketgetsockopt = socketdll.getsockopt +socketgetsockopt.argtypes = [c_int, c_int, c_int, + c_void_p, POINTER(socklen_t)] +socketgetsockopt.restype = c_int + +socketsetsockopt = socketdll.setsockopt +socketsetsockopt.argtypes = [c_int, c_int, c_int, + c_void_p, #this should be constant + socklen_t] +socketsetsockopt.restype = c_int + +socketrecv = socketdll.recv +socketrecv.argtypes = [c_int, c_void_p, c_int, c_int] +socketrecv.recv = ssize_t + +socketrecvfrom = socketdll.recvfrom +socketrecvfrom.argtypes = [c_int, c_void_p, size_t, + c_int, sockaddr_ptr, POINTER(socklen_t)] +socketrecvfrom.restype = ssize_t \ No newline at end of file From sanxiyn at codespeak.net Mon Apr 24 04:45:03 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 04:45:03 +0200 (CEST) Subject: [pypy-svn] r26214 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424024503.9AAD410081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 04:44:59 2006 New Revision: 26214 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: (sanxiyn, nik, yusei) Moved test utilities out of test script Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 04:44:59 2006 @@ -1,4 +1,6 @@ import autopath +import os +import py from pypy.objspace.flow import FlowObjSpace from pypy.translator.translator import TranslationContext @@ -6,6 +8,28 @@ from py.process import cmdexec from pypy import conftest +def is_on_path(name): + try: + py.path.local.sysfind(name) + except py.error.ENOENT: + return False + else: + return True + +def cl_detect(): + cl = os.getenv("PYPY_CL") + if cl: + return cl + if is_on_path("clisp"): + return "clisp" + if is_on_path("lisp"): + if is_on_path("cmuclinvoke.sh"): + return "cmuclinvoke.sh" + if is_on_path("sbcl"): + if is_on_path("sbclinvoke.sh"): + return "sbclinvoke.sh" + return None + class Literal: def __init__(self, val): self.val = val Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Mon Apr 24 04:44:59 2006 @@ -4,37 +4,18 @@ import os from pypy.objspace.flow import FlowObjSpace + +from pypy.translator.cl.buildcl import cl_detect, _make_cl_func +from pypy.translator.cl.buildcl import Literal + def setup_module(mod): - mod.global_cl = os.getenv("PYPY_CL") - if not mod.global_cl: - mod.global_cl = cl_detect() - -def cl_detect(): - if is_on_path("clisp"): - return "clisp" - if is_on_path("lisp"): - if is_on_path("cmuclinvoke.sh"): - return "cmuclinvoke.sh" - if is_on_path("sbcl"): - if is_on_path("sbclinvoke.sh"): - return "sbclinvoke.sh" - return None - -def is_on_path(name): - try: - py.path.local.sysfind(name) - except py.error.ENOENT: - return False - else: - return True + mod.global_cl = cl_detect() def make_cl_func(func, argtypes=[]): - from pypy.translator.cl.buildcl import _make_cl_func return _make_cl_func(func, global_cl, udir, argtypes) from pypy.translator.test import snippet as t -from pypy.translator.cl.buildcl import Literal class TestGenCLTestCase: def setup_class(cls): From dialtone at codespeak.net Mon Apr 24 04:51:49 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 04:51:49 +0200 (CEST) Subject: [pypy-svn] r26215 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424025149.D8AB210087@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 04:51:46 2006 New Revision: 26215 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (ericvrp, dialtone) finished socket object interface Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 04:51:46 2006 @@ -147,4 +147,20 @@ socketrecvfrom = socketdll.recvfrom socketrecvfrom.argtypes = [c_int, c_void_p, size_t, c_int, sockaddr_ptr, POINTER(socklen_t)] -socketrecvfrom.restype = ssize_t \ No newline at end of file +socketrecvfrom.restype = ssize_t + +socketsend = socketdll.send +socketsend.argtypes = [c_int, + c_void_p, #this should be constant + size_t, c_int] +socketsend.restype = ssize_t + +socketsendto = socketdll.sendto +socketsendto.argtypes = [c_int, c_void_p, #this should be constant + size_t, c_int, sockaddr_ptr, #this should be const + socklen_t] +socketsendto.restype = ssize_t + +socketshutdown = socketdll.shutdown +socketshutdown.argtypes = [c_int, c_int] +socketshutdown.restype = c_int \ No newline at end of file From nik at codespeak.net Mon Apr 24 04:54:31 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 24 Apr 2006 04:54:31 +0200 (CEST) Subject: [pypy-svn] r26216 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424025431.E939510087@code0.codespeak.net> Author: nik Date: Mon Apr 24 04:54:28 2006 New Revision: 26216 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: (nik, sanxiyn, yusei) moved py.test skipping into translation function to get rid of test boilerplate. Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 04:54:28 2006 @@ -2,12 +2,15 @@ import os import py +from pypy.tool.udir import udir from pypy.objspace.flow import FlowObjSpace from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL from py.process import cmdexec from pypy import conftest +global_cl = None + def is_on_path(name): try: py.path.local.sysfind(name) @@ -62,6 +65,14 @@ content = "'" + content # quote Lisp list return content +def make_cl_func(func, argtypes=[]): + global global_cl + if global_cl is None: + global_cl = cl_detect() + if not global_cl: + py.test.skip("Common Lisp neither configured nor detected.") + return _make_cl_func(func, global_cl, udir, argtypes) + def _make_cl_func(func, cl, path, argtypes=[]): t = TranslationContext() t.buildannotator().build_types(func, argtypes) Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Mon Apr 24 04:54:28 2006 @@ -1,29 +1,14 @@ import autopath -from pypy.tool.udir import udir import py import os -from pypy.objspace.flow import FlowObjSpace - -from pypy.translator.cl.buildcl import cl_detect, _make_cl_func +from pypy.translator.cl.buildcl import make_cl_func from pypy.translator.cl.buildcl import Literal -def setup_module(mod): - mod.global_cl = cl_detect() - -def make_cl_func(func, argtypes=[]): - return _make_cl_func(func, global_cl, udir, argtypes) - - from pypy.translator.test import snippet as t -class TestGenCLTestCase: - def setup_class(cls): - cls.space = FlowObjSpace() - def setup_method(self,method): - if not global_cl: - py.test.skip("Common Lisp neither configured nor detected.") +class TestGenCLTestCase: def test_if(self): py.test.skip("temporarily disabled") From sanxiyn at codespeak.net Mon Apr 24 05:26:58 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 05:26:58 +0200 (CEST) Subject: [pypy-svn] r26217 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424032658.2403F10081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 05:26:53 2006 New Revision: 26217 Added: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Get rid of debug output which fails now Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 05:26:53 2006 @@ -267,11 +267,6 @@ else: print self.repr_var(var), print ")" - print ";; DEBUG: type inference" - for var in vardict: - tp = vardict[var] - if tp: - print ";;", self.repr_var(var), "is", tp.__name__ print "(setq last-exc nil)" for block in blocklist: self.emit_block(block) Added: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Mon Apr 24 05:26:53 2006 @@ -0,0 +1,11 @@ +from pypy.translator.cl.buildcl import make_cl_func + +def test_simple(): + class C: + pass + def new_get_set(): + obj = C() + obj.answer = 42 + return obj.answer + cl_new_get_set = make_cl_func(new_get_set) + assert cl_new_get_set() == 42 From ac at codespeak.net Mon Apr 24 05:27:52 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 05:27:52 +0200 (CEST) Subject: [pypy-svn] r26218 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424032752.A6C9410081@code0.codespeak.net> Author: ac Date: Mon Apr 24 05:27:52 2006 New Revision: 26218 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Log: (aleale, arre) Implement some more of the socket interface. Fix some type problems. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 05:27:52 2006 @@ -1,6 +1,6 @@ from ctypes import c_char_p, POINTER, pointer, byref, cast, create_string_buffer, sizeof import ctypes_socket as _c - +import os globals().update(_c.constants) @@ -8,6 +8,14 @@ class error(Exception): pass +def makesockaddr(self, caddr): + family = caddr.sa_family + if family == AF_INET: + caddr = cast(pointer(caddr), POINTER(_c.sockaddr_in)).contents + return (_c.inet_ntoa(caddr.sin_addr), _c.ntohs(caddr.sin_port)) + else: + raise NotImplementedError('Unsupported address family') # XXX + class socket(object): @@ -50,14 +58,6 @@ else: raise NotImplementedError('Unsupported address family') # XXX - def _convert_from_caddr(self, caddr): - family = caddr.sa_family - if family == AF_INET: - caddr = cast(pointer(caddr), POINTER(_c.sockaddr_in)).contents - return (_c.inet_ntoa(caddr.sin_addr), _c.ntohs(caddr.sin_port)) - else: - raise NotImplementedError('Unsupported address family') # XXX - def listen(self, backlog): if self._fd != -1: fd = self._fd @@ -69,36 +69,55 @@ def accept(self): peeraddr = _c.sockaddr() - - newfd = _c.socketaccept(self._fd, pointer(peeraddr), sizeof(peeraddr)) + peeraddrlen = _c.socklen_t(sizeof(peeraddr)) + newfd = _c.socketaccept(self._fd, pointer(peeraddr), + pointer(peeraddrlen)) if newfd < 0: raise error(_c.errno.value) newsocket = socket(self.family, self.type, self.proto, newfd) - return (newsocket, self._convert_from_caddr(peeraddr)) + return (newsocket, makesockaddr(peeraddr)) - def connect_ex(self): - pass + def connect_ex(self, addr): + caddr = self._getsockaddr(addr) + paddr = cast(pointer(caddr), _c.sockaddr_ptr) + result = _c.socketconnect(self._fd, paddr, + _c.socklen_t(sizeof(caddr))) + if result == -1: + return _c.errno.value + return 0 def dup(self): - pass + raise NotImplementedError def fileno(self): - pass + return self._fd def getpeername(self): - pass + peeraddr = _c.sockaddr() + peeraddrlen = _c.socklen_t(sizeof(peeraddr)) + res = _c.socketgetpeername(self._fd, pointer(peeraddr), + pointer(peeraddrlen)) + if res < 0: + raise error(_c.errno.value) + return makesockaddr(peeraddr) def getsockname(self): - pass + peeraddr = _c.sockaddr() + peeraddrlen = _c.socklen_t(sizeof(peeraddr)) + res = _c.socketgetsockname(self._fd, pointer(peeraddr), + pointer(peeraddrlen)) + if res < 0: + raise error(_c.errno.value) + return makesockaddr(peeraddr) - def getsockopt(self): + def getsockopt(self, level, optname, buflen=-1): pass def gettimeout(self): pass def makefile(self): - pass + raise NotImplementedError def recv(self): pass @@ -128,12 +147,10 @@ pass def connect(self, addr): - caddr = self._getsockaddr(addr) - paddr = cast(pointer(caddr), _c.sockaddr_ptr) - result = _c.socketconnect(self._fd, paddr, sizeof(caddr)) - if result == -1: - raise error(_c.errno.value) - + err = self.connect_ex(addr) + if err: + raise error(err) + def makeipaddr(caddr, caddrlen): buf = create_string_buffer(NI_MAXHOST) error = _c.getnameinfo(caddr, caddrlen, buf, NI_MAXHOST, From sanxiyn at codespeak.net Mon Apr 24 05:32:42 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 05:32:42 +0200 (CEST) Subject: [pypy-svn] r26219 - pypy/dist/pypy/translator/cl Message-ID: <20060424033242.AAD4810081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 05:32:38 2006 New Revision: 26219 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Intermediate checkin Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 05:32:38 2006 @@ -96,6 +96,27 @@ def op_int_is_true(self): self.op_is_true(self.args[0]) + def declare_class(self, cls): + # cls is really type of Instance + name = cls._name + fields = cls._fields + fieldnames = fields.keys() + field_declaration = ' '.join(fieldnames) + class_declaration = "(defstruct %s (%s))" % (name, field_declaration) + return class_declaration + + def op_new(self): + cls = self.args[0].value + print self.declare_class(cls) + target = self.str(self.result) + print "(setq %s (make-%s))" % (target, cls._name) + + def op_oogetfield(self): + print "; oogetfield called" + + def op_oosetfield(self): + print "; oosetfield called" + def op_newtuple(self): s = self.str print "(setq", s(self.result), "(list", From yusei at codespeak.net Mon Apr 24 06:05:31 2006 From: yusei at codespeak.net (yusei at codespeak.net) Date: Mon, 24 Apr 2006 06:05:31 +0200 (CEST) Subject: [pypy-svn] r26220 - pypy/dist/pypy/translator/cl Message-ID: <20060424040531.156F310081@code0.codespeak.net> Author: yusei Date: Mon Apr 24 06:05:28 2006 New Revision: 26220 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (yusei, sanxiyn, nik) Implemented oogetfield and oosetfield. Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 06:05:28 2006 @@ -112,10 +112,21 @@ print "(setq %s (make-%s))" % (target, cls._name) def op_oogetfield(self): - print "; oogetfield called" + target = self.str(self.result) + clsname = self.args[0].concretetype._name + fieldname = self.args[1].value + obj = self.str(self.args[0]) + print "(setq %s (%s-%s %s))" % (target, clsname, fieldname, obj) def op_oosetfield(self): - print "; oosetfield called" + target = self.str(self.result) + clsname = self.args[0].concretetype._name + fieldname = self.args[1].value + if fieldname == "meta": # XXX + return + obj = self.str(self.args[0]) + fieldvalue = self.str(self.args[2]) + print "(setf (%s-%s %s) %s)" % (clsname, fieldname, obj, fieldvalue) def op_newtuple(self): s = self.str From dialtone at codespeak.net Mon Apr 24 06:14:07 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 06:14:07 +0200 (CEST) Subject: [pypy-svn] r26221 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060424041407.2E15410081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 06:14:02 2006 New Revision: 26221 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Log: (ericvrp, dialtone) Added an awful lot of constants Warning: testing takes some time because of the platform check. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 06:14:02 2006 @@ -151,6 +151,8 @@ if err: raise error(err) +SocketType = socket + def makeipaddr(caddr, caddrlen): buf = create_string_buffer(NI_MAXHOST) error = _c.getnameinfo(caddr, caddrlen, buf, NI_MAXHOST, Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 06:14:02 2006 @@ -1,4 +1,5 @@ import os +import distutils from pypy.rpython.rctypes.tool import ctypes_platform from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 from ctypes import * @@ -6,23 +7,76 @@ includes = ('sys/types.h', 'sys/socket.h', 'netinet/in.h', + 'netinet/tcp.h', 'unistd.h', 'stdio.h', 'netdb.h', - 'arpa/inet.h', + 'arpa/inet.h' ) HEADER = ''.join(['#include <%s>\n' % filename for filename in includes]) constants = {} # constants -for name in ['AF_INET', - 'AF_UNSPEC', - 'SOCK_STREAM', - 'SOCK_DGRAM', - 'NI_MAXHOST', - 'NI_NUMERICHOST', - ]: - constants[name] = ctypes_platform.getconstantinteger(name, HEADER) +for name in ['AF_APPLETALK', 'AF_INET', 'AF_INET6', 'AF_IPX','AF_ROUTE', +'AF_SNA', 'AF_UNIX', 'AF_UNSPEC', 'AI_ADDRCONFIG', 'AI_ALL', 'AI_CANONNAME', +'AI_DEFAULT', 'AI_MASK', 'AI_NUMERICHOST', 'AI_PASSIVE', 'AI_V4MAPPED', +'AI_V4MAPPED_CFG', 'EAI_ADDRFAMILY', 'EAI_AGAIN', 'EAI_BADFLAGS', +'EAI_BADHINTS', 'EAI_FAIL', 'EAI_FAMILY', 'EAI_MAX', 'EAI_MEMORY', +'EAI_NODATA', 'EAI_NONAME', 'EAI_PROTOCOL', 'EAI_SERVICE', 'EAI_SOCKTYPE', +'EAI_SYSTEM', 'INADDR_UNSPEC_GROUP', 'IPPROTO_AH', +'IPPROTO_DSTOPTS', 'IPPROTO_EGP', 'IPPROTO_EON', 'IPPROTO_ESP', +'IPPROTO_FRAGMENT', 'IPPROTO_GGP', 'IPPROTO_GRE', 'IPPROTO_HELLO', +'IPPROTO_HOPOPTS', 'IPPROTO_ICMP', 'IPPROTO_ICMPV6', 'IPPROTO_IDP', +'IPPROTO_IGMP', 'IPPROTO_IPCOMP', 'IPPROTO_IPIP', +'IPPROTO_IPV4', 'IPPROTO_IPV6', 'IPPROTO_MAX', 'IPPROTO_ND', 'IPPROTO_NONE', +'IPPROTO_PIM', 'IPPROTO_PUP', 'IPPROTO_ROUTING', +'IPPROTO_RSVP', 'IPPROTO_TCP', 'IPPROTO_TP', 'IPPROTO_XTP', +'IPV6_CHECKSUM', 'IPV6_DSTOPTS', 'IPV6_HOPLIMIT', 'IPV6_HOPOPTS', +'IPV6_JOIN_GROUP', 'IPV6_LEAVE_GROUP', 'IPV6_MULTICAST_HOPS', +'IPV6_MULTICAST_IF', 'IPV6_MULTICAST_LOOP', 'IPV6_NEXTHOP', 'IPV6_PKTINFO', +'IPV6_RTHDR', 'IPV6_RTHDR_TYPE_0', 'IPV6_UNICAST_HOPS', 'IPV6_V6ONLY', +'IP_ADD_MEMBERSHIP', 'IP_DEFAULT_MULTICAST_LOOP', 'IP_DEFAULT_MULTICAST_TTL', +'IP_DROP_MEMBERSHIP', 'IP_HDRINCL', 'IP_MAX_MEMBERSHIPS', 'IP_MULTICAST_IF', +'IP_MULTICAST_LOOP', 'IP_MULTICAST_TTL', 'IP_OPTIONS', 'IP_RECVDSTADDR', +'IP_RECVOPTS', 'IP_RECVRETOPTS', 'IP_RETOPTS', 'IP_TOS', 'IP_TTL', +'MSG_CTRUNC', 'MSG_DONTROUTE', 'MSG_DONTWAIT', 'MSG_EOR', 'MSG_OOB', +'MSG_PEEK', 'MSG_TRUNC', 'MSG_WAITALL', 'NI_DGRAM', 'NI_MAXHOST', +'NI_MAXSERV', 'NI_NAMEREQD', 'NI_NOFQDN', 'NI_NUMERICHOST', 'NI_NUMERICSERV', +'SOCK_DGRAM', 'SOCK_RAW', 'SOCK_RDM', +'SOCK_SEQPACKET', 'SOCK_STREAM', 'SOL_SOCKET', +'SO_ACCEPTCONN', 'SO_BROADCAST', 'SO_DEBUG', 'SO_DONTROUTE', +'SO_ERROR', 'SO_KEEPALIVE', 'SO_LINGER', 'SO_OOBINLINE', 'SO_RCVBUF', +'SO_RCVLOWAT', 'SO_RCVTIMEO', 'SO_REUSEADDR', 'SO_REUSEPORT', 'SO_SNDBUF', +'SO_SNDLOWAT', 'SO_SNDTIMEO', 'SO_TYPE', 'SO_USELOOPBACK', 'TCP_MAXSEG', +'TCP_NODELAY', 'AF_DECnet']: + try: + constants[name] = ctypes_platform.getconstantinteger(name, HEADER) + except distutils.errors.CompileError: + pass + +for special, default in [('SOL_IP', 0), + ('SOL_TCP', 6), + ('SOL_UDP', 17), + ('SOMAXCONN', 5), + ('IPPROTO_IP', 6), + ('IPPROTO_UDP', 17), + ('IPPROTO_RAW', 255), + ('IPPORT_RESERVED', 1024), + ('IPPORT_USERRESERVED', 5000), + ('INADDR_ANY', 0x00000000), + ('INADDR_BROADCAST', 0xffffffff), + ('INADDR_LOOPBACK', 0x7F000001), + ('INADDR_UNSPEC_GROUP', 0xe0000000), + ('INADDR_ALLHOSTS_GROUP', 0xe0000001), + ('INADDR_MAX_LOCAL_GROUP', 0xe00000ff), + ('INADDR_NONE', 0xffffffff), + ('SHUT_RD', 0), + ('SHUT_WR', 1), + ('SHUT_RDWR', 2)]: + try: + constants[special] = ctypes_platform.getconstantinteger(special, HEADER) + except distutils.errors.CompileError: + constants[special] = default # types uint16_t = ctypes_platform.getsimpletype('uint16_t', HEADER, c_ushort) @@ -57,6 +111,7 @@ FILE_ptr = ctypes_platform.getstruct('FILE *', HEADER, []) + # functions dllname = util.find_library('c') assert dllname is not None Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Mon Apr 24 06:14:02 2006 @@ -4,13 +4,14 @@ def interface_matcher(interface1, interface2): members = [member for member in dir(interface1) if not member.startswith('_')] + members.remove('CAPI') verifying_set = dir(interface2) for member in members: assert member in verifying_set def test_interfaces(): - #interface_matcher(_socket, _rsocket) - #interface_matcher(_rsocket, _socket) + interface_matcher(_socket, _rsocket) + interface_matcher(_rsocket, _socket) interface_matcher(_socket.socket, _rsocket.socket) interface_matcher(_rsocket.socket, _socket.socket) From sanxiyn at codespeak.net Mon Apr 24 07:46:28 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 07:46:28 +0200 (CEST) Subject: [pypy-svn] r26222 - pypy/dist/pypy/translator/cl Message-ID: <20060424054628.E63D810081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 07:46:26 2006 New Revision: 26222 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Parentheses are not needed. Thanks, Valentino. Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 07:46:26 2006 @@ -102,7 +102,7 @@ fields = cls._fields fieldnames = fields.keys() field_declaration = ' '.join(fieldnames) - class_declaration = "(defstruct %s (%s))" % (name, field_declaration) + class_declaration = "(defstruct %s %s)" % (name, field_declaration) return class_declaration def op_new(self): From sanxiyn at codespeak.net Mon Apr 24 07:48:39 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 07:48:39 +0200 (CEST) Subject: [pypy-svn] r26223 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424054839.603F910081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 07:48:35 2006 New Revision: 26223 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: Make more tests pass Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 07:48:35 2006 @@ -27,6 +27,11 @@ print ";;", self.op print ";; Op", self.opname, "is missing" + def op_same_as(self): + target = self.str(self.result) + origin = self.str(self.args[0]) + print "(setq %s %s)" % (target, origin) + binary_ops = { #"add": "+", "int_add": "+", @@ -36,8 +41,10 @@ "mod": "mod", "int_mod": "mod", "lt": "<", + "int_lt": "<", "le": "<=", "eq": "=", + "int_eq": "=", "gt": ">", "and_": "logand", "getitem": "elt", Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Mon Apr 24 07:48:35 2006 @@ -25,7 +25,6 @@ assert cl_gcd(96, 64) == 32 def test_is_perfect(self): # pun intended - py.test.skip("temporarily disabled") cl_perfect = make_cl_func(t.is_perfect_number, [int]) assert cl_perfect(24) == False assert cl_perfect(28) == True From dialtone at codespeak.net Mon Apr 24 08:01:49 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 08:01:49 +0200 (CEST) Subject: [pypy-svn] r26224 - pypy/dist/pypy/translator/cl Message-ID: <20060424060149.9CF4710081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 08:01:45 2006 New Revision: 26224 Added: pypy/dist/pypy/translator/cl/openmclinvoke.sh (contents, props changed) Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: add openmcl support Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 08:01:45 2006 @@ -23,6 +23,9 @@ cl = os.getenv("PYPY_CL") if cl: return cl + if is_on_path("openmcl"): + if is_on_path("openmclinvoke.sh"): + return "sbclinvoke.sh" if is_on_path("clisp"): return "clisp" if is_on_path("lisp"): Added: pypy/dist/pypy/translator/cl/openmclinvoke.sh ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/openmclinvoke.sh Mon Apr 24 08:01:45 2006 @@ -0,0 +1,3 @@ +#!/bin/sh +# Copy this to your PATH +openmcl --batch --load $1 From sanxiyn at codespeak.net Mon Apr 24 08:05:18 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 08:05:18 +0200 (CEST) Subject: [pypy-svn] r26225 - pypy/dist/pypy/translator/cl Message-ID: <20060424060518.91F0410081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 08:05:15 2006 New Revision: 26225 Modified: pypy/dist/pypy/translator/cl/cmuclinvoke.sh (props changed) pypy/dist/pypy/translator/cl/sbclinvoke.sh (props changed) Log: Make invoke scripts executable From sanxiyn at codespeak.net Mon Apr 24 08:44:09 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 08:44:09 +0200 (CEST) Subject: [pypy-svn] r26226 - pypy/dist/pypy/translator/cl Message-ID: <20060424064409.149D51006D@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 08:44:05 2006 New Revision: 26226 Added: pypy/dist/pypy/translator/cl/clrepr.py Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Refactor representation parts to clrepr.py Added: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 08:44:05 2006 @@ -0,0 +1,40 @@ +def repr_unknown(obj): + return '#<%r>' % (obj,) + +def repr_var(var): + return var.name + +def repr_const(val): + if isinstance(val, tuple): + val = map(repr_const, val) + return "'(%s)" % ' '.join(val) + elif isinstance(val, bool): # should precedes int + if val: + return "t" + else: + return "nil" + elif isinstance(val, (int, long)): + return str(val) + elif val is None: + return "nil" + elif isinstance(val, str): + val.replace("\\", "\\\\") + val.replace("\"", "\\\"") + val = '"' + val + '"' + return val + elif isinstance(val, type(Exception)) and issubclass(val, Exception): + return "'%s" % val.__name__ + elif val is last_exception: + return "last-exc" + elif val is last_exc_value: + return "'last-exc-value" + else: + return repr_unknown(val) + +def repr_arg(arg): + if isinstance(arg, Variable): + return repr_var(arg) + elif isinstance(arg, Constant): + return repr_const(arg.value) + else: + return repr_unknown(arg) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 08:44:05 2006 @@ -4,12 +4,14 @@ from pypy.translator.simplify import simplify_graph from pypy.translator.transform import transform_graph, default_extra_passes, transform_slice +from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const + class Op: def __init__(self, gen, op): self.gen = gen - self.str = gen.repr_arg + self.str = repr_arg self.op = op self.opname = op.opname self.args = op.args @@ -229,47 +231,6 @@ def get_type(self, var): return var.concretetype - def repr_unknown(self, obj): - return '#<%r>' % (obj,) - - def repr_var(self, var): - return var.name - - def repr_const(self, val): - if isinstance(val, tuple): - val = map(self.repr_const, val) - return "'(%s)" % ' '.join(val) - elif isinstance(val, bool): # should precedes int - if val: - return "t" - else: - return "nil" - elif isinstance(val, (int, long)): - return str(val) - elif val is None: - return "nil" - elif isinstance(val, str): - val.replace("\\", "\\\\") - val.replace("\"", "\\\"") - val = '"' + val + '"' - return val - elif isinstance(val, type(Exception)) and issubclass(val, Exception): - return "'%s" % val.__name__ - elif val is last_exception: - return "last-exc" - elif val is last_exc_value: - return "'last-exc-value" - else: - return self.repr_unknown(val) - - def repr_arg(self, arg): - if isinstance(arg, Variable): - return self.repr_var(arg) - elif isinstance(arg, Constant): - return self.repr_const(arg.value) - else: - return self.repr_unknown(arg) - def emitcode(self, public=True): import sys from cStringIO import StringIO @@ -289,7 +250,7 @@ arglist = fun.getargs() print "(", for arg in arglist: - print self.repr_var(arg), + print repr_var(arg), print ")" print "(prog" blocklist = list(fun.iterblocks()) @@ -302,9 +263,9 @@ print "( last-exc", for var in vardict: if var in arglist: - print "(", self.repr_var(var), self.repr_var(var), ")", + print "(", repr_var(var), repr_var(var), ")", else: - print self.repr_var(var), + print repr_var(var), print ")" print "(setq last-exc nil)" for block in blocklist: @@ -327,7 +288,7 @@ if (len(exits) == 2 and exits[0].exitcase == False and exits[1].exitcase == True): - print "(if", self.repr_arg(block.exitswitch) + print "(if", repr_arg(block.exitswitch) print "(progn" self.emit_link(exits[1]) print ") ; else" @@ -339,20 +300,20 @@ # shouldn't be needed but in Python 2.2 we can't tell apart # 0 vs nil and 1 vs t :-( for exit in exits[:-1]: - print "(if (equalp", self.repr_arg(block.exitswitch), - print self.repr_const(exit.exitcase), ')' + print "(if (equalp", repr_arg(block.exitswitch), + print repr_const(exit.exitcase), ')' print "(progn" self.emit_link(exit) print ")" - print "(progn ; else should be", self.repr_const(exits[-1].exitcase) + print "(progn ; else should be", repr_const(exits[-1].exitcase) self.emit_link(exits[-1]) print ")" * len(exits) elif len(block.inputargs) == 2: # exc_cls, exc_value - exc_cls = self.repr_var(block.inputargs[0]) - exc_value = self.repr_var(block.inputargs[1]) + exc_cls = repr_var(block.inputargs[0]) + exc_value = repr_var(block.inputargs[1]) print "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) else: - retval = self.repr_var(block.inputargs[0]) + retval = repr_var(block.inputargs[0]) print "(return", retval, ")" def emit_jump(self, block): @@ -360,8 +321,8 @@ print "(go", "tag" + str(tag), ")" def emit_link(self, link): - source = map(self.repr_arg, link.args) - target = map(self.repr_var, link.target.inputargs) + source = map(repr_arg, link.args) + target = map(repr_var, link.target.inputargs) print "(psetq", # parallel assignment for s, t in zip(source, target): print t, s @@ -377,7 +338,7 @@ } def emit_typecase(self, table, *args): - argreprs = tuple(map(self.repr_arg, args)) + argreprs = tuple(map(repr_arg, args)) argtypes = tuple(map(self.get_type, args)) if argtypes in table: trans = table[argtypes] From sanxiyn at codespeak.net Mon Apr 24 08:53:36 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 08:53:36 +0200 (CEST) Subject: [pypy-svn] r26227 - pypy/dist/pypy/translator/cl Message-ID: <20060424065336.942AE1006D@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 08:53:33 2006 New Revision: 26227 Modified: pypy/dist/pypy/translator/cl/clrepr.py Log: Missing imports Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 08:53:33 2006 @@ -1,3 +1,5 @@ +from pypy.objspace.flow.model import Constant, Variable, last_exception + def repr_unknown(obj): return '#<%r>' % (obj,) From sanxiyn at codespeak.net Mon Apr 24 08:58:23 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 08:58:23 +0200 (CEST) Subject: [pypy-svn] r26228 - pypy/dist/pypy/translator/cl Message-ID: <20060424065823.F1BC91007C@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 08:58:20 2006 New Revision: 26228 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Adapt buildcl to clrepr refactoring Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 08:58:20 2006 @@ -6,6 +6,7 @@ from pypy.objspace.flow import FlowObjSpace from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL +from pypy.translator.cl.clrepr import repr_const from py.process import cmdexec from pypy import conftest @@ -55,12 +56,11 @@ else: return Literal(s) -def writelisp(gen, obj): - #if isinstance(obj, (bool, int, type(None), str)): - if isinstance(obj, (int, type(None), str)): - return gen.repr_const(obj) +def writelisp(obj): + if isinstance(obj, (bool, int, type(None), str)): + return repr_const(obj) if isinstance(obj, (tuple, list)): - content = ' '.join([writelisp(gen, elt) for elt in obj]) + content = ' '.join([writelisp(elt) for elt in obj]) content = '(' + content + ')' if isinstance(obj, list): content = '#' + content @@ -95,7 +95,7 @@ fp = file(str(fpath), "a") print >>fp, "(write (", graph.name, for arg in args: - print >>fp, writelisp(gen, arg), + print >>fp, writelisp(arg), print >>fp, "))" fp.close() output = cmdexec("%s %s" % (cl, str(fpath))) @@ -110,7 +110,7 @@ gen = GenCL(fun) what = [True, "universe", 42, None, ("of", "them", ["eternal", 95])] - it = writelisp(gen, what) + it = writelisp(what) print what print it assert it == '#(t "universe" 42 nil \'("of" "them" #("eternal" 95)))' From sanxiyn at codespeak.net Mon Apr 24 08:58:58 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 08:58:58 +0200 (CEST) Subject: [pypy-svn] r26229 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424065858.4D1C810084@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 08:58:54 2006 New Revision: 26229 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (sanxiyn, nik, yusei) Start of list implementation in Common Lisp backend, but tests don't run yet. Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 08:58:54 2006 @@ -1,9 +1,10 @@ -from pypy.objspace.flow.model import Constant, Variable, last_exception +from pypy.objspace.flow.model import Constant from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.simplify import simplify_graph from pypy.translator.transform import transform_graph, default_extra_passes, transform_slice +from pypy.rpython.ootypesystem.ootype import Instance, List from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const @@ -116,10 +117,28 @@ def op_new(self): cls = self.args[0].value + if isinstance(cls, List): + self.op_new_list(cls) + else: + self.op_new_instance(cls) + + def op_new_list(self, cls): + target = self.str(self.result) + print "(setq %s (make-array 0 :adjustable t))" % (target,) + + def op_new_instance(self, cls): print self.declare_class(cls) target = self.str(self.result) print "(setq %s (make-%s))" % (target, cls._name) + def op_oosend(self): + method = self.args[0].value + receiver = self.args[1] + args = self.args[2:] + if isinstance(receiver.concretetype, List): + impl = ListImpl(receiver) + getattr(impl, method)(*args) + def op_oogetfield(self): target = self.str(self.result) clsname = self.args[0].concretetype._name @@ -211,6 +230,27 @@ print ")" +class ListImpl: + + def __init__(self, receiver): + self.obj = repr_arg(receiver) + + def ll_length(self): + pass + + def ll_getitem_fast(self, index): + pass + + def ll_setitem_fast(self, index, value): + index = repr_arg(index) + value = repr_arg(value) + return "(setf (aref %s %s) %s)" % (self.obj, index, value) + + def _ll_resize(self, size): + size = repr_arg(size) + return "(adjust-array %s %s)" % (self.obj, size) + + class GenCL: def __init__(self, fun, input_arg_types=[]): Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Mon Apr 24 08:58:54 2006 @@ -9,3 +9,17 @@ return obj.answer cl_new_get_set = make_cl_func(new_get_set) assert cl_new_get_set() == 42 + +def dont_test_list_length(): + def list_length_one(number): + lst = [number] + return len(lst) + cl_list_length_one = make_cl_func(list_length_one, [int]) + assert cl_list_length_one(0) == 1 + +def dont_test_list_get(): + def list_and_get(number): + lst = [number] + return lst[0] + cl_list_and_get = make_cl_func(list_and_get, [int]) + assert cl_list_and_get(1985) == 1985 From ac at codespeak.net Mon Apr 24 09:02:35 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 09:02:35 +0200 (CEST) Subject: [pypy-svn] r26230 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060424070235.E225B1007F@code0.codespeak.net> Author: ac Date: Mon Apr 24 09:02:35 2006 New Revision: 26230 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Log: (aleale, arre) Add error message to socket.error. Fix some typos in ctypes_socket. Add some tests and fix the bugs they exposed. Added some constants missing on Linux (there are more to add). Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 09:02:35 2006 @@ -6,15 +6,8 @@ class error(Exception): - pass - -def makesockaddr(self, caddr): - family = caddr.sa_family - if family == AF_INET: - caddr = cast(pointer(caddr), POINTER(_c.sockaddr_in)).contents - return (_c.inet_ntoa(caddr.sin_addr), _c.ntohs(caddr.sin_port)) - else: - raise NotImplementedError('Unsupported address family') # XXX + def __init__(self, errno): + Exception.__init__(self, errno, _c.strerror(errno)) class socket(object): @@ -59,25 +52,25 @@ raise NotImplementedError('Unsupported address family') # XXX def listen(self, backlog): - if self._fd != -1: - fd = self._fd - res = _c.listen(fd, backlog) - if res == -1: - raise error(_c.errno.value) - else: - XXX + fd = self._fd + if backlog < 1: + backlog = 1 + res = _c.listen(fd, backlog) + if res == -1: + raise error(_c.errno.value) def accept(self): - peeraddr = _c.sockaddr() - peeraddrlen = _c.socklen_t(sizeof(peeraddr)) - newfd = _c.socketaccept(self._fd, pointer(peeraddr), + peeraddr = pointer(_c.sockaddr()) + peeraddrlen = _c.socklen_t(sizeof(_c.sockaddr)) + newfd = _c.socketaccept(self._fd, peeraddr, pointer(peeraddrlen)) if newfd < 0: raise error(_c.errno.value) newsocket = socket(self.family, self.type, self.proto, newfd) - return (newsocket, makesockaddr(peeraddr)) + return (newsocket, makesockaddr(peeraddr, peeraddrlen, self.proto)) def connect_ex(self, addr): + host, port = addr caddr = self._getsockaddr(addr) paddr = cast(pointer(caddr), _c.sockaddr_ptr) result = _c.socketconnect(self._fd, paddr, @@ -93,22 +86,22 @@ return self._fd def getpeername(self): - peeraddr = _c.sockaddr() - peeraddrlen = _c.socklen_t(sizeof(peeraddr)) - res = _c.socketgetpeername(self._fd, pointer(peeraddr), + peeraddr = pointer(_c.sockaddr()) + peeraddrlen = _c.socklen_t(sizeof(_c.sockaddr)) + res = _c.socketgetpeername(self._fd, peeraddr, pointer(peeraddrlen)) if res < 0: raise error(_c.errno.value) - return makesockaddr(peeraddr) + return makesockaddr(peeraddr, peeraddrlen, self.proto) def getsockname(self): - peeraddr = _c.sockaddr() - peeraddrlen = _c.socklen_t(sizeof(peeraddr)) - res = _c.socketgetsockname(self._fd, pointer(peeraddr), + peeraddr = pointer(_c.sockaddr()) + peeraddrlen = _c.socklen_t(sizeof(_c.sockaddr)) + res = _c.socketgetsockname(self._fd, peeraddr, pointer(peeraddrlen)) if res < 0: raise error(_c.errno.value) - return makesockaddr(peeraddr) + return makesockaddr(peeraddr, peeraddrlen, self.proto) def getsockopt(self, level, optname, buflen=-1): pass @@ -169,7 +162,7 @@ a = cast(caddr, POINTER(_c.sockaddr_in)) return makeipaddr(caddr, caddrlen), _c.ntohs(a.contents.sin_port) else: - XXX + raise NotImplementedError("Unsupported address family %d" % caddr.contents.sa_family) def getaddrinfo(host, port, family=AF_UNSPEC, socktype=0, proto=0, flags=0): if isinstance(port, (int, long)): Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 09:02:35 2006 @@ -14,13 +14,18 @@ 'arpa/inet.h' ) HEADER = ''.join(['#include <%s>\n' % filename for filename in includes]) -constants = {} +constants = {"BDADDR_ANY": "00:00:00:00:00:00", + "BDADDR_LOCAL": "00:00:00:FF:FF:FF"} # constants -for name in ['AF_APPLETALK', 'AF_INET', 'AF_INET6', 'AF_IPX','AF_ROUTE', -'AF_SNA', 'AF_UNIX', 'AF_UNSPEC', 'AI_ADDRCONFIG', 'AI_ALL', 'AI_CANONNAME', -'AI_DEFAULT', 'AI_MASK', 'AI_NUMERICHOST', 'AI_PASSIVE', 'AI_V4MAPPED', -'AI_V4MAPPED_CFG', 'EAI_ADDRFAMILY', 'EAI_AGAIN', 'EAI_BADFLAGS', +for name in ['AF_APPLETALK', 'AF_ASH', 'AF_ATMPVC', 'AF_ATMSVC', 'AF_AX25', + 'AF_BLUETOOTH', 'AF_BRIDGE', 'AF_ECONET', 'AF_INET', 'AF_INET6', + 'AF_IPX', 'AF_IRDA', 'AF_KEY', 'AF_NETBEUI', 'AF_NETLINK', + 'AF_NETROM', 'AF_PACKET', 'AF_PPPOX', 'AF_ROSE', 'AF_ROUTE', + 'AF_SECURITY', 'AF_WANPIPE', 'AF_SNA', 'AF_UNIX', 'AF_X25', + 'AF_UNSPEC', 'AI_ADDRCONFIG', 'AI_ALL', 'AI_CANONNAME', +'AI_DEFAULT', 'AI_MASK', 'AI_NUMERICHOST', 'AI_NUMERICSERV', 'AI_PASSIVE', 'AI_V4MAPPED', +'AI_V4MAPPED_CFG', 'BDADDR_ANY', 'EAI_ADDRFAMILY', 'EAI_AGAIN', 'EAI_BADFLAGS', 'EAI_BADHINTS', 'EAI_FAIL', 'EAI_FAMILY', 'EAI_MAX', 'EAI_MEMORY', 'EAI_NODATA', 'EAI_NONAME', 'EAI_PROTOCOL', 'EAI_SERVICE', 'EAI_SOCKTYPE', 'EAI_SYSTEM', 'INADDR_UNSPEC_GROUP', 'IPPROTO_AH', @@ -118,6 +123,9 @@ socketdll = cdll.LoadLibrary(dllname) errno = c_int.in_dll(socketdll, 'errno') +strerror = socketdll.strerror +strerror.argtypes = [c_int] +strerror.restype = c_char_p socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] @@ -148,15 +156,15 @@ htonl.argtypes = [uint32_t] htonl.restype = uint32_t -htons = socketdll.htonl +htons = socketdll.htons htons.argtypes = [uint16_t] htons.restype = uint16_t -ntohl = socketdll.htonl +ntohl = socketdll.ntohl ntohl.argtypes = [uint32_t] ntohl.restype = uint32_t -ntohs = socketdll.htonl +ntohs = socketdll.ntohs ntohs.argtypes = [uint16_t] ntohs.restype = uint16_t @@ -218,4 +226,4 @@ socketshutdown = socketdll.shutdown socketshutdown.argtypes = [c_int, c_int] -socketshutdown.restype = c_int \ No newline at end of file +socketshutdown.restype = c_int Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Mon Apr 24 09:02:35 2006 @@ -1,23 +1,51 @@ import py -import _socket, errno +import _socket, errno, thread from pypy.rpython.rctypes.socketmodule import _socket as _rsocket +from pypy.module._socket.test import echoserver def interface_matcher(interface1, interface2): - members = [member for member in dir(interface1) if not member.startswith('_')] - members.remove('CAPI') + members = [member for member in dir(interface1) if member != "CAPI" and not member.startswith('_')] verifying_set = dir(interface2) for member in members: assert member in verifying_set def test_interfaces(): + py.test.skip("In progress.") interface_matcher(_socket, _rsocket) interface_matcher(_rsocket, _socket) interface_matcher(_socket.socket, _rsocket.socket) interface_matcher(_rsocket.socket, _socket.socket) -def test_accept(): - s = _rsocket.socket(_rsocket.AF_INET, _rsocket.SOCK_STREAM, - _socket.IPPROTO_TCP) - e = py.test.raises(_rsocket.error, s.accept).value - assert e.args[0] == errno.EINVAL +class TestSocket: + + HOST = "127.0.0.1" + PORT = echoserver.PORT + family = _socket.AF_INET + + def setup_class(cls): + thread.start_new_thread(echoserver.start_server, (), + {"address_family": cls.family}) + import time + time.sleep(1) + def teardown_class(cls): + import telnetlib + tn = telnetlib.Telnet(cls.HOST, cls.PORT) + tn.write("shutdown\n") + tn.close() + + def test_accept_no_bind(self): + s = _rsocket.socket(_rsocket.AF_INET, _rsocket.SOCK_STREAM, + _socket.IPPROTO_TCP) + e = py.test.raises(_rsocket.error, s.accept).value + assert e.args[0] == errno.EINVAL + def test_getpeername(self): + s = _rsocket.socket(self.family, _socket.SOCK_STREAM, 0) + s.connect((self.HOST, self.PORT)) + sockname = s.getpeername() + s.close() + host, port = sockname + assert host == self.HOST + assert port == self.PORT + + From sanxiyn at codespeak.net Mon Apr 24 09:06:20 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 09:06:20 +0200 (CEST) Subject: [pypy-svn] r26231 - pypy/dist/pypy/translator/cl Message-ID: <20060424070620.5E3141007F@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 09:06:17 2006 New Revision: 26231 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Filter lines starting with semicolon in readlisp() Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 09:06:17 2006 @@ -43,6 +43,10 @@ def readlisp(s): # Return bool/int/str or give up + lines = s.splitlines() + lines = [ line for line in lines if line and not line.startswith(';') ] + assert len(lines) == 1 + s = lines[0] import string s = s.strip() if s == "T": From ac at codespeak.net Mon Apr 24 09:18:35 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 09:18:35 +0200 (CEST) Subject: [pypy-svn] r26232 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060424071835.C6B3E1007F@code0.codespeak.net> Author: ac Date: Mon Apr 24 09:18:35 2006 New Revision: 26232 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Log: (aleale, arre) Progresssing on _socket Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 09:18:35 2006 @@ -34,8 +34,9 @@ _c.socketclose(fd) def bind(self, addr): - caddr, caddrlen = self._getsockaddr(addr) - res = _c.bind(self._fd, caddr, caddrlen) + caddr = self._getsockaddr(addr) + paddr = cast(pointer(caddr), _c.sockaddr_ptr) + res = _c.socketbind(self._fd, paddr, sizeof(caddr)) if res < 0: raise error(_c.errno.value) Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 09:18:35 2006 @@ -176,13 +176,9 @@ socketaccept.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] socketaccept.restype = c_int -socketdup = socketdll.dup -socketdup.argtypes = [c_int] -socketdup.restype = c_int - -socketfileno = socketdll.fileno -socketfileno.argtypes = [FILE_ptr] -socketfileno.restype = c_int +socketbind = socketdll.bind +socketbind.argtypes = [c_int, sockaddr_ptr, socklen_t] +socketbind.restype = c_int socketgetpeername = socketdll.getpeername socketgetpeername.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Mon Apr 24 09:18:35 2006 @@ -47,5 +47,16 @@ host, port = sockname assert host == self.HOST assert port == self.PORT - + + def test_getsockname(self): + s = _rsocket.socket(self.family, _socket.SOCK_STREAM, 0) + HOST = "127.0.0.1" + PORT = 1025 + s.bind((HOST, PORT)) + sockname = s.getsockname() + s.close() + host, port = sockname + assert host == HOST + assert port == PORT + From dialtone at codespeak.net Mon Apr 24 10:02:03 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 10:02:03 +0200 (CEST) Subject: [pypy-svn] r26235 - pypy/dist/pypy/translator/cl Message-ID: <20060424080203.075C410081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 10:01:59 2006 New Revision: 26235 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/openmclinvoke.sh Log: use setf and not setq as recommended everywhere in lisp documentation, fix a typo Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 10:01:59 2006 @@ -26,7 +26,7 @@ return cl if is_on_path("openmcl"): if is_on_path("openmclinvoke.sh"): - return "sbclinvoke.sh" + return "openmclinvoke.sh" if is_on_path("clisp"): return "clisp" if is_on_path("lisp"): Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 10:01:59 2006 @@ -33,7 +33,7 @@ def op_same_as(self): target = self.str(self.result) origin = self.str(self.args[0]) - print "(setq %s %s)" % (target, origin) + print "(setf %s %s)" % (target, origin) binary_ops = { #"add": "+", @@ -57,17 +57,17 @@ s = self.str result, (arg1, arg2) = self.result, self.args cl_op = self.binary_ops[op] - print "(setq", s(result), "(", cl_op, s(arg1), s(arg2), "))" + print "(setf", s(result), "(", cl_op, s(arg1), s(arg2), "))" def op_contains(self): s = self.str result, (arg1, arg2) = self.result, self.args - print "(setq", s(result), "(not (not (find", s(arg2), s(arg1), "))))" + print "(setf", s(result), "(not (not (find", s(arg2), s(arg1), "))))" def op_add(self): s = self.str result, (arg1, arg2) = self.result, self.args - print "(setq", s(result) + print "(setf", s(result) table = { (int, int): "(+ %s %s)", (int, long): "(+ %s %s)", @@ -82,7 +82,7 @@ def op_not_(self): s = self.str result, (arg1,) = self.result, self.args - print "(setq", s(result), "(not" + print "(setf", s(result), "(not" table = { (bool,): "(not %s)", (int,): "(zerop %s)", @@ -93,7 +93,7 @@ print "))" def op_is_true(self, arg): - print "(setq", self.str(self.result) + print "(setf", self.str(self.result) table = { (bool,): "%s", (int,): "(not (zerop %s))", @@ -124,12 +124,12 @@ def op_new_list(self, cls): target = self.str(self.result) - print "(setq %s (make-array 0 :adjustable t))" % (target,) + print "(setf %s (make-array 0 :adjustable t))" % (target,) def op_new_instance(self, cls): print self.declare_class(cls) target = self.str(self.result) - print "(setq %s (make-%s))" % (target, cls._name) + print "(setf %s (make-%s))" % (target, cls._name) def op_oosend(self): method = self.args[0].value @@ -144,7 +144,7 @@ clsname = self.args[0].concretetype._name fieldname = self.args[1].value obj = self.str(self.args[0]) - print "(setq %s (%s-%s %s))" % (target, clsname, fieldname, obj) + print "(setf %s (%s-%s %s))" % (target, clsname, fieldname, obj) def op_oosetfield(self): target = self.str(self.result) @@ -158,14 +158,14 @@ def op_newtuple(self): s = self.str - print "(setq", s(self.result), "(list", + print "(setf", s(self.result), "(list", for arg in self.args: print s(arg), print "))" def op_newlist(self): s = self.str - print "(setq", s(self.result), "(vector", + print "(setf", s(self.result), "(vector", for arg in self.args: print s(arg), print "))" @@ -173,7 +173,7 @@ def op_alloc_and_set(self): s = self.str result, (size, init) = self.result, self.args - print "(setq", s(result), "(make-array", s(size), "))" + print "(setf", s(result), "(make-array", s(size), "))" print "(fill", s(result), s(init), ")" def op_setitem(self): @@ -184,14 +184,14 @@ def op_iter(self): s = self.str result, (seq,) = self.result, self.args - print "(setq", s(result), "(make-iterator", s(seq), "))" + print "(setf", s(result), "(make-iterator", s(seq), "))" def op_next(self): s = self.str result, (iterator,) = self.result, self.args print "(let ((result (funcall", s(iterator), ")))" - print " (setq", s(result), "(car result))" - print " (setq last-exc (cdr result)))" + print " (setf", s(result), "(car result))" + print " (setf last-exc (cdr result)))" builtin_map = { pow: "expt", @@ -209,7 +209,7 @@ return s = self.str args = self.args[1:] - print "(setq", s(self.result), "(", self.builtin_map[func], + print "(setf", s(self.result), "(", self.builtin_map[func], for arg in args: print s(arg), print "))" @@ -217,12 +217,12 @@ def op_getslice(self): s = self.str result, (seq, start, end) = self.result, self.args - print "(setq", s(result), "(python-slice", s(seq), s(start), s(end), "))" + print "(setf", s(result), "(python-slice", s(seq), s(start), s(end), "))" def op_pow(self): s = self.str result, (x,y,z) = self.result, self.args - print "(setq", s(result) + print "(setf", s(result) table = { (int, int, type(None)): (lambda args: args[:2], "(expt %s %s)"), } @@ -286,7 +286,7 @@ def emit_defun(self, fun): print ";;;; Main" - print "(defun", fun.name + print "(defun", fun.name, arglist = fun.getargs() print "(", for arg in arglist: @@ -307,11 +307,11 @@ else: print repr_var(var), print ")" - print "(setq last-exc nil)" + print "(setf last-exc nil)" for block in blocklist: + print self.emit_block(block) - print ")" - print ")" + print "))" def emit_block(self, block): self.cur_block = block @@ -354,7 +354,7 @@ print "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) else: retval = repr_var(block.inputargs[0]) - print "(return", retval, ")" + print "(return", retval, ")", def emit_jump(self, block): tag = self.blockref[block] @@ -363,10 +363,13 @@ def emit_link(self, link): source = map(repr_arg, link.args) target = map(repr_var, link.target.inputargs) - print "(psetq", # parallel assignment - for s, t in zip(source, target): + print "(setf", + couples = zip(source, target) + for s, t in couples[:-1]: print t, s - print ")" + else: + s, t = couples[-1] + print t, s, ")" self.emit_jump(link.target) typemap = { Modified: pypy/dist/pypy/translator/cl/openmclinvoke.sh ============================================================================== --- pypy/dist/pypy/translator/cl/openmclinvoke.sh (original) +++ pypy/dist/pypy/translator/cl/openmclinvoke.sh Mon Apr 24 10:01:59 2006 @@ -1,3 +1,3 @@ #!/bin/sh # Copy this to your PATH -openmcl --batch --load $1 +openmcl --batch --quiet --load $1 From sanxiyn at codespeak.net Mon Apr 24 10:20:07 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 10:20:07 +0200 (CEST) Subject: [pypy-svn] r26236 - pypy/tag/before-ootype-cl Message-ID: <20060424082007.9568610081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 10:20:03 2006 New Revision: 26236 Added: pypy/tag/before-ootype-cl/ - copied from r26235, pypy/dist/pypy/translator/cl/ Log: Tag CL backend From arigo at codespeak.net Mon Apr 24 10:27:19 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 24 Apr 2006 10:27:19 +0200 (CEST) Subject: [pypy-svn] r26237 - pypy/dist/pypy/objspace/cpy/test Message-ID: <20060424082719.283F210081@code0.codespeak.net> Author: arigo Date: Mon Apr 24 10:27:17 2006 New Revision: 26237 Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py Log: Added skipped tests: the next goals Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Mon Apr 24 10:27:17 2006 @@ -1,7 +1,12 @@ +import py from pypy.translator.c.test.test_genc import compile +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext, graphof from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy from pypy.objspace.cpy.objspace import CPyObjSpace import pypy.rpython.rctypes.implementation +from pypy.interpreter.function import BuiltinFunction +from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root def test_demo(): @@ -16,3 +21,40 @@ res = fn(10, long) assert isinstance(res, int) + +# ____________________________________________________________ + +def myfunc(space, w_x): + x = space.int_w(w_x) + result = x * 7 + return space.wrap(result) +myfunc.unwrap_spec = [ObjSpace, W_Root] + +def maketest(): + space = CPyObjSpace() + func = interp2app(myfunc).__spacebind__(space) + bltin = BuiltinFunction(func) + w_myfunc = space.wrap(bltin) + def entrypoint(n): + w_result = space.call_function(w_myfunc, space.wrap(n)) + return space.int_w(w_result) + return entrypoint + +def test_annotate(): + py.test.skip("in-progress") + entrypoint = maketest() + t = TranslationContext() + a = t.buildannotator(policy=PyPyAnnotatorPolicy()) + s = a.build_types(entrypoint, [int]) + assert s.knowntype == int + graph = graphof(t, myfunc) + assert len(graph.inputargs()) == 2 + s = a.getbinding(graph.inputargs()[1]) + assert s.knowntype == CPyObjSpace.W_Object + +def test_compile(): + py.test.skip("in-progress") + entrypoint = maketest() + fn = compile(entrypoint, [int], annotatorpolicy=PyPyAnnotatorPolicy()) + res = fn(-6) + assert res == -42 From arigo at codespeak.net Mon Apr 24 10:27:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 24 Apr 2006 10:27:46 +0200 (CEST) Subject: [pypy-svn] r26238 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424082746.2280410084@code0.codespeak.net> Author: arigo Date: Mon Apr 24 10:27:44 2006 New Revision: 26238 Modified: pypy/dist/pypy/translator/cl/ (props changed) pypy/dist/pypy/translator/cl/__init__.py (props changed) pypy/dist/pypy/translator/cl/clrepr.py (props changed) pypy/dist/pypy/translator/cl/test/ (props changed) pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (props changed) Log: fixeol From mwh at codespeak.net Mon Apr 24 10:34:39 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 24 Apr 2006 10:34:39 +0200 (CEST) Subject: [pypy-svn] r26239 - pypy/dist/pypy/translator/stackless Message-ID: <20060424083439.5C90110081@code0.codespeak.net> Author: mwh Date: Mon Apr 24 10:34:38 2006 New Revision: 26239 Modified: pypy/dist/pypy/translator/stackless/transform.py Log: don't make a functionptr out of add_frame_state's graph until i'm sure it's been rtyped. this fixes the failures in test_transform.py. Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 24 10:34:38 2006 @@ -100,9 +100,6 @@ ADD_FRAME_STATE_TYPE = lltype.FuncType( [self.unwind_exception_type, lltype.Ptr(STATE_HEADER)], lltype.Void) - self.add_frame_state_ptr = model.Constant( - getfunctionptr(add_frame_state_graph), - lltype.Ptr(ADD_FRAME_STATE_TYPE)) RESUME_STATE_TYPE = lltype.FuncType([], lltype.Signed) resume_state_graph = mixlevelannotator.getgraph( @@ -154,6 +151,10 @@ mixlevelannotator.finish() + self.add_frame_state_ptr = model.Constant( + getfunctionptr(add_frame_state_graph), + lltype.Ptr(ADD_FRAME_STATE_TYPE)) + s_global_state = bk.immutablevalue(code.global_state) r_global_state = translator.rtyper.getrepr(s_global_state) self.ll_global_state = model.Constant( From arigo at codespeak.net Mon Apr 24 10:40:08 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 24 Apr 2006 10:40:08 +0200 (CEST) Subject: [pypy-svn] r26240 - in pypy/dist/pypy/objspace/cpy: . test Message-ID: <20060424084008.BEC9510081@code0.codespeak.net> Author: arigo Date: Mon Apr 24 10:40:07 2006 New Revision: 26240 Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/objspace/cpy/wrappable.py Log: Annotate built-in functions via a bookkeeper.emulate_pbc_call() that is triggered when the annotator sees the W_Object instance. The trick is done with the extregistry. It's not really obvious what is going on when, I admit... Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Mon Apr 24 10:40:07 2006 @@ -40,19 +40,18 @@ return space.int_w(w_result) return entrypoint -def test_annotate(): - py.test.skip("in-progress") +def test_annotate_bltinfunc(): entrypoint = maketest() t = TranslationContext() a = t.buildannotator(policy=PyPyAnnotatorPolicy()) s = a.build_types(entrypoint, [int]) assert s.knowntype == int graph = graphof(t, myfunc) - assert len(graph.inputargs()) == 2 - s = a.getbinding(graph.inputargs()[1]) + assert len(graph.getargs()) == 2 + s = a.binding(graph.getargs()[1]) assert s.knowntype == CPyObjSpace.W_Object -def test_compile(): +def test_compile_bltinfunc(): py.test.skip("in-progress") entrypoint = maketest() fn = compile(entrypoint, [int], annotatorpolicy=PyPyAnnotatorPolicy()) Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Mon Apr 24 10:40:07 2006 @@ -9,6 +9,7 @@ from pypy.objspace.cpy.objspace import CPyObjSpace from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root +from pypy.rpython import extregistry class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): @@ -29,4 +30,28 @@ w_result = bltin(space, *args_w) return space.unwrap(w_result) - return W_Object(trampoline) + w_result = W_Object(trampoline) + + # override the annotation behavior of 'w_result' + # to emulate a call to the bltin function at interp-level + BaseEntry = extregistry._lookup_cls(w_result) + uniquekey = bltin + nb_args = len(unwrap_spec) - 1 + + class TrampolineEntry(BaseEntry): + _about_ = w_result + def compute_annotation(self): + from pypy.annotation.bookkeeper import getbookkeeper + bookkeeper = getbookkeeper() + s_bltin = bookkeeper.immutablevalue(bltin) + s_space = bookkeeper.immutablevalue(space) + s_w_obj = bookkeeper.valueoftype(W_Object) + args_s = [s_space] + [s_w_obj]*nb_args + s_result = bookkeeper.emulate_pbc_call(uniquekey, s_bltin, + args_s) + assert s_w_obj.contains(s_result), ( + "%r should return a wrapped obj, got %r instead" % ( + bltin, s_result)) + return super(TrampolineEntry, self).compute_annotation() + + return w_result From nik at codespeak.net Mon Apr 24 10:50:56 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Mon, 24 Apr 2006 10:50:56 +0200 (CEST) Subject: [pypy-svn] r26241 - pypy/dist/pypy/translator/cl Message-ID: <20060424085056.6532210087@code0.codespeak.net> Author: nik Date: Mon Apr 24 10:50:51 2006 New Revision: 26241 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/gencl.py Log: (sanxiyn, nik) get rid of prints and stdout capturing for code generation (we can use pdb again, yay!). delete a lot of code that is not valid anymore for specialized graphs. the code before this revision is tagged at http://codespeak.net/svn/pypy/tag/before-ootype-cl Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 10:50:51 2006 @@ -91,7 +91,7 @@ graph = t.graphs[0] gen = GenCL(graph, argtypes) - out = gen.globaldeclarations() + '\n' + gen.emitcode() + out = gen.emitcode() i = 1 fpath = path.join("%s.lisp" % graph.name) def _(*args): Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 10:50:51 2006 @@ -18,22 +18,19 @@ self.args = op.args self.result = op.result - def __call__(self): + def __iter__(self): if self.opname in self.binary_ops: - self.op_binary(self.opname) + for line in self.op_binary(self.opname): + yield line else: - default = self.op_default - meth = getattr(self, "op_" + self.opname, default) - meth() - - def op_default(self): - print ";;", self.op - print ";; Op", self.opname, "is missing" + meth = getattr(self, "op_" + self.opname) + for line in meth(): + yield line def op_same_as(self): target = self.str(self.result) origin = self.str(self.args[0]) - print "(setf %s %s)" % (target, origin) + yield "(setf %s %s)" % (target, origin) binary_ops = { #"add": "+", @@ -57,54 +54,12 @@ s = self.str result, (arg1, arg2) = self.result, self.args cl_op = self.binary_ops[op] - print "(setf", s(result), "(", cl_op, s(arg1), s(arg2), "))" + yield "(setf %s (%s %s %s))" % (s(result), cl_op, s(arg1), s(arg2)) - def op_contains(self): - s = self.str - result, (arg1, arg2) = self.result, self.args - print "(setf", s(result), "(not (not (find", s(arg2), s(arg1), "))))" - - def op_add(self): - s = self.str - result, (arg1, arg2) = self.result, self.args - print "(setf", s(result) - table = { - (int, int): "(+ %s %s)", - (int, long): "(+ %s %s)", - (long, int): "(+ %s %s)", - (long, long): "(+ %s %s)", - (str, str): "(concatenate 'string %s %s)", - (list, list): "(concatenate 'vector %s %s)", - } - self.gen.emit_typecase(table, arg1, arg2) - print ")" - - def op_not_(self): - s = self.str - result, (arg1,) = self.result, self.args - print "(setf", s(result), "(not" - table = { - (bool,): "(not %s)", - (int,): "(zerop %s)", - (long,): "(zerop %s)", - (list,): "(zerop (length %s))", - } - self.gen.emit_typecase(table, arg1) - print "))" - - def op_is_true(self, arg): - print "(setf", self.str(self.result) - table = { - (bool,): "%s", - (int,): "(not (zerop %s))", - (long,): "(not (zerop %s))", - (list,): "(not (zerop (length %s)))", - } - self.gen.emit_typecase(table, arg) - print ")" - def op_int_is_true(self): - self.op_is_true(self.args[0]) + target = self.str(self.result) + arg = self.str(self.args[0]) + yield "(setf %s (not (zerop %s)))" % (target, arg) def declare_class(self, cls): # cls is really type of Instance @@ -118,18 +73,20 @@ def op_new(self): cls = self.args[0].value if isinstance(cls, List): - self.op_new_list(cls) + generator = self.op_new_list else: - self.op_new_instance(cls) + generator = self.op_new_instance + for line in generator(cls): + yield line def op_new_list(self, cls): target = self.str(self.result) - print "(setf %s (make-array 0 :adjustable t))" % (target,) + yield "(setf %s (make-array 0 :adjustable t))" % (target,) def op_new_instance(self, cls): - print self.declare_class(cls) + yield self.declare_class(cls) target = self.str(self.result) - print "(setf %s (make-%s))" % (target, cls._name) + yield "(setf %s (make-%s))" % (target, cls._name) def op_oosend(self): method = self.args[0].value @@ -144,90 +101,17 @@ clsname = self.args[0].concretetype._name fieldname = self.args[1].value obj = self.str(self.args[0]) - print "(setf %s (%s-%s %s))" % (target, clsname, fieldname, obj) + yield "(setf %s (%s-%s %s))" % (target, clsname, fieldname, obj) def op_oosetfield(self): target = self.str(self.result) clsname = self.args[0].concretetype._name fieldname = self.args[1].value if fieldname == "meta": # XXX - return + raise StopIteration obj = self.str(self.args[0]) fieldvalue = self.str(self.args[2]) - print "(setf (%s-%s %s) %s)" % (clsname, fieldname, obj, fieldvalue) - - def op_newtuple(self): - s = self.str - print "(setf", s(self.result), "(list", - for arg in self.args: - print s(arg), - print "))" - - def op_newlist(self): - s = self.str - print "(setf", s(self.result), "(vector", - for arg in self.args: - print s(arg), - print "))" - - def op_alloc_and_set(self): - s = self.str - result, (size, init) = self.result, self.args - print "(setf", s(result), "(make-array", s(size), "))" - print "(fill", s(result), s(init), ")" - - def op_setitem(self): - s = self.str - (seq, index, element) = self.args - print "(setf (elt", s(seq), s(index), ")", s(element), ")" - - def op_iter(self): - s = self.str - result, (seq,) = self.result, self.args - print "(setf", s(result), "(make-iterator", s(seq), "))" - - def op_next(self): - s = self.str - result, (iterator,) = self.result, self.args - print "(let ((result (funcall", s(iterator), ")))" - print " (setf", s(result), "(car result))" - print " (setf last-exc (cdr result)))" - - builtin_map = { - pow: "expt", - range: "python-range", - } - - def op_simple_call(self): - func = self.args[0] - if not isinstance(func, Constant): - self.op_default() - return - func = func.value - if func not in self.builtin_map: - self.op_default() - return - s = self.str - args = self.args[1:] - print "(setf", s(self.result), "(", self.builtin_map[func], - for arg in args: - print s(arg), - print "))" - - def op_getslice(self): - s = self.str - result, (seq, start, end) = self.result, self.args - print "(setf", s(result), "(python-slice", s(seq), s(start), s(end), "))" - - def op_pow(self): - s = self.str - result, (x,y,z) = self.result, self.args - print "(setf", s(result) - table = { - (int, int, type(None)): (lambda args: args[:2], "(expt %s %s)"), - } - self.gen.emit_typecase(table, x, y, z) - print ")" + yield "(setf (%s-%s %s) %s)" % (clsname, fieldname, obj, fieldvalue) class ListImpl: @@ -272,27 +156,22 @@ return var.concretetype def emitcode(self, public=True): - import sys - from cStringIO import StringIO - out = StringIO() - oldstdout = sys.stdout - sys.stdout = out - self.emit() - sys.stdout = oldstdout - return out.getvalue() + code = "\n".join(list(self.emit())) + return code def emit(self): - self.emit_defun(self.fun) + for line in self.emit_defun(self.fun): + yield line def emit_defun(self, fun): - print ";;;; Main" - print "(defun", fun.name, + yield ";;;; Main" + yield "(defun " + fun.name arglist = fun.getargs() - print "(", + yield "(" for arg in arglist: - print repr_var(arg), - print ")" - print "(prog" + yield repr_var(arg) + yield ")" + yield "(prog" blocklist = list(fun.iterblocks()) vardict = {} for block in blocklist: @@ -300,130 +179,82 @@ self.blockref[block] = tag for var in block.getvariables(): vardict[var] = self.get_type(var) - print "( last-exc", + yield "( last-exc" for var in vardict: if var in arglist: - print "(", repr_var(var), repr_var(var), ")", + yield "(%s %s)" % (repr_var(var), repr_var(var)) else: - print repr_var(var), - print ")" - print "(setf last-exc nil)" + yield repr_var(var) + yield ")" + yield "(setf last-exc nil)" for block in blocklist: - print - self.emit_block(block) - print "))" + yield "" + for line in self.emit_block(block): + yield line + yield "))" def emit_block(self, block): self.cur_block = block tag = self.blockref[block] - print "tag" + str(tag) + yield "tag" + str(tag) for op in block.operations: emit_op = Op(self, op) - emit_op() + for line in emit_op: + yield line exits = block.exits if len(exits) == 1: - self.emit_link(exits[0]) + for line in self.emit_link(exits[0]): + yield line elif len(exits) > 1: # only works in the current special case if (len(exits) == 2 and exits[0].exitcase == False and exits[1].exitcase == True): - print "(if", repr_arg(block.exitswitch) - print "(progn" - self.emit_link(exits[1]) - print ") ; else" - print "(progn" - self.emit_link(exits[0]) - print "))" + yield "(if " + repr_arg(block.exitswitch) + yield "(progn" + for line in self.emit_link(exits[1]): + yield line + yield ") ; else" + yield "(progn" + for line in self.emit_link(exits[0]): + yield line + yield "))" else: # this is for the more general case. The previous special case # shouldn't be needed but in Python 2.2 we can't tell apart # 0 vs nil and 1 vs t :-( for exit in exits[:-1]: - print "(if (equalp", repr_arg(block.exitswitch), - print repr_const(exit.exitcase), ')' - print "(progn" - self.emit_link(exit) - print ")" - print "(progn ; else should be", repr_const(exits[-1].exitcase) - self.emit_link(exits[-1]) - print ")" * len(exits) + yield "(if (equalp " + repr_arg(block.exitswitch) + yield repr_const(exit.exitcase) + ')' + yield "(progn" + for line in self.emit_link(exit): + yield line + yield ")" + yield "(progn ; else should be %s" % repr_const(exits[-1].exitcase) + for line in self.emit_link(exits[-1]): + yield line + yield ")" * len(exits) elif len(block.inputargs) == 2: # exc_cls, exc_value exc_cls = repr_var(block.inputargs[0]) exc_value = repr_var(block.inputargs[1]) - print "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) + yield "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) else: retval = repr_var(block.inputargs[0]) - print "(return", retval, ")", + yield "(return %s )" % retval - def emit_jump(self, block): + def format_jump(self, block): tag = self.blockref[block] - print "(go", "tag" + str(tag), ")" + return "(go tag" + str(tag) + ")" def emit_link(self, link): source = map(repr_arg, link.args) target = map(repr_var, link.target.inputargs) - print "(setf", + yield "(setf" couples = zip(source, target) for s, t in couples[:-1]: - print t, s + yield "%s %s" % (t, s) else: s, t = couples[-1] - print t, s, ")" - self.emit_jump(link.target) + yield "%s %s)" % (t, s) + yield self.format_jump(link.target) - typemap = { - bool: "boolean", - int: "fixnum", - long: "bignum", - str: "string", - list: "vector", - } - - def emit_typecase(self, table, *args): - argreprs = tuple(map(repr_arg, args)) - argtypes = tuple(map(self.get_type, args)) - if argtypes in table: - trans = table[argtypes] - if isinstance(trans, tuple): # (manip-args, code-template) - manip, trans = trans - argreprs = manip(argreprs) - print trans % argreprs - else: - print "(cond" - for argtypes in table: - print "((and", - for tp, s in zip(argtypes, argreprs): - cl_tp = "'" + self.typemap[tp] - print "(typep", s, cl_tp, ")", - print ")" - trans = table[argtypes] - print trans % argreprs, - print ")" - print ")" - - def globaldeclarations(self): - return prelude - - -prelude = """;;;; Prelude -(defun make-iterator (seq) - (let ((i 0)) - (lambda () - (if (< i (length seq)) - (let ((v (elt seq i))) (incf i) (cons v nil)) - (cons nil 'StopIteration))))) -(defun python-slice (seq start end) - (let ((l (length seq))) - (if (not start) (setf start 0)) - (if (not end) (setf end l)) - (if (minusp start) (incf start l)) - (if (minusp end) (incf end l)) - (subseq seq start end))) -;; temporary -(defun python-range (end) - (let ((a (make-array end))) - (loop for i below end - do (setf (elt a i) i) - finally (return a)))) -""" From mwh at codespeak.net Mon Apr 24 10:58:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 24 Apr 2006 10:58:53 +0200 (CEST) Subject: [pypy-svn] r26242 - pypy/dist/pypy/translator/stackless Message-ID: <20060424085853.23B1310084@code0.codespeak.net> Author: mwh Date: Mon Apr 24 10:58:52 2006 New Revision: 26242 Modified: pypy/dist/pypy/translator/stackless/transform.py Log: a long-needed refactoring of how the stackless transform makes function pointers out of it's helper functions. Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 24 10:58:52 2006 @@ -90,70 +90,44 @@ mixlevelannotator = MixLevelHelperAnnotator(translator.rtyper) l2a = annmodel.lltype_to_annotation - annotations = [ - annmodel.SomeInstance(bk.getuniqueclassdef(code.UnwindException)), - annmodel.SomePtr(lltype.Ptr(STATE_HEADER))] - + unwinddef = bk.getuniqueclassdef(code.UnwindException) add_frame_state_graph = mixlevelannotator.getgraph( code.add_frame_state, - annotations, l2a(lltype.Void)) - ADD_FRAME_STATE_TYPE = lltype.FuncType( - [self.unwind_exception_type, lltype.Ptr(STATE_HEADER)], - lltype.Void) + [annmodel.SomeInstance(unwinddef), + annmodel.SomePtr(lltype.Ptr(STATE_HEADER))], + l2a(lltype.Void)) - RESUME_STATE_TYPE = lltype.FuncType([], lltype.Signed) resume_state_graph = mixlevelannotator.getgraph( code.resume_state, [], annmodel.SomeInteger()) - self.resume_state_ptr = model.Constant(lltype.functionptr( - RESUME_STATE_TYPE, "resume_state", - graph=resume_state_graph), - lltype.Ptr(RESUME_STATE_TYPE)) - FETCH_RETVAL_VOID_TYPE = lltype.FuncType([], lltype.Void) fetch_retval_void_graph = mixlevelannotator.getgraph( code.fetch_retval_void, [], annmodel.s_None) - self.fetch_retval_void_ptr = model.Constant(lltype.functionptr( - FETCH_RETVAL_VOID_TYPE, "fetch_retval_void", - graph=fetch_retval_void_graph), - lltype.Ptr(FETCH_RETVAL_VOID_TYPE)) - - FETCH_RETVAL_LONG_TYPE = lltype.FuncType([], lltype.Signed) fetch_retval_long_graph = mixlevelannotator.getgraph( code.fetch_retval_long, [], annmodel.SomeInteger()) - self.fetch_retval_long_ptr = model.Constant(lltype.functionptr( - FETCH_RETVAL_LONG_TYPE, "fetch_retval_long", - graph=fetch_retval_long_graph), - lltype.Ptr(FETCH_RETVAL_LONG_TYPE)) - - FETCH_RETVAL_LONGLONG_TYPE = lltype.FuncType([], lltype.Signed) fetch_retval_longlong_graph = mixlevelannotator.getgraph( # WAA! code.fetch_retval_longlong, [], annmodel.SomeInteger(size=2)) - self.fetch_retval_longlong_ptr = model.Constant(lltype.functionptr( - FETCH_RETVAL_LONGLONG_TYPE, "fetch_retval_longlong", - graph=fetch_retval_longlong_graph), - lltype.Ptr(FETCH_RETVAL_LONGLONG_TYPE)) - - FETCH_RETVAL_FLOAT_TYPE = lltype.FuncType([], lltype.Float) fetch_retval_float_graph = mixlevelannotator.getgraph( code.fetch_retval_float, [], annmodel.SomeFloat()) - self.fetch_retval_float_ptr = model.Constant(lltype.functionptr( - FETCH_RETVAL_FLOAT_TYPE, "fetch_retval_float", - graph=fetch_retval_float_graph), - lltype.Ptr(FETCH_RETVAL_FLOAT_TYPE)) - - FETCH_RETVAL_VOID_P_TYPE = lltype.FuncType([], llmemory.Address) fetch_retval_void_p_graph = mixlevelannotator.getgraph( code.fetch_retval_void_p, [], annmodel.SomeAddress()) - self.fetch_retval_void_p_ptr = model.Constant(lltype.functionptr( - FETCH_RETVAL_VOID_P_TYPE, "fetch_retval_void_p", - graph=fetch_retval_void_p_graph), - lltype.Ptr(FETCH_RETVAL_VOID_P_TYPE)) mixlevelannotator.finish() - self.add_frame_state_ptr = model.Constant( - getfunctionptr(add_frame_state_graph), - lltype.Ptr(ADD_FRAME_STATE_TYPE)) + def fptr(graph): + FTYPE = lltype.FuncType( + [v.concretetype for v in graph.startblock.inputargs], + graph.returnblock.inputargs[0].concretetype) + return model.Constant(getfunctionptr(graph), lltype.Ptr(FTYPE)) + + self.add_frame_state_ptr = fptr(add_frame_state_graph) + + self.resume_state_ptr = fptr(resume_state_graph) + + self.fetch_retval_void_ptr = fptr(fetch_retval_void_graph) + self.fetch_retval_long_ptr = fptr(fetch_retval_long_graph) + self.fetch_retval_longlong_ptr = fptr(fetch_retval_longlong_graph) + self.fetch_retval_float_ptr = fptr(fetch_retval_float_graph) + self.fetch_retval_void_p_ptr = fptr(fetch_retval_void_p_graph) s_global_state = bk.immutablevalue(code.global_state) r_global_state = translator.rtyper.getrepr(s_global_state) From sanxiyn at codespeak.net Mon Apr 24 11:01:53 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 11:01:53 +0200 (CEST) Subject: [pypy-svn] r26243 - pypy/dist/pypy/translator/cl Message-ID: <20060424090153.00A6810084@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 11:01:51 2006 New Revision: 26243 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Use isdigit() string method in buildcl Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 11:01:51 2006 @@ -47,7 +47,6 @@ lines = [ line for line in lines if line and not line.startswith(';') ] assert len(lines) == 1 s = lines[0] - import string s = s.strip() if s == "T": return True @@ -55,7 +54,7 @@ return False elif s[0] == '"': return s[1:-1] - elif s.strip(string.digits) == '': + elif s.isdigit(): return int(s) else: return Literal(s) From bea at codespeak.net Mon Apr 24 11:06:59 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 24 Apr 2006 11:06:59 +0200 (CEST) Subject: [pypy-svn] r26244 - pypy/extradoc/talk Message-ID: <20060424090659.49A4610084@code0.codespeak.net> Author: bea Date: Mon Apr 24 11:06:53 2006 New Revision: 26244 Added: pypy/extradoc/talk/pypypaper_xp2006_final.pdf (contents, props changed) Log: the final original version for the xp 2006 paper Added: pypy/extradoc/talk/pypypaper_xp2006_final.pdf ============================================================================== Binary file. No diff available. From sanxiyn at codespeak.net Mon Apr 24 11:08:37 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 11:08:37 +0200 (CEST) Subject: [pypy-svn] r26245 - pypy/dist/pypy/translator/cl Message-ID: <20060424090837.955E210084@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 11:08:34 2006 New Revision: 26245 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (nik, sanxiyn) Record class declarations and emit it before the code Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 11:08:34 2006 @@ -71,22 +71,14 @@ return class_declaration def op_new(self): + target = self.str(self.result) cls = self.args[0].value if isinstance(cls, List): - generator = self.op_new_list + yield "(setf %s (make-array 0 :adjustable t))" % (target,) else: - generator = self.op_new_instance - for line in generator(cls): - yield line - - def op_new_list(self, cls): - target = self.str(self.result) - yield "(setf %s (make-array 0 :adjustable t))" % (target,) - - def op_new_instance(self, cls): - yield self.declare_class(cls) - target = self.str(self.result) - yield "(setf %s (make-%s))" % (target, cls._name) + declaration = self.declare_class(cls) + self.gen.declarations.append(declaration) + yield "(setf %s (make-%s))" % (target, cls._name) def op_oosend(self): method = self.args[0].value @@ -142,6 +134,7 @@ simplify_graph(fun) self.fun = fun self.blockref = {} + self.declarations = [] def annotate(self, input_arg_types): ann = RPythonAnnotator() @@ -156,8 +149,10 @@ return var.concretetype def emitcode(self, public=True): - code = "\n".join(list(self.emit())) - return code + lines = list(self.emit()) + declarations = "\n".join(self.declarations) + code = "\n".join(lines) + return declarations + "\n" + code def emit(self): for line in self.emit_defun(self.fun): From sanxiyn at codespeak.net Mon Apr 24 11:30:03 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 11:30:03 +0200 (CEST) Subject: [pypy-svn] r26246 - pypy/dist/pypy/translator/cl Message-ID: <20060424093003.0A6831007D@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 11:30:00 2006 New Revision: 26246 Modified: pypy/dist/pypy/translator/cl/clrepr.py Log: Remove obsolete stuffs from clrepr Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 11:30:00 2006 @@ -24,12 +24,6 @@ val.replace("\"", "\\\"") val = '"' + val + '"' return val - elif isinstance(val, type(Exception)) and issubclass(val, Exception): - return "'%s" % val.__name__ - elif val is last_exception: - return "last-exc" - elif val is last_exc_value: - return "'last-exc-value" else: return repr_unknown(val) From sanxiyn at codespeak.net Mon Apr 24 11:32:28 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 11:32:28 +0200 (CEST) Subject: [pypy-svn] r26247 - pypy/dist/pypy/translator/cl Message-ID: <20060424093228.9D8551007D@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 11:32:25 2006 New Revision: 26247 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (nik, sanxiyn) Refactor Op class to pass string version of arguments to methods Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 11:32:25 2006 @@ -24,13 +24,13 @@ yield line else: meth = getattr(self, "op_" + self.opname) - for line in meth(): + result = repr_arg(self.result) + args = map(repr_arg, self.args) + for line in meth(result, *args): yield line - def op_same_as(self): - target = self.str(self.result) - origin = self.str(self.args[0]) - yield "(setf %s %s)" % (target, origin) + def op_same_as(self, result, arg): + yield "(setf %s %s)" % (result, arg) binary_ops = { #"add": "+", @@ -56,10 +56,8 @@ cl_op = self.binary_ops[op] yield "(setf %s (%s %s %s))" % (s(result), cl_op, s(arg1), s(arg2)) - def op_int_is_true(self): - target = self.str(self.result) - arg = self.str(self.args[0]) - yield "(setf %s (not (zerop %s)))" % (target, arg) + def op_int_is_true(self, result, arg): + yield "(setf %s (not (zerop %s)))" % (result, arg) def declare_class(self, cls): # cls is really type of Instance @@ -70,40 +68,35 @@ class_declaration = "(defstruct %s %s)" % (name, field_declaration) return class_declaration - def op_new(self): - target = self.str(self.result) + def op_new(self, result, _): cls = self.args[0].value if isinstance(cls, List): - yield "(setf %s (make-array 0 :adjustable t))" % (target,) + yield "(setf %s (make-array 0 :adjustable t))" % (result,) else: declaration = self.declare_class(cls) self.gen.declarations.append(declaration) - yield "(setf %s (make-%s))" % (target, cls._name) + yield "(setf %s (make-%s))" % (result, cls._name) - def op_oosend(self): + def op_oosend(self, result, *ignore): method = self.args[0].value receiver = self.args[1] args = self.args[2:] if isinstance(receiver.concretetype, List): impl = ListImpl(receiver) - getattr(impl, method)(*args) + code = getattr(impl, method)(*args) + yield "(setf %s %s)" % (result, code) - def op_oogetfield(self): - target = self.str(self.result) + def op_oogetfield(self, result, obj, _): clsname = self.args[0].concretetype._name fieldname = self.args[1].value - obj = self.str(self.args[0]) - yield "(setf %s (%s-%s %s))" % (target, clsname, fieldname, obj) + yield "(setf %s (%s-%s %s))" % (result, clsname, fieldname, obj) - def op_oosetfield(self): - target = self.str(self.result) + def op_oosetfield(self, result, obj, _, value): clsname = self.args[0].concretetype._name fieldname = self.args[1].value if fieldname == "meta": # XXX raise StopIteration - obj = self.str(self.args[0]) - fieldvalue = self.str(self.args[2]) - yield "(setf (%s-%s %s) %s)" % (clsname, fieldname, obj, fieldvalue) + yield "(setf (%s-%s %s) %s)" % (clsname, fieldname, obj, value) class ListImpl: From ac at codespeak.net Mon Apr 24 11:43:56 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 11:43:56 +0200 (CEST) Subject: [pypy-svn] r26248 - in pypy/dist/pypy/rpython/rctypes/tool: . test Message-ID: <20060424094356.0FDF21007C@code0.codespeak.net> Author: ac Date: Mon Apr 24 11:43:55 2006 New Revision: 26248 Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py Log: (aleale, arre) Support nested structures. (Needs refactoring later). Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Mon Apr 24 11:43:55 2006 @@ -37,6 +37,26 @@ # # General interface +class ConfigResult: + def __init__(self, CConfig, info, entries): + self.CConfig = CConfig + self.result = {} + self.info = info + self.entries = entries + + def get_entry_result(self, entry): + try: + return self.result[entry] + except KeyError: + pass + name = self.entries[entry] + info = self.info[name] + self.result[entry] = entry.build_result(info, self) + + def get_result(self): + return dict([(name, self.result[entry]) + for entry, name in self.entries.iteritems()]) + def configure(CConfig): """Examine the local system by running the C compiler. The CConfig class contains CConfigEntry attribues that describe @@ -77,10 +97,16 @@ infolist = list(run_example_code(filepath, include_dirs)) assert len(infolist) == len(entries) - result = {} + resultinfo = {} + resultentries = {} for info, (key, entry) in zip(infolist, entries): - result[key] = entry.build_result(info) - return result + resultinfo[key] = info + resultentries[entry] = key + + result = ConfigResult(CConfig, resultinfo, resultentries) + for name, entry in entries: + result.get_entry_result(entry) + return result.get_result() # ____________________________________________________________ @@ -119,23 +145,30 @@ yield 'dump("fldunsigned %s", s.%s > 0);' % (fieldname, fieldname) - def build_result(self, info): + def build_result(self, info, config_result): alignment = 1 layout = [None] * info['size'] for fieldname, fieldtype in self.interesting_fields: - offset = info['fldofs ' + fieldname] - size = info['fldsize ' + fieldname] - sign = info.get('fldunsigned ' + fieldname, False) - if (size, sign) != size_and_sign(fieldtype): - fieldtype = fixup_ctype(fieldtype, fieldname, (size, sign)) - layout_addfield(layout, offset, fieldtype, fieldname) - alignment = max(alignment, ctypes.alignment(fieldtype)) + if isinstance(fieldtype, Struct): + offset = info['fldofs ' + fieldname] + size = info['fldsize ' + fieldname] + c_fieldtype = config_result.get_entry_result(fieldtype) + layout_addfield(layout, offset, c_fieldtype, fieldname) + alignment = max(alignment, ctype_alignment(c_fieldtype)) + else: + offset = info['fldofs ' + fieldname] + size = info['fldsize ' + fieldname] + sign = info.get('fldunsigned ' + fieldname, False) + if (size, sign) != size_and_sign(fieldtype): + fieldtype = fixup_ctype(fieldtype, fieldname, (size, sign)) + layout_addfield(layout, offset, fieldtype, fieldname) + alignment = max(alignment, ctype_alignment(fieldtype)) # try to enforce the same alignment as the one of the original # structure if alignment < info['align']: choices = [ctype for ctype in alignment_types - if ctypes.alignment(ctype) == info['align']] + if ctype_alignment(ctype) == info['align']] assert choices, "unsupported alignment %d" % (info['align'],) choices = [(ctypes.sizeof(ctype), i, ctype) for i, ctype in enumerate(choices)] @@ -190,7 +223,7 @@ yield 'x = 0; x = ~x;' yield 'dump("unsigned", x > 0);' - def build_result(self, info): + def build_result(self, info, config_result): size = info['size'] sign = info.get('unsigned', False) ctype = self.ctype_hint @@ -216,9 +249,36 @@ yield ' printf("value: %llu\\n", x);' yield '}' - def build_result(self, info): + def build_result(self, info, config_result): return info['value'] +class DefinedConstantInteger(CConfigEntry): + """An entry in a CConfig class that stands for an externally + defined integer constant. If not #defined the value will be None. + """ + def __init__(self, macro): + self.name = self.macro = macro + + def prepare_code(self): + yield '#ifdef %s' % self.macro + yield 'dump("defined", 1);' + yield 'if ((%s) < 0) {' % (self.macro,) + yield ' long long x = (long long)(%s);' % (self.macro,) + yield ' printf("value: %lld\\n", x);' + yield '} else {' + yield ' unsigned long long x = (unsigned long long)(%s);' % ( + self.macro,) + yield ' printf("value: %llu\\n", x);' + yield '}' + yield '#else' + yield 'dump("defined", 0);' + yield '#endif' + + def build_result(self, info, config_result): + if info["defined"]: + return info['value'] + return None + class Defined(CConfigEntry): """A boolean, corresponding to an #ifdef. @@ -234,13 +294,20 @@ yield 'dump("defined", 0);' yield '#endif' - def build_result(self, info): + def build_result(self, info, config_result): return bool(info['defined']) # ____________________________________________________________ # # internal helpers +def ctype_alignment(c_type): + if issubclass(c_type, ctypes.Structure): + return max([ctype_alignment(fld_type) + for fld_name, fld_type in c_type._fields_]) + + return ctypes.alignment(c_type) + def uniquefilepath(LAST=[0]): i = LAST[0] LAST[0] += 1 Modified: pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/test/test_ctypes_platform.py Mon Apr 24 11:43:55 2006 @@ -124,3 +124,25 @@ assert res == {'FILE': res['FILE'], 'ushort': ctypes.c_ushort, 'XYZZY': 42} +def test_nested_structs(): + class CConfig: + _header_ = """ +struct x { + int foo; + unsigned long bar; + }; +struct y { + char c; + struct x x; + }; +""" + x = ctypes_platform.Struct("struct x", [("bar", ctypes.c_short)]) + y = ctypes_platform.Struct("struct y", [("x", x)]) + + res = ctypes_platform.configure(CConfig) + c_x = res["x"] + c_y = res["y"] + c_y_fields = dict(c_y._fields_) + assert issubclass(c_x , ctypes.Structure) + assert issubclass(c_y, ctypes.Structure) + assert c_y_fields["x"] is c_x From ac at codespeak.net Mon Apr 24 11:46:09 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 11:46:09 +0200 (CEST) Subject: [pypy-svn] r26249 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060424094609.E1F1A1007C@code0.codespeak.net> Author: ac Date: Mon Apr 24 11:46:09 2006 New Revision: 26249 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (aleale, arre) Refactor to new ctypes_platform interface. Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 11:46:09 2006 @@ -14,108 +14,133 @@ 'arpa/inet.h' ) HEADER = ''.join(['#include <%s>\n' % filename for filename in includes]) -constants = {"BDADDR_ANY": "00:00:00:00:00:00", - "BDADDR_LOCAL": "00:00:00:FF:FF:FF"} - -# constants -for name in ['AF_APPLETALK', 'AF_ASH', 'AF_ATMPVC', 'AF_ATMSVC', 'AF_AX25', - 'AF_BLUETOOTH', 'AF_BRIDGE', 'AF_ECONET', 'AF_INET', 'AF_INET6', - 'AF_IPX', 'AF_IRDA', 'AF_KEY', 'AF_NETBEUI', 'AF_NETLINK', - 'AF_NETROM', 'AF_PACKET', 'AF_PPPOX', 'AF_ROSE', 'AF_ROUTE', - 'AF_SECURITY', 'AF_WANPIPE', 'AF_SNA', 'AF_UNIX', 'AF_X25', - 'AF_UNSPEC', 'AI_ADDRCONFIG', 'AI_ALL', 'AI_CANONNAME', -'AI_DEFAULT', 'AI_MASK', 'AI_NUMERICHOST', 'AI_NUMERICSERV', 'AI_PASSIVE', 'AI_V4MAPPED', -'AI_V4MAPPED_CFG', 'BDADDR_ANY', 'EAI_ADDRFAMILY', 'EAI_AGAIN', 'EAI_BADFLAGS', -'EAI_BADHINTS', 'EAI_FAIL', 'EAI_FAMILY', 'EAI_MAX', 'EAI_MEMORY', -'EAI_NODATA', 'EAI_NONAME', 'EAI_PROTOCOL', 'EAI_SERVICE', 'EAI_SOCKTYPE', -'EAI_SYSTEM', 'INADDR_UNSPEC_GROUP', 'IPPROTO_AH', -'IPPROTO_DSTOPTS', 'IPPROTO_EGP', 'IPPROTO_EON', 'IPPROTO_ESP', -'IPPROTO_FRAGMENT', 'IPPROTO_GGP', 'IPPROTO_GRE', 'IPPROTO_HELLO', -'IPPROTO_HOPOPTS', 'IPPROTO_ICMP', 'IPPROTO_ICMPV6', 'IPPROTO_IDP', -'IPPROTO_IGMP', 'IPPROTO_IPCOMP', 'IPPROTO_IPIP', -'IPPROTO_IPV4', 'IPPROTO_IPV6', 'IPPROTO_MAX', 'IPPROTO_ND', 'IPPROTO_NONE', -'IPPROTO_PIM', 'IPPROTO_PUP', 'IPPROTO_ROUTING', -'IPPROTO_RSVP', 'IPPROTO_TCP', 'IPPROTO_TP', 'IPPROTO_XTP', -'IPV6_CHECKSUM', 'IPV6_DSTOPTS', 'IPV6_HOPLIMIT', 'IPV6_HOPOPTS', -'IPV6_JOIN_GROUP', 'IPV6_LEAVE_GROUP', 'IPV6_MULTICAST_HOPS', -'IPV6_MULTICAST_IF', 'IPV6_MULTICAST_LOOP', 'IPV6_NEXTHOP', 'IPV6_PKTINFO', -'IPV6_RTHDR', 'IPV6_RTHDR_TYPE_0', 'IPV6_UNICAST_HOPS', 'IPV6_V6ONLY', -'IP_ADD_MEMBERSHIP', 'IP_DEFAULT_MULTICAST_LOOP', 'IP_DEFAULT_MULTICAST_TTL', -'IP_DROP_MEMBERSHIP', 'IP_HDRINCL', 'IP_MAX_MEMBERSHIPS', 'IP_MULTICAST_IF', -'IP_MULTICAST_LOOP', 'IP_MULTICAST_TTL', 'IP_OPTIONS', 'IP_RECVDSTADDR', -'IP_RECVOPTS', 'IP_RECVRETOPTS', 'IP_RETOPTS', 'IP_TOS', 'IP_TTL', -'MSG_CTRUNC', 'MSG_DONTROUTE', 'MSG_DONTWAIT', 'MSG_EOR', 'MSG_OOB', -'MSG_PEEK', 'MSG_TRUNC', 'MSG_WAITALL', 'NI_DGRAM', 'NI_MAXHOST', -'NI_MAXSERV', 'NI_NAMEREQD', 'NI_NOFQDN', 'NI_NUMERICHOST', 'NI_NUMERICSERV', -'SOCK_DGRAM', 'SOCK_RAW', 'SOCK_RDM', -'SOCK_SEQPACKET', 'SOCK_STREAM', 'SOL_SOCKET', -'SO_ACCEPTCONN', 'SO_BROADCAST', 'SO_DEBUG', 'SO_DONTROUTE', -'SO_ERROR', 'SO_KEEPALIVE', 'SO_LINGER', 'SO_OOBINLINE', 'SO_RCVBUF', -'SO_RCVLOWAT', 'SO_RCVTIMEO', 'SO_REUSEADDR', 'SO_REUSEPORT', 'SO_SNDBUF', -'SO_SNDLOWAT', 'SO_SNDTIMEO', 'SO_TYPE', 'SO_USELOOPBACK', 'TCP_MAXSEG', -'TCP_NODELAY', 'AF_DECnet']: - try: - constants[name] = ctypes_platform.getconstantinteger(name, HEADER) - except distutils.errors.CompileError: - pass - -for special, default in [('SOL_IP', 0), - ('SOL_TCP', 6), - ('SOL_UDP', 17), - ('SOMAXCONN', 5), - ('IPPROTO_IP', 6), - ('IPPROTO_UDP', 17), - ('IPPROTO_RAW', 255), - ('IPPORT_RESERVED', 1024), - ('IPPORT_USERRESERVED', 5000), - ('INADDR_ANY', 0x00000000), - ('INADDR_BROADCAST', 0xffffffff), - ('INADDR_LOOPBACK', 0x7F000001), - ('INADDR_UNSPEC_GROUP', 0xe0000000), - ('INADDR_ALLHOSTS_GROUP', 0xe0000001), - ('INADDR_MAX_LOCAL_GROUP', 0xe00000ff), - ('INADDR_NONE', 0xffffffff), - ('SHUT_RD', 0), - ('SHUT_WR', 1), - ('SHUT_RDWR', 2)]: - try: - constants[special] = ctypes_platform.getconstantinteger(special, HEADER) - except distutils.errors.CompileError: - constants[special] = default +constants = {} +class CConfig: + _header_ = HEADER + # constants + +constant_names = ['AF_APPLETALK', 'AF_ASH', 'AF_ATMPVC', 'AF_ATMSVC', 'AF_AX25', + 'AF_BLUETOOTH', 'AF_BRIDGE', 'AF_ECONET', 'AF_INET', 'AF_INET6', + 'AF_IPX', 'AF_IRDA', 'AF_KEY', 'AF_NETBEUI', 'AF_NETLINK', + 'AF_NETROM', 'AF_PACKET', 'AF_PPPOX', 'AF_ROSE', 'AF_ROUTE', + 'AF_SECURITY', 'AF_WANPIPE', 'AF_SNA', 'AF_UNIX', 'AF_X25', + 'AF_UNSPEC', 'AI_ADDRCONFIG', 'AI_ALL', 'AI_CANONNAME', + 'AI_DEFAULT', 'AI_MASK', 'AI_NUMERICHOST', 'AI_NUMERICSERV', 'AI_PASSIVE', 'AI_V4MAPPED', + 'AI_V4MAPPED_CFG', 'BDADDR_ANY', 'EAI_ADDRFAMILY', 'EAI_AGAIN', 'EAI_BADFLAGS', + 'EAI_BADHINTS', 'EAI_FAIL', 'EAI_FAMILY', 'EAI_MAX', 'EAI_MEMORY', + 'EAI_NODATA', 'EAI_NONAME', 'EAI_PROTOCOL', 'EAI_SERVICE', 'EAI_SOCKTYPE', + 'EAI_SYSTEM', 'INADDR_UNSPEC_GROUP', 'IPPROTO_AH', + 'IPPROTO_DSTOPTS', 'IPPROTO_EGP', 'IPPROTO_EON', 'IPPROTO_ESP', + 'IPPROTO_FRAGMENT', 'IPPROTO_GGP', 'IPPROTO_GRE', 'IPPROTO_HELLO', + 'IPPROTO_HOPOPTS', 'IPPROTO_ICMP', 'IPPROTO_ICMPV6', 'IPPROTO_IDP', + 'IPPROTO_IGMP', 'IPPROTO_IPCOMP', 'IPPROTO_IPIP', + 'IPPROTO_IPV4', 'IPPROTO_IPV6', 'IPPROTO_MAX', 'IPPROTO_ND', 'IPPROTO_NONE', + 'IPPROTO_PIM', 'IPPROTO_PUP', 'IPPROTO_ROUTING', + 'IPPROTO_RSVP', 'IPPROTO_TCP', 'IPPROTO_TP', 'IPPROTO_XTP', + 'IPV6_CHECKSUM', 'IPV6_DSTOPTS', 'IPV6_HOPLIMIT', 'IPV6_HOPOPTS', + 'IPV6_JOIN_GROUP', 'IPV6_LEAVE_GROUP', 'IPV6_MULTICAST_HOPS', + 'IPV6_MULTICAST_IF', 'IPV6_MULTICAST_LOOP', 'IPV6_NEXTHOP', 'IPV6_PKTINFO', + 'IPV6_RTHDR', 'IPV6_RTHDR_TYPE_0', 'IPV6_UNICAST_HOPS', 'IPV6_V6ONLY', + 'IP_ADD_MEMBERSHIP', 'IP_DEFAULT_MULTICAST_LOOP', 'IP_DEFAULT_MULTICAST_TTL', + 'IP_DROP_MEMBERSHIP', 'IP_HDRINCL', 'IP_MAX_MEMBERSHIPS', 'IP_MULTICAST_IF', + 'IP_MULTICAST_LOOP', 'IP_MULTICAST_TTL', 'IP_OPTIONS', 'IP_RECVDSTADDR', + 'IP_RECVOPTS', 'IP_RECVRETOPTS', 'IP_RETOPTS', 'IP_TOS', 'IP_TTL', + 'MSG_CTRUNC', 'MSG_DONTROUTE', 'MSG_DONTWAIT', 'MSG_EOR', 'MSG_OOB', + 'MSG_PEEK', 'MSG_TRUNC', 'MSG_WAITALL', 'NI_DGRAM', 'NI_MAXHOST', + 'NI_MAXSERV', 'NI_NAMEREQD', 'NI_NOFQDN', 'NI_NUMERICHOST', 'NI_NUMERICSERV', + 'SOCK_DGRAM', 'SOCK_RAW', 'SOCK_RDM', + 'SOCK_SEQPACKET', 'SOCK_STREAM', 'SOL_SOCKET', + 'SO_ACCEPTCONN', 'SO_BROADCAST', 'SO_DEBUG', 'SO_DONTROUTE', + 'SO_ERROR', 'SO_KEEPALIVE', 'SO_LINGER', 'SO_OOBINLINE', 'SO_RCVBUF', + 'SO_RCVLOWAT', 'SO_RCVTIMEO', 'SO_REUSEADDR', 'SO_REUSEPORT', 'SO_SNDBUF', + 'SO_SNDLOWAT', 'SO_SNDTIMEO', 'SO_TYPE', 'SO_USELOOPBACK', 'TCP_MAXSEG', + 'TCP_NODELAY', 'AF_DECnet'] +for name in constant_names: + setattr(CConfig, name, ctypes_platform.DefinedConstantInteger(name)) + +constants["BDADDR_ANY"] = "00:00:00:00:00:00" +constants["BDADDR_LOCAL"] = "00:00:00:FF:FF:FF" + +constants_w_defaults = [('SOL_IP', 0), + ('SOL_TCP', 6), + ('SOL_UDP', 17), + ('SOMAXCONN', 5), + ('IPPROTO_IP', 6), + ('IPPROTO_UDP', 17), + ('IPPROTO_RAW', 255), + ('IPPORT_RESERVED', 1024), + ('IPPORT_USERRESERVED', 5000), + ('INADDR_ANY', 0x00000000), + ('INADDR_BROADCAST', 0xffffffff), + ('INADDR_LOOPBACK', 0x7F000001), + ('INADDR_UNSPEC_GROUP', 0xe0000000), + ('INADDR_ALLHOSTS_GROUP', 0xe0000001), + ('INADDR_MAX_LOCAL_GROUP', 0xe00000ff), + ('INADDR_NONE', 0xffffffff), + ('SHUT_RD', 0), + ('SHUT_WR', 1), + ('SHUT_RDWR', 2)] +for name, default in constants_w_defaults: + setattr(CConfig, name, ctypes_platform.DefinedConstantInteger(name)) + # types -uint16_t = ctypes_platform.getsimpletype('uint16_t', HEADER, c_ushort) -uint32_t = ctypes_platform.getsimpletype('uint32_t', HEADER, c_uint) -size_t = ctypes_platform.getsimpletype('size_t', HEADER, c_int) -ssize_t = ctypes_platform.getsimpletype('ssize_t', HEADER, c_int) -socklen_t = ctypes_platform.getsimpletype('socklen_t', HEADER, c_int) +CConfig.uint16_t = ctypes_platform.SimpleType('uint16_t', c_ushort) +CConfig.uint32_t = ctypes_platform.SimpleType('uint32_t', c_uint) +CConfig.size_t = ctypes_platform.SimpleType('size_t', c_int) +CConfig.ssize_t = ctypes_platform.SimpleType('ssize_t', c_int) +CConfig.socklen_t = ctypes_platform.SimpleType('socklen_t', c_int) # struct types -sockaddr = ctypes_platform.getstruct('struct sockaddr', HEADER, - [('sa_family', c_int), - # unknown and variable fields follow - ]) -sockaddr_ptr = POINTER(sockaddr) -in_addr = ctypes_platform.getstruct('struct in_addr', HEADER, - [('s_addr', c_uint)]) -sockaddr_in = ctypes_platform.getstruct('struct sockaddr_in', HEADER, +CConfig.sockaddr = ctypes_platform.Struct('struct sockaddr', + [('sa_family', c_int), + ]) +sockaddr_ptr = POINTER('sockaddr') +CConfig.in_addr = ctypes_platform.Struct('struct in_addr', + [('s_addr', c_uint)]) +CConfig.sockaddr_in = ctypes_platform.Struct('struct sockaddr_in', [('sin_family', c_int), ('sin_port', c_ushort), - ('sin_addr', in_addr)]) + ('sin_addr', CConfig.in_addr)]) addrinfo_ptr = POINTER("addrinfo") -addrinfo = ctypes_platform.getstruct('struct addrinfo', HEADER, +CConfig.addrinfo = ctypes_platform.Struct('struct addrinfo', [('ai_flags', c_int), ('ai_family', c_int), ('ai_socktype', c_int), ('ai_protocol', c_int), ('ai_addrlen', c_int), - ('ai_addr', POINTER(sockaddr)), + ('ai_addr', sockaddr_ptr), ('ai_canonname', c_char_p), ('ai_next', addrinfo_ptr)]) -SetPointerType(addrinfo_ptr, addrinfo) -FILE_ptr = ctypes_platform.getstruct('FILE *', HEADER, - []) + +class cConfig: + pass +cConfig.__dict__.update(ctypes_platform.configure(CConfig)) + +for name in constant_names: + value = getattr(cConfig, name) + if value is not None: + constants[name] = value +for name, default in constants_w_defaults: + value = getattr(cConfig, name) + if value is not None: + constants[name] = value + else: + constants[name] = default + +uint16_t = cConfig.uint16_t +uint32_t = cConfig.uint32_t +size_t = cConfig.size_t +ssize_t = cConfig.ssize_t +socklen_t = cConfig.socklen_t +sockaddr = cConfig.sockaddr +sockaddr_in = cConfig.sockaddr_in +in_addr = cConfig.in_addr +addrinfo = cConfig.addrinfo + +SetPointerType(addrinfo_ptr, addrinfo) +SetPointerType(sockaddr_ptr, sockaddr) # functions dllname = util.find_library('c') From sanxiyn at codespeak.net Mon Apr 24 11:52:40 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 11:52:40 +0200 (CEST) Subject: [pypy-svn] r26250 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424095240.5215A1007C@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 11:52:35 2006 New Revision: 26250 Added: pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py Log: (nik, sanxiyn) Implement direct_call and multiple function graphs Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 11:52:35 2006 @@ -1,4 +1,5 @@ from pypy.objspace.flow.model import Constant, Variable, last_exception +from pypy.rpython.ootypesystem.ootype import _static_meth def repr_unknown(obj): return '#<%r>' % (obj,) @@ -7,6 +8,8 @@ return var.name def repr_const(val): + if isinstance(val, _static_meth): + return val._name # XXX make sure function names are unique if isinstance(val, tuple): val = map(repr_const, val) return "'(%s)" % ' '.join(val) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 11:52:35 2006 @@ -59,6 +59,12 @@ def op_int_is_true(self, result, arg): yield "(setf %s (not (zerop %s)))" % (result, arg) + def op_direct_call(self, result, fun, *args): + graph = self.args[0].value.graph + self.gen.pendinggraphs.append(graph) + args = " ".join(args) + yield "(setf %s (%s %s))" % (result, fun, args) + def declare_class(self, cls): # cls is really type of Instance name = cls._name @@ -122,11 +128,8 @@ class GenCL: - def __init__(self, fun, input_arg_types=[]): - # NB. 'fun' is a graph! - simplify_graph(fun) - self.fun = fun - self.blockref = {} + def __init__(self, entry_point, input_arg_types=[]): + self.pendinggraphs = [entry_point] self.declarations = [] def annotate(self, input_arg_types): @@ -148,8 +151,10 @@ return declarations + "\n" + code def emit(self): - for line in self.emit_defun(self.fun): - yield line + while self.pendinggraphs: + graph = self.pendinggraphs.pop() + for line in self.emit_defun(graph): + yield line def emit_defun(self, fun): yield ";;;; Main" @@ -162,6 +167,7 @@ yield "(prog" blocklist = list(fun.iterblocks()) vardict = {} + self.blockref = {} for block in blocklist: tag = len(self.blockref) self.blockref[block] = tag Added: pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py Mon Apr 24 11:52:35 2006 @@ -0,0 +1,10 @@ +from pypy.translator.cl.buildcl import make_cl_func + +def test_call(): + def add_one(n): + n = add_one_really(n) + return n + def add_one_really(n): + return n + 1 + cl_add_one = make_cl_func(add_one, [int]) + assert cl_add_one(1) == 2 From antocuni at codespeak.net Mon Apr 24 11:56:27 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 24 Apr 2006 11:56:27 +0200 (CEST) Subject: [pypy-svn] r26251 - in pypy/dist/pypy/rpython: ootypesystem test Message-ID: <20060424095627.5939310081@code0.codespeak.net> Author: antocuni Date: Mon Apr 24 11:56:21 2006 New Revision: 26251 Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: Added support for methods 'values' and 'items' Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Mon Apr 24 11:56:21 2006 @@ -3,12 +3,15 @@ from pypy.objspace.flow.model import Constant from pypy.rpython.rdict import AbstractDictRepr, rtype_newdict from pypy.rpython.ootypesystem import ootype +from pypy.rpython.ootypesystem.rlist import ll_newlist from pypy.rpython.rarithmetic import r_uint from pypy.rpython.objectmodel import hlinvoke from pypy.rpython import robject from pypy.rpython import objectmodel from pypy.rpython import rmodel +def dum_values(): pass +def dum_items():pass class DictRepr(AbstractDictRepr): def __init__(self, rtyper, key_repr, value_repr, dictkey, dictvalue, @@ -89,6 +92,26 @@ hop.exception_cannot_occur() return self.send_message(hop, 'll_keys') + def rtype_method_values(self, hop): + return self._rtype_method_vi(hop, dum_values) + + def rtype_method_items(self, hop): + return self._rtype_method_vi(hop, dum_items) + + def _rtype_method_vi(self, hop, spec): + v_dict, = hop.inputargs(self) + r_list = hop.r_result + cLIST = hop.inputconst(ootype.Void, r_list.lowleveltype) + c_func = hop.inputconst(ootype.Void, spec) + c_dummy_default = hop.inputconst(self.value_repr.lowleveltype, + self.value_repr.ll_dummy_value) + return hop.gendirectcall(ll_dict_values_items, v_dict, cLIST, c_func, c_dummy_default) + +## def rtype_method_items(self, hop): +## v_dict, = hop.inputargs(self) +## return hop.gendirectcall(ll_dict_items, v_dict) + + class __extend__(pairtype(DictRepr, rmodel.Repr)): @@ -147,3 +170,22 @@ except KeyError: d.ll_set(key, default) return default + +def ll_dict_values_items(d, LIST, func, dummy_default): + keys = d.ll_keys() + length = keys.ll_length() + result = ll_newlist(LIST, length) + i = 0 + while i < length: + key = keys.ll_getitem_fast(i) + value = d.ll_get(key, dummy_default) # dummy_default is never returned + if func is dum_items: + r = ootype.new(LIST._ITEMTYPE) + r.item0 = key # TODO: do we need casting? + r.item1 = value + result.ll_setitem_fast(i, r) + elif func is dum_values: + result.ll_setitem_fast(i, value) + + i += 1 + return result Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Mon Apr 24 11:56:21 2006 @@ -729,6 +729,10 @@ def ll_to_list(self, l): return l._list[:] + def ll_to_tuple(self, t, num): + lst = [getattr(t, 'item%d' % i) for i in range(num)] + return tuple(lst) + # these tests are similar to those above, but they don't use strings def test_dict_creation(self): def createdict(i): @@ -799,3 +803,19 @@ return d.keys() res = self.ll_to_list(self.interpret(func, [42, 13])) assert res == [42, 13] or res == [13, 42] + + def test_values(self): + def func(x, y): + d = {x: x+1, y: y+1} + return d.values() + res = self.ll_to_list(self.interpret(func, [42, 13])) + assert res == [43, 14] or res == [14, 43] + + def test_items(self): + def func(x, y): + d = {x: x+1, y: y+1} + return d.items() + res = self.ll_to_list(self.interpret(func, [42, 13])) + res = [self.ll_to_tuple(item, 2) for item in res] + assert res == [(42, 43), (13, 14)] or res == [(13, 14), (42, 43)] + From sanxiyn at codespeak.net Mon Apr 24 12:00:03 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 12:00:03 +0200 (CEST) Subject: [pypy-svn] r26252 - pypy/dist/pypy/translator/cl Message-ID: <20060424100003.51F3210081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 12:00:00 2006 New Revision: 26252 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Remove more obsolete stuffs Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 12:00:00 2006 @@ -132,18 +132,6 @@ self.pendinggraphs = [entry_point] self.declarations = [] - def annotate(self, input_arg_types): - ann = RPythonAnnotator() - inputcells = [ann.typeannotation(t) for t in input_arg_types] - ann.build_graph_types(self.fun, inputcells) - self.setannotator(ann) - - def setannotator(self, annotator): - self.ann = annotator - - def get_type(self, var): - return var.concretetype - def emitcode(self, public=True): lines = list(self.emit()) declarations = "\n".join(self.declarations) @@ -172,7 +160,8 @@ tag = len(self.blockref) self.blockref[block] = tag for var in block.getvariables(): - vardict[var] = self.get_type(var) + # In the future, we could assign type information here + vardict[var] = None yield "( last-exc" for var in vardict: if var in arglist: From sanxiyn at codespeak.net Mon Apr 24 12:22:46 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 12:22:46 +0200 (CEST) Subject: [pypy-svn] r26253 - pypy/dist/pypy/translator/cl Message-ID: <20060424102246.19ED61007C@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 12:22:43 2006 New Revision: 26253 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: (dialtone, sanxiyn) Pretty-print lisp sources Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 12:22:43 2006 @@ -79,6 +79,15 @@ py.test.skip("Common Lisp neither configured nor detected.") return _make_cl_func(func, global_cl, udir, argtypes) +pretty_printer = """ +(let* ((filename "%s") + (content (with-open-file (f filename) + (loop for sexp = (read f nil) while sexp collect sexp)))) + (with-open-file (out filename + :direction :output :if-does-not-exist :create :if-exists :supersede) + (loop for sexp in content do (pprint sexp out)))) +""" + def _make_cl_func(func, cl, path, argtypes=[]): t = TranslationContext() t.buildannotator().build_types(func, argtypes) @@ -93,6 +102,11 @@ out = gen.emitcode() i = 1 fpath = path.join("%s.lisp" % graph.name) + script = path.join(".printer.lisp") + fp = file(str(script), "w") + fp.write(pretty_printer % (fpath,)) + fp.close() + def _(*args): fpath.write(out) fp = file(str(fpath), "a") @@ -101,6 +115,7 @@ print >>fp, writelisp(arg), print >>fp, "))" fp.close() + cmdexec("%s %s" % (cl, str(script))) output = cmdexec("%s %s" % (cl, str(fpath))) return readlisp(output) return _ From sanxiyn at codespeak.net Mon Apr 24 12:35:40 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 12:35:40 +0200 (CEST) Subject: [pypy-svn] r26254 - pypy/dist/pypy/translator/cl Message-ID: <20060424103540.C6FCF1007C@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 12:35:36 2006 New Revision: 26254 Added: pypy/dist/pypy/translator/cl/conftest.py Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Add --prettyprint option to py.test to pretty-print Common Lisp sources. Off by default. Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 12:35:36 2006 @@ -102,10 +102,12 @@ out = gen.emitcode() i = 1 fpath = path.join("%s.lisp" % graph.name) - script = path.join(".printer.lisp") - fp = file(str(script), "w") - fp.write(pretty_printer % (fpath,)) - fp.close() + + if conftest.option.prettyprint: + script = path.join(".printer.lisp") + fp = file(str(script), "w") + fp.write(pretty_printer % (fpath,)) + fp.close() def _(*args): fpath.write(out) @@ -115,7 +117,8 @@ print >>fp, writelisp(arg), print >>fp, "))" fp.close() - cmdexec("%s %s" % (cl, str(script))) + if conftest.option.prettyprint: + cmdexec("%s %s" % (cl, str(script))) output = cmdexec("%s %s" % (cl, str(fpath))) return readlisp(output) return _ Added: pypy/dist/pypy/translator/cl/conftest.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/conftest.py Mon Apr 24 12:35:36 2006 @@ -0,0 +1,7 @@ +import py + +Option = py.test.Config.Option + +option = py.test.Config.addoptions('pypy-cl options', + Option('--prettyprint', action='store_true', dest='prettyprint', + default=False, help='pretty-print Common Lisp source')) From dialtone at codespeak.net Mon Apr 24 12:44:05 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 12:44:05 +0200 (CEST) Subject: [pypy-svn] r26255 - pypy/dist/pypy/translator/cl Message-ID: <20060424104405.EFEA21007C@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 12:44:03 2006 New Revision: 26255 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: use a class instead of a struct to represent a class Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 12:44:03 2006 @@ -69,9 +69,9 @@ # cls is really type of Instance name = cls._name fields = cls._fields - fieldnames = fields.keys() + fieldnames = ['('+field+')' for field in fields.keys()] field_declaration = ' '.join(fieldnames) - class_declaration = "(defstruct %s %s)" % (name, field_declaration) + class_declaration = "(defclass %s () (%s))" % (name, field_declaration) return class_declaration def op_new(self, result, _): @@ -81,7 +81,7 @@ else: declaration = self.declare_class(cls) self.gen.declarations.append(declaration) - yield "(setf %s (make-%s))" % (result, cls._name) + yield "(setf %s (make-instance '%s))" % (result, cls._name) def op_oosend(self, result, *ignore): method = self.args[0].value @@ -93,16 +93,15 @@ yield "(setf %s %s)" % (result, code) def op_oogetfield(self, result, obj, _): - clsname = self.args[0].concretetype._name fieldname = self.args[1].value - yield "(setf %s (%s-%s %s))" % (result, clsname, fieldname, obj) + yield "(setf %s (slot-value %s '%s))" % (result, obj, fieldname) def op_oosetfield(self, result, obj, _, value): clsname = self.args[0].concretetype._name fieldname = self.args[1].value if fieldname == "meta": # XXX raise StopIteration - yield "(setf (%s-%s %s) %s)" % (clsname, fieldname, obj, value) + yield "(setf (slot-value %s '%s) %s)" % (obj, fieldname, value) class ListImpl: From ac at codespeak.net Mon Apr 24 12:47:15 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 12:47:15 +0200 (CEST) Subject: [pypy-svn] r26256 - pypy/dist/pypy/module/_socket/test Message-ID: <20060424104715.F00EC1007C@code0.codespeak.net> Author: ac Date: Mon Apr 24 12:47:15 2006 New Revision: 26256 Modified: pypy/dist/pypy/module/_socket/test/echoserver.py Log: (aleale, arre) Make echoserver allow reuse of its address. Add an interface for instantiating a server. Modified: pypy/dist/pypy/module/_socket/test/echoserver.py ============================================================================== --- pypy/dist/pypy/module/_socket/test/echoserver.py (original) +++ pypy/dist/pypy/module/_socket/test/echoserver.py Mon Apr 24 12:47:15 2006 @@ -5,7 +5,7 @@ PORT = 8037 class EchoServer(SocketServer.TCPServer): - + allow_reuse_address = True def __init__(self, *args, **kwargs): self.address_family = kwargs["address_family"] del kwargs["address_family"] @@ -32,8 +32,10 @@ raise RuntimeError() self.wfile.write(client_string) +def create_server(address_family=socket.AF_INET): + return EchoServer(("", PORT), EchoRequestHandler, address_family=address_family) def start_server(address_family=socket.AF_INET): - server = EchoServer(("", PORT), EchoRequestHandler, address_family=address_family) + server = create_server(address_family) server.serve() if __name__ == "__main__": From ac at codespeak.net Mon Apr 24 12:48:51 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 24 Apr 2006 12:48:51 +0200 (CEST) Subject: [pypy-svn] r26257 - in pypy/dist/pypy/rpython/rctypes/socketmodule: . test Message-ID: <20060424104851.B983B10081@code0.codespeak.net> Author: ac Date: Mon Apr 24 12:48:51 2006 New Revision: 26257 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Log: (aleale, arre) accept, listen, recv! Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Mon Apr 24 12:48:51 2006 @@ -56,7 +56,7 @@ fd = self._fd if backlog < 1: backlog = 1 - res = _c.listen(fd, backlog) + res = _c.socketlisten(fd, backlog) if res == -1: raise error(_c.errno.value) @@ -113,8 +113,12 @@ def makefile(self): raise NotImplementedError - def recv(self): - pass + def recv(self, bufsize, flags=0): + buf = create_string_buffer(bufsize) + read_bytes = _c.socketrecv(self._fd, buf, bufsize, flags) + if read_bytes < 0: + raise error(_c.errno) + return buf[:read_bytes] def recvfrom(self): pass Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Mon Apr 24 12:48:51 2006 @@ -205,6 +205,10 @@ socketbind.argtypes = [c_int, sockaddr_ptr, socklen_t] socketbind.restype = c_int +socketlisten = socketdll.listen +socketlisten.argtypes = [c_int, c_int] +socketlisten.restype = c_int + socketgetpeername = socketdll.getpeername socketgetpeername.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] socketgetpeername.restype = c_int Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Mon Apr 24 12:48:51 2006 @@ -16,20 +16,20 @@ interface_matcher(_socket.socket, _rsocket.socket) interface_matcher(_rsocket.socket, _socket.socket) -class TestSocket: +class TestSocketWithServer: HOST = "127.0.0.1" PORT = echoserver.PORT family = _socket.AF_INET - def setup_class(cls): - thread.start_new_thread(echoserver.start_server, (), - {"address_family": cls.family}) - import time - time.sleep(1) - def teardown_class(cls): + def setup_class(self): + self.server = echoserver.create_server() + thread.start_new_thread(self.server.serve, (), + {"address_family": self.family}) + + def teardown_class(self): import telnetlib - tn = telnetlib.Telnet(cls.HOST, cls.PORT) + tn = telnetlib.Telnet(self.HOST, self.PORT) tn.write("shutdown\n") tn.close() @@ -40,7 +40,7 @@ assert e.args[0] == errno.EINVAL def test_getpeername(self): - s = _rsocket.socket(self.family, _socket.SOCK_STREAM, 0) + s = _rsocket.socket(self.family, _rsocket.SOCK_STREAM, 0) s.connect((self.HOST, self.PORT)) sockname = s.getpeername() s.close() @@ -48,15 +48,39 @@ assert host == self.HOST assert port == self.PORT + +class TestSocket: + HOST = "127.0.0.1" + PORT = 1025 + ADDR = (HOST, PORT) + + def connecting_client(self, addr): + socket = __import__("socket", {}, {}, []) + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + s.connect(addr) + self.client_addr = s.getsockname() + s.send("@") + data = s.recv(1024) + while data: + data = s.recv(1024) + s.close() + def test_getsockname(self): - s = _rsocket.socket(self.family, _socket.SOCK_STREAM, 0) - HOST = "127.0.0.1" - PORT = 1025 - s.bind((HOST, PORT)) + s = _rsocket.socket(_rsocket.AF_INET, _rsocket.SOCK_STREAM, 0) + s.bind(self.ADDR) sockname = s.getsockname() s.close() - host, port = sockname - assert host == HOST - assert port == PORT - + assert sockname == self.ADDR + def test_accept(self): + s = _rsocket.socket(_rsocket.AF_INET, _rsocket.SOCK_STREAM, 0) + s.bind((self.HOST, 0)) + s.listen(1) + ADDR = s.getsockname() + thread.start_new_thread(self.connecting_client, (ADDR,)) + client, clientaddr = s.accept() + res = client.recv(1) + assert res == "@" + assert clientaddr == self.client_addr + client.close() + From sanxiyn at codespeak.net Mon Apr 24 12:54:29 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 12:54:29 +0200 (CEST) Subject: [pypy-svn] r26258 - pypy/dist/pypy/translator/cl Message-ID: <20060424105429.7EEF710081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 12:54:26 2006 New Revision: 26258 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: General cleanup Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 12:54:26 2006 @@ -97,7 +97,6 @@ yield "(setf %s (slot-value %s '%s))" % (result, obj, fieldname) def op_oosetfield(self, result, obj, _, value): - clsname = self.args[0].concretetype._name fieldname = self.args[1].value if fieldname == "meta": # XXX raise StopIteration @@ -135,7 +134,7 @@ lines = list(self.emit()) declarations = "\n".join(self.declarations) code = "\n".join(lines) - return declarations + "\n" + code + return declarations + "\n" + code + "\n" def emit(self): while self.pendinggraphs: @@ -144,13 +143,10 @@ yield line def emit_defun(self, fun): - yield ";;;; Main" yield "(defun " + fun.name arglist = fun.getargs() - yield "(" - for arg in arglist: - yield repr_var(arg) - yield ")" + args = " ".join(map(repr_var, arglist)) + yield "(%s)" % (args,) yield "(prog" blocklist = list(fun.iterblocks()) vardict = {} @@ -161,16 +157,14 @@ for var in block.getvariables(): # In the future, we could assign type information here vardict[var] = None - yield "( last-exc" + yield "(" for var in vardict: if var in arglist: yield "(%s %s)" % (repr_var(var), repr_var(var)) else: yield repr_var(var) yield ")" - yield "(setf last-exc nil)" for block in blocklist: - yield "" for line in self.emit_block(block): yield line yield "))" @@ -239,4 +233,3 @@ s, t = couples[-1] yield "%s %s)" % (t, s) yield self.format_jump(link.target) - From dialtone at codespeak.net Mon Apr 24 13:05:33 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Mon, 24 Apr 2006 13:05:33 +0200 (CEST) Subject: [pypy-svn] r26259 - pypy/dist/pypy/translator/cl Message-ID: <20060424110533.A5D1010081@code0.codespeak.net> Author: dialtone Date: Mon Apr 24 13:05:29 2006 New Revision: 26259 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py Log: (dialtone, nik) more lispy than ever Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 13:05:29 2006 @@ -6,7 +6,7 @@ from pypy.objspace.flow import FlowObjSpace from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL -from pypy.translator.cl.clrepr import repr_const +from pypy.translator.cl.clrepr import repr_const, repr_fun_name from py.process import cmdexec from pypy import conftest @@ -112,7 +112,7 @@ def _(*args): fpath.write(out) fp = file(str(fpath), "a") - print >>fp, "(write (", graph.name, + print >>fp, "(write (", repr_fun_name(graph.name), for arg in args: print >>fp, writelisp(arg), print >>fp, "))" Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 13:05:29 2006 @@ -1,15 +1,21 @@ from pypy.objspace.flow.model import Constant, Variable, last_exception -from pypy.rpython.ootypesystem.ootype import _static_meth +from pypy.rpython.ootypesystem.ootype import _static_meth, Instance def repr_unknown(obj): return '#<%r>' % (obj,) def repr_var(var): return var.name + +def repr_class_name(name): + return name.replace('_', '-') + +def repr_fun_name(name): + return name.replace('_', '-') def repr_const(val): if isinstance(val, _static_meth): - return val._name # XXX make sure function names are unique + return repr_fun_name(val._name) # XXX make sure function names are unique if isinstance(val, tuple): val = map(repr_const, val) return "'(%s)" % ' '.join(val) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 13:05:29 2006 @@ -5,7 +5,7 @@ from pypy.translator.transform import transform_graph, default_extra_passes, transform_slice from pypy.rpython.ootypesystem.ootype import Instance, List -from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const +from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name class Op: @@ -71,7 +71,7 @@ fields = cls._fields fieldnames = ['('+field+')' for field in fields.keys()] field_declaration = ' '.join(fieldnames) - class_declaration = "(defclass %s () (%s))" % (name, field_declaration) + class_declaration = "(defclass %s () (%s))" % (repr_class_name(name), field_declaration) return class_declaration def op_new(self, result, _): @@ -81,7 +81,7 @@ else: declaration = self.declare_class(cls) self.gen.declarations.append(declaration) - yield "(setf %s (make-instance '%s))" % (result, cls._name) + yield "(setf %s (make-instance '%s))" % (result, repr_class_name(cls._name)) def op_oosend(self, result, *ignore): method = self.args[0].value @@ -143,7 +143,7 @@ yield line def emit_defun(self, fun): - yield "(defun " + fun.name + yield "(defun " + repr_fun_name(fun.name) arglist = fun.getargs() args = " ".join(map(repr_var, arglist)) yield "(%s)" % (args,) From sanxiyn at codespeak.net Mon Apr 24 13:10:11 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 13:10:11 +0200 (CEST) Subject: [pypy-svn] r26260 - pypy/dist/pypy/translator/cl/test Message-ID: <20060424111011.E233910081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 13:10:09 2006 New Revision: 26260 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: Rename a test function f to add_two Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Mon Apr 24 13:10:09 2006 @@ -108,10 +108,10 @@ assert cl_sum(range(12)) == 66 def test_int_add(self): - def f(number): + def add_two(number): return number + 2 - cl_add = make_cl_func(f, [int]) - assert cl_add(5) == 7 + cl_add_two = make_cl_func(add_two, [int]) + assert cl_add_two(5) == 7 # TODO From auc at codespeak.net Mon Apr 24 13:23:34 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 24 Apr 2006 13:23:34 +0200 (CEST) Subject: [pypy-svn] r26261 - pypy/dist/pypy/translator/cl Message-ID: <20060424112334.B386310081@code0.codespeak.net> Author: auc Date: Mon Apr 24 13:23:33 2006 New Revision: 26261 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: removed unused autopath Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 13:23:33 2006 @@ -1,4 +1,3 @@ -import autopath import os import py From antocuni at codespeak.net Mon Apr 24 14:25:43 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 24 Apr 2006 14:25:43 +0200 (CEST) Subject: [pypy-svn] r26262 - pypy/dist/pypy/rpython/ootypesystem/test Message-ID: <20060424122543.BB9E610087@code0.codespeak.net> Author: antocuni Date: Mon Apr 24 14:25:39 2006 New Revision: 26262 Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Log: Fixed some little bugs in tests. Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Mon Apr 24 14:25:39 2006 @@ -10,11 +10,12 @@ def test_length(): DT = Dict(Signed, Float) d = new(DT) - d.ll_setitem(42, 123.45) + d.ll_set(42, 123.45) assert d.ll_length() == 1 def test_setitem_getitem(): DT = Dict(Signed, Float) d = new(DT) - d.ll_setitem(42, 123.45) - assert d.ll_getitem(42) == 123.45 + d.ll_set(42, 123.45) + assert d.ll_get(42, 0.0) == 123.45 + assert d.ll_get(43, 0.0) == 0.0 From sanxiyn at codespeak.net Mon Apr 24 16:16:16 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 16:16:16 +0200 (CEST) Subject: [pypy-svn] r26264 - pypy/dist/pypy/translator/cl Message-ID: <20060424141616.BE9621007F@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 16:16:13 2006 New Revision: 26264 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Get rid of the initial newline in generated lisp sources Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 16:16:13 2006 @@ -134,7 +134,10 @@ lines = list(self.emit()) declarations = "\n".join(self.declarations) code = "\n".join(lines) - return declarations + "\n" + code + "\n" + if declarations: + return declarations + "\n" + code + "\n" + else: + return code + "\n" def emit(self): while self.pendinggraphs: From sanxiyn at codespeak.net Mon Apr 24 16:25:27 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 16:25:27 +0200 (CEST) Subject: [pypy-svn] r26265 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424142527.7FD4110081@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 16:25:22 2006 New Revision: 26265 Added: pypy/dist/pypy/translator/cl/test/test_buildcl.py Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Move tests from buildcl.py's __name__ == '__main__' block to the proper place Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Mon Apr 24 16:25:22 2006 @@ -121,16 +121,3 @@ output = cmdexec("%s %s" % (cl, str(fpath))) return readlisp(output) return _ - -if __name__ == '__main__': - # for test - # ultimately, GenCL's str and conv will move to here - def f(): pass - fun = FlowObjSpace().build_flow(f) - gen = GenCL(fun) - - what = [True, "universe", 42, None, ("of", "them", ["eternal", 95])] - it = writelisp(what) - print what - print it - assert it == '#(t "universe" 42 nil \'("of" "them" #("eternal" 95)))' Added: pypy/dist/pypy/translator/cl/test/test_buildcl.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/test/test_buildcl.py Mon Apr 24 16:25:22 2006 @@ -0,0 +1,10 @@ +from pypy.translator.cl.buildcl import writelisp + +def test_write(): + assert writelisp(True) == 't' + assert writelisp(False) == 'nil' + assert writelisp(42) == '42' + assert writelisp(None) == 'nil' + assert writelisp('answer') == '"answer"' + assert writelisp((2, 3)) == "'(2 3)" + assert writelisp([2, 3]) == "#(2 3)" From antocuni at codespeak.net Mon Apr 24 16:31:50 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 24 Apr 2006 16:31:50 +0200 (CEST) Subject: [pypy-svn] r26266 - in pypy/dist/pypy/rpython: . lltypesystem ootypesystem ootypesystem/test test Message-ID: <20060424143150.9C50110081@code0.codespeak.net> Author: antocuni Date: Mon Apr 24 16:31:40 2006 New Revision: 26266 Modified: pypy/dist/pypy/rpython/lltypesystem/rdict.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py pypy/dist/pypy/rpython/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: - The signature of ootype.Dict.ll_get has been changed - some code has been factored out from lltypesystem.rlist.DictIteratorRepr to rlist.AbstractDictIteratorRepr - added support for iteration and iter* methods to ootypesystem.rdict. The current implementation of itervalues() and iteritems() is far from optimal because it does a dict lookup at every iteration. Probabily we will need a refactoring to get rid of this. Modified: pypy/dist/pypy/rpython/lltypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rdict.py Mon Apr 24 16:31:40 2006 @@ -1,7 +1,8 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.objspace.flow.model import Constant -from pypy.rpython.rdict import AbstractDictRepr, rtype_newdict +from pypy.rpython.rdict import AbstractDictRepr, AbstractDictIteratorRepr,\ + rtype_newdict, dum_variant, dum_keys, dum_values, dum_items from pypy.rpython.lltypesystem import lltype from pypy.rpython.rarithmetic import r_uint from pypy.rpython.objectmodel import hlinvoke @@ -377,13 +378,6 @@ DICT = lltype.typeOf(d).TO return hlinvoke(DICT.r_rdict_eqfn, d.fnkeyeq, key1, key2) -def dum_keys(): pass -def dum_values(): pass -def dum_items():pass -dum_variant = {"keys": dum_keys, - "values": dum_values, - "items": dum_items} - def ll_dict_len(d): return d.num_items @@ -591,7 +585,7 @@ # # Iteration. -class DictIteratorRepr(rmodel.IteratorRepr): +class DictIteratorRepr(AbstractDictIteratorRepr): def __init__(self, r_dict, variant="keys"): self.r_dict = r_dict @@ -599,29 +593,9 @@ self.lowleveltype = lltype.Ptr(lltype.GcStruct('dictiter', ('dict', r_dict.lowleveltype), ('index', lltype.Signed))) + self.ll_dictiter = ll_dictiter + self.ll_dictnext = ll_dictnext - def newiter(self, hop): - v_dict, = hop.inputargs(self.r_dict) - citerptr = hop.inputconst(lltype.Void, self.lowleveltype) - return hop.gendirectcall(ll_dictiter, citerptr, v_dict) - - def rtype_next(self, hop): - variant = self.variant - v_iter, = hop.inputargs(self) - v_func = hop.inputconst(lltype.Void, dum_variant[self.variant]) - if variant in ('keys', 'values'): - c1 = hop.inputconst(lltype.Void, None) - else: - c1 = hop.inputconst(lltype.Void, hop.r_result.lowleveltype) - hop.has_implicit_exception(StopIteration) # record that we know about it - hop.exception_is_here() - v = hop.gendirectcall(ll_dictnext, v_iter, v_func, c1) - if variant == 'keys': - return self.r_dict.recast_key(hop.llops, v) - elif variant == 'values': - return self.r_dict.recast_value(hop.llops, v) - else: - return v def ll_dictiter(ITERPTR, d): iter = lltype.malloc(ITERPTR.TO) Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Mon Apr 24 16:31:40 2006 @@ -297,7 +297,7 @@ self._GENERIC_METHODS = frozendict({ "ll_length": Meth([], Signed), - "ll_get": Meth([self.KEYTYPE_T, self.VALUETYPE_T], self.VALUETYPE_T), # ll_get(key, default) + "ll_get": Meth([self.KEYTYPE_T], self.VALUETYPE_T), "ll_set": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), "ll_remove": Meth([self.KEYTYPE_T], Bool), # return False is key was not present "ll_contains": Meth([self.KEYTYPE_T], Bool), @@ -664,11 +664,11 @@ # NOT_RPYTHON return len(self._dict) - def ll_get(self, key, default): + def ll_get(self, key): # NOT_RPYTHON assert typeOf(key) == self._TYPE._KEYTYPE - assert typeOf(key) == self._TYPE._VALUETYPE - return self._dict.get(key, default) + assert key in self._dict + return self._dict[key] def ll_set(self, key, value): # NOT_RPYTHON Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Mon Apr 24 16:31:40 2006 @@ -1,17 +1,17 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.objspace.flow.model import Constant -from pypy.rpython.rdict import AbstractDictRepr, rtype_newdict +from pypy.rpython.rdict import AbstractDictRepr, AbstractDictIteratorRepr,\ + rtype_newdict, dum_variant, dum_keys, dum_values, dum_items from pypy.rpython.ootypesystem import ootype from pypy.rpython.ootypesystem.rlist import ll_newlist +from pypy.rpython.ootypesystem.riterable import iterator_type from pypy.rpython.rarithmetic import r_uint from pypy.rpython.objectmodel import hlinvoke from pypy.rpython import robject from pypy.rpython import objectmodel from pypy.rpython import rmodel -def dum_values(): pass -def dum_items():pass class DictRepr(AbstractDictRepr): def __init__(self, rtyper, key_repr, value_repr, dictkey, dictvalue, @@ -65,6 +65,9 @@ return hop.genop("oosend", [c_name] + v_args, resulttype=hop.r_result.lowleveltype) + def make_iterator_repr(self, *variant): + return DictIteratorRepr(self, *variant) + def rtype_len(self, hop): v_dict, = hop.inputargs(self) return self.send_message(hop, 'll_length') @@ -77,8 +80,7 @@ v_dict, v_key, v_default = hop.inputargs(self, self.key_repr, self.value_repr) hop.exception_cannot_occur() - v_res = self.send_message(hop, 'll_get') - #v_res = hop.gendirectcall(ll_get, v_dict, v_key, v_default) + v_res = hop.gendirectcall(ll_dict_get, v_dict, v_key, v_default) return self.recast_value(hop.llops, v_res) def rtype_method_setdefault(self, hop): @@ -103,14 +105,19 @@ r_list = hop.r_result cLIST = hop.inputconst(ootype.Void, r_list.lowleveltype) c_func = hop.inputconst(ootype.Void, spec) - c_dummy_default = hop.inputconst(self.value_repr.lowleveltype, - self.value_repr.ll_dummy_value) - return hop.gendirectcall(ll_dict_values_items, v_dict, cLIST, c_func, c_dummy_default) - -## def rtype_method_items(self, hop): -## v_dict, = hop.inputargs(self) -## return hop.gendirectcall(ll_dict_items, v_dict) + return hop.gendirectcall(ll_dict_values_items, v_dict, cLIST, c_func) + def rtype_method_iterkeys(self, hop): + hop.exception_cannot_occur() + return DictIteratorRepr(self, "keys").newiter(hop) + + def rtype_method_itervalues(self, hop): + hop.exception_cannot_occur() + return DictIteratorRepr(self, "values").newiter(hop) + + def rtype_method_iteritems(self, hop): + hop.exception_cannot_occur() + return DictIteratorRepr(self, "items").newiter(hop) class __extend__(pairtype(DictRepr, rmodel.Repr)): @@ -120,8 +127,7 @@ if not r_dict.custom_eq_hash: # TODO: why only in this case? hop.has_implicit_exception(KeyError) # record that we know about it hop.exception_is_here() - c_dummy_default = hop.inputconst(r_dict.value_repr.lowleveltype, r_dict.value_repr.ll_dummy_value) - v_res = hop.gendirectcall(ll_dict_getitem, v_dict, v_key, c_dummy_default) + v_res = hop.gendirectcall(ll_dict_getitem, v_dict, v_key) return r_dict.recast_value(hop.llops, v_res) def rtype_delitem((r_dict, r_key), hop): @@ -153,10 +159,10 @@ # check if a dict is True, allowing for None return bool(d) and d.ll_length() != 0 -def ll_dict_getitem(d, key, dummy_default): +def ll_dict_getitem(d, key): # TODO: this is inefficient because it does two lookups if d.ll_contains(key): - return d.ll_get(key, dummy_default) # dummy_default is never returned + return d.ll_get(key) else: raise KeyError @@ -164,21 +170,28 @@ if not d.ll_remove(key): raise KeyError +def ll_dict_get(d, key, default): + # TODO: this is inefficient because it does two lookups + if d.ll_contains(key): + return d.ll_get(key) + else: + return default + def ll_dict_setdefault(d, key, default): try: - return ll_dict_getitem(d, key, default) + return ll_dict_getitem(d, key) except KeyError: d.ll_set(key, default) return default -def ll_dict_values_items(d, LIST, func, dummy_default): +def ll_dict_values_items(d, LIST, func): keys = d.ll_keys() length = keys.ll_length() result = ll_newlist(LIST, length) i = 0 while i < length: key = keys.ll_getitem_fast(i) - value = d.ll_get(key, dummy_default) # dummy_default is never returned + value = d.ll_get(key) if func is dum_items: r = ootype.new(LIST._ITEMTYPE) r.item0 = key # TODO: do we need casting? @@ -189,3 +202,45 @@ i += 1 return result + + +# ____________________________________________________________ +# +# Iteration. + +class DictIteratorRepr(AbstractDictIteratorRepr): + + def __init__(self, r_dict, variant="keys"): + self.r_dict = r_dict + self.variant = variant + self.lowleveltype = iterator_type(r_dict, r_dict.key_repr) + self.ll_dictiter = ll_dictiter + self.ll_dictnext = ll_dictnext + +def ll_dictiter(ITER, d): + iter = ootype.new(ITER) + iter.iterable = d + iter.index = 0 + return iter + +# TODO: this is very inefficient for values and items because it does +# a dict lookup at every iteration. Need to be refactored. +def ll_dictnext(iter, func, RETURNTYPE): + d = iter.iterable + keys = d.ll_keys() + index = iter.index + if index >= keys.ll_length(): + raise StopIteration + iter.index = index + 1 + + key = keys.ll_getitem_fast(index) + if func is dum_keys: + return key + elif func is dum_values: + return d.ll_get(key) + elif func is dum_items: + res = ootype.new(RETURNTYPE) + res.item0 = key + res.item1 = d.ll_get(key) + return res + Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Mon Apr 24 16:31:40 2006 @@ -17,5 +17,4 @@ DT = Dict(Signed, Float) d = new(DT) d.ll_set(42, 123.45) - assert d.ll_get(42, 0.0) == 123.45 - assert d.ll_get(43, 0.0) == 0.0 + assert d.ll_get(42) == 123.45 Modified: pypy/dist/pypy/rpython/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/rdict.py (original) +++ pypy/dist/pypy/rpython/rdict.py Mon Apr 24 16:31:40 2006 @@ -8,6 +8,13 @@ from pypy.rpython import objectmodel from pypy.rpython import rmodel +def dum_keys(): pass +def dum_values(): pass +def dum_items():pass +dum_variant = {"keys": dum_keys, + "values": dum_values, + "items": dum_items} + class __extend__(annmodel.SomeDict): def rtyper_makerepr(self, rtyper): @@ -71,3 +78,29 @@ cDICT = hop.inputconst(lltype.Void, r_dict.DICT) v_result = hop.gendirectcall(hop.rtyper.type_system.rdict.ll_newdict, cDICT) return v_result + + +class AbstractDictIteratorRepr(rmodel.IteratorRepr): + def newiter(self, hop): + v_dict, = hop.inputargs(self.r_dict) + citerptr = hop.inputconst(lltype.Void, self.lowleveltype) + return hop.gendirectcall(self.ll_dictiter, citerptr, v_dict) + + def rtype_next(self, hop): + variant = self.variant + v_iter, = hop.inputargs(self) + v_func = hop.inputconst(lltype.Void, dum_variant[self.variant]) + if variant in ('keys', 'values'): + c1 = hop.inputconst(lltype.Void, None) + else: + c1 = hop.inputconst(lltype.Void, hop.r_result.lowleveltype) + hop.has_implicit_exception(StopIteration) # record that we know about it + hop.exception_is_here() + v = hop.gendirectcall(self.ll_dictnext, v_iter, v_func, c1) + if variant == 'keys': + return self.r_dict.recast_key(hop.llops, v) + elif variant == 'values': + return self.r_dict.recast_value(hop.llops, v) + else: + return v + Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Mon Apr 24 16:31:40 2006 @@ -819,3 +819,41 @@ res = [self.ll_to_tuple(item, 2) for item in res] assert res == [(42, 43), (13, 14)] or res == [(13, 14), (42, 43)] + def test_iteration(self): + def func(x, y): + d = {x: x+1, y: y+1} + tot = 0 + for key in d: + tot += key + return tot + assert self.interpret(func, [42, 13]) == 55 + + def test_iterkeys(self): + def func(x, y): + d = {x: x+1, y: y+1} + tot = 0 + for key in d.iterkeys(): + tot += key + return tot + assert self.interpret(func, [42, 13]) == 55 + + def test_itervalues(self): + def func(x, y): + d = {x: x+1, y: y+1} + tot = 0 + for value in d.itervalues(): + tot += value + return tot + assert self.interpret(func, [42, 13]) == 57 + + def test_iteritems(self): + def func(x, y): + d = {x: x+1, y: y+1} + tot1 = 0 + tot2 = 0 + for key, value in d.iteritems(): + tot1 += key + tot2 += value + return tot1, tot2 + res = self.ll_to_tuple(self.interpret(func, [42, 13]), 2) + assert res == (55, 57) From sanxiyn at codespeak.net Mon Apr 24 16:32:15 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 16:32:15 +0200 (CEST) Subject: [pypy-svn] r26267 - pypy/dist/pypy/translator/cl Message-ID: <20060424143215.B28FB1008A@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 16:32:12 2006 New Revision: 26267 Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py Log: Import cleanup Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 16:32:12 2006 @@ -1,5 +1,5 @@ -from pypy.objspace.flow.model import Constant, Variable, last_exception -from pypy.rpython.ootypesystem.ootype import _static_meth, Instance +from pypy.objspace.flow.model import Constant, Variable +from pypy.rpython.ootypesystem.ootype import _static_meth def repr_unknown(obj): return '#<%r>' % (obj,) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 16:32:12 2006 @@ -1,9 +1,3 @@ -from pypy.objspace.flow.model import Constant -from pypy.annotation.annrpython import RPythonAnnotator - -from pypy.translator.simplify import simplify_graph -from pypy.translator.transform import transform_graph, default_extra_passes, transform_slice - from pypy.rpython.ootypesystem.ootype import Instance, List from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name From sanxiyn at codespeak.net Mon Apr 24 16:45:45 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 16:45:45 +0200 (CEST) Subject: [pypy-svn] r26269 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424144545.946931008D@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 16:45:41 2006 New Revision: 26269 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: Implement ll_length Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 16:45:41 2006 @@ -103,7 +103,7 @@ self.obj = repr_arg(receiver) def ll_length(self): - pass + return "(length %s)" % (self.obj,) def ll_getitem_fast(self, index): pass Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Mon Apr 24 16:45:41 2006 @@ -10,7 +10,7 @@ cl_new_get_set = make_cl_func(new_get_set) assert cl_new_get_set() == 42 -def dont_test_list_length(): +def test_list_length(): def list_length_one(number): lst = [number] return len(lst) From sanxiyn at codespeak.net Mon Apr 24 17:03:51 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Mon, 24 Apr 2006 17:03:51 +0200 (CEST) Subject: [pypy-svn] r26270 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060424150351.7B77B10091@code0.codespeak.net> Author: sanxiyn Date: Mon Apr 24 17:03:46 2006 New Revision: 26270 Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: Implement ll_getitem_fast and enable the corresponding test Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Mon Apr 24 17:03:46 2006 @@ -1,3 +1,5 @@ +import types + from pypy.objspace.flow.model import Constant, Variable from pypy.rpython.ootypesystem.ootype import _static_meth @@ -14,6 +16,9 @@ return name.replace('_', '-') def repr_const(val): + if isinstance(val, types.FunctionType): + if val.func_name == 'dum_nocheck': # XXX + return "'dummy" if isinstance(val, _static_meth): return repr_fun_name(val._name) # XXX make sure function names are unique if isinstance(val, tuple): Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Mon Apr 24 17:03:46 2006 @@ -106,7 +106,8 @@ return "(length %s)" % (self.obj,) def ll_getitem_fast(self, index): - pass + index = repr_arg(index) + return "(aref %s %s)" % (self.obj, index) def ll_setitem_fast(self, index, value): index = repr_arg(index) Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Mon Apr 24 17:03:46 2006 @@ -17,7 +17,7 @@ cl_list_length_one = make_cl_func(list_length_one, [int]) assert cl_list_length_one(0) == 1 -def dont_test_list_get(): +def test_list_get(): def list_and_get(number): lst = [number] return lst[0] From mwh at codespeak.net Mon Apr 24 17:44:15 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 24 Apr 2006 17:44:15 +0200 (CEST) Subject: [pypy-svn] r26272 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060424154415.0FDAD10081@code0.codespeak.net> Author: mwh Date: Mon Apr 24 17:44:13 2006 New Revision: 26272 Modified: pypy/dist/pypy/translator/stackless/test/test_depth.py pypy/dist/pypy/translator/stackless/transform.py Log: a new test for the stackless transform. this one required a bout of hacking on the transformer: because we do a limited form of type erasure for the return value at least we may need to do some conversion before resumption. this is most striking in the case of addresses/pointers, which is the bulk of this work. unfortunately, i had to worry about annotations and i'm not happy about that. Modified: pypy/dist/pypy/translator/stackless/test/test_depth.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_depth.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_depth.py Mon Apr 24 17:44:13 2006 @@ -28,3 +28,22 @@ res = run_stackless_function(fn, fn, f, g2, g1) assert res.strip() == "10" + +def test_with_ptr(): + def f(n): + if n > 0: + res = f(n-1) + else: + res = code.stack_frames_depth(), 1 + return res + + def fn(ignored): + count0, _ = f(0) + count10, _ = f(10) + return count10 - count0 + + res = llinterp_stackless_function(fn, fn, f) + assert res == 10 + + res = run_stackless_function(fn, fn, f) + assert res.strip() == "10" Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Mon Apr 24 17:44:13 2006 @@ -3,7 +3,7 @@ from pypy.translator.backendopt import support from pypy.objspace.flow import model from pypy.rpython.memory.gctransform import varoftype -from pypy.translator.unsimplify import copyvar +from pypy.translator import unsimplify from pypy.annotation import model as annmodel from pypy.rpython.annlowlevel import MixLevelHelperAnnotator from pypy.translator.stackless import code @@ -170,6 +170,8 @@ if self.resume_points: self.insert_resume_handling(graph) + model.checkgraph(graph) + self.curr_graph = None def ops_read_global_state_field(self, targetvar, fieldname): @@ -195,7 +197,7 @@ def insert_resume_handling(self, graph): old_start_block = graph.startblock - newinputargs = [copyvar(self.translator, v) + newinputargs = [unsimplify.copyvar(self.translator, v) for v in old_start_block.inputargs] new_start_block = model.Block(newinputargs) var_resume_state = varoftype(lltype.Signed) @@ -221,7 +223,7 @@ varoftype(lltype.Void))) varmap = {} for i, arg in enumerate(resume_point.args): - newarg = varmap[arg] = copyvar(self.translator, arg) + newarg = varmap[arg] = unsimplify.copyvar(self.translator, arg) assert arg is not resume_point.var_result fname = model.Constant(frame_state_type._names[i+1], lltype.Void) ops.append(model.SpaceOperation( @@ -232,6 +234,9 @@ rettype = STORAGE_TYPES[r] else: rettype = lltype.Void + + need_address_conversion = False + if rettype == lltype.Signed: getretval = self.fetch_retval_long_ptr if rettype == lltype.SignedLongLong: @@ -241,9 +246,21 @@ elif rettype == lltype.Float: getretval = self.fetch_retval_float_ptr elif rettype == llmemory.Address: + if resume_point.var_result.concretetype is not \ + llmemory.Address: + if resume_point.var_result in \ + resume_point.links_to_resumption[0].args: + need_address_conversion = True getretval = self.fetch_retval_void_p_ptr - varmap[resume_point.var_result] = retval = ( - copyvar(self.translator, resume_point.var_result)) + + if need_address_conversion: + varmap[resume_point.var_result] = retval = ( + varoftype(llmemory.Address)) + self.translator.annotator.setbinding( + retval, annmodel.SomeAddress()) + else: + varmap[resume_point.var_result] = retval = ( + unsimplify.copyvar(self.translator, resume_point.var_result)) ops.append(model.SpaceOperation("direct_call", [getretval], retval)) newblock.operations.extend(ops) @@ -254,7 +271,7 @@ return varmap[arg] else: assert arg in [l.last_exception, l.last_exc_value] - r = copyvar(self.translator, arg) + r = unsimplify.copyvar(self.translator, arg) varmap[arg] = r return r else: @@ -267,6 +284,20 @@ newblock.exitswitch = model.c_last_exception else: newblock.exitswitch = None + + if need_address_conversion: + convertblock = unsimplify.insert_empty_block( + self.translator, newblock.exits[0]) + returnvarindex = newblock.exits[0].args.index(retval) + newvar = unsimplify.copyvar(self.translator, resume_point.var_result) + convertblock.operations.append( + model.SpaceOperation("cast_adr_to_ptr", + [convertblock.inputargs[returnvarindex]], + newvar)) + convertblock.exits[0].args[returnvarindex] = newvar + self.translator.rtyper.insert_link_conversions(convertblock) + + self.translator.rtyper.insert_link_conversions(newblock) resuming_links.append( model.Link([], newblock, resume_point_index+1)) @@ -293,6 +324,18 @@ link = block.exits[0] else: link = support.split_block_with_keepalive(block, i+1) + # this section deserves a whinge: + + # i want to use rtyper.insert_link_conversions() in + # insert_resume_handling(). insert_link_conversions() + # calls bindingrepr(), which depends on variables having + # annotations. split_block called copyvar(None, ...) + # which doesn't preserve the annotation. so put it back + # here. it certainly sucks that this module has to worry + # about annotations :( + ann = self.translator.annotator + for f, t in zip(link.args, link.target.inputargs): + ann.setbinding(t, ann.binding(f)) block.exitswitch = model.c_last_exception link.llexitcase = None var_unwind_exception = varoftype(evalue) @@ -345,8 +388,9 @@ edata = rtyper.getexceptiondata() etype = edata.lltype_of_exception_type evalue = edata.lltype_of_exception_value - inputargs = [copyvar(self.translator, v) for v in varstosave] - var_unwind_exception = copyvar(self.translator, var_unwind_exception) + inputargs = [unsimplify.copyvar(self.translator, v) for v in varstosave] + var_unwind_exception = unsimplify.copyvar( + self.translator, var_unwind_exception) fields = [] n = [] From arigo at codespeak.net Mon Apr 24 17:56:02 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 24 Apr 2006 17:56:02 +0200 (CEST) Subject: [pypy-svn] r26273 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060424155602.0F40410087@code0.codespeak.net> Author: arigo Date: Mon Apr 24 17:56:00 2006 New Revision: 26273 Modified: pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: rctypes: annotation of user-defined RPython-level callback functions. Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Mon Apr 24 17:56:00 2006 @@ -1,7 +1,7 @@ from pypy.annotation.model import SomeCTypesObject from pypy.annotation import model as annmodel from pypy.rpython.error import TyperError -from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.rpython.rctypes.implementation import CTypesEntry from pypy.rpython.lltypesystem import lltype import ctypes @@ -10,7 +10,7 @@ CFuncPtrType = type(ctypes.CFUNCTYPE(None)) -class CallEntry(ExtRegistryEntry): +class CallEntry(CTypesEntry): """Annotation and rtyping of calls to external functions declared with ctypes. """ @@ -33,6 +33,33 @@ s_result = SomeCTypesObject(result_ctype, SomeCTypesObject.OWNSMEMORY) return s_result.return_annotation() + def object_seen(self, bookkeeper): + "Called when the annotator sees this ctypes function object." + # if the function is a Python callback, emulate a call to it + # so that the callback is properly annotated + if hasattr(self.instance, 'callback'): + callback = self.instance.callback + argtypes = self.instance.argtypes + restype = self.instance.restype + s_callback = bookkeeper.immutablevalue(callback) + # the input arg annotations, which are automatically unwrapped + args_s = [bookkeeper.valueoftype(ctype).return_annotation() + for ctype in argtypes] + uniquekey = (callback, argtypes, restype) + s_res = bookkeeper.emulate_pbc_call(uniquekey, s_callback, args_s) + # check the result type + if restype is None: + s_expected = annmodel.s_None + else: + s_expected = bookkeeper.valueoftype(restype) + # can also return the unwrapped version of the ctype, + # e.g. an int instead of a c_int + s_orelse = s_expected.return_annotation() + assert s_expected.contains(s_res) or s_orelse.contains(s_res), ( + "%r should return a %s but returned %s" % (callback, + restype, + s_res)) + def specialize_call(self, hop): from pypy.rpython.rctypes.rmodel import CTypesValueRepr cfuncptr = self.instance Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Mon Apr 24 17:56:00 2006 @@ -2,20 +2,103 @@ from pypy.annotation.model import SomeCTypesObject from pypy.rpython import extregistry from pypy.rpython.extregistry import ExtRegistryEntry +import ctypes +# rctypes version of ctypes.CFUNCTYPE. +# It's required to work around three limitations of CFUNCTYPE: +# +# * There is no PY_ version to make callbacks for CPython, which +# expects the callback to follow the usual conventions (NULL = error). +# +# * The wrapped callback is not exposed in any special attribute, so +# if rctypes sees a CFunctionType object it can't find the Python callback +# +# * I would expect a return type of py_object to mean that if the +# callback Python function returns a py_object, the C caller sees the +# PyObject* inside. Wrong: it sees the py_object wrapper itself. For +# consistency -- and also because unwrapping the py_object manually is +# not allowed annotation-wise -- we change the semantics here under +# the nose of the annotator. + +_c_callback_functype_cache = {} +def CALLBACK_FUNCTYPE(restype, *argtypes, **flags): + if 'callconv' in flags: + callconv = flags.pop('callconv') + else: + callconv = ctypes.CDLL + assert not flags, "unknown keyword arguments %r" % (flags.keys(),) + try: + return _c_callback_functype_cache[(restype, argtypes)] + except KeyError: + class CallbackFunctionType(ctypes._CFuncPtr): + _argtypes_ = argtypes + _restype_ = restype + _flags_ = callconv._FuncPtr._flags_ + + def __new__(cls, callback): + assert callable(callback) + if issubclass(restype, ctypes.py_object): + def func(*args, **kwds): + w_res = callback(*args, **kwds) + assert isinstance(w_res, py_object) + return w_res.value + else: + func = callback + res = super(CallbackFunctionType, cls).__new__(cls, func) + res.callback = callback + return res -class CTypesCallEntry(ExtRegistryEntry): + _c_callback_functype_cache[(restype, argtypes)] = CallbackFunctionType + return CallbackFunctionType + +# ____________________________________________________________ + +class CTypesEntry(ExtRegistryEntry): + + def compute_annotation(self): + self.ctype_object_discovered() + return super(CTypesEntry, self).compute_annotation() + + def ctype_object_discovered(self): + if self.instance is None: + return + from pypy.annotation.bookkeeper import getbookkeeper + bookkeeper = getbookkeeper() + if bookkeeper is None: + return + # follow all dependent ctypes objects in order to discover + # all callback functions + memo = {} + def recfind(o): + if id(o) in memo: + return + memo[id(o)] = o + if isinstance(o, dict): + for x in o.itervalues(): + recfind(x) + elif extregistry.is_registered(o): + entry = extregistry.lookup(o) + if isinstance(entry, CTypesEntry): + entry.object_seen(bookkeeper) + recfind(self.instance._objects) + + def object_seen(self, bookkeeper): + """To be overriden for ctypes objects whose mere presence influences + annotation, e.g. callback functions.""" + + +class CTypesCallEntry(CTypesEntry): "Annotation and rtyping of calls to ctypes types." def compute_result_annotation(self, *args_s, **kwds_s): ctype = self.instance # the ctype is the called object return SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) - -class CTypesObjEntry(ExtRegistryEntry): +class CTypesObjEntry(CTypesEntry): "Annotation and rtyping of ctypes instances." def compute_annotation(self): + self.ctype_object_discovered() ctype = self.type return SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Mon Apr 24 17:56:00 2006 @@ -4,17 +4,18 @@ import py import sys -import pypy.rpython.rctypes.implementation +from pypy.rpython.rctypes.implementation import CALLBACK_FUNCTYPE from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.test.test_llinterp import interpret from pypy.translator.c.test.test_genc import compile from pypy import conftest from pypy.rpython.rstr import string_repr from pypy.rpython.lltypesystem import lltype -from ctypes import cdll, pythonapi, _FUNCFLAG_PYTHONAPI +from ctypes import cdll, pythonapi, PyDLL, _FUNCFLAG_PYTHONAPI from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer -from ctypes import POINTER, py_object, byref +from ctypes import POINTER, py_object, byref, Structure from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 # __________ the standard C library __________ @@ -54,6 +55,14 @@ ctime.restype = c_char_p #ctimes.argtypes: omitted for this test +IntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int) +def mycallback(n): + return n+3 +callback = IntIntCallback(mycallback) + +PyIntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int, callconv=PyDLL) +pycallback = PyIntIntCallback(mycallback) + def test_labs(n=6): assert labs(n) == abs(n) @@ -97,6 +106,10 @@ s2 = ctime(byref(c_long(N))) assert s1.strip() == s2.strip() +def test_callback(): + assert callback(100) == 103 + assert pycallback(100) == 103 + class Test_annotation: def test_annotate_labs(self): a = RPythonAnnotator() @@ -132,6 +145,41 @@ if conftest.option.view: a.translator.view() + def test_annotate_callback(self): + def fn(n): + return callback(n) + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(fn, [int]) + assert s.knowntype == int + if conftest.option.view: + a.translator.view() + graph = graphof(t, mycallback) + [v1] = graph.getargs() + v2 = graph.getreturnvar() + assert a.binding(v1).knowntype == int + assert a.binding(v2).knowntype == int + + def test_annotation_indirectly_found_callback(self): + class S(Structure): + _fields_ = [('vtable', IntIntCallback*5), + ('z', c_int)] + s = S(z=3) + s.vtable[2] = callback + def fn(): + return s.z + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(fn, []) + assert s.knowntype == int + if conftest.option.view: + a.translator.view() + graph = graphof(t, mycallback) + [v1] = graph.getargs() + v2 = graph.getreturnvar() + assert a.binding(v1).knowntype == int + assert a.binding(v2).knowntype == int + class Test_specialization: def test_specialize_labs(self): res = interpret(test_labs, [-11]) From arigo at codespeak.net Mon Apr 24 19:24:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 24 Apr 2006 19:24:58 +0200 (CEST) Subject: [pypy-svn] r26275 - in pypy/dist/pypy: objspace/cpy objspace/cpy/test rpython/rctypes translator/goal Message-ID: <20060424172458.7457910084@code0.codespeak.net> Author: arigo Date: Mon Apr 24 19:24:56 2006 New Revision: 26275 Added: pypy/dist/pypy/objspace/cpy/ann_policy.py (contents, props changed) Modified: pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/objspace/cpy/wrappable.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Remove the hacks from wrappable.py and build a real CPython built-in function object, using a freshly built PyMethodDef structure that points to a ctypes callback. Runs and (with some efforts) annotates correctly. Added: pypy/dist/pypy/objspace/cpy/ann_policy.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/ann_policy.py Mon Apr 24 19:24:56 2006 @@ -0,0 +1,3 @@ +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy + +CPyAnnotatorPolicy = PyPyAnnotatorPolicy # for now Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Mon Apr 24 19:24:56 2006 @@ -1,7 +1,9 @@ import sys +import ctypes from ctypes import * from pypy.rpython.rctypes.tool import ctypes_platform from pypy.rpython.rctypes import apyobject +from pypy.rpython.rctypes.implementation import CALLBACK_FUNCTYPE class W_Object(py_object): "A py_object subclass, representing wrapped objects for the CPyObjSpace." @@ -14,6 +16,12 @@ ############################################################### # ____________________ Types and constants ____________________ +PyCFunction = CALLBACK_FUNCTYPE(W_Object, W_Object, W_Object, callconv=PyDLL) +PyNoArgsFunction = CALLBACK_FUNCTYPE(W_Object, W_Object, callconv=PyDLL) +PyCFunctionWithKeywords = CALLBACK_FUNCTYPE(W_Object, + W_Object, W_Object, W_Object, + callconv=PyDLL) + class CConfig: _header_ = """ #include @@ -34,7 +42,7 @@ PyMethodDef = ctypes_platform.Struct('PyMethodDef', [('ml_name', c_char_p), - ('ml_meth', c_void_p), + ('ml_meth', PyCFunction), ('ml_flags', c_int), ('ml_doc', c_char_p)]) METH_VARARGS = ctypes_platform.ConstantInteger('METH_VARARGS') Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Mon Apr 24 19:24:56 2006 @@ -2,11 +2,12 @@ from pypy.translator.c.test.test_genc import compile from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext, graphof -from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy +from pypy.objspace.cpy.ann_policy import CPyAnnotatorPolicy from pypy.objspace.cpy.objspace import CPyObjSpace import pypy.rpython.rctypes.implementation from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root +from pypy import conftest def test_demo(): @@ -17,7 +18,7 @@ return demo.measuretime(space, n, w_callable) fn = compile(entry_point, [int, CPyObjSpace.W_Object], - annotatorpolicy = PyPyAnnotatorPolicy()) + annotatorpolicy = CPyAnnotatorPolicy()) res = fn(10, long) assert isinstance(res, int) @@ -43,17 +44,22 @@ def test_annotate_bltinfunc(): entrypoint = maketest() t = TranslationContext() - a = t.buildannotator(policy=PyPyAnnotatorPolicy()) + a = t.buildannotator(policy=CPyAnnotatorPolicy()) s = a.build_types(entrypoint, [int]) + if conftest.option.view: + t.view() assert s.knowntype == int graph = graphof(t, myfunc) assert len(graph.getargs()) == 2 s = a.binding(graph.getargs()[1]) assert s.knowntype == CPyObjSpace.W_Object + s = a.binding(graph.getreturnvar()) + assert s.knowntype == CPyObjSpace.W_Object def test_compile_bltinfunc(): py.test.skip("in-progress") entrypoint = maketest() - fn = compile(entrypoint, [int], annotatorpolicy=PyPyAnnotatorPolicy()) + fn = compile(entrypoint, [int], + annotatorpolicy = CPyAnnotatorPolicy()) res = fn(-6) assert res == -42 Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Mon Apr 24 19:24:56 2006 @@ -9,7 +9,6 @@ from pypy.objspace.cpy.objspace import CPyObjSpace from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root -from pypy.rpython import extregistry class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): @@ -21,37 +20,20 @@ bltin = factory.behavior unwrap_spec = factory.unwrap_spec - assert unwrap_spec[0] == ObjSpace - for spec in unwrap_spec[1:]: - assert spec == W_Root # XXX - - def trampoline(*args): - args_w = [space.wrap(a) for a in args] - w_result = bltin(space, *args_w) - return space.unwrap(w_result) - - w_result = W_Object(trampoline) - - # override the annotation behavior of 'w_result' - # to emulate a call to the bltin function at interp-level - BaseEntry = extregistry._lookup_cls(w_result) - uniquekey = bltin - nb_args = len(unwrap_spec) - 1 - - class TrampolineEntry(BaseEntry): - _about_ = w_result - def compute_annotation(self): - from pypy.annotation.bookkeeper import getbookkeeper - bookkeeper = getbookkeeper() - s_bltin = bookkeeper.immutablevalue(bltin) - s_space = bookkeeper.immutablevalue(space) - s_w_obj = bookkeeper.valueoftype(W_Object) - args_s = [s_space] + [s_w_obj]*nb_args - s_result = bookkeeper.emulate_pbc_call(uniquekey, s_bltin, - args_s) - assert s_w_obj.contains(s_result), ( - "%r should return a wrapped obj, got %r instead" % ( - bltin, s_result)) - return super(TrampolineEntry, self).compute_annotation() + assert unwrap_spec == [ObjSpace, W_Root] # XXX for now + # make a real CPython built-in function from a PyMethodDef + def callback(w_self, w_args): + "XXX minimalistic" + w_a = PyObject_GetItem(w_args, 0) + w_result = bltin(space, w_a) + return w_result + + ml = PyMethodDef(ml_name = factory.b_name, + ml_meth = PyCFunction(callback), + ml_flags = METH_VARARGS, + #ml_doc = ..., + ) + w_result = PyCFunction_NewEx(byref(ml), None, func.w_module) + w_result.ml = ml # keep ml alive as long as w_result is around return w_result Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Mon Apr 24 19:24:56 2006 @@ -80,7 +80,9 @@ entry = extregistry.lookup(o) if isinstance(entry, CTypesEntry): entry.object_seen(bookkeeper) - recfind(self.instance._objects) + recfind(o._objects) + recfind(o.__dict__) # for extra keepalives + recfind(self.instance) def object_seen(self, bookkeeper): """To be overriden for ctypes objects whose mere presence influences Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Mon Apr 24 19:24:56 2006 @@ -1,5 +1,5 @@ from pypy.module._demo import demo -from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy +from pypy.objspace.cpy.ann_policy import CPyAnnotatorPolicy from pypy.objspace.cpy.objspace import CPyObjSpace import pypy.rpython.rctypes.implementation @@ -12,7 +12,7 @@ # _____ Define and setup target ___ def target(*args): - return entry_point, [int, CPyObjSpace.W_Object], PyPyAnnotatorPolicy() + return entry_point, [int, CPyObjSpace.W_Object], CPyAnnotatorPolicy() if __name__ == '__main__': From arigo at codespeak.net Mon Apr 24 20:06:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 24 Apr 2006 20:06:55 +0200 (CEST) Subject: [pypy-svn] r26276 - pypy/dist/pypy/objspace/cpy Message-ID: <20060424180655.3893B10087@code0.codespeak.net> Author: arigo Date: Mon Apr 24 20:06:54 2006 New Revision: 26276 Modified: pypy/dist/pypy/objspace/cpy/wrappable.py Log: Make test_wrappable pass. The problem is that ctypes doesn't support callbacks of CPython calling conventions -- only of general C calling conventions. Hack hack hack needed. Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Mon Apr 24 20:06:54 2006 @@ -36,4 +36,15 @@ ) w_result = PyCFunction_NewEx(byref(ml), None, func.w_module) w_result.ml = ml # keep ml alive as long as w_result is around - return w_result + + # argh! callbacks of mode PyDLL are not supported by ctypes so far + # (as of 0.9.9.4). XXX hack. I am not happy. + + def hackish_trampoline(*args): + args_w = [space.W_Object(a) for a in args] + w_result = bltin(space, *args_w) + return w_result.value + + w_pseudoresult = W_Object(hackish_trampoline) + w_pseudoresult._hack_replace_with_ = w_result + return w_pseudoresult From sanxiyn at codespeak.net Tue Apr 25 01:36:01 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 01:36:01 +0200 (CEST) Subject: [pypy-svn] r26279 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060424233601.0408710073@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 01:35:53 2006 New Revision: 26279 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Insert newlines here and there to correct ReST issues Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Tue Apr 25 01:35:53 2006 @@ -6,8 +6,8 @@ Break day: Wednesday -Sunday, 23. April 2006: -*********************** +Sunday, 23. April 2006 +********************** Present: Samuele, Arre, Valentino, Eric, Yutaka, Sanghyeon, Yusei, Bea, Nik, George @@ -19,8 +19,10 @@ - Progress further on an 'rctypes' module aiming at letting us use a ctypes implementation of an extension module from the compiled pypy-c. + - check status - ctypes time and socket (and select ;)) + (Arre, Valentino, Eric) - rctypes support for LLVM (Eric) @@ -46,7 +48,7 @@ - general wizardry (Samuele) -Monday, 24th April 2006: +Monday, 24th April 2006 *********************** Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Bea, Nik, Anders, Daiki @@ -54,6 +56,7 @@ Status after day 1: - gensqueak/ootypesystem (Nik) + * introduction done * steps towards making genclisp use current infrastructure @@ -61,26 +64,32 @@ - Progress further on an 'rctypes' module aiming at letting us use a ctypes implementation of an extension module from the compiled pypy-c. + - check status (DONE) - ctypes time and socket (and select ;)) + * added errno functionality * refactor to latest ctypes version * wrote test for finding missing socket methods+added stubs + (Arre, Valentino, Eric) Pairings today: - ctypes time and socket (and select ;)) (Arre, Valentino, Eric, Sanghyeon) + * Anders will join arre, Valentino and Eric - ootypesystem introduction, gensqueak/gencl (Nik, Yutaka, Yusei, George) + * more steps towards making genclisp use current infrastructure (Nik, Sanghyeon) * during sprint some ootype issues will be resolved by - Samuele and Nik (tuples list...) + Samuele and Nik (tuples list...) - general wizardry (Samuele) + * bit running around and working with Nik in the afternoon on ootype From sanxiyn at codespeak.net Tue Apr 25 01:41:08 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 01:41:08 +0200 (CEST) Subject: [pypy-svn] r26280 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060424234108.9D2DE10073@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 01:41:05 2006 New Revision: 26280 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Correct various factual errors in the sprint report Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Tue Apr 25 01:41:05 2006 @@ -48,10 +48,11 @@ - general wizardry (Samuele) + Monday, 24th April 2006 *********************** -Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Bea, Nik, Anders, Daiki +Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Bea, Nik, Anders, Daiki, and later, Yutaka and Yusei Status after day 1: @@ -72,20 +73,20 @@ * refactor to latest ctypes version * wrote test for finding missing socket methods+added stubs - (Arre, Valentino, Eric) + (Arre, Valentino, Eric, Sanghyeon) Pairings today: - ctypes time and socket (and select ;)) - (Arre, Valentino, Eric, Sanghyeon) + (Arre, Valentino, Eric) - * Anders will join arre, Valentino and Eric + * Anders will join Arre, Valentino and Eric - ootypesystem introduction, gensqueak/gencl - (Nik, Yutaka, Yusei, George) + (Nik, Yusei, Sanghyeon) - * more steps towards making genclisp use current infrastructure - (Nik, Sanghyeon) + * more steps towards making gencl use current infrastructure + (Nik, Yusei, Sanghyeon) * during sprint some ootype issues will be resolved by Samuele and Nik (tuples list...) From ericvrp at codespeak.net Tue Apr 25 03:13:33 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 25 Apr 2006 03:13:33 +0200 (CEST) Subject: [pypy-svn] r26281 - pypy/dist/pypy/interpreter/test Message-ID: <20060425011333.90F861007E@code0.codespeak.net> Author: ericvrp Date: Tue Apr 25 03:13:23 2006 New Revision: 26281 Added: pypy/dist/pypy/interpreter/test/test_pickle.py (contents, props changed) Log: First pickle tests for cells and generators. Still disabled because we have no support for this yet. Added: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Tue Apr 25 03:13:23 2006 @@ -0,0 +1,49 @@ +import py +import pickle + +def test_pickle_cell(): + py.test.skip("cell pickling is work in progress") + def g(): + x = None + def f(): + return x + return f.func_closure[0] + try: + cell = g() + pickle.dumps(cell) + except IndexError, e: + raise + +def test_pickle_generator(): + py.test.skip("generator pickling is work in progress") + def giveme(n): + x = 0 + while x < n: + yield x + generator = giveme(10) + pickle.dumps(generator) + +#TODO: test pickling of code objects +#TODO: test pickling of function objects +#TODO: test pickling of frame objects +#TODO: test pickling of tracebacks +#TODO: test pickling of modules + +''' +etc. etc. etc. +init_codetype() +init_functype() +init_celltype() +init_frametype() +init_tracebacktype() +init_moduletype() +init_moduledicttype() +init_itertype() +init_methodtype() +init_dictitertype() +init_enumtype() +init_enumfactorytype() +init_listitertype() +init_rangeitertype() +init_tupleitertype() +''' From dialtone at codespeak.net Tue Apr 25 03:26:22 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Tue, 25 Apr 2006 03:26:22 +0200 (CEST) Subject: [pypy-svn] r26282 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060425012622.43A1A10084@code0.codespeak.net> Author: dialtone Date: Tue Apr 25 03:26:18 2006 New Revision: 26282 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: planning Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Tue Apr 25 03:26:18 2006 @@ -94,3 +94,42 @@ * bit running around and working with Nik in the afternoon on ootype + +Tuesday, 25th April 2006 +************************ + +Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Bea, Nik, Anders, Daiki + +Status after day 2: + + - Orientation on pickling support for cells and generators for being able + to pickle tasklets (Eric) + + - gencl (Sanghyeon, Nik, Yusei) + * instance creation and function call + * code reorganization and cleanup (and Valentino) + + - Progress further on an 'rctypes' module aiming at letting us use a ctypes + implementation of an extension module from the compiled pypy-c. + + - ctypes time and socket (and select ;)) + + * sockets now listen for connections and connect correctly + * they also actually send and receive data + * lower level ctypes wrapper almost complete + + (Arre, Valentino, Eric, Anders) + +Pairings today: + + - Work on pickling support for cells and generators for being able + to pickle tasklets (Eric) + + - reorganize the new socket module in the right project location (Arre, Anders) + + - gencl + * compile pystone in gencl + (Valentino, Sanghyeon) + + - Coneptual design decisions about ootypesystem (tuples list...) (Samuele, Nik) + From sanxiyn at codespeak.net Tue Apr 25 04:53:27 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 04:53:27 +0200 (CEST) Subject: [pypy-svn] r26283 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060425025327.314561007E@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 04:53:21 2006 New Revision: 26283 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (dialtone, sanxiyn) Support generation of methods of classes Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Tue Apr 25 04:53:21 2006 @@ -95,12 +95,10 @@ if conftest.option.view: t.view() - graph = t.graphs[0] - - gen = GenCL(graph, argtypes) + gen = GenCL(t, func, argtypes) out = gen.emitcode() i = 1 - fpath = path.join("%s.lisp" % graph.name) + fpath = path.join("%s.lisp" % func.func_name) if conftest.option.prettyprint: script = path.join(".printer.lisp") @@ -111,7 +109,7 @@ def _(*args): fpath.write(out) fp = file(str(fpath), "a") - print >>fp, "(write (", repr_fun_name(graph.name), + print >>fp, "(write (", repr_fun_name(func.func_name), for arg in args: print >>fp, writelisp(arg), print >>fp, "))" Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 04:53:21 2006 @@ -1,4 +1,7 @@ -from pypy.rpython.ootypesystem.ootype import Instance, List +import types + +from pypy.translator.translator import graphof +from pypy.rpython.ootypesystem.ootype import Instance, List, _static_meth, _meth from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name @@ -54,8 +57,8 @@ yield "(setf %s (not (zerop %s)))" % (result, arg) def op_direct_call(self, result, fun, *args): - graph = self.args[0].value.graph - self.gen.pendinggraphs.append(graph) + funobj = self.args[0].value + self.gen.pendinggraphs.append(funobj) args = " ".join(args) yield "(setf %s (%s %s))" % (result, fun, args) @@ -80,11 +83,19 @@ def op_oosend(self, result, *ignore): method = self.args[0].value receiver = self.args[1] + cls = receiver.concretetype args = self.args[2:] - if isinstance(receiver.concretetype, List): + if isinstance(cls, List): impl = ListImpl(receiver) code = getattr(impl, method)(*args) - yield "(setf %s %s)" % (result, code) + yield "(setf %s %s)" % (result, code) + elif isinstance(cls, Instance): + methodobj = cls._methods[method] + methodobj._method_name = method # XXX + self.gen.pendinggraphs.append(methodobj) + args = map(repr_arg, args) + args = " ".join(args) + yield "(setf %s (%s %s %s))" % (result, repr_fun_name(method), repr_arg(receiver), args) def op_oogetfield(self, result, obj, _): fieldname = self.args[1].value @@ -121,8 +132,9 @@ class GenCL: - def __init__(self, entry_point, input_arg_types=[]): - self.pendinggraphs = [entry_point] + def __init__(self, context, funobj, input_arg_types=[]): + self.context = context + self.pendinggraphs = [funobj] self.declarations = [] def emitcode(self, public=True): @@ -136,15 +148,40 @@ def emit(self): while self.pendinggraphs: - graph = self.pendinggraphs.pop() - for line in self.emit_defun(graph): - yield line + obj = self.pendinggraphs.pop() + if isinstance(obj, types.FunctionType): + graph = graphof(self.context, obj) + for line in self.emit_defun(graph): + yield line + elif isinstance(obj, _static_meth): + graph = obj.graph + for line in self.emit_defun(graph): + yield line + elif isinstance(obj, _meth): + graph = obj.graph + name = obj._method_name # XXX + for line in self.emit_defmethod(graph, name): + yield line def emit_defun(self, fun): yield "(defun " + repr_fun_name(fun.name) arglist = fun.getargs() args = " ".join(map(repr_var, arglist)) yield "(%s)" % (args,) + for line in self.emit_body(fun, arglist): + yield line + + def emit_defmethod(self, fun, name): + yield "(defmethod %s" % (repr_fun_name(name)) + arglist = fun.getargs() + cls = arglist[0].concretetype + selfvar = repr_var(arglist[0]) + args = " ".join(map(repr_var, arglist[1:])) + yield "((%s %s) %s)" % (selfvar, repr_class_name(cls._name), args) + for line in self.emit_body(fun, arglist): + yield line + + def emit_body(self, fun, arglist): yield "(prog" blocklist = list(fun.iterblocks()) vardict = {} Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Tue Apr 25 04:53:21 2006 @@ -10,6 +10,21 @@ cl_new_get_set = make_cl_func(new_get_set) assert cl_new_get_set() == 42 +def test_inc(): + class IntHolder: + def __init__(self, number): + self.number = number + def inc(self): + self.number += 1 + def get(self): + return self.number + def inc(number): + obj = IntHolder(number) + obj.inc() + return obj.get() + cl_inc = make_cl_func(inc, [int]) + assert cl_inc(5) == 6 + def test_list_length(): def list_length_one(number): lst = [number] From dialtone at codespeak.net Tue Apr 25 05:22:33 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Tue, 25 Apr 2006 05:22:33 +0200 (CEST) Subject: [pypy-svn] r26284 - pypy/dist/pypy/translator Message-ID: <20060425032233.041271007E@code0.codespeak.net> Author: dialtone Date: Tue Apr 25 05:22:30 2006 New Revision: 26284 Modified: pypy/dist/pypy/translator/interactive.py Log: (sanxiyn, dialtone) add source_cl to interactive.py, first step towards a cl backend target Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Tue Apr 25 05:22:30 2006 @@ -56,6 +56,7 @@ 'source_c': [], 'compile_c': [], 'compile_llvm': [], + 'source_cl': [], } def view(self): @@ -153,6 +154,11 @@ self.ensure_backend('llvm') self.driver.source_llvm() + def source_cl(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) + self.ensure_backend('cl') + self.driver.source_cl() + def compile(self, argtypes=None, **kwds): self.update_options(argtypes, kwds) backend = self.ensure_backend() From sanxiyn at codespeak.net Tue Apr 25 05:24:29 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 05:24:29 +0200 (CEST) Subject: [pypy-svn] r26285 - in pypy/dist/pypy/translator: . goal Message-ID: <20060425032429.D098F1007E@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 05:24:25 2006 New Revision: 26285 Modified: pypy/dist/pypy/translator/driver.py pypy/dist/pypy/translator/goal/translate.py Log: (dialtone, sanxiyn) Intermediate checkin, add Common Lisp source generation task to the translation driver Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Tue Apr 25 05:24:25 2006 @@ -357,6 +357,13 @@ "Running compiled llvm source", idemp=True) + def task_source_cl(self): + from pypy.translator.cl.gencl import GenCL + self.gen = GenCL(self.translator, self.entry_point) + self.gen.emitcode() + task_source_cl = taskdef(task_source_cl, ['backendopt', 'rtype'], + 'Generating Common Lisp source') + def proceed(self, goals): if not goals: if self.default_goal: Modified: pypy/dist/pypy/translator/goal/translate.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate.py (original) +++ pypy/dist/pypy/translator/goal/translate.py Tue Apr 25 05:24:25 2006 @@ -46,7 +46,7 @@ '0_source': [OPT(('-s', '--source'), "Generate source code", GOAL), OPT(('--no-source',), "Don't generate source code", SKIP_GOAL)], - '1_backend': [OPT(('-b', '--backend'), "Backend", ['c', 'llvm'])], + '1_backend': [OPT(('-b', '--backend'), "Backend", ['c', 'llvm', 'cl'])], '2_gc': [OPT(('--gc',), "Garbage collector", ['boehm', 'ref', 'framework', 'none'])], '3_stackless': [OPT(('--stackless',), "Stackless code generation", True)], From ac at codespeak.net Tue Apr 25 06:19:02 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 25 Apr 2006 06:19:02 +0200 (CEST) Subject: [pypy-svn] r26286 - in pypy/dist/pypy: module/_socket module/_socket/test rpython/rctypes/socketmodule rpython/rctypes/socketmodule/test Message-ID: <20060425041902.2C7091007D@code0.codespeak.net> Author: ac Date: Tue Apr 25 06:19:00 2006 New Revision: 26286 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/module/_socket/test/test_socket2.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Log: (aleale, arre) Start on changing _socket module to use ctypes. Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Tue Apr 25 06:19:00 2006 @@ -1,13 +1,11 @@ -import _socket, socket, errno, os, sys +import sys from pypy.interpreter.typedef import TypeDef from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import W_Root, NoneNotWrapped from pypy.interpreter.gateway import ObjSpace, interp2app -from pypy.module._socket.rpython import rsocket - -# Force the declarations of external functions -import pypy.module._socket.rpython.exttable +from pypy.rpython.rctypes.socketmodule import ctypes_socket as _c +import ctypes IPV4_ADDRESS_SIZE = 4 IPV6_ADDRESS_SIZE = 16 @@ -75,33 +73,28 @@ def socket_strerror(errno): return WIN32_ERROR_MESSAGES.get(errno, "winsock error") else: - import os def socket_strerror(errno): - try: - return os.strerror(errno) - except ValueError: - return "socket error %d" % (errno,) - -def wrap_socketerror(space, e): - assert isinstance(e, socket.error) - errno = e.args[0] - msg = socket_strerror(errno) + return _c.strerror(errno) +def w_get_socketerror(space, message, errno=-1): w_module = space.getbuiltinmodule('_socket') - if isinstance(e, socket.gaierror): - w_errortype = space.getattr(w_module, space.wrap('gaierror')) - elif isinstance(e, socket.herror): - w_errortype = space.getattr(w_module, space.wrap('herror')) + w_errortype = space.getattr(w_module, space.wrap('error')) + if errno > -1: + return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) else: - w_errortype = space.getattr(w_module, space.wrap('error')) + return OperationError(w_errortype, space.wrap(message)) - return OperationError(w_errortype, - space.wrap(errno), - space.wrap(msg)) +def w_get_socketgaierror(space, message, errno=-1): + w_module = space.getbuiltinmodule('_socket') + w_errortype = space.getattr(w_module, space.wrap('gaierror')) + if errno > -1: + return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) + else: + return OperationError(w_errortype, space.wrap(message)) -def w_get_socketerror(space, message, errno=-1): +def w_get_socketherror(space, message, errno=-1): w_module = space.getbuiltinmodule('_socket') - w_errortype = space.getattr(w_module, space.wrap('error')) + w_errortype = space.getattr(w_module, space.wrap('herror')) if errno > -1: return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) else: @@ -116,15 +109,26 @@ space.wrap("timed out")) return w_error +def w_makesockaddr(space, caddr, caddrlen, proto): + if caddr.contents.sa_family == AF_INET: + a = cast(caddr, POINTER(_c.sockaddr_in)) + return space.newtuple([space.wrap(_c.inet_ntoa(a.contents.sin_addr)), + space.wrap(_c.ntohs(a.contents.sin_port))]) + else: + raise OperationError(space.w_NotImplementedError, + space.wrap("Unsupported address family %d" % caddr.contents.sa_family)) + def gethostname(space): """gethostname() -> string Return the current host name. """ - try: - return space.wrap(_socket.gethostname()) - except socket.error, e: - raise wrap_socketerror(space, e) + BUFFLEN = 1024 + namebuff = ctypes.create_string_buffer(BUFFLEN) + res = _c.gethostname(namebuff, BUFFLEN - 1) + if res < 0: + raise w_get_socketerror(_c.errno.value) + return namebuff.value gethostname.unwrap_spec = [ObjSpace] def gethostbyname(space, name): @@ -505,16 +509,6 @@ ip += "%x" % part return ip -def enumerateaddrinfo(space, addr): - result = [] - while True: - addrinfo = addr.nextinfo() - if addrinfo[0] == 0: - break - info = addrinfo[:4] + (addrinfo[4:],) - result.append(space.wrap(info)) - return space.newlist(result) - def getaddrinfo(space, w_host, w_port, family=0, socktype=0, proto=0, flags=0): """getaddrinfo(host, port [, family, socktype, proto, flags]) -> list of (family, socktype, proto, canonname, sockaddr) @@ -545,15 +539,32 @@ raise OperationError(space.w_TypeError, space.wrap("Int or String expected")) - try: - addr = rsocket.getaddrinfo(host, port, family, socktype, proto, flags) - except socket.error, e: - raise wrap_socketerror(space, e) + res = addrinfo_ptr + hints = _c.addrinfo + hints.ai_flags = flags + hints.ai_family = family + hints.ai_socktype = socktype + hints.ai_protocol = proto + retval = _c.getaddrinfo(host, port, pointer(hints), pointer(res)) + if retval < 0: + raise w_get_socketgaierror(_c.gai_strerror(_c.errno), _c.errno.value) try: - return enumerateaddrinfo(space, addr) + result = [] + next = res + while next: + info = next.contents + next = info.ai_next + w_family = space.wrap(info.ai_family.value) + w_socktype = space.wrap(info.ai_socktype.value) + w_proto = space.wrap(info.ai_protocol.value) + w_canonname = space.wrap(info.ai_canonname.value) + w_addr = w_makesockaddr(sapce, info.ai_canonname, info.ai_addrlen.value) + results.append(space.newtuple([w_family, w_socktype, w_proto, + w_canonname, w_addr])) + return space.newlist(result) finally: - addr.free() + _c.freeaddrinfo(res) getaddrinfo.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int, int, int] def getnameinfo(space, w_sockaddr, flags): @@ -679,13 +690,10 @@ Close the socket. It cannot be used after this call. """ if not self.closed: - try: - # Reusing the os.close primitive to save us from writing a - # socket-specific close primitive. This might not be perfectly - # cross-platform (Windows?). - os.close(self.fd) - except OSError, e: - raise w_get_socketerror(space, e.strerror, e.errno) + res = _c.close(self.fd) + if res < 0: + errno = _c.errno.value + raise w_get_socketerror(space, socket_strerror(errno), errno) self.closed = True close.unwrap_spec = ['self', ObjSpace] Modified: pypy/dist/pypy/module/_socket/test/test_socket2.py ============================================================================== --- pypy/dist/pypy/module/_socket/test/test_socket2.py (original) +++ pypy/dist/pypy/module/_socket/test/test_socket2.py Tue Apr 25 06:19:00 2006 @@ -4,6 +4,8 @@ import py import socket, sys +py.test.skip('In-progress') + def setup_module(mod): mod.space = StdObjSpace(usemodules=['_socket']) mod.w_socket = space.appexec([], "(): import _socket as m; return m") Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Tue Apr 25 06:19:00 2006 @@ -148,10 +148,21 @@ socketdll = cdll.LoadLibrary(dllname) errno = c_int.in_dll(socketdll, 'errno') + strerror = socketdll.strerror strerror.argtypes = [c_int] strerror.restype = c_char_p +gai_strerror = socketdll.gai_strerror +gai_strerror.argtypes = [c_int] +gai_strerror.restype = c_char_p + +h_errno = c_int.in_dll(socketdll, 'h_errno') + +hstrerror = socketdll.hstrerror +hstrerror.argtypes = [c_int] +hstrerror.restype = c_char_p + socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] socket.restype = c_int @@ -197,6 +208,10 @@ inet_aton.argtypes = [c_char_p, POINTER(in_addr)] inet_aton.restype = c_int +close = socketdll.close +close.argtypes = [c_int] +close.restype = c_int + socketaccept = socketdll.accept socketaccept.argtypes = [c_int, sockaddr_ptr, POINTER(socklen_t)] socketaccept.restype = c_int @@ -252,3 +267,12 @@ socketshutdown = socketdll.shutdown socketshutdown.argtypes = [c_int, c_int] socketshutdown.restype = c_int + + +getaddrinfo = socketdll.getaddrinfo +getaddrinfo.argtypes = [ c_char_p, c_char_p, addrinfo_ptr, POINTER(addrinfo_ptr)] +getaddrinfo.restype = c_int + +gethostname = socketdll.gethostname +gethostname.argtypes = [POINTER(c_char), c_int] +gethostname.restype = c_int Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/test/test__socket.py Tue Apr 25 06:19:00 2006 @@ -60,9 +60,9 @@ s.connect(addr) self.client_addr = s.getsockname() s.send("@") - data = s.recv(1024) + self.received = s.recv(1024) while data: - data = s.recv(1024) + self.received += s.recv(1024) s.close() def test_getsockname(self): From nik at codespeak.net Tue Apr 25 06:48:29 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 06:48:29 +0200 (CEST) Subject: [pypy-svn] r26287 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060425044829.3105010036@code0.codespeak.net> Author: nik Date: Tue Apr 25 06:48:23 2006 New Revision: 26287 Added: pypy/dist/pypy/rpython/ootypesystem/test/test_ootuple.py (contents, props changed) Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rtuple.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: (nik, pedronis consulting) introduce a new low-level type Tuple in ootypesystem. it's to be used for structure-like things such as tuples and iterators. this will resolve the recursion and ForwardReference problems that we currently have. these are mostly caused by the fact that Instances are compared by identity, the new Tuple type compares by structure. rtyping doesn't yet use the new type anywhere. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 06:48:23 2006 @@ -183,6 +183,40 @@ class BuiltinType(OOType): + def _example(self): + return new(self) + + def _defl(self): + return self._null + + def _get_interp_class(self): + raise NotImplementedError + +class Tuple(BuiltinType): + + def __init__(self, ITEMTYPES): + self._items = frozendict(ITEMTYPES) + self._null = _null_tuple(self) + + def _defl(self): + return self._null + + def _get_interp_class(self): + return _tuple + + def _field_type(self, name): + try: + return self._items[name] + except KeyError: + raise TypeError("No field names %r" % name) + + _check_field = _field_type + + def _lookup(self, meth_name): + return self, None + +class BuiltinADTType(BuiltinType): + def _setup_methods(self, generic_types): methods = {} for name, meth in self._GENERIC_METHODS.iteritems(): @@ -194,7 +228,7 @@ self._METHODS = frozendict(methods) def _specialize_type(self, type_, generic_types): - if isinstance(type_, BuiltinType): + if isinstance(type_, BuiltinADTType): return type_._specialize(generic_types) else: return generic_types.get(type_, type_) @@ -210,17 +244,8 @@ meth = _meth(METH, _name=meth_name, _callable=getattr(cls, meth_name)) return self, meth - def _example(self): - return new(self) - - def _defl(self): - return self._null - def _get_interp_class(self): - raise NotImplementedError - - -class List(BuiltinType): +class List(BuiltinADTType): # placeholders for types # make sure that each derived class has his own SELFTYPE_T # placeholder, because we want backends to distinguish that. @@ -276,9 +301,12 @@ def _specialize(self, generic_types): ITEMTYPE = self._specialize_type(self._ITEMTYPE, generic_types) return self.__class__(ITEMTYPE) + + def _defl(self): + return self._null -class Dict(BuiltinType): +class Dict(BuiltinADTType): # placeholders for types SELFTYPE_T = object() KEYTYPE_T = object() @@ -327,7 +355,7 @@ class ForwardReference(OOType): def become(self, real_instance): - if not isinstance(real_instance, (Instance, BuiltinType)): + if not isinstance(real_instance, (Instance, BuiltinADTType)): raise TypeError("ForwardReference can only be to an instance, " "not %r" % (real_instance,)) self.__class__ = real_instance.__class__ @@ -600,7 +628,7 @@ return meth._bound(TYPE, self) return object.__getattribute__(self, name) - + class _list(_builtin_type): @@ -701,6 +729,36 @@ def __init__(self, DICT): self.__dict__["_TYPE"] = DICT +class _tuple(object): + + def __init__(self, TYPE): + self._items = {} + self._TYPE = TYPE + for name, ITEMTYPE in TYPE._items.items(): + self._items[name] = ITEMTYPE._defl() + + def __getattr__(self, name): + items = self.__dict__["_items"] + if name in items: + return items[name] + return self.__dict__[name] + + def __setattr__(self, name, value): + if hasattr(self, "_items") and name in self._items: + self._items[name] = value + else: + self.__dict__[name] = value + + def _identityhash(self): + if self: + return id(self) + else: + return 0 # for all null tuples + +class _null_tuple(_null_mixin(_tuple), _tuple): + + def __init__(self, TUPLE): + self.__dict__["_TYPE"] = TUPLE def new(TYPE): if isinstance(TYPE, Instance): @@ -776,7 +834,7 @@ return instance._downcast(INSTANCE) def ooidentityhash(inst): - assert isinstance(typeOf(inst), Instance) + assert isinstance(typeOf(inst), (Instance, Tuple)) return inst._identityhash() Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Tue Apr 25 06:48:23 2006 @@ -8,6 +8,7 @@ def __init__(self, rtyper, items_r): AbstractTupleRepr.__init__(self, rtyper, items_r) + #self.lowleveltype = ootype.Tuple(dict(zip(self.fieldnames, self.lltypes))) self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) def newtuple(cls, llops, r_tuple, items_v): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 25 06:48:23 2006 @@ -2,6 +2,7 @@ from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.ootypesystem.ootype import * from pypy.rpython.rlist import ll_append +from pypy.translator.translator import TranslationContext def test_new(): LT = List(Signed) @@ -111,3 +112,27 @@ res = interpret(f, [3], type_system="ootype") assert res == 3 + def test_initialize(self): + def f(x): + l = [1, 2] + l.append(x) + return l[2] + res = interpret(f, [3], type_system="ootype") + assert res == 3 + + def dont_test_listtype_explosion(self): + def f(x): + l1 = [x] + l2 = [x] + return l1, l2 + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(f, [int]) + typer = t.buildrtyper(type_system="ootype") + typer.specialize() + + s_l1, s_l2 = s.items + r_l1 = typer.getrepr(s_l1) + r_l2 = typer.getrepr(s_l2) + assert r_l1.lowleveltype == r_l2.lowleveltype + Added: pypy/dist/pypy/rpython/ootypesystem/test/test_ootuple.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_ootuple.py Tue Apr 25 06:48:23 2006 @@ -0,0 +1,40 @@ +from pypy.rpython.ootypesystem.ootype import * + +def test_type_comparison(): + T = Tuple({"a": Signed, "b": Signed}) + T2 = Tuple({"a": Signed, "b": Signed}) + T3 = Tuple({"a": Signed, "b": Unsigned}) + + assert T == T2 + assert T2 != T3 + assert hash(T) == hash(T2) + +def test_new(): + T = Tuple({"a": Signed, "b": Signed}) + t = new(T) + assert t.a == 0 + assert t.b == 0 + +def test_getsetitem(): + T = Tuple({"a": Signed, "b": Signed}) + t = new(T) + t.a = 2 + t.b = 3 + assert t.a == 2 + assert t.b == 3 + +def test_null(): + T = Tuple({"a": Signed}) + n = null(T) + n2 = null(T) + assert n == n2 + +def test_ooidentityhash(): + T = Tuple({"a": Signed, "b": Signed}) + t = new(T) + t.a = 1 + t.b = 2 + t2 = new(T) + t.a = 1 + t.b = 2 + assert ooidentityhash(t) != ooidentityhash(t2) From nik at codespeak.net Tue Apr 25 08:09:02 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 08:09:02 +0200 (CEST) Subject: [pypy-svn] r26289 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060425060902.CE9791007E@code0.codespeak.net> Author: nik Date: Tue Apr 25 08:08:59 2006 New Revision: 26289 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: make Tuple more similar to Instance to make updating the existing backends to the new scheme easier. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 08:08:59 2006 @@ -193,11 +193,21 @@ raise NotImplementedError class Tuple(BuiltinType): + + # We try to keep Tuple as similar to Instance as possible, so backends + # can treat them polymorphically, if they choose to do so. - def __init__(self, ITEMTYPES): - self._items = frozendict(ITEMTYPES) + def __init__(self, fields): + self._fields = frozendict() + self._add_fields(fields) self._null = _null_tuple(self) + def _add_fields(self, fields): + fields.copy() + for name, ITEMTYPE in fields.items(): + fields[name] = ITEMTYPE, ITEMTYPE._defl() + self._fields.update(fields) + def _defl(self): return self._null @@ -734,8 +744,8 @@ def __init__(self, TYPE): self._items = {} self._TYPE = TYPE - for name, ITEMTYPE in TYPE._items.items(): - self._items[name] = ITEMTYPE._defl() + for name, (_, default) in TYPE._fields.items(): + self._items[name] = default def __getattr__(self, name): items = self.__dict__["_items"] From sanxiyn at codespeak.net Tue Apr 25 08:24:45 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 08:24:45 +0200 (CEST) Subject: [pypy-svn] r26290 - pypy/dist/pypy/translator/cl Message-ID: <20060425062445.C19B01007E@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 08:24:40 2006 New Revision: 26290 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/gencl.py Log: Remove crufts Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Tue Apr 25 08:24:40 2006 @@ -95,7 +95,7 @@ if conftest.option.view: t.view() - gen = GenCL(t, func, argtypes) + gen = GenCL(t, func) out = gen.emitcode() i = 1 fpath = path.join("%s.lisp" % func.func_name) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 08:24:40 2006 @@ -132,12 +132,12 @@ class GenCL: - def __init__(self, context, funobj, input_arg_types=[]): + def __init__(self, context, funobj): self.context = context self.pendinggraphs = [funobj] self.declarations = [] - def emitcode(self, public=True): + def emitcode(self): lines = list(self.emit()) declarations = "\n".join(self.declarations) code = "\n".join(lines) From sanxiyn at codespeak.net Tue Apr 25 08:26:14 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 08:26:14 +0200 (CEST) Subject: [pypy-svn] r26291 - pypy/dist/pypy/translator/cl Message-ID: <20060425062614.A937A10085@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 08:26:10 2006 New Revision: 26291 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Add emitfile() to gencl that writes the code to the file and returns the file name Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 08:26:10 2006 @@ -1,5 +1,6 @@ import types +from pypy.tool.udir import udir from pypy.translator.translator import graphof from pypy.rpython.ootypesystem.ootype import Instance, List, _static_meth, _meth from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name @@ -134,9 +135,17 @@ def __init__(self, context, funobj): self.context = context + self.entry_point = funobj self.pendinggraphs = [funobj] self.declarations = [] + def emitfile(self): + name = self.entry_point.func_name + path = udir.join("%s.lisp" % (name,)) + code = self.emitcode() + path.write(code) + return str(path) + def emitcode(self): lines = list(self.emit()) declarations = "\n".join(self.declarations) From sanxiyn at codespeak.net Tue Apr 25 08:28:44 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 08:28:44 +0200 (CEST) Subject: [pypy-svn] r26292 - pypy/dist/pypy/translator Message-ID: <20060425062844.6CA8210087@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 08:28:38 2006 New Revision: 26292 Modified: pypy/dist/pypy/translator/driver.py Log: In the translation driver, use emitfile(), depend on ootype task instead of rtype task, and add stubs for compiling and running lisp sources. Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Tue Apr 25 08:28:38 2006 @@ -360,10 +360,21 @@ def task_source_cl(self): from pypy.translator.cl.gencl import GenCL self.gen = GenCL(self.translator, self.entry_point) - self.gen.emitcode() - task_source_cl = taskdef(task_source_cl, ['backendopt', 'rtype'], + filename = self.gen.emitfile() + self.log.info("Wrote %s" % (filename,)) + task_source_cl = taskdef(task_source_cl, ['ootype'], 'Generating Common Lisp source') + def task_compile_cl(self): + pass + task_compile_cl = taskdef(task_compile_cl, ['source_cl'], + 'XXX') + + def task_run_cl(self): + pass + task_run_cl = taskdef(task_run_cl, ['compile_cl'], + 'XXX') + def proceed(self, goals): if not goals: if self.default_goal: From sanxiyn at codespeak.net Tue Apr 25 08:32:50 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 08:32:50 +0200 (CEST) Subject: [pypy-svn] r26293 - pypy/dist/pypy/translator/cl/test Message-ID: <20060425063250.0962D10087@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 08:32:47 2006 New Revision: 26293 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: De-class-ify tests Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Tue Apr 25 08:32:47 2006 @@ -8,110 +8,108 @@ from pypy.translator.test import snippet as t -class TestGenCLTestCase: - - def test_if(self): - py.test.skip("temporarily disabled") - cl_if = make_cl_func(t.if_then_else, [object, object, object]) - assert cl_if(True, 50, 100) == 50 - assert cl_if(False, 50, 100) == 100 - assert cl_if(0, 50, 100) == 100 - assert cl_if(1, 50, 100) == 50 - assert cl_if([], 50, 100) == 100 - assert cl_if([[]], 50, 100) == 50 - - def test_gcd(self): - cl_gcd = make_cl_func(t.my_gcd, [int, int]) - assert cl_gcd(96, 64) == 32 - - def test_is_perfect(self): # pun intended - cl_perfect = make_cl_func(t.is_perfect_number, [int]) - assert cl_perfect(24) == False - assert cl_perfect(28) == True - - def test_bool(self): - py.test.skip("temporarily disabled") - cl_bool = make_cl_func(t.my_bool, [object]) - assert cl_bool(0) == False - assert cl_bool(42) == True - assert cl_bool(True) == True - - def test_contains(self): - py.test.skip("temporarily disabled") - my_contains = make_cl_func(t.my_contains, [list, int]) - assert my_contains([1, 2, 3], 1) - assert not my_contains([1, 2, 3], 0) - is_one_or_two = make_cl_func(t.is_one_or_two, [int]) - assert is_one_or_two(2) - assert not is_one_or_two(3) - - def test_array(self): - py.test.skip("temporarily disabled") - cl_four = make_cl_func(t.two_plus_two) - assert cl_four() == 4 - - def test_sieve(self): - py.test.skip("temporarily disabled") - cl_sieve = make_cl_func(t.sieve_of_eratosthenes) - assert cl_sieve() == 1028 - - def test_easy(self): - py.test.skip("temporarily disabled") - # These are the Pyrex tests which were easy to adopt. - f1 = make_cl_func(t.simple_func, [int]) - assert f1(1) == 2 - f2 = make_cl_func(t.while_func, [int]) - assert f2(10) == 55 - f3 = make_cl_func(t.simple_id, [int]) - assert f3(9) == 9 - f4 = make_cl_func(t.branch_id, [int, int, int]) - assert f4(1, 2, 3) == 2 - assert f4(0, 2, 3) == 3 - f5 = make_cl_func(t.int_id, [int]) - assert f5(3) == 3 - f6 = make_cl_func(t.time_waster, [int]) - assert f6(30) == 3657 - - def test_string(self): - py.test.skip("temporarily disabled") - cl_greet = make_cl_func(t.greet, [str]) - assert cl_greet("world") == "helloworld" - cl_stringmaker = make_cl_func(t.nested_whiles, [int, int]) - assert cl_stringmaker(111, 114) == ( - "...!...!...!...!...!") - - def test_for(self): - py.test.skip("temporarily disabled") - cl_python = make_cl_func(t.choose_last) - assert cl_python() == "python" - - def test_builtin(self): - py.test.skip("temporarily disabled") - cl_builtinusage = make_cl_func(t.builtinusage) - assert cl_builtinusage() == 4 - - def test_slice(self): - py.test.skip("temporarily disabled") - cl_half = make_cl_func(t.half_of_n, [int]) - assert cl_half(10) == 5 - - def test_powerset(self): - py.test.skip("temporarily disabled") - cl_powerset = make_cl_func(t.powerset, [int]) - result = cl_powerset(3) - assert result.__class__ == Literal - assert result.val == ( - '#(#() #(0) #(1) #(0 1) #(2) #(0 2) #(1 2) #(0 1 2))') - def test_yast(self): - py.test.skip("temporarily disabled") - cl_sum = make_cl_func(t.yast, [list]) # yet another sum test - assert cl_sum(range(12)) == 66 - - def test_int_add(self): - def add_two(number): - return number + 2 - cl_add_two = make_cl_func(add_two, [int]) - assert cl_add_two(5) == 7 +def test_if(): + py.test.skip("temporarily disabled") + cl_if = make_cl_func(t.if_then_else, [object, object, object]) + assert cl_if(True, 50, 100) == 50 + assert cl_if(False, 50, 100) == 100 + assert cl_if(0, 50, 100) == 100 + assert cl_if(1, 50, 100) == 50 + assert cl_if([], 50, 100) == 100 + assert cl_if([[]], 50, 100) == 50 + +def test_gcd(): + cl_gcd = make_cl_func(t.my_gcd, [int, int]) + assert cl_gcd(96, 64) == 32 + +def test_is_perfect(): # pun intended + cl_perfect = make_cl_func(t.is_perfect_number, [int]) + assert cl_perfect(24) == False + assert cl_perfect(28) == True + +def test_bool(): + py.test.skip("temporarily disabled") + cl_bool = make_cl_func(t.my_bool, [object]) + assert cl_bool(0) == False + assert cl_bool(42) == True + assert cl_bool(True) == True + +def test_contains(): + py.test.skip("temporarily disabled") + my_contains = make_cl_func(t.my_contains, [list, int]) + assert my_contains([1, 2, 3], 1) + assert not my_contains([1, 2, 3], 0) + is_one_or_two = make_cl_func(t.is_one_or_two, [int]) + assert is_one_or_two(2) + assert not is_one_or_two(3) + +def test_array(): + py.test.skip("temporarily disabled") + cl_four = make_cl_func(t.two_plus_two) + assert cl_four() == 4 + +def test_sieve(): + py.test.skip("temporarily disabled") + cl_sieve = make_cl_func(t.sieve_of_eratosthenes) + assert cl_sieve() == 1028 + +def test_easy(): + py.test.skip("temporarily disabled") + # These are the Pyrex tests which were easy to adopt. + f1 = make_cl_func(t.simple_func, [int]) + assert f1(1) == 2 + f2 = make_cl_func(t.while_func, [int]) + assert f2(10) == 55 + f3 = make_cl_func(t.simple_id, [int]) + assert f3(9) == 9 + f4 = make_cl_func(t.branch_id, [int, int, int]) + assert f4(1, 2, 3) == 2 + assert f4(0, 2, 3) == 3 + f5 = make_cl_func(t.int_id, [int]) + assert f5(3) == 3 + f6 = make_cl_func(t.time_waster, [int]) + assert f6(30) == 3657 + +def test_string(): + py.test.skip("temporarily disabled") + cl_greet = make_cl_func(t.greet, [str]) + assert cl_greet("world") == "helloworld" + cl_stringmaker = make_cl_func(t.nested_whiles, [int, int]) + assert cl_stringmaker(111, 114) == ( + "...!...!...!...!...!") + +def test_for(): + py.test.skip("temporarily disabled") + cl_python = make_cl_func(t.choose_last) + assert cl_python() == "python" + +def test_builtin(): + py.test.skip("temporarily disabled") + cl_builtinusage = make_cl_func(t.builtinusage) + assert cl_builtinusage() == 4 + +def test_slice(): + py.test.skip("temporarily disabled") + cl_half = make_cl_func(t.half_of_n, [int]) + assert cl_half(10) == 5 + +def test_powerset(): + py.test.skip("temporarily disabled") + cl_powerset = make_cl_func(t.powerset, [int]) + result = cl_powerset(3) + assert result.__class__ == Literal + assert result.val == ( + '#(#() #(0) #(1) #(0 1) #(2) #(0 2) #(1 2) #(0 1 2))') +def test_yast(): + py.test.skip("temporarily disabled") + cl_sum = make_cl_func(t.yast, [list]) # yet another sum test + assert cl_sum(range(12)) == 66 + +def test_int_add(): + def add_two(number): + return number + 2 + cl_add_two = make_cl_func(add_two, [int]) + assert cl_add_two(5) == 7 # TODO From ale at codespeak.net Tue Apr 25 08:34:22 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 25 Apr 2006 08:34:22 +0200 (CEST) Subject: [pypy-svn] r26294 - pypy/dist/pypy/module/_socket Message-ID: <20060425063422.14B2310087@code0.codespeak.net> Author: ale Date: Tue Apr 25 08:34:16 2006 New Revision: 26294 Modified: pypy/dist/pypy/module/_socket/__init__.py pypy/dist/pypy/module/_socket/app_socket.py pypy/dist/pypy/module/_socket/interp_socket.py Log: Intermediate check in Modified: pypy/dist/pypy/module/_socket/__init__.py ============================================================================== --- pypy/dist/pypy/module/_socket/__init__.py (original) +++ pypy/dist/pypy/module/_socket/__init__.py Tue Apr 25 08:34:16 2006 @@ -1,6 +1,6 @@ # Package initialisation from pypy.interpreter.mixedmodule import MixedModule -import _socket +from pypy.rpython.rctypes.socketmodule import ctypes_socket as _c import sys class Module(MixedModule): @@ -25,12 +25,9 @@ getdefaulttimeout setdefaulttimeout """.split(): - if hasattr(_socket, name): - Module.interpleveldefs[name] = 'interp_socket.%s' % (name, ) + Module.interpleveldefs[name] = 'interp_socket.%s' % (name, ) -for constant in dir(_socket): - value = getattr(_socket, constant) - if constant.isupper() and type(value) in (int, str): - Module.interpleveldefs[constant] = "space.wrap(%r)" % value +for constant, value in _c.constants.iteritems(): + Module.interpleveldefs[constant] = "space.wrap(%r)" % value -Module.interpleveldefs['has_ipv6'] = "space.wrap(%s)" % _socket.has_ipv6 +#Module.interpleveldefs['has_ipv6'] = "space.wrap(%s)" % _socket.has_ipv6 Modified: pypy/dist/pypy/module/_socket/app_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/app_socket.py (original) +++ pypy/dist/pypy/module/_socket/app_socket.py Tue Apr 25 08:34:16 2006 @@ -19,3 +19,5 @@ socket = SocketType + +def setipaddress() Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Tue Apr 25 08:34:16 2006 @@ -109,6 +109,21 @@ space.wrap("timed out")) return w_error +def setipaddr(space, name, addr_ret, addr_ret_size, af): + hints = _c.addr_info + hints.ai_family = af + hints.ai_socktype = _c.SOCK_DGRAM + hints.ai_flags = _c.AI_PASSIVE + res = _c.addr_info_ptr + err = _c.getaddrinfo( None, "0", pointer(hints), pointer(res)) + if err: + raise w_get_socketgaierror(_c.errno) + if res.contents.ai_next: + raise OperationError(_socket.error, space.wrap("wildcard resolved to multiple address")) + addr = res.contents.ai_addr + _c.freeaddrinfo(res) + return addr + def w_makesockaddr(space, caddr, caddrlen, proto): if caddr.contents.sa_family == AF_INET: a = cast(caddr, POINTER(_c.sockaddr_in)) @@ -128,7 +143,7 @@ res = _c.gethostname(namebuff, BUFFLEN - 1) if res < 0: raise w_get_socketerror(_c.errno.value) - return namebuff.value + return space.wrap(namebuff.value) gethostname.unwrap_spec = [ObjSpace] def gethostbyname(space, name): @@ -136,10 +151,10 @@ Return the IP address (a string of the form '255.255.255.255') for a host. """ - try: - return space.wrap(_socket.gethostbyname(name)) - except socket.error, e: - raise wrap_socketerror(space, e) + hostent = _c.gethostbyname(name) + if not hostent: + raise w_get_socketherror(_c.h_errno.value) + return space.wrap(hostent.contents.h_name) gethostbyname.unwrap_spec = [ObjSpace, str] def gethostbyname_ex(space, name): @@ -253,7 +268,7 @@ Convert a 16-bit integer from network to host byte order. """ - return space.wrap(_socket.ntohs(x)) + return space.wrap(_c.ntohs(x)) ntohs.unwrap_spec = [ObjSpace, int] def ntohl(space, w_x): @@ -270,7 +285,7 @@ space.wrap("expected int/long, %s found" % (space.type(w_x).getname(space, "?")))) - return space.wrap(_socket.ntohl(x)) + return space.wrap(_c.ntohl(x)) ntohl.unwrap_spec = [ObjSpace, W_Root] def htons(space, x): @@ -278,7 +293,7 @@ Convert a 16-bit integer from host to network byte order. """ - return space.wrap(_socket.htons(x)) + return space.wrap(_c.htons(x)) htons.unwrap_spec = [ObjSpace, int] def htonl(space, w_x): @@ -295,7 +310,7 @@ space.wrap("expected int/long, %s found" % (space.type(w_x).getname(space, "?")))) - return space.wrap(_socket.htonl(x)) + return space.wrap(_c.htonl(x)) htonl.unwrap_spec = [ObjSpace, W_Root] def inet_aton(space, ip): @@ -335,14 +350,14 @@ Convert an IP address from string format to a packed string suitable for use with low-level network functions. """ - if family == socket.AF_INET: + if family == _c.AF_INET: packed = inet_pton_ipv4(space, ip) if len(packed) == 0: raise w_get_socketerror(space, "illegal IP address string passed to inet_pton") else: return space.wrap(packed) - elif socket.has_ipv6 and family == socket.AF_INET6: + elif _c.has_ipv6 and family == _c.AF_INET6: packed = inet_pton_ipv6(space, ip) if len(packed) == 0: raise w_get_socketerror(space, @@ -449,9 +464,9 @@ Convert a packed IP address of the given family to string format. """ - if family == socket.AF_INET: + if family == _c.AF_INET: return space.wrap(inet_ntop_ipv4(space, packed)) - elif socket.has_ipv6 and family == socket.AF_INET6: + elif _c.has_ipv6 and family == _c.AF_INET6: return space.wrap(inet_ntop_ipv6(space, packed)) else: raise OperationError(space.w_ValueError, @@ -573,8 +588,8 @@ Get host and port for a sockaddr.""" sockaddr = space.unwrap(w_sockaddr) try: - return space.wrap(socket.getnameinfo(sockaddr, flags)) - except socket.error, e: + return space.wrap(_c.getnameinfo(sockaddr, flags)) + except _c.error, e: raise wrap_socketerror(space, e) getnameinfo.unwrap_spec = [ObjSpace, W_Root, int] @@ -619,18 +634,18 @@ def getsockettype(space): return space.gettypeobject(Socket.typedef) -def newsocket(space, w_subtype, family=socket.AF_INET, - type=socket.SOCK_STREAM, proto=0): +def newsocket(space, w_subtype, family=_c.AF_INET, + type=_c.SOCK_STREAM, proto=0): # sets the timeout for the CPython implementation timeout = getstate(space).defaulttimeout if timeout < 0.0: - socket.setdefaulttimeout(None) + _c.setdefaulttimeout(None) else: - socket.setdefaulttimeout(timeout) + _c.setdefaulttimeout(timeout) try: - fd = rsocket.newsocket(family, type, proto) - except socket.error, e: # On untranslated PyPy + fd = _c.socket(family, type, proto) + except _c.error, e: # On untranslated PyPy raise wrap_socketerror(space, e) except OSError, e: # On translated PyPy raise w_get_socketerror(space, e.strerror, e.errno) @@ -987,11 +1002,10 @@ """.split() socketmethods = {} for methodname in socketmethodnames: - if hasattr(_socket.socket, methodname): - method = getattr(Socket, methodname) - assert hasattr(method,'unwrap_spec'), methodname - assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname - socketmethods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec) + method = getattr(Socket, methodname) + assert hasattr(method,'unwrap_spec'), methodname + assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname + socketmethods[methodname] = interp2app(method, unwrap_spec=method.unwrap_spec) Socket.typedef = TypeDef("_socket.socket", __doc__ = """\ From ale at codespeak.net Tue Apr 25 08:36:05 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 25 Apr 2006 08:36:05 +0200 (CEST) Subject: [pypy-svn] r26295 - pypy/dist/pypy/module/_socket Message-ID: <20060425063605.4652410087@code0.codespeak.net> Author: ale Date: Tue Apr 25 08:36:01 2006 New Revision: 26295 Modified: pypy/dist/pypy/module/_socket/app_socket.py Log: Ups - didn't mean to check that in Modified: pypy/dist/pypy/module/_socket/app_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/app_socket.py (original) +++ pypy/dist/pypy/module/_socket/app_socket.py Tue Apr 25 08:36:01 2006 @@ -20,4 +20,3 @@ socket = SocketType -def setipaddress() From nik at codespeak.net Tue Apr 25 08:42:46 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 08:42:46 +0200 (CEST) Subject: [pypy-svn] r26296 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060425064246.373C510087@code0.codespeak.net> Author: nik Date: Tue Apr 25 08:42:42 2006 New Revision: 26296 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: (dialtone, nik) make it possible to set the item type of a list lazily. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 08:42:42 2006 @@ -216,7 +216,7 @@ def _field_type(self, name): try: - return self._items[name] + return self._fields[name][0] except KeyError: raise TypeError("No field names %r" % name) @@ -262,10 +262,13 @@ SELFTYPE_T = object() ITEMTYPE_T = object() - def __init__(self, ITEMTYPE): + def __init__(self, ITEMTYPE=None): self._ITEMTYPE = ITEMTYPE self._null = _null_list(self) + if ITEMTYPE is not None: + self._init_methods() + def _init_methods(self): # This defines the abstract list interface that backends will # have to map to their native list implementations. # 'ITEMTYPE_T' is used as a placeholder for indicating @@ -273,7 +276,7 @@ generic_types = { self.SELFTYPE_T: self, - self.ITEMTYPE_T: ITEMTYPE, + self.ITEMTYPE_T: self._ITEMTYPE, } # the methods are named after the ADT methods of lltypesystem's lists @@ -301,6 +304,18 @@ # data structures. But it is important to make sure that attributes # of supposedly equal Lists compare/hash equal. + def __eq__(self, other): + if not isinstance(other, List): + return False + if self._ITEMTYPE is None or other._ITEMTYPE is None: + raise TypeError("Can't compare uninitialized List type.") + return BuiltinADTType.__eq__(self, other) + + def __hash__(self): + if self._ITEMTYPE is None: + raise TypeError("Can't hash uninitialized List type.") + return BuiltinADTType.__hash__(self) + def __str__(self): return '%s(%s)' % (self.__class__.__name__, saferecursive(str, "...")(self._ITEMTYPE)) @@ -315,6 +330,10 @@ def _defl(self): return self._null + def _set_itemtype(self, ITEMTYPE): + self._ITEMTYPE = ITEMTYPE + self._init_methods() + class Dict(BuiltinADTType): # placeholders for types @@ -847,5 +866,8 @@ assert isinstance(typeOf(inst), (Instance, Tuple)) return inst._identityhash() +def setItemType(LIST, ITEMTYPE): + return LIST._set_itemtype(ITEMTYPE) + ROOT = Instance('Root', None, _is_root=True) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 25 08:42:42 2006 @@ -60,6 +60,18 @@ assert LT1 != LT3 assert hash(LT1) == hash(LT2) +def test_optional_itemtype(): + LT = List() + LT2 = List(Signed) + assert LT != Signed + py.test.raises(TypeError, "LT == LT2") + py.test.raises(TypeError, "LT2 == LT") + py.test.raises(TypeError, hash, LT) + setItemType(LT, Signed) + assert LT == LT2 + assert LT2 == LT + assert hash(LT) == hash(LT2) + def test_recursive(): FORWARD = ForwardReference() LT = List(FORWARD) From ale at codespeak.net Tue Apr 25 08:44:03 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 25 Apr 2006 08:44:03 +0200 (CEST) Subject: [pypy-svn] r26297 - pypy/dist/pypy/rpython/rctypes/socketmodule Message-ID: <20060425064403.95C3A10087@code0.codespeak.net> Author: ale Date: Tue Apr 25 08:44:00 2006 New Revision: 26297 Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: Small change to ctypes_socket to expose constants at module level Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Tue Apr 25 08:44:00 2006 @@ -113,6 +113,15 @@ ('ai_canonname', c_char_p), ('ai_next', addrinfo_ptr)]) +CConfig.hostent = ctypes_platform.Struct('struct hostent', + [('h_name', c_char_p), + ('h_aliases', POINTER(c_char_p)), + ('h_addrtype', c_int), + ('h_length', c_int), + ('h_addr_list', POINTER(c_char_p)) + ]) + + class cConfig: pass @@ -129,6 +138,8 @@ else: constants[name] = default +locals().update(constants) + uint16_t = cConfig.uint16_t uint32_t = cConfig.uint32_t size_t = cConfig.size_t @@ -276,3 +287,7 @@ gethostname = socketdll.gethostname gethostname.argtypes = [POINTER(c_char), c_int] gethostname.restype = c_int + +gethostbyname = socketdll.gethostbyname +gethostbyname.argtypes = [POINTER(c_char)] +gethostbyname.restype = POINTER(cConfig.hostent) From sanxiyn at codespeak.net Tue Apr 25 08:45:37 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 08:45:37 +0200 (CEST) Subject: [pypy-svn] r26298 - pypy/dist/pypy/translator/cl Message-ID: <20060425064537.D6F7810087@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 08:45:34 2006 New Revision: 26298 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Import cleanup in buildcl Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Tue Apr 25 08:45:34 2006 @@ -2,11 +2,9 @@ import py from pypy.tool.udir import udir -from pypy.objspace.flow import FlowObjSpace from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL from pypy.translator.cl.clrepr import repr_const, repr_fun_name -from py.process import cmdexec from pypy import conftest global_cl = None @@ -97,7 +95,6 @@ gen = GenCL(t, func) out = gen.emitcode() - i = 1 fpath = path.join("%s.lisp" % func.func_name) if conftest.option.prettyprint: @@ -115,7 +112,7 @@ print >>fp, "))" fp.close() if conftest.option.prettyprint: - cmdexec("%s %s" % (cl, str(script))) - output = cmdexec("%s %s" % (cl, str(fpath))) + py.process.cmdexec("%s %s" % (cl, str(script))) + output = py.process.cmdexec("%s %s" % (cl, str(fpath))) return readlisp(output) return _ From sanxiyn at codespeak.net Tue Apr 25 08:53:02 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 08:53:02 +0200 (CEST) Subject: [pypy-svn] r26299 - pypy/dist/pypy/translator/cl Message-ID: <20060425065302.C8B4D10078@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 08:53:00 2006 New Revision: 26299 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Refactor binary_ops Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 08:53:00 2006 @@ -10,49 +10,30 @@ def __init__(self, gen, op): self.gen = gen - self.str = repr_arg self.op = op self.opname = op.opname self.args = op.args self.result = op.result def __iter__(self): - if self.opname in self.binary_ops: - for line in self.op_binary(self.opname): - yield line - else: - meth = getattr(self, "op_" + self.opname) - result = repr_arg(self.result) - args = map(repr_arg, self.args) - for line in meth(result, *args): - yield line + method = getattr(self, "op_" + self.opname) + result = repr_arg(self.result) + args = map(repr_arg, self.args) + for line in method(result, *args): + yield line def op_same_as(self, result, arg): yield "(setf %s %s)" % (result, arg) - binary_ops = { - #"add": "+", - "int_add": "+", - "sub": "-", - "inplace_add": "+", # weird, but it works - "inplace_lshift": "ash", - "mod": "mod", - "int_mod": "mod", - "lt": "<", - "int_lt": "<", - "le": "<=", - "eq": "=", - "int_eq": "=", - "gt": ">", - "and_": "logand", - "getitem": "elt", - } - - def op_binary(self, op): - s = self.str - result, (arg1, arg2) = self.result, self.args - cl_op = self.binary_ops[op] - yield "(setf %s (%s %s %s))" % (s(result), cl_op, s(arg1), s(arg2)) + def make_binary_op(cl_op): + def _(self, result, arg1, arg2): + yield "(setf %s (%s %s %s))" % (result, cl_op, arg1, arg2) + return _ + + op_int_add = make_binary_op("+") + op_int_eq = make_binary_op("=") + op_int_lt = make_binary_op("<") + op_int_mod = make_binary_op("mod") def op_int_is_true(self, result, arg): yield "(setf %s (not (zerop %s)))" % (result, arg) From sanxiyn at codespeak.net Tue Apr 25 09:09:59 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 09:09:59 +0200 (CEST) Subject: [pypy-svn] r26300 - pypy/dist/pypy/translator/cl Message-ID: <20060425070959.378991007E@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 09:09:56 2006 New Revision: 26300 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Various cleanups Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 09:09:56 2006 @@ -164,10 +164,13 @@ def emit_defmethod(self, fun, name): yield "(defmethod %s" % (repr_fun_name(name)) arglist = fun.getargs() - cls = arglist[0].concretetype selfvar = repr_var(arglist[0]) + clsname = repr_class_name(arglist[0].concretetype._name) args = " ".join(map(repr_var, arglist[1:])) - yield "((%s %s) %s)" % (selfvar, repr_class_name(cls._name), args) + if args: + yield "((%s %s) %s)" % (selfvar, clsname, args) + else: + yield "((%s %s))" % (selfvar, clsname) for line in self.emit_body(fun, arglist): yield line @@ -182,13 +185,15 @@ for var in block.getvariables(): # In the future, we could assign type information here vardict[var] = None - yield "(" + varnames = [] for var in vardict: + varname = repr_var(var) if var in arglist: - yield "(%s %s)" % (repr_var(var), repr_var(var)) + varnames.append("(%s %s)" % (varname, varname)) else: - yield repr_var(var) - yield ")" + varnames.append(varname) + varnames = " ".join(varnames) + yield "(%s)" % (varnames,) for block in blocklist: for line in self.emit_block(block): yield line @@ -241,7 +246,7 @@ yield "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) else: retval = repr_var(block.inputargs[0]) - yield "(return %s )" % retval + yield "(return %s)" % retval def format_jump(self, block): tag = self.blockref[block] @@ -250,11 +255,7 @@ def emit_link(self, link): source = map(repr_arg, link.args) target = map(repr_var, link.target.inputargs) - yield "(setf" - couples = zip(source, target) - for s, t in couples[:-1]: - yield "%s %s" % (t, s) - else: - s, t = couples[-1] - yield "%s %s)" % (t, s) + couples = [ "%s %s" % (t, s) for (s, t) in zip(source, target)] + couples = " ".join(couples) + yield "(setf %s)" % (couples,) yield self.format_jump(link.target) From sanxiyn at codespeak.net Tue Apr 25 09:14:44 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 09:14:44 +0200 (CEST) Subject: [pypy-svn] r26301 - pypy/dist/pypy/translator/cl Message-ID: <20060425071444.5D0921007F@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 09:14:41 2006 New Revision: 26301 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Cleanup a bit more Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 09:14:41 2006 @@ -75,9 +75,14 @@ methodobj = cls._methods[method] methodobj._method_name = method # XXX self.gen.pendinggraphs.append(methodobj) + name = repr_fun_name(method) + selfvar = repr_arg(receiver) args = map(repr_arg, args) args = " ".join(args) - yield "(setf %s (%s %s %s))" % (result, repr_fun_name(method), repr_arg(receiver), args) + if args: + yield "(setf %s (%s %s %s))" % (result, name, selfvar, args) + else: + yield "(setf %s (%s %s))" % (result, name, selfvar) def op_oogetfield(self, result, obj, _): fieldname = self.args[1].value From nik at codespeak.net Tue Apr 25 09:25:23 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 09:25:23 +0200 (CEST) Subject: [pypy-svn] r26302 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060425072523.DACB710078@code0.codespeak.net> Author: nik Date: Tue Apr 25 09:25:18 2006 New Revision: 26302 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rlist.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: (dialtone, nik) get rid of ForwardReferences in ListReprs. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 09:25:18 2006 @@ -869,5 +869,8 @@ def setItemType(LIST, ITEMTYPE): return LIST._set_itemtype(ITEMTYPE) +def hasItemType(LIST): + return LIST._ITEMTYPE is not None + ROOT = Instance('Root', None, _is_root=True) Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Tue Apr 25 09:25:18 2006 @@ -14,14 +14,13 @@ def __init__(self, rtyper, item_repr, listitem=None): self.rtyper = rtyper - if not isinstance(item_repr, Repr): # not computed yet, done by setup() + if not isinstance(item_repr, Repr): assert callable(item_repr) self._item_repr_computer = item_repr - self.LIST = ootype.ForwardReference() else: - self.LIST = ootype.List(item_repr.lowleveltype) self.external_item_repr, self.item_repr = \ externalvsinternal(rtyper, item_repr) + self.LIST = ootype.List() self.lowleveltype = self.LIST self.listitem = listitem self.list_cache = {} @@ -31,8 +30,8 @@ if 'item_repr' not in self.__dict__: self.external_item_repr, self.item_repr = \ externalvsinternal(self.rtyper, self._item_repr_computer()) - if isinstance(self.lowleveltype, ootype.ForwardReference): - self.lowleveltype.become(ootype.List(self.item_repr.lowleveltype)) + if not ootype.hasItemType(self.lowleveltype): + ootype.setItemType(self.lowleveltype, self.item_repr.lowleveltype) def null_const(self): return self.LIST._null Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Tue Apr 25 09:25:18 2006 @@ -149,15 +149,6 @@ res = interpret(oof, [], type_system='ootype') assert res is -1 -def test_list_lltype_identity(): - t = TranslationContext() - t.buildannotator() - rtyper = t.buildrtyper() - repr1 = ListRepr(rtyper, signed_repr) - repr2 = ListRepr(rtyper, signed_repr) - assert repr1.lowleveltype == repr2.lowleveltype - assert hash(repr1.lowleveltype) == hash(repr2.lowleveltype) - def test_list_annotation(): LIST = List(Signed) From tismer at codespeak.net Tue Apr 25 09:25:52 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 25 Apr 2006 09:25:52 +0200 (CEST) Subject: [pypy-svn] r26303 - pypy/dist/pypy/doc/discussion Message-ID: <20060425072552.2A60110078@code0.codespeak.net> Author: tismer Date: Tue Apr 25 09:25:48 2006 New Revision: 26303 Added: pypy/dist/pypy/doc/discussion/howtoimplementpickling.txt (contents, props changed) Log: some mockup about thread pickling, not really complete, yet. Added: pypy/dist/pypy/doc/discussion/howtoimplementpickling.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/discussion/howtoimplementpickling.txt Tue Apr 25 09:25:48 2006 @@ -0,0 +1,118 @@ +Designing thread pickling +------------------------- + +Thread pickling is a unique feature in Stackless Python +and should be implemented for PyPy pretty soon. + +What is meant by pickling? +.......................... + +I'd like to define thread pickling as a restartable subset +of a running program. The re-runnable part should be based +upon Python frame chains, represented by coroutines, tasklets +or any other application level switchable subcontext. +It is surely possible to support pickling of arbitrary +interplevel state, but this seems to be not mandatory as long +as we consider Stackless as the reference implementation. +Extensions of this might be considered when the basic task +is fulfilled. + +Pickling should create a re-startable coroutine-alike thing +that can run on a different machine, same Python version, +but not necessarily the same PyPy translation. This belongs +to the harder parts. + +What is not meant by pickling? +.............................. + +Saving the whole memory state and writing a loader that +reconstructs the whole binary with its state im memory +is not what I consider a real solution. In some sense, +this can be a fall-back if we fail in every other case, +but I consider it really nasty for the C backend. + +If we had a dynamic backend that supports direct creation +of the program and its state (example: a Forth backend), +I would see it as a valid solution, since it is +relocatable. It is of course a possible fall-back to write +sucn a backend of we fail otherwise. + +There are some simple steps and some more difficult ones. +Let's start with the simple. + +Basic necessities +................. + +Pickling of a running thread involves a bit more than normal +object pickling, because there exist many objects which +don't have a pickling interface, and people would not care +aboutpickling them at all. But with thread pickling, these +objects simply exist as local variables and are needed +to restore the current runtime environment, and the user +should not have to know what goes into the pickle. + +Examples are + +- generators +- frames +- cells +- iterators +- tracebacks + +to name just a few. Fortunately most of these objects already have +got a pickling implementation in Stackless Python, namely the +prickelpit.c file. + +It should be simple and straightforward to redo these implementations. +Nevertheless there is a complication. The most natural to support +pickling is providing a __getstate__/__setstate__ method pair. +This is ok for extension types like coroutines/tasklets, but +not allowed to use for existing types, since adding new methods +would change the interface of these objects. For Stackless, +I used the copyreg module, instead, and created special surrogate +objects as placeholders, which replace the type of the object +after unpickling with the right type pointer. For details, see +the prickelpit.c file in the Stackless distribution. + +The real problem +................ + +There are currently some crucial differences between Stackless +Python (SLP for now) and the PyPy Stackless support (PyPy for now) +as far as it is grown. +When CPython does a call to a Python function, there are several +helper functions involved for adjusting parameters, unpacking +methods and some more. SLP takes a hard time to remove all these +C functions from the C stack before starting the Python interpreter +for the function. This change of behavior is done manually for +all the helper functions by figuring out, which variables are +still needed after the call. It turns out that in most cases, +it is possible to let all the helper functions finish their +work and return form the function call before the interpreter +is started at all. + +This is the major difference which needs to be tackled for PyPy. +Whenever we run a Python function, quite a number of functions +incarnate on the C stack, and they get *not* finished before +running the new frame. In case of a coroutine switch, we just +save the whole chain of activation records - c function +entrypoints with the saved block variables. This is ok for +coroutine switching, but in the sense of SLP, it is rather +incomplete and not stackless at all. The stack still exists, +we can unwind and rebuild it, but it is a problem. + +Why a problem? +.............. + +In an ideal world, thread pickling would just be building +chains of pickled frames and nothing else. For every different +extra activation record like mentioned above, we have the +problem of how to save this information. We need a representation +which is not machine or compiler dependent. Right now, PyPy +is quite unstable in terms of which blocks it will produce, +what gets inlined, etc. The best solution possible is to try +to get completely rid of these extra structures. + +Unfortunately this is not even possible with SLP, because +there are different flavors of state which make it hard +to go without extra information. From sanxiyn at codespeak.net Tue Apr 25 09:54:26 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 09:54:26 +0200 (CEST) Subject: [pypy-svn] r26304 - pypy/dist/pypy/translator/cl Message-ID: <20060425075426.5E1EC10078@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 09:54:21 2006 New Revision: 26304 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (dialtone, sanxiyn) Add bunch of missing operations found while trying to compile pystone Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 09:54:21 2006 @@ -32,8 +32,27 @@ op_int_add = make_binary_op("+") op_int_eq = make_binary_op("=") + op_int_gt = make_binary_op(">") op_int_lt = make_binary_op("<") op_int_mod = make_binary_op("mod") + op_int_sub = make_binary_op("-") + op_float_sub = make_binary_op("-") + op_float_truediv = make_binary_op("/") + op_char_eq = make_binary_op("char=") + op_char_le = make_binary_op("char<=") + op_char_ne = make_binary_op("char/=") + + def op_cast_char_to_int(self, result, arg): + yield "(setf %s (char-code (char %s 0)))" % (result, arg) + + def op_cast_int_to_char(self, result, arg): + yield "(setf %s (string (code-char %s)))" % (result, arg) + + def op_cast_int_to_float(self, result, arg): + yield "(setf %s (float %s))" % (result, arg) + + def op_bool_not(self, result, arg): + yield "(setf %s (not %s))" % (result, arg) def op_int_is_true(self, result, arg): yield "(setf %s (not (zerop %s)))" % (result, arg) @@ -94,6 +113,12 @@ raise StopIteration yield "(setf (slot-value %s '%s) %s)" % (obj, fieldname, value) + def op_ooidentityhash(self, result, arg): + yield "(setf %s (sxhash %s))" % (result, arg) + + def op_oononnull(self, result, arg): + yield "(setf %s (not (null %s)))" % (result, arg) + class ListImpl: From sanxiyn at codespeak.net Tue Apr 25 10:15:54 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 10:15:54 +0200 (CEST) Subject: [pypy-svn] r26305 - pypy/dist/pypy/translator/cl/test Message-ID: <20060425081554.4131F1007D@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 10:15:48 2006 New Revision: 26305 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: Returning string doesn't work in ootypesystem. Add a disabled test to remind this. Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Tue Apr 25 10:15:48 2006 @@ -8,6 +8,12 @@ from pypy.translator.test import snippet as t +def dont_test_return_str(): + def return_str(): + return 'test' + cl_return_str = make_cl_func(return_str) + assert cl_return_str() == 'test' + def test_if(): py.test.skip("temporarily disabled") cl_if = make_cl_func(t.if_then_else, [object, object, object]) From sanxiyn at codespeak.net Tue Apr 25 10:47:34 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 10:47:34 +0200 (CEST) Subject: [pypy-svn] r26307 - pypy/dist/pypy/translator/cl Message-ID: <20060425084734.434031007D@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 10:47:29 2006 New Revision: 26307 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: (dialtone, sanxiyn) Refactor generate_cl_func out of _make_cl_func Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Tue Apr 25 10:47:29 2006 @@ -86,15 +86,7 @@ """ def _make_cl_func(func, cl, path, argtypes=[]): - t = TranslationContext() - t.buildannotator().build_types(func, argtypes) - t.buildrtyper(type_system="ootype").specialize() - - if conftest.option.view: - t.view() - - gen = GenCL(t, func) - out = gen.emitcode() + out = generate_cl_func(func, argtypes) fpath = path.join("%s.lisp" % func.func_name) if conftest.option.prettyprint: @@ -116,3 +108,15 @@ output = py.process.cmdexec("%s %s" % (cl, str(fpath))) return readlisp(output) return _ + +def generate_cl_func(func, argtypes=[]): + t = TranslationContext() + t.buildannotator().build_types(func, argtypes) + t.buildrtyper(type_system="ootype").specialize() + + if conftest.option.view: + t.view() + + gen = GenCL(t, func) + code = gen.emitcode() + return code From sanxiyn at codespeak.net Tue Apr 25 10:48:34 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 10:48:34 +0200 (CEST) Subject: [pypy-svn] r26308 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060425084834.16CCE1007D@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 10:48:29 2006 New Revision: 26308 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (dialtone, sanxiyn) Declare superclasses. Tests. Add a disabled test to check isinstance(). Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 10:48:29 2006 @@ -2,7 +2,8 @@ from pypy.tool.udir import udir from pypy.translator.translator import graphof -from pypy.rpython.ootypesystem.ootype import Instance, List, _static_meth, _meth +from pypy.rpython.ootypesystem.ootype import Instance, List, _static_meth, _meth, ROOT +from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name @@ -65,20 +66,23 @@ def declare_class(self, cls): # cls is really type of Instance - name = cls._name - fields = cls._fields - fieldnames = ['('+field+')' for field in fields.keys()] - field_declaration = ' '.join(fieldnames) - class_declaration = "(defclass %s () (%s))" % (repr_class_name(name), field_declaration) - return class_declaration + name = repr_class_name(cls._name) + field_declaration = ['('+field+')' for field in cls._fields] + field_declaration = " ".join(field_declaration) + if cls._superclass in (OBJECT, ROOT): + class_declaration = "(defclass %s () (%s))" % (name, field_declaration) + else: + self.declare_class(cls._superclass) + supername = repr_class_name(cls._superclass._name) + class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) + self.gen.declarations.append(class_declaration) def op_new(self, result, _): cls = self.args[0].value if isinstance(cls, List): yield "(setf %s (make-array 0 :adjustable t))" % (result,) else: - declaration = self.declare_class(cls) - self.gen.declarations.append(declaration) + self.declare_class(cls) yield "(setf %s (make-instance '%s))" % (result, repr_class_name(cls._name)) def op_oosend(self, result, *ignore): Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Tue Apr 25 10:48:29 2006 @@ -1,4 +1,4 @@ -from pypy.translator.cl.buildcl import make_cl_func +from pypy.translator.cl.buildcl import make_cl_func, generate_cl_func def test_simple(): class C: @@ -25,6 +25,33 @@ cl_inc = make_cl_func(inc, [int]) assert cl_inc(5) == 6 +def test_inherit(): + class Foo: + pass + class Bar(Foo): + pass + def check_inheritance(): + obj = Bar() + code = generate_cl_func(check_inheritance) + print code + assert code.count("defclass") == 2 + +def dont_test_isinstance(): + class Foo: + pass + class Bar(Foo): + pass + class Baz: + pass + def check_isinstance(flag): + if flag: + obj = Bar() + else: + obj = Baz() + return isinstance(obj, Foo) + cl_check_isinstance = make_cl_func(check_isinstance, [bool]) + assert cl_check_isinstance(True) == True + def test_list_length(): def list_length_one(number): lst = [number] From ale at codespeak.net Tue Apr 25 10:50:55 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 25 Apr 2006 10:50:55 +0200 (CEST) Subject: [pypy-svn] r26309 - pypy/dist/pypy/module/_socket Message-ID: <20060425085055.36FBE10088@code0.codespeak.net> Author: ale Date: Tue Apr 25 10:50:51 2006 New Revision: 26309 Modified: pypy/dist/pypy/module/_socket/interp_socket.py Log: Progress on gethostbyname_ex Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Tue Apr 25 10:50:51 2006 @@ -110,22 +110,22 @@ return w_error def setipaddr(space, name, addr_ret, addr_ret_size, af): - hints = _c.addr_info - hints.ai_family = af - hints.ai_socktype = _c.SOCK_DGRAM - hints.ai_flags = _c.AI_PASSIVE - res = _c.addr_info_ptr - err = _c.getaddrinfo( None, "0", pointer(hints), pointer(res)) + hints = _c.addr_info + hints.ai_family = af + hints.ai_socktype = _c.SOCK_DGRAM + hints.ai_flags = _c.AI_PASSIVE + res = _c.addr_info_ptr + err = _c.getaddrinfo( None, "0", pointer(hints), pointer(res)) if err: - raise w_get_socketgaierror(_c.errno) - if res.contents.ai_next: - raise OperationError(_socket.error, space.wrap("wildcard resolved to multiple address")) - addr = res.contents.ai_addr - _c.freeaddrinfo(res) + raise w_get_socketgaierror(_c.errno) + if res.contents.ai_next: + raise OperationError(_socket.error, space.wrap("wildcard resolved to multiple address")) + addr = res.contents.ai_addr + _c.freeaddrinfo(res) return addr def w_makesockaddr(space, caddr, caddrlen, proto): - if caddr.contents.sa_family == AF_INET: + if caddr.contents.sa_family == _c.AF_INET: a = cast(caddr, POINTER(_c.sockaddr_in)) return space.newtuple([space.wrap(_c.inet_ntoa(a.contents.sin_addr)), space.wrap(_c.ntohs(a.contents.sin_port))]) @@ -163,12 +163,24 @@ Return the true host name, a list of aliases, and a list of IP addresses, for a host. The host argument is a string giving a host name or IP number. """ - try: - return space.wrap(socket.gethostbyname_ex(name)) - except socket.error, e: - raise wrap_socketerror(space, e) -gethostbyname_ex.unwrap_spec = [ObjSpace, str] + hostent = _c.gethostbyname(name) + if not hostent: + raise w_get_socketherror(_c.h_errno.value) + aliases = [] + for alias in hostent.contents.h_aliases: + if alias is None: + break + aliases.append(space.wrap(alias)) + address_list = [] + for addr in hostent.contents.h_addr_list: + if addr is None: + break + address_list.append(space.wrap(addr)) + return space.newtuple([space.wrap(name), space.newlist(aliases), + space.newlist(address_list)]) +gethostbyname_ex.unwrap_spec = [ObjSpace, str] + def gethostbyaddr(space, ip_num): """gethostbyaddr(host) -> (name, aliaslist, addresslist) From ericvrp at codespeak.net Tue Apr 25 10:51:22 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 25 Apr 2006 10:51:22 +0200 (CEST) Subject: [pypy-svn] r26310 - in pypy/dist/pypy: interpreter interpreter/test module/_pickle_support Message-ID: <20060425085122.048C11008F@code0.codespeak.net> Author: ericvrp Date: Tue Apr 25 10:51:14 2006 New Revision: 26310 Added: pypy/dist/pypy/module/_pickle_support/ pypy/dist/pypy/module/_pickle_support/__init__.py (contents, props changed) pypy/dist/pypy/module/_pickle_support/maker.py (contents, props changed) Modified: pypy/dist/pypy/interpreter/baseobjspace.py pypy/dist/pypy/interpreter/nestedscope.py pypy/dist/pypy/interpreter/test/test_pickle.py pypy/dist/pypy/interpreter/typedef.py Log: (pedronis, ericvrp) Added Cell pickling! We had to jump though a few loops but all-in-all it was not too hard. Added Cell.__eq__ as well. Modified: pypy/dist/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/dist/pypy/interpreter/baseobjspace.py (original) +++ pypy/dist/pypy/interpreter/baseobjspace.py Tue Apr 25 10:51:14 2006 @@ -193,6 +193,7 @@ modules.extend(['unicodedata', '_codecs', 'array', 'marshal', 'errno', 'math', '_sre']) + modules.append('_pickle_support') if self.options.nofaking: modules.append('posix') Modified: pypy/dist/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/dist/pypy/interpreter/nestedscope.py (original) +++ pypy/dist/pypy/interpreter/nestedscope.py Tue Apr 25 10:51:14 2006 @@ -27,7 +27,23 @@ if self.w_value is None: raise ValueError, "delete() on an empty cell" self.w_value = None - + + def descr__eq__(self, space, w_other): + other = space.interpclass_w(w_other) + if not isinstance(other, Cell): + return space.w_False + return space.eq(self.w_value, other.w_value) + + def descr__reduce__(self, space): + cell_new = space.getbuiltinmodule('_pickle_support').get('cell_new') + if self.w_value is None: #when would this happen? + return space.newtuple([cell_new, space.newtuple([])]) + return space.newtuple([cell_new, space.newtuple([]), + space.newtuple([self.w_value])]) + + def descr__setstate__(self, space, w_state): + self.w_value = space.getitem(w_state, space.wrap(0)) + def __repr__(self): """ representation for debugging purposes """ if self.w_value is None: Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Tue Apr 25 10:51:14 2006 @@ -1,27 +1,27 @@ -import py -import pickle - -def test_pickle_cell(): - py.test.skip("cell pickling is work in progress") - def g(): - x = None - def f(): - return x - return f.func_closure[0] - try: +class AppTestInterpObjectPickling: + + def test_pickle_cell(self): + import pickle + def g(): + x = [42] + def f(): + x[0] += 1 + return x + return f.func_closure[0] cell = g() - pickle.dumps(cell) - except IndexError, e: - raise + pckl = pickle.dumps(g()) + result = pickle.loads(pckl) + assert cell == result + assert not (cell != result) -def test_pickle_generator(): - py.test.skip("generator pickling is work in progress") - def giveme(n): - x = 0 - while x < n: - yield x - generator = giveme(10) - pickle.dumps(generator) + #def test_pickle_generator(self): + # import pickle + # def giveme(n): + # x = 0 + # while x < n: + # yield x + # generator = giveme(10) + # print pickle.dumps(generator) #TODO: test pickling of code objects #TODO: test pickling of function objects Modified: pypy/dist/pypy/interpreter/typedef.py ============================================================================== --- pypy/dist/pypy/interpreter/typedef.py (original) +++ pypy/dist/pypy/interpreter/typedef.py Tue Apr 25 10:51:14 2006 @@ -539,7 +539,15 @@ gi_frame = interp_attrproperty('frame', cls=GeneratorIterator), ) -Cell.typedef = TypeDef("cell") +Cell.typedef = TypeDef("cell", + __eq__ = interp2app(Cell.descr__eq__, + unwrap_spec=['self', ObjSpace, W_Root]), + __ne__ = descr_generic_ne, + __reduce__ = interp2app(Cell.descr__reduce__, + unwrap_spec=['self', ObjSpace]), + __setstate__ = interp2app(Cell.descr__setstate__, + unwrap_spec=['self', ObjSpace, W_Root]), +) Ellipsis.typedef = TypeDef("Ellipsis", __repr__ = interp2app(Ellipsis.descr__repr__), Added: pypy/dist/pypy/module/_pickle_support/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_pickle_support/__init__.py Tue Apr 25 10:51:14 2006 @@ -0,0 +1,11 @@ +from pypy.interpreter.mixedmodule import MixedModule + +class Module(MixedModule): + """Built-in functions, exceptions, and other objects.""" + + appleveldefs = { + } + + interpleveldefs = { + 'cell_new': 'maker.cell_new' + } Added: pypy/dist/pypy/module/_pickle_support/maker.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_pickle_support/maker.py Tue Apr 25 10:51:14 2006 @@ -0,0 +1,9 @@ +from pypy.interpreter.nestedscope import Cell + +#note: for now we don't use the actual value when creating the Cell. +# (i.e. we assume it will be handled by __setstate__) +# Stackless does use this so it might be needed here as well. + +def cell_new(space): + return space.wrap(Cell()) +#cell_new.unwrap_spec = [...] From antocuni at codespeak.net Tue Apr 25 10:53:06 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 25 Apr 2006 10:53:06 +0200 (CEST) Subject: [pypy-svn] r26311 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060425085306.5463010088@code0.codespeak.net> Author: antocuni Date: Tue Apr 25 10:52:59 2006 New Revision: 26311 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Log: The Dict interface has been changed to support a more efficient iteration. The ll_get_items_iterator ADT method has been added: it returns a DictItemsIterator that allow a pseudo-iteration over dict items. Methods such as keys(), values(), items() and their iter* correspondent will be implemented in term of it. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 10:52:59 2006 @@ -358,7 +358,8 @@ "ll_set": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), "ll_remove": Meth([self.KEYTYPE_T], Bool), # return False is key was not present "ll_contains": Meth([self.KEYTYPE_T], Bool), - "ll_keys": Meth([], List(self.KEYTYPE_T)), + "ll_keys": Meth([], List(self.KEYTYPE_T)), # deprecated + "ll_get_items_iterator": Meth([], DictItemsIterator(self.KEYTYPE_T, self.VALUETYPE_T)), }) self._setup_methods(generic_types) @@ -382,6 +383,54 @@ VALUETYPE = self._specialize_type(self._VALUETYPE, generic_types) return self.__class__(KEYTYPE, VALUETYPE) +class DictItemsIterator(BuiltinADTType): + SELFTYPE_T = object() + KEYTYPE_T = object() + VALUETYPE_T = object() + + def __init__(self, KEYTYPE, VALUETYPE): + self._KEYTYPE = KEYTYPE + self._VALUETYPE = VALUETYPE + self._null = _null_dict_items_iterator(self) + + generic_types = { + self.SELFTYPE_T: self, + self.KEYTYPE_T: KEYTYPE, + self.VALUETYPE_T: VALUETYPE + } + + # some words about the interface of the iterator: we can't + # write the next() method directly in the backend because of + # two reasons: + # 1) tuples aren't BuiltinType, yet, so we can't make them + # generic; moreover, they are constructed on-the-fly, so the + # two-item tuple type that an hypotetic next() would return + # isn't available at this time. + # 2) StopIteration is generated at translation time too, so + # this would prevent backends to do a precompilation of + # support code. + + self._GENERIC_METHODS = frozendict({ + "ll_go_next": Meth([], Bool), # move forward; return False is there is no more data available + "ll_current_key": Meth([], self.KEYTYPE_T), + "ll_current_value": Meth([], self.VALUETYPE_T), + }) + + self._setup_methods(generic_types) + + def __str__(self): + return '%s%s' % (self.__class__.__name__, + saferecursive(str, "(...)")((self._KEYTYPE, self._VALUETYPE))) + + def _get_interp_class(self): + return _dict_items_iterator + + def _specialize(self, generic_types): + KEYTYPE = self._specialize_type(self._KEYTYPE, generic_types) + VALUETYPE = self._specialize_type(self._VALUETYPE, generic_types) + return self.__class__(KEYTYPE, VALUETYPE) + + class ForwardReference(OOType): def become(self, real_instance): if not isinstance(real_instance, (Instance, BuiltinADTType)): @@ -753,11 +802,45 @@ keys._list = self._dict.keys() return keys -class _null_dict(_null_mixin(_dict), _dict): + def ll_get_items_iterator(self): + # NOT_RPYTHON + ITER = DictItemsIterator(self._TYPE._KEYTYPE, self._TYPE._VALUETYPE) + return _dict_items_iterator(ITER, self._dict) +class _null_dict(_null_mixin(_dict), _dict): def __init__(self, DICT): self.__dict__["_TYPE"] = DICT + +class _dict_items_iterator(_builtin_type): + def __init__(self, ITER, d): + self._TYPE = ITER + self._items = d.items() + self._index = -1 + + def ll_go_next(self): + # NOT_RPYTHON + self._index += 1 + if self._index >= len(self._items): + return False + else: + return True + + def ll_current_key(self): + # NOT_RPYTHON + assert 0 <= self._index < len(self._items) + return self._items[self._index][0] + + def ll_current_value(self): + # NOT_RPYTHON + assert 0 <= self._index < len(self._items) + return self._items[self._index][1] + +class _null_dict_items_iterator(_null_mixin(_dict_items_iterator), _dict_items_iterator): + def __init__(self, ITER): + self.__dict__["_TYPE"] = ITER + + class _tuple(object): def __init__(self, TYPE): @@ -789,6 +872,7 @@ def __init__(self, TUPLE): self.__dict__["_TYPE"] = TUPLE + def new(TYPE): if isinstance(TYPE, Instance): return make_instance(TYPE) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Tue Apr 25 10:52:59 2006 @@ -18,3 +18,15 @@ d = new(DT) d.ll_set(42, 123.45) assert d.ll_get(42) == 123.45 + +def test_iteritems(): + DT = Dict(Signed, Float) + d = new(DT) + d.ll_set(42, 43.0) + d.ll_set(52, 53.0) + it = d.ll_get_items_iterator() + items = [] + while it.ll_go_next(): + items.append((it.ll_current_key(), it.ll_current_value())) + items.sort() + assert items == [(42, 43.0), (52, 53.0)] From sanxiyn at codespeak.net Tue Apr 25 11:10:55 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 11:10:55 +0200 (CEST) Subject: [pypy-svn] r26312 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060425091055.32DEA1007E@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 11:10:51 2006 New Revision: 26312 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (dialtone, sanxiyn) Fix isinstance() test. Pass it. (Implemented ooupcast and instanceof) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 11:10:51 2006 @@ -23,9 +23,12 @@ for line in method(result, *args): yield line - def op_same_as(self, result, arg): + def nop(self, result, arg): yield "(setf %s %s)" % (result, arg) + op_same_as = nop + op_ooupcast = nop + def make_binary_op(cl_op): def _(self, result, arg1, arg2): yield "(setf %s (%s %s %s))" % (result, cl_op, arg1, arg2) @@ -83,7 +86,12 @@ yield "(setf %s (make-array 0 :adjustable t))" % (result,) else: self.declare_class(cls) - yield "(setf %s (make-instance '%s))" % (result, repr_class_name(cls._name)) + clsname = repr_class_name(cls._name) + yield "(setf %s (make-instance '%s))" % (result, clsname) + + def op_instanceof(self, result, arg, _): + clsname = repr_class_name(self.args[1].value._name) + yield "(setf %s (typep %s '%s))" % (result, arg, clsname) def op_oosend(self, result, *ignore): method = self.args[0].value Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Tue Apr 25 11:10:51 2006 @@ -36,19 +36,19 @@ print code assert code.count("defclass") == 2 -def dont_test_isinstance(): +def test_isinstance(): class Foo: pass class Bar(Foo): pass - class Baz: + class Baz(Foo): pass def check_isinstance(flag): if flag: obj = Bar() else: obj = Baz() - return isinstance(obj, Foo) + return isinstance(obj, Bar) cl_check_isinstance = make_cl_func(check_isinstance, [bool]) assert cl_check_isinstance(True) == True From ac at codespeak.net Tue Apr 25 11:13:36 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 25 Apr 2006 11:13:36 +0200 (CEST) Subject: [pypy-svn] r26313 - in pypy/dist/pypy: module/_socket rpython/rctypes/socketmodule rpython/rctypes/tool Message-ID: <20060425091336.C26A21007E@code0.codespeak.net> Author: ac Date: Tue Apr 25 11:13:31 2006 New Revision: 26313 Modified: pypy/dist/pypy/module/_socket/__init__.py pypy/dist/pypy/module/_socket/app_socket.py pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Log: (aleale, arre) Fix gethostbyname{,_ex}. Modified: pypy/dist/pypy/module/_socket/__init__.py ============================================================================== --- pypy/dist/pypy/module/_socket/__init__.py (original) +++ pypy/dist/pypy/module/_socket/__init__.py Tue Apr 25 11:13:31 2006 @@ -9,6 +9,7 @@ 'herror' : 'app_socket.herror', 'gaierror' : 'app_socket.gaierror', 'timeout' : 'app_socket.timeout', + 'gethostbyname': 'app_socket.gethostbyname', } interpleveldefs = { @@ -17,7 +18,7 @@ } for name in """ - gethostbyname gethostbyname_ex gethostbyaddr gethostname + gethostbyname_ex gethostbyaddr gethostname getservbyname getservbyport getprotobyname fromfd socketpair ntohs ntohl htons htonl inet_aton inet_ntoa inet_pton inet_ntop Modified: pypy/dist/pypy/module/_socket/app_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/app_socket.py (original) +++ pypy/dist/pypy/module/_socket/app_socket.py Tue Apr 25 11:13:31 2006 @@ -2,6 +2,8 @@ See the socket module for documentation.""" +import _socket + class error(Exception): pass @@ -19,4 +21,5 @@ socket = SocketType - +def gethostbyname(name): + return _socket.gethostbyname_ex(name)[2][0] Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Tue Apr 25 11:13:31 2006 @@ -126,7 +126,7 @@ def w_makesockaddr(space, caddr, caddrlen, proto): if caddr.contents.sa_family == _c.AF_INET: - a = cast(caddr, POINTER(_c.sockaddr_in)) + a = _c.cast(caddr, ctypes.POINTER(_c.sockaddr_in)) return space.newtuple([space.wrap(_c.inet_ntoa(a.contents.sin_addr)), space.wrap(_c.ntohs(a.contents.sin_port))]) else: @@ -154,7 +154,7 @@ hostent = _c.gethostbyname(name) if not hostent: raise w_get_socketherror(_c.h_errno.value) - return space.wrap(hostent.contents.h_name) + return space.wrap(hostent.contents.h_addr) gethostbyname.unwrap_spec = [ObjSpace, str] def gethostbyname_ex(space, name): @@ -172,12 +172,13 @@ break aliases.append(space.wrap(alias)) address_list = [] - for addr in hostent.contents.h_addr_list: - if addr is None: + h_addr_list = ctypes.cast(hostent.contents.h_addr_list, ctypes.POINTER(ctypes.POINTER(_c.in_addr))) + for addr in h_addr_list: + if not addr: break - address_list.append(space.wrap(addr)) + address_list.append(space.wrap(_c.inet_ntoa(addr.contents))) - return space.newtuple([space.wrap(name), space.newlist(aliases), + return space.newtuple([space.wrap(hostent.contents.h_name), space.newlist(aliases), space.newlist(address_list)]) gethostbyname_ex.unwrap_spec = [ObjSpace, str] @@ -566,13 +567,13 @@ raise OperationError(space.w_TypeError, space.wrap("Int or String expected")) - res = addrinfo_ptr - hints = _c.addrinfo + res = _c.addrinfo_ptr() + hints = _c.addrinfo() hints.ai_flags = flags hints.ai_family = family hints.ai_socktype = socktype hints.ai_protocol = proto - retval = _c.getaddrinfo(host, port, pointer(hints), pointer(res)) + retval = _c.getaddrinfo(host, port, ctypes.pointer(hints), ctypes.pointer(res)) if retval < 0: raise w_get_socketgaierror(_c.gai_strerror(_c.errno), _c.errno.value) @@ -582,14 +583,18 @@ while next: info = next.contents next = info.ai_next - w_family = space.wrap(info.ai_family.value) - w_socktype = space.wrap(info.ai_socktype.value) - w_proto = space.wrap(info.ai_protocol.value) - w_canonname = space.wrap(info.ai_canonname.value) - w_addr = w_makesockaddr(sapce, info.ai_canonname, info.ai_addrlen.value) - results.append(space.newtuple([w_family, w_socktype, w_proto, + w_family = space.wrap(info.ai_family) + w_socktype = space.wrap(info.ai_socktype) + w_proto = space.wrap(info.ai_protocol) + if info.ai_canonname: + w_canonname = space.wrap(info.ai_canonname) + else: + w_canonname = space.wrap('') + w_addr = w_makesockaddr(space, info.ai_addr, info.ai_addrlen, + info.ai_protocol) + result.append(space.newtuple([w_family, w_socktype, w_proto, w_canonname, w_addr])) - return space.newlist(result) + return space.newlist(result) finally: _c.freeaddrinfo(res) getaddrinfo.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int, int, int] Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Tue Apr 25 11:13:31 2006 @@ -118,7 +118,7 @@ ('h_aliases', POINTER(c_char_p)), ('h_addrtype', c_int), ('h_length', c_int), - ('h_addr_list', POINTER(c_char_p)) + ('h_addr_list', POINTER(POINTER(c_char))) ]) @@ -219,6 +219,10 @@ inet_aton.argtypes = [c_char_p, POINTER(in_addr)] inet_aton.restype = c_int +inet_ntoa = socketdll.inet_ntoa +inet_ntoa.argtypes = [in_addr] +inet_ntoa.restype = c_char_p + close = socketdll.close close.argtypes = [c_int] close.restype = c_int Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Tue Apr 25 11:13:31 2006 @@ -1,10 +1,12 @@ #! /usr/bin/env python -import os, py +import os, py, sys import ctypes from pypy.translator.tool.cbuild import build_executable from pypy.tool.udir import udir +del sys.modules['pypy.tool.udir'] # Don't expose udir + # ____________________________________________________________ # # Helpers for simple cases From dialtone at codespeak.net Tue Apr 25 11:20:03 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Tue, 25 Apr 2006 11:20:03 +0200 (CEST) Subject: [pypy-svn] r26314 - pypy/dist/pypy/translator/cl Message-ID: <20060425092003.C56FD10088@code0.codespeak.net> Author: dialtone Date: Tue Apr 25 11:19:58 2006 New Revision: 26314 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (sanxiyn, dialtone) avoid duplicate superclass declaration Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 11:19:58 2006 @@ -78,7 +78,8 @@ self.declare_class(cls._superclass) supername = repr_class_name(cls._superclass._name) class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) - self.gen.declarations.append(class_declaration) + if class_declaration not in self.gen.declarations: + self.gen.declarations.append(class_declaration) def op_new(self, result, _): cls = self.args[0].value From antocuni at codespeak.net Tue Apr 25 11:37:42 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 25 Apr 2006 11:37:42 +0200 (CEST) Subject: [pypy-svn] r26315 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060425093742.C3D0010088@code0.codespeak.net> Author: antocuni Date: Tue Apr 25 11:37:36 2006 New Revision: 26315 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py Log: rdict's iter* methods are now implemented on top of the new ll_get_items_iterator() insted of the deprecated ll_keys(). Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 11:37:36 2006 @@ -805,7 +805,9 @@ def ll_get_items_iterator(self): # NOT_RPYTHON ITER = DictItemsIterator(self._TYPE._KEYTYPE, self._TYPE._VALUETYPE) - return _dict_items_iterator(ITER, self._dict) + iter = _dict_items_iterator(ITER) + iter._items = self._dict.items() + return iter class _null_dict(_null_mixin(_dict), _dict): def __init__(self, DICT): @@ -813,9 +815,9 @@ class _dict_items_iterator(_builtin_type): - def __init__(self, ITER, d): + def __init__(self, ITER): self._TYPE = ITER - self._items = d.items() + self._items = [] self._index = -1 def ll_go_next(self): Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Tue Apr 25 11:37:36 2006 @@ -5,7 +5,6 @@ rtype_newdict, dum_variant, dum_keys, dum_values, dum_items from pypy.rpython.ootypesystem import ootype from pypy.rpython.ootypesystem.rlist import ll_newlist -from pypy.rpython.ootypesystem.riterable import iterator_type from pypy.rpython.rarithmetic import r_uint from pypy.rpython.objectmodel import hlinvoke from pypy.rpython import robject @@ -213,34 +212,33 @@ def __init__(self, r_dict, variant="keys"): self.r_dict = r_dict self.variant = variant - self.lowleveltype = iterator_type(r_dict, r_dict.key_repr) + self.lowleveltype = self._get_type() self.ll_dictiter = ll_dictiter self.ll_dictnext = ll_dictnext + def _get_type(self): + KEYTYPE = self.r_dict.key_repr.lowleveltype + VALUETYPE = self.r_dict.value_repr.lowleveltype + ITER = ootype.DictItemsIterator(KEYTYPE, VALUETYPE) + return ootype.Tuple({"iterator": ITER}) + + def ll_dictiter(ITER, d): iter = ootype.new(ITER) - iter.iterable = d - iter.index = 0 + iter.iterator = d.ll_get_items_iterator() return iter -# TODO: this is very inefficient for values and items because it does -# a dict lookup at every iteration. Need to be refactored. def ll_dictnext(iter, func, RETURNTYPE): - d = iter.iterable - keys = d.ll_keys() - index = iter.index - if index >= keys.ll_length(): + it = iter.iterator + if not it.ll_go_next(): raise StopIteration - iter.index = index + 1 - key = keys.ll_getitem_fast(index) if func is dum_keys: - return key + return it.ll_current_key() elif func is dum_values: - return d.ll_get(key) + return it.ll_current_value() elif func is dum_items: res = ootype.new(RETURNTYPE) - res.item0 = key - res.item1 = d.ll_get(key) + res.item0 = it.ll_current_key() + res.item1 = it.ll_current_value() return res - From dialtone at codespeak.net Tue Apr 25 11:40:46 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Tue, 25 Apr 2006 11:40:46 +0200 (CEST) Subject: [pypy-svn] r26316 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060425094046.6D75410088@code0.codespeak.net> Author: dialtone Date: Tue Apr 25 11:40:41 2006 New Revision: 26316 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: (sanxiyon, dialtone) more passing tests, more operations implemented Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Tue Apr 25 11:40:41 2006 @@ -38,6 +38,8 @@ op_int_eq = make_binary_op("=") op_int_gt = make_binary_op(">") op_int_lt = make_binary_op("<") + op_int_le = make_binary_op("<=") + op_int_and = make_binary_op("logand") op_int_mod = make_binary_op("mod") op_int_sub = make_binary_op("-") op_float_sub = make_binary_op("-") Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Tue Apr 25 11:40:41 2006 @@ -7,7 +7,6 @@ from pypy.translator.test import snippet as t - def dont_test_return_str(): def return_str(): return 'test' @@ -15,14 +14,12 @@ assert cl_return_str() == 'test' def test_if(): - py.test.skip("temporarily disabled") - cl_if = make_cl_func(t.if_then_else, [object, object, object]) + cl_if = make_cl_func(t.if_then_else, [bool, int, int]) assert cl_if(True, 50, 100) == 50 assert cl_if(False, 50, 100) == 100 + cl_if = make_cl_func(t.if_then_else, [int, int, int]) assert cl_if(0, 50, 100) == 100 assert cl_if(1, 50, 100) == 50 - assert cl_if([], 50, 100) == 100 - assert cl_if([[]], 50, 100) == 50 def test_gcd(): cl_gcd = make_cl_func(t.my_gcd, [int, int]) @@ -34,17 +31,19 @@ assert cl_perfect(28) == True def test_bool(): - py.test.skip("temporarily disabled") - cl_bool = make_cl_func(t.my_bool, [object]) + cl_bool = make_cl_func(t.my_bool, [int]) assert cl_bool(0) == False assert cl_bool(42) == True + cl_bool = make_cl_func(t.my_bool, [bool]) assert cl_bool(True) == True def test_contains(): - py.test.skip("temporarily disabled") - my_contains = make_cl_func(t.my_contains, [list, int]) - assert my_contains([1, 2, 3], 1) - assert not my_contains([1, 2, 3], 0) + py.test.skip("fails for obscure dict-related reasons") + def contains_int(num): + return t.my_contains([1,2,3], num) + my_contains = make_cl_func(contains_int, [int]) + assert my_contains(1) + assert not my_contains(0) is_one_or_two = make_cl_func(t.is_one_or_two, [int]) assert is_one_or_two(2) assert not is_one_or_two(3) @@ -60,7 +59,7 @@ assert cl_sieve() == 1028 def test_easy(): - py.test.skip("temporarily disabled") + #py.test.skip("temporarily disabled") # These are the Pyrex tests which were easy to adopt. f1 = make_cl_func(t.simple_func, [int]) assert f1(1) == 2 From nik at codespeak.net Tue Apr 25 11:53:24 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 11:53:24 +0200 (CEST) Subject: [pypy-svn] r26317 - in pypy/dist/pypy/rpython: . lltypesystem ootypesystem test Message-ID: <20060425095324.6107E10078@code0.codespeak.net> Author: nik Date: Tue Apr 25 11:53:05 2006 New Revision: 26317 Added: pypy/dist/pypy/rpython/lltypesystem/rrange.py (contents, props changed) pypy/dist/pypy/rpython/ootypesystem/rrange.py (contents, props changed) Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/rrange.py pypy/dist/pypy/rpython/test/test_rrange.py pypy/dist/pypy/rpython/typesystem.py Log: bring rrange support to ootypesystem. Added: pypy/dist/pypy/rpython/lltypesystem/rrange.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/rrange.py Tue Apr 25 11:53:05 2006 @@ -0,0 +1,71 @@ +from pypy.rpython.lltypesystem.lltype import Ptr, GcStruct, Signed, malloc, Void +from pypy.rpython.rrange import AbstractRangeRepr, AbstractRangeIteratorRepr + +# ____________________________________________________________ +# +# Concrete implementation of RPython lists that are returned by range() +# and never mutated afterwards: +# +# struct range { +# Signed start, stop; // step is always constant +# } +# +# struct rangest { +# Signed start, stop, step; // rare case, for completeness +# } + +RANGE = GcStruct("range", ("start", Signed), ("stop", Signed)) +RANGEITER = GcStruct("range", ("next", Signed), ("stop", Signed)) + +RANGEST = GcStruct("range", ("start", Signed), ("stop", Signed),("step", Signed)) +RANGESTITER = GcStruct("range", ("next", Signed), ("stop", Signed), ("step", Signed)) + +class RangeRepr(AbstractRangeRepr): + + RANGE = Ptr(RANGE) + RANGEITER = Ptr(RANGEITER) + + RANGEST = Ptr(RANGEST) + RANGESTITER = Ptr(RANGESTITER) + + getfield_opname = "getfield" + + def __init__(self, *args): + AbstractRangeRepr.__init__(self, *args) + self.ll_newrange = ll_newrange + self.ll_newrangest = ll_newrangest + + def make_iterator_repr(self): + return RangeIteratorRepr(self) + + +def ll_newrange(start, stop): + l = malloc(RANGE) + l.start = start + l.stop = stop + return l + +def ll_newrangest(start, stop, step): + if step == 0: + raise ValueError + l = malloc(RANGEST) + l.start = start + l.stop = stop + l.step = step + return l + + +class RangeIteratorRepr(AbstractRangeIteratorRepr): + + def __init__(self, *args): + AbstractRangeIteratorRepr.__init__(self, *args) + self.ll_rangeiter = ll_rangeiter + +def ll_rangeiter(ITERPTR, rng): + iter = malloc(ITERPTR.TO) + iter.next = rng.start + iter.stop = rng.stop + if ITERPTR.TO is RANGESTITER: + iter.step = rng.step + return iter + Added: pypy/dist/pypy/rpython/ootypesystem/rrange.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/rrange.py Tue Apr 25 11:53:05 2006 @@ -0,0 +1,57 @@ +from pypy.rpython.ootypesystem.ootype import Signed, Tuple, new +from pypy.rpython.rrange import AbstractRangeRepr, AbstractRangeIteratorRepr + +RANGE = Tuple({"start": Signed, "stop": Signed}) +RANGEITER = Tuple({"next": Signed, "stop": Signed}) + +RANGEST = Tuple({"start": Signed, "stop": Signed, "step": Signed}) +RANGESTITER = Tuple({"next": Signed, "stop": Signed, "step": Signed}) + +class RangeRepr(AbstractRangeRepr): + + RANGE = RANGE + RANGEITER = RANGEITER + RANGEST = RANGEST + RANGESTITER = RANGESTITER + + getfield_opname = "oogetfield" + + def __init__(self, *args): + AbstractRangeRepr.__init__(self, *args) + self.ll_newrange = ll_newrange + self.ll_newrangest = ll_newrangest + + def make_iterator_repr(self): + return RangeIteratorRepr(self) + + +def ll_newrange(start, stop): + l = new(RANGE) + l.start = start + l.stop = stop + return l + +def ll_newrangest(start, stop, step): + if step == 0: + raise ValueError + l = new(RANGEST) + l.start = start + l.stop = stop + l.step = step + return l + + +class RangeIteratorRepr(AbstractRangeIteratorRepr): + + def __init__(self, *args): + AbstractRangeIteratorRepr.__init__(self, *args) + self.ll_rangeiter = ll_rangeiter + +def ll_rangeiter(ITER, rng): + iter = new(ITER) + iter.next = rng.start + iter.stop = rng.stop + if ITER is RANGESTITER: + iter.step = rng.step + return iter + Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Tue Apr 25 11:53:05 2006 @@ -8,8 +8,8 @@ def __init__(self, rtyper, items_r): AbstractTupleRepr.__init__(self, rtyper, items_r) - #self.lowleveltype = ootype.Tuple(dict(zip(self.fieldnames, self.lltypes))) - self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) + self.lowleveltype = ootype.Tuple(dict(zip(self.fieldnames, self.lltypes))) + #self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) def newtuple(cls, llops, r_tuple, items_v): # items_v should have the lowleveltype of the internal reprs Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Tue Apr 25 11:53:05 2006 @@ -12,11 +12,10 @@ class __extend__(annmodel.SomeList): def rtyper_makerepr(self, rtyper): - from pypy.rpython import rrange listitem = self.listdef.listitem s_value = listitem.s_value if listitem.range_step is not None and not listitem.mutated: - return rrange.RangeRepr(listitem.range_step) + return rtyper.type_system.rrange.RangeRepr(listitem.range_step) elif (s_value.__class__ is annmodel.SomeObject and s_value.knowntype == object): return robject.pyobj_repr else: Modified: pypy/dist/pypy/rpython/rrange.py ============================================================================== --- pypy/dist/pypy/rpython/rrange.py (original) +++ pypy/dist/pypy/rpython/rrange.py Tue Apr 25 11:53:05 2006 @@ -1,40 +1,22 @@ from pypy.annotation.pairtype import pairtype from pypy.rpython.error import TyperError +from pypy.rpython.lltypesystem.lltype import Signed, Void, Ptr from pypy.rpython.rmodel import Repr, IntegerRepr, IteratorRepr -from pypy.rpython.lltypesystem.lltype import Ptr, GcStruct, Signed, malloc, Void from pypy.objspace.flow.model import Constant from pypy.rpython.rlist import dum_nocheck, dum_checkidx -# ____________________________________________________________ -# -# Concrete implementation of RPython lists that are returned by range() -# and never mutated afterwards: -# -# struct range { -# Signed start, stop; // step is always constant -# } -# -# struct rangest { -# Signed start, stop, step; // rare case, for completeness -# } -RANGE = GcStruct("range", ("start", Signed), ("stop", Signed)) -RANGEITER = GcStruct("range", ("next", Signed), ("stop", Signed)) - -RANGEST = GcStruct("range", ("start", Signed), ("stop", Signed),("step", Signed)) -RANGESTITER = GcStruct("range", ("next", Signed), ("stop", Signed), ("step", Signed)) - -class RangeRepr(Repr): +class AbstractRangeRepr(Repr): def __init__(self, step): self.step = step if step != 0: - self.lowleveltype = Ptr(RANGE) + self.lowleveltype = self.RANGE else: - self.lowleveltype = Ptr(RANGEST) + self.lowleveltype = self.RANGEST def _getstep(self, v_rng, hop): - return hop.genop('getfield', [v_rng, hop.inputconst(Void, 'step')], - resulttype=Signed) + return hop.genop(self.getfield_opname, + [v_rng, hop.inputconst(Void, 'step')], resulttype=Signed) def rtype_len(self, hop): v_rng, = hop.inputargs(self) @@ -44,10 +26,7 @@ cstep = self._getstep(v_rng, hop) return hop.gendirectcall(ll_rangelen, v_rng, cstep) - def make_iterator_repr(self): - return RangeIteratorRepr(self) - -class __extend__(pairtype(RangeRepr, IntegerRepr)): +class __extend__(pairtype(AbstractRangeRepr, IntegerRepr)): def rtype_getitem((r_rng, r_int), hop): if hop.has_implicit_exception(IndexError): @@ -105,21 +84,6 @@ # # Irregular operations. -def ll_newrange(start, stop): - l = malloc(RANGE) - l.start = start - l.stop = stop - return l - -def ll_newrangest(start, stop, step): - if step == 0: - raise ValueError - l = malloc(RANGEST) - l.start = start - l.stop = stop - l.step = step - return l - def rtype_builtin_range(hop): vstep = hop.inputconst(Signed, 1) if hop.nb_args == 1: @@ -132,15 +96,18 @@ if isinstance(vstep, Constant) and vstep.value == 0: # not really needed, annotator catches it. Just in case... raise TyperError("range cannot have a const step of zero") - if isinstance(hop.r_result, RangeRepr): + if isinstance(hop.r_result, AbstractRangeRepr): if hop.r_result.step != 0: - return hop.gendirectcall(ll_newrange, vstart, vstop) + return hop.gendirectcall(hop.r_result.ll_newrange, vstart, vstop) else: - return hop.gendirectcall(ll_newrangest, vstart, vstop, vstep) + return hop.gendirectcall(hop.r_result.ll_newrangest, vstart, vstop, vstep) else: # cannot build a RANGE object, needs a real list r_list = hop.r_result - cLIST = hop.inputconst(Void, r_list.lowleveltype.TO) + ITEMTYPE = r_list.lowleveltype + if isinstance(ITEMTYPE, Ptr): + ITEMTYPE = ITEMTYPE.TO + cLIST = hop.inputconst(Void, ITEMTYPE) return hop.gendirectcall(ll_range2list, cLIST, vstart, vstop, vstep) rtype_builtin_xrange = rtype_builtin_range @@ -151,9 +118,8 @@ length = _ll_rangelen(start, stop, step) l = LIST.ll_newlist(length) idx = 0 - items = l.ll_items() while idx < length: - items[idx] = start + l.ll_setitem_fast(idx, start) start += step idx += 1 return l @@ -162,18 +128,18 @@ # # Iteration. -class RangeIteratorRepr(IteratorRepr): +class AbstractRangeIteratorRepr(IteratorRepr): def __init__(self, r_rng): self.r_rng = r_rng if r_rng.step != 0: - self.lowleveltype = Ptr(RANGEITER) + self.lowleveltype = r_rng.RANGEITER else: - self.lowleveltype = Ptr(RANGESTITER) + self.lowleveltype = r_rng.RANGESTITER def newiter(self, hop): v_rng, = hop.inputargs(self.r_rng) citerptr = hop.inputconst(Void, self.lowleveltype) - return hop.gendirectcall(ll_rangeiter, citerptr, v_rng) + return hop.gendirectcall(self.ll_rangeiter, citerptr, v_rng) def rtype_next(self, hop): v_iter, = hop.inputargs(self) @@ -189,14 +155,6 @@ hop.exception_is_here() return hop.gendirectcall(llfn, v_iter, *args) -def ll_rangeiter(ITERPTR, rng): - iter = malloc(ITERPTR.TO) - iter.next = rng.start - iter.stop = rng.stop - if ITERPTR.TO is RANGESTITER: - iter.step = rng.step - return iter - def ll_rangenext_up(iter, step): next = iter.next if next >= iter.stop: Modified: pypy/dist/pypy/rpython/test/test_rrange.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rrange.py (original) +++ pypy/dist/pypy/rpython/test/test_rrange.py Tue Apr 25 11:53:05 2006 @@ -2,122 +2,136 @@ from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.rarithmetic import intmask -def test_rlist_range(): - def test1(start, stop, step, varstep): - expected = range(start, stop, step) - length = len(expected) - if varstep: - l = ll_newrangest(start, stop, step) - step = l.step - else: - l = ll_newrange(start,stop) - assert ll_rangelen(l, step) == length - lst = [ll_rangeitem(dum_nocheck, l, i, step) for i in range(length)] - assert lst == expected - lst = [ll_rangeitem_nonneg(dum_nocheck, l, i, step) for i in range(length)] - assert lst == expected - lst = [ll_rangeitem(dum_nocheck, l, i-length, step) for i in range(length)] - assert lst == expected - - for start in (-10, 0, 1, 10): - for stop in (-8, 0, 4, 8, 25): - for step in (1, 2, 3, -1, -2): - for varstep in False, True: - test1(start, stop, step, varstep) - -# ____________________________________________________________ - -def test_range(): - def dummyfn(N): - total = 0 - for i in range(N): - total += i - return total - res = interpret(dummyfn, [10]) - assert res == 45 - -def test_range_is_lazy(): - def dummyfn(N, M): - total = 0 - for i in range(M): - if i == N: - break - total += i - return total - res = interpret(dummyfn, [10, 2147418112]) - assert res == 45 - -def test_range_item(): - def dummyfn(start, stop, i): - r = range(start, stop) - return r[i] - res = interpret(dummyfn, [10, 17, 4]) - assert res == 14 - res = interpret(dummyfn, [10, 17, -2]) - assert res == 15 - -def test_xrange(): - def dummyfn(N): - total = 0 - for i in xrange(N): - total += i - return total - res = interpret(dummyfn, [10]) - assert res == 45 - -def test_range_len(): - def dummyfn(start, stop): - r = range(start, stop) - return len(r) - start, stop = 10, 17 - res = interpret(dummyfn, [start, stop]) - assert res == dummyfn(start, stop) - -def test_range2list(): - def dummyfn(start, stop): - r = range(start, stop) - r.reverse() - return r[0] - start, stop = 10, 17 - res = interpret(dummyfn, [start, stop]) - assert res == dummyfn(start, stop) - -def check_failed(func, *args): - try: - interpret(func, *args) - except: - return True - else: - return False - -def test_range_extra(): - def failingfn_const(): - r = range(10, 17, 0) - return r[-1] - assert check_failed(failingfn_const, []) - - def failingfn_var(step): - r = range(10, 17, step) - return r[-1] - step = 3 - res = interpret(failingfn_var, [step]) - assert res == failingfn_var(step) - step = 0 - assert check_failed(failingfn_var, [step]) - -def test_range_iter(): - def fn(start, stop, step): - res = 0 - if step == 0: - if stop >= start: - r = range(start, stop, 1) + +class AbstractTestRange: + + def test_rlist_range(self): + def test1(start, stop, step, varstep): + expected = range(start, stop, step) + length = len(expected) + if varstep: + l = self.rrange.ll_newrangest(start, stop, step) + step = l.step else: - r = range(start, stop, -1) + l = self.rrange.ll_newrange(start,stop) + assert ll_rangelen(l, step) == length + lst = [ll_rangeitem(dum_nocheck, l, i, step) for i in range(length)] + assert lst == expected + lst = [ll_rangeitem_nonneg(dum_nocheck, l, i, step) for i in range(length)] + assert lst == expected + lst = [ll_rangeitem(dum_nocheck, l, i-length, step) for i in range(length)] + assert lst == expected + + for start in (-10, 0, 1, 10): + for stop in (-8, 0, 4, 8, 25): + for step in (1, 2, 3, -1, -2): + for varstep in False, True: + test1(start, stop, step, varstep) + + def test_range(self): + def dummyfn(N): + total = 0 + for i in range(N): + total += i + return total + res = interpret(dummyfn, [10], type_system=self.ts) + assert res == 45 + + def test_range_is_lazy(self): + def dummyfn(N, M): + total = 0 + for i in range(M): + if i == N: + break + total += i + return total + res = interpret(dummyfn, [10, 2147418112], type_system=self.ts) + assert res == 45 + + def test_range_item(self): + def dummyfn(start, stop, i): + r = range(start, stop) + return r[i] + res = interpret(dummyfn, [10, 17, 4], type_system=self.ts) + assert res == 14 + res = interpret(dummyfn, [10, 17, -2], type_system=self.ts) + assert res == 15 + + def test_xrange(self): + def dummyfn(N): + total = 0 + for i in xrange(N): + total += i + return total + res = interpret(dummyfn, [10], type_system=self.ts) + assert res == 45 + + def test_range_len(self): + def dummyfn(start, stop): + r = range(start, stop) + return len(r) + start, stop = 10, 17 + res = interpret(dummyfn, [start, stop], type_system=self.ts) + assert res == dummyfn(start, stop) + + def test_range2list(self): + def dummyfn(start, stop): + r = range(start, stop) + r.reverse() + return r[0] + start, stop = 10, 17 + res = interpret(dummyfn, [start, stop], type_system=self.ts) + assert res == dummyfn(start, stop) + + def check_failed(self, func, *args): + try: + kwargs = {"type_system": self.ts} + interpret(func, *args, **kwargs) + except: + return True else: - r = range(start, stop, step) - for i in r: - res = res * 51 + i - return res - for args in [2, 7, 0], [7, 2, 0], [10, 50, 7], [50, -10, -3]: - res = interpret(fn, args)#, view=True) - assert res == intmask(fn(*args)) + return False + + def test_range_extra(self): + def failingfn_const(): + r = range(10, 17, 0) + return r[-1] + assert self.check_failed(failingfn_const, []) + + def failingfn_var(step): + r = range(10, 17, step) + return r[-1] + step = 3 + res = interpret(failingfn_var, [step], type_system=self.ts) + assert res == failingfn_var(step) + step = 0 + assert self.check_failed(failingfn_var, [step]) + + def test_range_iter(self): + def fn(start, stop, step): + res = 0 + if step == 0: + if stop >= start: + r = range(start, stop, 1) + else: + r = range(start, stop, -1) + else: + r = range(start, stop, step) + for i in r: + res = res * 51 + i + return res + for args in [2, 7, 0], [7, 2, 0], [10, 50, 7], [50, -10, -3]: + res = interpret(fn, args, type_system=self.ts) + assert res == intmask(fn(*args)) + + +class TestRangeLltype(AbstractTestRange): + + ts = "lltype" + from pypy.rpython.lltypesystem import rrange + + +class TestRangeOotype(AbstractTestRange): + + ts = "ootype" + from pypy.rpython.ootypesystem import rrange Modified: pypy/dist/pypy/rpython/typesystem.py ============================================================================== --- pypy/dist/pypy/rpython/typesystem.py (original) +++ pypy/dist/pypy/rpython/typesystem.py Tue Apr 25 11:53:05 2006 @@ -20,8 +20,8 @@ None, None, ['__doc__']) except ImportError: return None - if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', 'rslice', 'rdict', - 'exceptiondata'): + if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', + 'rslice', 'rdict', 'rrange', 'exceptiondata'): mod = load(name) if mod is not None: setattr(self, name, mod) From antocuni at codespeak.net Tue Apr 25 11:54:10 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 25 Apr 2006 11:54:10 +0200 (CEST) Subject: [pypy-svn] r26318 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060425095410.E9BB110088@code0.codespeak.net> Author: antocuni Date: Tue Apr 25 11:53:55 2006 New Revision: 26318 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py Log: rdict's keys(), values() and items() are now implemented on top of ll_get_items_iterator(). The deprecated ll_keys() has been removed from Dict's interface. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 11:53:55 2006 @@ -358,7 +358,6 @@ "ll_set": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), "ll_remove": Meth([self.KEYTYPE_T], Bool), # return False is key was not present "ll_contains": Meth([self.KEYTYPE_T], Bool), - "ll_keys": Meth([], List(self.KEYTYPE_T)), # deprecated "ll_get_items_iterator": Meth([], DictItemsIterator(self.KEYTYPE_T, self.VALUETYPE_T)), }) @@ -796,12 +795,6 @@ assert typeOf(key) == self._TYPE._KEYTYPE return key in self._dict - def ll_keys(self): - # NOT_RPYTHON - keys = _list(List(self._TYPE._KEYTYPE)) - keys._list = self._dict.keys() - return keys - def ll_get_items_iterator(self): # NOT_RPYTHON ITER = DictItemsIterator(self._TYPE._KEYTYPE, self._TYPE._VALUETYPE) Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Tue Apr 25 11:53:55 2006 @@ -90,21 +90,21 @@ return self.recast_value(hop.llops, v_res) def rtype_method_keys(self, hop): - hop.exception_cannot_occur() - return self.send_message(hop, 'll_keys') + return self._rtype_method_kvi(hop, dum_keys) def rtype_method_values(self, hop): - return self._rtype_method_vi(hop, dum_values) + return self._rtype_method_kvi(hop, dum_values) def rtype_method_items(self, hop): - return self._rtype_method_vi(hop, dum_items) + return self._rtype_method_kvi(hop, dum_items) - def _rtype_method_vi(self, hop, spec): + def _rtype_method_kvi(self, hop, spec): v_dict, = hop.inputargs(self) r_list = hop.r_result cLIST = hop.inputconst(ootype.Void, r_list.lowleveltype) - c_func = hop.inputconst(ootype.Void, spec) - return hop.gendirectcall(ll_dict_values_items, v_dict, cLIST, c_func) + c_func = hop.inputconst(ootype.Void, spec) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_dict_kvi, v_dict, cLIST, c_func) def rtype_method_iterkeys(self, hop): hop.exception_cannot_occur() @@ -183,23 +183,23 @@ d.ll_set(key, default) return default -def ll_dict_values_items(d, LIST, func): - keys = d.ll_keys() - length = keys.ll_length() +def ll_dict_kvi(d, LIST, func): + length = d.ll_length() result = ll_newlist(LIST, length) + it = d.ll_get_items_iterator() i = 0 - while i < length: - key = keys.ll_getitem_fast(i) - value = d.ll_get(key) + while it.ll_go_next(): + if func is dum_keys: + result.ll_setitem_fast(i, it.ll_current_key()) + elif func is dum_values: + result.ll_setitem_fast(i, it.ll_current_value()) if func is dum_items: r = ootype.new(LIST._ITEMTYPE) - r.item0 = key # TODO: do we need casting? - r.item1 = value + r.item0 = it.ll_current_key() # TODO: do we need casting? + r.item1 = it.ll_current_value() result.ll_setitem_fast(i, r) - elif func is dum_values: - result.ll_setitem_fast(i, value) - i += 1 + #assert i == length return result @@ -242,3 +242,4 @@ res.item0 = it.ll_current_key() res.item1 = it.ll_current_value() return res + From nik at codespeak.net Tue Apr 25 11:55:49 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 11:55:49 +0200 (CEST) Subject: [pypy-svn] r26319 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060425095549.25A741008F@code0.codespeak.net> Author: nik Date: Tue Apr 25 11:55:44 2006 New Revision: 26319 Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py Log: oops, i didn't mean to check this in with the last commit. Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Tue Apr 25 11:55:44 2006 @@ -8,8 +8,8 @@ def __init__(self, rtyper, items_r): AbstractTupleRepr.__init__(self, rtyper, items_r) - self.lowleveltype = ootype.Tuple(dict(zip(self.fieldnames, self.lltypes))) - #self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) + #self.lowleveltype = ootype.Tuple(dict(zip(self.fieldnames, self.lltypes))) + self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) def newtuple(cls, llops, r_tuple, items_v): # items_v should have the lowleveltype of the internal reprs From ale at codespeak.net Tue Apr 25 12:17:54 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Tue, 25 Apr 2006 12:17:54 +0200 (CEST) Subject: [pypy-svn] r26320 - pypy/dist/pypy/module/_socket Message-ID: <20060425101754.D0E761007E@code0.codespeak.net> Author: ale Date: Tue Apr 25 12:17:51 2006 New Revision: 26320 Modified: pypy/dist/pypy/module/_socket/interp_socket.py Log: (aleale, arre) First try at getting gethostbyaddr to work - not finished Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Tue Apr 25 12:17:51 2006 @@ -164,8 +164,14 @@ for a host. The host argument is a string giving a host name or IP number. """ hostent = _c.gethostbyname(name) + return common_gethost(space,hostent) + +gethostbyname_ex.unwrap_spec = [ObjSpace, str] + +def common_gethost(space, hostent): + if not hostent: - raise w_get_socketherror(_c.h_errno.value) + raise w_get_socketherror(space, _c.hstrerror(_c.h_errno.value), _c.h_errno.value) aliases = [] for alias in hostent.contents.h_aliases: if alias is None: @@ -178,20 +184,22 @@ break address_list.append(space.wrap(_c.inet_ntoa(addr.contents))) - return space.newtuple([space.wrap(hostent.contents.h_name), space.newlist(aliases), - space.newlist(address_list)]) -gethostbyname_ex.unwrap_spec = [ObjSpace, str] + return space.newtuple([space.wrap(hostent.contents.h_name), + space.newlist(aliases), space.newlist(address_list)]) -def gethostbyaddr(space, ip_num): +def gethostbyaddr(space, name): """gethostbyaddr(host) -> (name, aliaslist, addresslist) Return the true host name, a list of aliases, and a list of IP addresses, for a host. The host argument is a string giving a host name or IP number. """ - try: - return space.wrap(socket.gethostbyaddr(ip_num)) - except socket.error, e: - raise wrap_socketerror(space, e) + hostent = _c.gethostbyname(name) + if not hostent: + raise w_get_socketherror(_c.h_errno.value) + p_addr = hostent.contents.h_addr_list[0] + + hostent = _c.gethostbyaddr(p_addr, ctypes.sizeof(_c.in_addr), _c.AF_INET) + return common_gethost(space, hostent) gethostbyaddr.unwrap_spec = [ObjSpace, str] def getservbyname(space, name, w_proto=NoneNotWrapped): From arigo at codespeak.net Tue Apr 25 12:25:15 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 12:25:15 +0200 (CEST) Subject: [pypy-svn] r26321 - in pypy/dist/pypy: objspace/cpy objspace/cpy/test rpython/rctypes Message-ID: <20060425102515.0B1301007E@code0.codespeak.net> Author: arigo Date: Tue Apr 25 12:25:12 2006 New Revision: 26321 Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/objspace/cpy/wrappable.py pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/implementation.py Log: A possibly dead-end check-in: trying to compile built-in functions via a 'builder' attribute on W_Objects that specify how the Python object should be created. The builder's intent is to be called when the compiled extension module is first imported, but this is not implemented and I think I will try another approach instead. Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Tue Apr 25 12:25:12 2006 @@ -4,6 +4,7 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.objspace.cpy.ann_policy import CPyAnnotatorPolicy from pypy.objspace.cpy.objspace import CPyObjSpace +from pypy.objspace.cpy import wrappable, capi import pypy.rpython.rctypes.implementation from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root @@ -55,6 +56,12 @@ assert s.knowntype == CPyObjSpace.W_Object s = a.binding(graph.getreturnvar()) assert s.knowntype == CPyObjSpace.W_Object + graph = graphof(t, wrappable.builtin_function_builder) + assert len(graph.getargs()) == 2 + s = a.binding(graph.getargs()[0]) + assert s.knowntype == capi.PyMethodDef + s = a.binding(graph.getreturnvar()) + assert s.knowntype == CPyObjSpace.W_Object def test_compile_bltinfunc(): py.test.skip("in-progress") Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Tue Apr 25 12:25:12 2006 @@ -11,6 +11,14 @@ from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root +def builtin_function_builder(ml, w_module): + """Build and initialize a real CPython built-in function object. + Compiled into the extension module and called when it is + first imported. + """ + return PyCFunction_NewEx(byref(ml), None, w_module) + + class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): def wrap((space, func)): @@ -22,9 +30,21 @@ assert unwrap_spec == [ObjSpace, W_Root] # XXX for now - # make a real CPython built-in function from a PyMethodDef + # argh! callbacks of mode PyDLL are not supported by ctypes so far + # (as of 0.9.9.4). XXX hack. I am not happy. + + # as a workaround, we use an interface that lets us return a normal + # Python function used for testing, with an attached 'builder' + # function that is executed at start-up to construct the object + + def trampoline(*args): + # only called during testing; not compiled + args_w = [space.W_Object(a) for a in args] + w_result = bltin(space, *args_w) + return w_result.value + def callback(w_self, w_args): - "XXX minimalistic" + # only called when compiled into the extension module w_a = PyObject_GetItem(w_args, 0) w_result = bltin(space, w_a) return w_result @@ -34,17 +54,7 @@ ml_flags = METH_VARARGS, #ml_doc = ..., ) - w_result = PyCFunction_NewEx(byref(ml), None, func.w_module) - w_result.ml = ml # keep ml alive as long as w_result is around - - # argh! callbacks of mode PyDLL are not supported by ctypes so far - # (as of 0.9.9.4). XXX hack. I am not happy. - - def hackish_trampoline(*args): - args_w = [space.W_Object(a) for a in args] - w_result = bltin(space, *args_w) - return w_result.value - w_pseudoresult = W_Object(hackish_trampoline) - w_pseudoresult._hack_replace_with_ = w_result - return w_pseudoresult + w_result = W_Object(trampoline) + w_result.builder = builtin_function_builder, (ml, func.w_module) + return w_result Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Tue Apr 25 12:25:12 2006 @@ -1,7 +1,8 @@ from ctypes import py_object -from pypy.annotation.model import SomeCTypesObject +from pypy.annotation.model import SomeCTypesObject, SomeImpossibleValue from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.rpython.lltypesystem import lltype +from pypy.tool.uid import Hashable class CallEntry(CTypesCallEntry): @@ -22,6 +23,22 @@ "Annotation and rtyping of py_object instances." _type_ = py_object + def object_seen(self, bookkeeper): + "Called when the annotator sees this py_object." + # extension: if the py_object instance has a 'builder' attribute, + # it must be a pair (callable, args) which is meant to be called + # at initialization-time when the compiled extension module is + # first imported. It returns the "real" Python object. + if hasattr(self.instance, 'builder'): + # emulate a call so that the callable is properly annotated + callable, args = self.instance.builder + s_callable = bookkeeper.immutablevalue(callable) + args_s = [bookkeeper.immutablevalue(a) for a in args] + uniquekey = Hashable(self.instance) + s_res = bookkeeper.emulate_pbc_call(uniquekey, s_callable, args_s) + assert (issubclass(s_res.knowntype, py_object) or + isinstance(s_res, SomeImpossibleValue)) + def get_repr(self, rtyper, s_pyobject): from pypy.rpython.rctypes.rpyobject import CTypesPyObjRepr lowleveltype = lltype.Ptr(lltype.PyObject) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Apr 25 12:25:12 2006 @@ -76,6 +76,9 @@ if isinstance(o, dict): for x in o.itervalues(): recfind(x) + elif isinstance(o, (list, tuple)): + for x in o: + recfind(x) elif extregistry.is_registered(o): entry = extregistry.lookup(o) if isinstance(entry, CTypesEntry): From nik at codespeak.net Tue Apr 25 12:47:31 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Tue, 25 Apr 2006 12:47:31 +0200 (CEST) Subject: [pypy-svn] r26322 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060425104731.C651A1007E@code0.codespeak.net> Author: nik Date: Tue Apr 25 12:47:23 2006 New Revision: 26322 Added: pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py - copied, changed from r26319, pypy/dist/pypy/rpython/ootypesystem/test/test_ootuple.py Removed: pypy/dist/pypy/rpython/ootypesystem/test/test_ootuple.py Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/ootypesystem/rrange.py pypy/dist/pypy/rpython/ootypesystem/rtuple.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: rename the Tuple type to "Record". Tuple was really too misleading. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 12:47:23 2006 @@ -192,15 +192,15 @@ def _get_interp_class(self): raise NotImplementedError -class Tuple(BuiltinType): +class Record(BuiltinType): - # We try to keep Tuple as similar to Instance as possible, so backends + # We try to keep Record as similar to Instance as possible, so backends # can treat them polymorphically, if they choose to do so. def __init__(self, fields): self._fields = frozendict() self._add_fields(fields) - self._null = _null_tuple(self) + self._null = _null_record(self) def _add_fields(self, fields): fields.copy() @@ -212,7 +212,7 @@ return self._null def _get_interp_class(self): - return _tuple + return _record def _field_type(self, name): try: @@ -836,7 +836,7 @@ self.__dict__["_TYPE"] = ITER -class _tuple(object): +class _record(object): def __init__(self, TYPE): self._items = {} @@ -862,10 +862,10 @@ else: return 0 # for all null tuples -class _null_tuple(_null_mixin(_tuple), _tuple): +class _null_record(_null_mixin(_record), _record): - def __init__(self, TUPLE): - self.__dict__["_TYPE"] = TUPLE + def __init__(self, RECORD): + self.__dict__["_TYPE"] = RECORD def new(TYPE): @@ -942,7 +942,7 @@ return instance._downcast(INSTANCE) def ooidentityhash(inst): - assert isinstance(typeOf(inst), (Instance, Tuple)) + assert isinstance(typeOf(inst), (Instance, Record)) return inst._identityhash() def setItemType(LIST, ITEMTYPE): Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Tue Apr 25 12:47:23 2006 @@ -220,7 +220,7 @@ KEYTYPE = self.r_dict.key_repr.lowleveltype VALUETYPE = self.r_dict.value_repr.lowleveltype ITER = ootype.DictItemsIterator(KEYTYPE, VALUETYPE) - return ootype.Tuple({"iterator": ITER}) + return ootype.Record({"iterator": ITER}) def ll_dictiter(ITER, d): Modified: pypy/dist/pypy/rpython/ootypesystem/rrange.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rrange.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rrange.py Tue Apr 25 12:47:23 2006 @@ -1,11 +1,11 @@ -from pypy.rpython.ootypesystem.ootype import Signed, Tuple, new +from pypy.rpython.ootypesystem.ootype import Signed, Record, new from pypy.rpython.rrange import AbstractRangeRepr, AbstractRangeIteratorRepr -RANGE = Tuple({"start": Signed, "stop": Signed}) -RANGEITER = Tuple({"next": Signed, "stop": Signed}) +RANGE = Record({"start": Signed, "stop": Signed}) +RANGEITER = Record({"next": Signed, "stop": Signed}) -RANGEST = Tuple({"start": Signed, "stop": Signed, "step": Signed}) -RANGESTITER = Tuple({"next": Signed, "stop": Signed, "step": Signed}) +RANGEST = Record({"start": Signed, "stop": Signed, "step": Signed}) +RANGESTITER = Record({"next": Signed, "stop": Signed, "step": Signed}) class RangeRepr(AbstractRangeRepr): Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Tue Apr 25 12:47:23 2006 @@ -8,7 +8,7 @@ def __init__(self, rtyper, items_r): AbstractTupleRepr.__init__(self, rtyper, items_r) - #self.lowleveltype = ootype.Tuple(dict(zip(self.fieldnames, self.lltypes))) + #self.lowleveltype = ootype.Record(dict(zip(self.fieldnames, self.lltypes))) self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) def newtuple(cls, llops, r_tuple, items_v): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Tue Apr 25 12:47:23 2006 @@ -132,7 +132,7 @@ res = interpret(f, [3], type_system="ootype") assert res == 3 - def dont_test_listtype_explosion(self): + def test_listtype_explosion(self): def f(x): l1 = [x] l2 = [x] @@ -148,3 +148,19 @@ r_l2 = typer.getrepr(s_l2) assert r_l1.lowleveltype == r_l2.lowleveltype + def test_tupletype_explosion(self): + def f(x): + t1 = ([x], [x, x]) + t2 = ([x, x], [x]) + return t1, t2 + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(f, [int]) + typer = t.buildrtyper(type_system="ootype") + typer.specialize() + + s_t1, s_t2 = s.items + r_t1 = typer.getrepr(s_t1) + r_t2 = typer.getrepr(s_t2) + assert r_t1.lowleveltype == r_t2.lowleveltype + From arigo at codespeak.net Tue Apr 25 13:20:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 13:20:01 +0200 (CEST) Subject: [pypy-svn] r26324 - in pypy/dist/pypy: objspace/cpy objspace/cpy/test rpython/rctypes rpython/rctypes/test Message-ID: <20060425112001.8B6D010088@code0.codespeak.net> Author: arigo Date: Tue Apr 25 13:19:59 2006 New Revision: 26324 Modified: pypy/dist/pypy/objspace/cpy/ann_policy.py pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/objspace/cpy/wrappable.py pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: Dropping the latest approach and come back to the annotated trampoline. For clarity, commented out all the callback-related code in rctypes; it can be re-enabled later if necessary. Modified: pypy/dist/pypy/objspace/cpy/ann_policy.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/ann_policy.py (original) +++ pypy/dist/pypy/objspace/cpy/ann_policy.py Tue Apr 25 13:19:59 2006 @@ -1,3 +1,8 @@ from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy -CPyAnnotatorPolicy = PyPyAnnotatorPolicy # for now +class CPyAnnotatorPolicy(PyPyAnnotatorPolicy): + allow_someobjects = True + + # XXX make it more subtle: only allow SomeObjects in + # some specific functions, not all of them. + # Currently only trampoline() in wrapper.py should need them. Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Tue Apr 25 13:19:59 2006 @@ -3,7 +3,7 @@ from ctypes import * from pypy.rpython.rctypes.tool import ctypes_platform from pypy.rpython.rctypes import apyobject -from pypy.rpython.rctypes.implementation import CALLBACK_FUNCTYPE +##from pypy.rpython.rctypes.implementation import CALLBACK_FUNCTYPE class W_Object(py_object): "A py_object subclass, representing wrapped objects for the CPyObjSpace." @@ -16,11 +16,11 @@ ############################################################### # ____________________ Types and constants ____________________ -PyCFunction = CALLBACK_FUNCTYPE(W_Object, W_Object, W_Object, callconv=PyDLL) -PyNoArgsFunction = CALLBACK_FUNCTYPE(W_Object, W_Object, callconv=PyDLL) -PyCFunctionWithKeywords = CALLBACK_FUNCTYPE(W_Object, - W_Object, W_Object, W_Object, - callconv=PyDLL) +##PyCFunction = CALLBACK_FUNCTYPE(W_Object, W_Object, W_Object, callconv=PyDLL) +##PyNoArgsFunction = CALLBACK_FUNCTYPE(W_Object, W_Object, callconv=PyDLL) +##PyCFunctionWithKeywords = CALLBACK_FUNCTYPE(W_Object, +## W_Object, W_Object, W_Object, +## callconv=PyDLL) class CConfig: _header_ = """ @@ -40,12 +40,12 @@ Py_GT = ctypes_platform.ConstantInteger('Py_GT') Py_GE = ctypes_platform.ConstantInteger('Py_GE') - PyMethodDef = ctypes_platform.Struct('PyMethodDef', - [('ml_name', c_char_p), - ('ml_meth', PyCFunction), - ('ml_flags', c_int), - ('ml_doc', c_char_p)]) - METH_VARARGS = ctypes_platform.ConstantInteger('METH_VARARGS') +## PyMethodDef = ctypes_platform.Struct('PyMethodDef', +## [('ml_name', c_char_p), +## ('ml_meth', PyCFunction), +## ('ml_flags', c_int), +## ('ml_doc', c_char_p)]) +## METH_VARARGS = ctypes_platform.ConstantInteger('METH_VARARGS') globals().update(ctypes_platform.configure(CConfig)) del CConfig @@ -171,6 +171,6 @@ #PyArg_ParseTupleAndKeywords.argtypes = [W_Object, W_Object, # c_char_p, POINTER(c_char_p), ...] -PyCFunction_NewEx = pythonapi.PyCFunction_NewEx -PyCFunction_NewEx.argtypes = [POINTER(PyMethodDef), W_Object, W_Object] -PyCFunction_NewEx.restype = W_Object +##PyCFunction_NewEx = pythonapi.PyCFunction_NewEx +##PyCFunction_NewEx.argtypes = [POINTER(PyMethodDef), W_Object, W_Object] +##PyCFunction_NewEx.restype = W_Object Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Tue Apr 25 13:19:59 2006 @@ -4,7 +4,6 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.objspace.cpy.ann_policy import CPyAnnotatorPolicy from pypy.objspace.cpy.objspace import CPyObjSpace -from pypy.objspace.cpy import wrappable, capi import pypy.rpython.rctypes.implementation from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root @@ -43,6 +42,7 @@ return entrypoint def test_annotate_bltinfunc(): + py.test.skip("in-progress") entrypoint = maketest() t = TranslationContext() a = t.buildannotator(policy=CPyAnnotatorPolicy()) @@ -56,12 +56,6 @@ assert s.knowntype == CPyObjSpace.W_Object s = a.binding(graph.getreturnvar()) assert s.knowntype == CPyObjSpace.W_Object - graph = graphof(t, wrappable.builtin_function_builder) - assert len(graph.getargs()) == 2 - s = a.binding(graph.getargs()[0]) - assert s.knowntype == capi.PyMethodDef - s = a.binding(graph.getreturnvar()) - assert s.knowntype == CPyObjSpace.W_Object def test_compile_bltinfunc(): py.test.skip("in-progress") Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Tue Apr 25 13:19:59 2006 @@ -9,14 +9,7 @@ from pypy.objspace.cpy.objspace import CPyObjSpace from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root - - -def builtin_function_builder(ml, w_module): - """Build and initialize a real CPython built-in function object. - Compiled into the extension module and called when it is - first imported. - """ - return PyCFunction_NewEx(byref(ml), None, w_module) +from pypy.rpython import extregistry class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): @@ -30,31 +23,32 @@ assert unwrap_spec == [ObjSpace, W_Root] # XXX for now - # argh! callbacks of mode PyDLL are not supported by ctypes so far - # (as of 0.9.9.4). XXX hack. I am not happy. - - # as a workaround, we use an interface that lets us return a normal - # Python function used for testing, with an attached 'builder' - # function that is executed at start-up to construct the object - - def trampoline(*args): - # only called during testing; not compiled - args_w = [space.W_Object(a) for a in args] - w_result = bltin(space, *args_w) + def trampoline(a): + w_arg = space.W_Object(a) + w_result = bltin(space, w_arg) return w_result.value - def callback(w_self, w_args): - # only called when compiled into the extension module - w_a = PyObject_GetItem(w_args, 0) - w_result = bltin(space, w_a) - return w_result - - ml = PyMethodDef(ml_name = factory.b_name, - ml_meth = PyCFunction(callback), - ml_flags = METH_VARARGS, - #ml_doc = ..., - ) - w_result = W_Object(trampoline) - w_result.builder = builtin_function_builder, (ml, func.w_module) + + # override the annotation behavior of 'w_result' + # to emulate a call to the trampoline() function at interp-level + BaseEntry = extregistry._lookup_cls(w_result) + uniquekey = trampoline + nb_args = len(unwrap_spec) - 1 + + class TrampolineEntry(BaseEntry): + _about_ = w_result + + def compute_annotation(self): + from pypy.annotation import model as annmodel + from pypy.annotation.bookkeeper import getbookkeeper + + bookkeeper = getbookkeeper() + if bookkeeper is not None: # else probably already rtyping + s_trampoline = bookkeeper.immutablevalue(trampoline) + args_s = [annmodel.SomeObject()]*nb_args + bookkeeper.emulate_pbc_call(uniquekey, s_trampoline, + args_s) + return super(TrampolineEntry, self).compute_annotation() + return w_result Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Tue Apr 25 13:19:59 2006 @@ -33,32 +33,32 @@ s_result = SomeCTypesObject(result_ctype, SomeCTypesObject.OWNSMEMORY) return s_result.return_annotation() - def object_seen(self, bookkeeper): - "Called when the annotator sees this ctypes function object." - # if the function is a Python callback, emulate a call to it - # so that the callback is properly annotated - if hasattr(self.instance, 'callback'): - callback = self.instance.callback - argtypes = self.instance.argtypes - restype = self.instance.restype - s_callback = bookkeeper.immutablevalue(callback) - # the input arg annotations, which are automatically unwrapped - args_s = [bookkeeper.valueoftype(ctype).return_annotation() - for ctype in argtypes] - uniquekey = (callback, argtypes, restype) - s_res = bookkeeper.emulate_pbc_call(uniquekey, s_callback, args_s) - # check the result type - if restype is None: - s_expected = annmodel.s_None - else: - s_expected = bookkeeper.valueoftype(restype) - # can also return the unwrapped version of the ctype, - # e.g. an int instead of a c_int - s_orelse = s_expected.return_annotation() - assert s_expected.contains(s_res) or s_orelse.contains(s_res), ( - "%r should return a %s but returned %s" % (callback, - restype, - s_res)) +## def object_seen(self, bookkeeper): +## "Called when the annotator sees this ctypes function object." +## # if the function is a Python callback, emulate a call to it +## # so that the callback is properly annotated +## if hasattr(self.instance, 'callback'): +## callback = self.instance.callback +## argtypes = self.instance.argtypes +## restype = self.instance.restype +## s_callback = bookkeeper.immutablevalue(callback) +## # the input arg annotations, which are automatically unwrapped +## args_s = [bookkeeper.valueoftype(ctype).return_annotation() +## for ctype in argtypes] +## uniquekey = (callback, argtypes, restype) +## s_res = bookkeeper.emulate_pbc_call(uniquekey, s_callback, args_s) +## # check the result type +## if restype is None: +## s_expected = annmodel.s_None +## else: +## s_expected = bookkeeper.valueoftype(restype) +## # can also return the unwrapped version of the ctype, +## # e.g. an int instead of a c_int +## s_orelse = s_expected.return_annotation() +## assert s_expected.contains(s_res) or s_orelse.contains(s_res), ( +## "%r should return a %s but returned %s" % (callback, +## restype, +## s_res)) def specialize_call(self, hop): from pypy.rpython.rctypes.rmodel import CTypesValueRepr Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Tue Apr 25 13:19:59 2006 @@ -23,21 +23,21 @@ "Annotation and rtyping of py_object instances." _type_ = py_object - def object_seen(self, bookkeeper): - "Called when the annotator sees this py_object." - # extension: if the py_object instance has a 'builder' attribute, - # it must be a pair (callable, args) which is meant to be called - # at initialization-time when the compiled extension module is - # first imported. It returns the "real" Python object. - if hasattr(self.instance, 'builder'): - # emulate a call so that the callable is properly annotated - callable, args = self.instance.builder - s_callable = bookkeeper.immutablevalue(callable) - args_s = [bookkeeper.immutablevalue(a) for a in args] - uniquekey = Hashable(self.instance) - s_res = bookkeeper.emulate_pbc_call(uniquekey, s_callable, args_s) - assert (issubclass(s_res.knowntype, py_object) or - isinstance(s_res, SomeImpossibleValue)) +## def object_seen(self, bookkeeper): +## "Called when the annotator sees this py_object." +## # extension: if the py_object instance has a 'builder' attribute, +## # it must be a pair (callable, args) which is meant to be called +## # at initialization-time when the compiled extension module is +## # first imported. It returns the "real" Python object. +## if hasattr(self.instance, 'builder'): +## # emulate a call so that the callable is properly annotated +## callable, args = self.instance.builder +## s_callable = bookkeeper.immutablevalue(callable) +## args_s = [bookkeeper.immutablevalue(a) for a in args] +## uniquekey = Hashable(self.instance) +## s_res = bookkeeper.emulate_pbc_call(uniquekey, s_callable, args_s) +## assert (issubclass(s_res.knowntype, py_object) or +## isinstance(s_res, SomeImpossibleValue)) def get_repr(self, rtyper, s_pyobject): from pypy.rpython.rctypes.rpyobject import CTypesPyObjRepr Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Apr 25 13:19:59 2006 @@ -2,7 +2,6 @@ from pypy.annotation.model import SomeCTypesObject from pypy.rpython import extregistry from pypy.rpython.extregistry import ExtRegistryEntry -import ctypes # rctypes version of ctypes.CFUNCTYPE. # It's required to work around three limitations of CFUNCTYPE: @@ -20,76 +19,77 @@ # not allowed annotation-wise -- we change the semantics here under # the nose of the annotator. -_c_callback_functype_cache = {} -def CALLBACK_FUNCTYPE(restype, *argtypes, **flags): - if 'callconv' in flags: - callconv = flags.pop('callconv') - else: - callconv = ctypes.CDLL - assert not flags, "unknown keyword arguments %r" % (flags.keys(),) - try: - return _c_callback_functype_cache[(restype, argtypes)] - except KeyError: - class CallbackFunctionType(ctypes._CFuncPtr): - _argtypes_ = argtypes - _restype_ = restype - _flags_ = callconv._FuncPtr._flags_ - - def __new__(cls, callback): - assert callable(callback) - if issubclass(restype, ctypes.py_object): - def func(*args, **kwds): - w_res = callback(*args, **kwds) - assert isinstance(w_res, py_object) - return w_res.value - else: - func = callback - res = super(CallbackFunctionType, cls).__new__(cls, func) - res.callback = callback - return res +##_c_callback_functype_cache = {} +##def CALLBACK_FUNCTYPE(restype, *argtypes, **flags): +## if 'callconv' in flags: +## callconv = flags.pop('callconv') +## else: +## callconv = ctypes.CDLL +## assert not flags, "unknown keyword arguments %r" % (flags.keys(),) +## try: +## return _c_callback_functype_cache[(restype, argtypes)] +## except KeyError: +## class CallbackFunctionType(ctypes._CFuncPtr): +## _argtypes_ = argtypes +## _restype_ = restype +## _flags_ = callconv._FuncPtr._flags_ + +## def __new__(cls, callback): +## assert callable(callback) +## if issubclass(restype, ctypes.py_object): +## def func(*args, **kwds): +## w_res = callback(*args, **kwds) +## assert isinstance(w_res, py_object) +## return w_res.value +## else: +## func = callback +## res = super(CallbackFunctionType, cls).__new__(cls, func) +## res.callback = callback +## return res - _c_callback_functype_cache[(restype, argtypes)] = CallbackFunctionType - return CallbackFunctionType +## _c_callback_functype_cache[(restype, argtypes)] = CallbackFunctionType +## return CallbackFunctionType # ____________________________________________________________ class CTypesEntry(ExtRegistryEntry): + pass - def compute_annotation(self): - self.ctype_object_discovered() - return super(CTypesEntry, self).compute_annotation() - - def ctype_object_discovered(self): - if self.instance is None: - return - from pypy.annotation.bookkeeper import getbookkeeper - bookkeeper = getbookkeeper() - if bookkeeper is None: - return - # follow all dependent ctypes objects in order to discover - # all callback functions - memo = {} - def recfind(o): - if id(o) in memo: - return - memo[id(o)] = o - if isinstance(o, dict): - for x in o.itervalues(): - recfind(x) - elif isinstance(o, (list, tuple)): - for x in o: - recfind(x) - elif extregistry.is_registered(o): - entry = extregistry.lookup(o) - if isinstance(entry, CTypesEntry): - entry.object_seen(bookkeeper) - recfind(o._objects) - recfind(o.__dict__) # for extra keepalives - recfind(self.instance) - - def object_seen(self, bookkeeper): - """To be overriden for ctypes objects whose mere presence influences - annotation, e.g. callback functions.""" +## def compute_annotation(self): +## self.ctype_object_discovered() +## return super(CTypesEntry, self).compute_annotation() + +## def ctype_object_discovered(self): +## if self.instance is None: +## return +## from pypy.annotation.bookkeeper import getbookkeeper +## bookkeeper = getbookkeeper() +## if bookkeeper is None: +## return +## # follow all dependent ctypes objects in order to discover +## # all callback functions +## memo = {} +## def recfind(o): +## if id(o) in memo: +## return +## memo[id(o)] = o +## if isinstance(o, dict): +## for x in o.itervalues(): +## recfind(x) +## elif isinstance(o, (list, tuple)): +## for x in o: +## recfind(x) +## elif extregistry.is_registered(o): +## entry = extregistry.lookup(o) +## if isinstance(entry, CTypesEntry): +## entry.object_seen(bookkeeper) +## recfind(o._objects) +## recfind(o.__dict__) # for extra keepalives +## recfind(self.instance) + +## def object_seen(self, bookkeeper): +## """To be overriden for ctypes objects whose mere presence influences +## annotation, e.g. callback functions.""" class CTypesCallEntry(CTypesEntry): @@ -103,7 +103,7 @@ "Annotation and rtyping of ctypes instances." def compute_annotation(self): - self.ctype_object_discovered() + #self.ctype_object_discovered() ctype = self.type return SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Tue Apr 25 13:19:59 2006 @@ -4,7 +4,7 @@ import py import sys -from pypy.rpython.rctypes.implementation import CALLBACK_FUNCTYPE +import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.test.test_llinterp import interpret @@ -55,13 +55,13 @@ ctime.restype = c_char_p #ctimes.argtypes: omitted for this test -IntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int) -def mycallback(n): - return n+3 -callback = IntIntCallback(mycallback) +##IntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int) +##def mycallback(n): +## return n+3 +##callback = IntIntCallback(mycallback) -PyIntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int, callconv=PyDLL) -pycallback = PyIntIntCallback(mycallback) +##PyIntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int, callconv=PyDLL) +##pycallback = PyIntIntCallback(mycallback) def test_labs(n=6): @@ -106,9 +106,9 @@ s2 = ctime(byref(c_long(N))) assert s1.strip() == s2.strip() -def test_callback(): - assert callback(100) == 103 - assert pycallback(100) == 103 +##def test_callback(): +## assert callback(100) == 103 +## assert pycallback(100) == 103 class Test_annotation: def test_annotate_labs(self): @@ -145,40 +145,40 @@ if conftest.option.view: a.translator.view() - def test_annotate_callback(self): - def fn(n): - return callback(n) - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(fn, [int]) - assert s.knowntype == int - if conftest.option.view: - a.translator.view() - graph = graphof(t, mycallback) - [v1] = graph.getargs() - v2 = graph.getreturnvar() - assert a.binding(v1).knowntype == int - assert a.binding(v2).knowntype == int - - def test_annotation_indirectly_found_callback(self): - class S(Structure): - _fields_ = [('vtable', IntIntCallback*5), - ('z', c_int)] - s = S(z=3) - s.vtable[2] = callback - def fn(): - return s.z - t = TranslationContext() - a = t.buildannotator() - s = a.build_types(fn, []) - assert s.knowntype == int - if conftest.option.view: - a.translator.view() - graph = graphof(t, mycallback) - [v1] = graph.getargs() - v2 = graph.getreturnvar() - assert a.binding(v1).knowntype == int - assert a.binding(v2).knowntype == int +## def test_annotate_callback(self): +## def fn(n): +## return callback(n) +## t = TranslationContext() +## a = t.buildannotator() +## s = a.build_types(fn, [int]) +## assert s.knowntype == int +## if conftest.option.view: +## a.translator.view() +## graph = graphof(t, mycallback) +## [v1] = graph.getargs() +## v2 = graph.getreturnvar() +## assert a.binding(v1).knowntype == int +## assert a.binding(v2).knowntype == int + +## def test_annotation_indirectly_found_callback(self): +## class S(Structure): +## _fields_ = [('vtable', IntIntCallback*5), +## ('z', c_int)] +## s = S(z=3) +## s.vtable[2] = callback +## def fn(): +## return s.z +## t = TranslationContext() +## a = t.buildannotator() +## s = a.build_types(fn, []) +## assert s.knowntype == int +## if conftest.option.view: +## a.translator.view() +## graph = graphof(t, mycallback) +## [v1] = graph.getargs() +## v2 = graph.getreturnvar() +## assert a.binding(v1).knowntype == int +## assert a.binding(v2).knowntype == int class Test_specialization: def test_specialize_labs(self): From arigo at codespeak.net Tue Apr 25 13:22:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 13:22:16 +0200 (CEST) Subject: [pypy-svn] r26325 - in pypy/dist/pypy: objspace/cpy/test rpython/rctypes Message-ID: <20060425112216.AEBC61008A@code0.codespeak.net> Author: arigo Date: Tue Apr 25 13:22:15 2006 New Revision: 26325 Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/rpython/rctypes/apyobject.py Log: Annotate the trampoline() function with SomeObjects. Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Tue Apr 25 13:22:15 2006 @@ -42,7 +42,6 @@ return entrypoint def test_annotate_bltinfunc(): - py.test.skip("in-progress") entrypoint = maketest() t = TranslationContext() a = t.buildannotator(policy=CPyAnnotatorPolicy()) Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Tue Apr 25 13:22:15 2006 @@ -1,5 +1,5 @@ from ctypes import py_object -from pypy.annotation.model import SomeCTypesObject, SomeImpossibleValue +from pypy.annotation.model import SomeCTypesObject, SomeObject from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.rpython.lltypesystem import lltype from pypy.tool.uid import Hashable @@ -23,6 +23,13 @@ "Annotation and rtyping of py_object instances." _type_ = py_object + def get_field_annotation(self, s_pyobject, fieldname): + assert fieldname == "value" + # reading the .value field results in an object of + # completely unknown type. This crashes the annotator if + # it is not in allow_someobjects mode. + return SomeObject() + ## def object_seen(self, bookkeeper): ## "Called when the annotator sees this py_object." ## # extension: if the py_object instance has a 'builder' attribute, From arigo at codespeak.net Tue Apr 25 13:26:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 13:26:18 +0200 (CEST) Subject: [pypy-svn] r26326 - in pypy/dist/pypy: objspace/cpy/test rpython/rctypes Message-ID: <20060425112618.DBF6A1008A@code0.codespeak.net> Author: arigo Date: Tue Apr 25 13:26:17 2006 New Revision: 26326 Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/rpython/rctypes/rpyobject.py Log: Actually, genc is happy with this right out of the box. Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Tue Apr 25 13:26:17 2006 @@ -57,7 +57,6 @@ assert s.knowntype == CPyObjSpace.W_Object def test_compile_bltinfunc(): - py.test.skip("in-progress") entrypoint = maketest() fn = compile(entrypoint, [int], annotatorpolicy = CPyAnnotatorPolicy()) Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Tue Apr 25 13:26:17 2006 @@ -12,7 +12,13 @@ value = value.value p.c_data[0] = lltype.pyobjectptr(value) - # reading .value is not allowed, as it can't be annotated! + def rtype_getattr(self, hop): + # only for 'allow_someobjects' annotations + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + v_pyobj = hop.inputarg(self, 0) + return self.getvalue(hop.llops, v_pyobj) def rtype_setattr(self, hop): s_attr = hop.args_s[1] From arigo at codespeak.net Tue Apr 25 13:51:29 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 13:51:29 +0200 (CEST) Subject: [pypy-svn] r26327 - in pypy/dist/pypy: annotation objspace/cpy Message-ID: <20060425115129.487351007E@code0.codespeak.net> Author: arigo Date: Tue Apr 25 13:51:27 2006 New Revision: 26327 Modified: pypy/dist/pypy/annotation/annrpython.py pypy/dist/pypy/objspace/cpy/ann_policy.py pypy/dist/pypy/objspace/cpy/wrappable.py Log: Disallow SomeObjects again, with an exception: they are allowed inside functions with an attribute 'allow_someobjects=True'. Modified: pypy/dist/pypy/annotation/annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/annrpython.py (original) +++ pypy/dist/pypy/annotation/annrpython.py Tue Apr 25 13:51:27 2006 @@ -243,12 +243,19 @@ def ondegenerated(self, what, s_value, where=None, called_from_graph=None): if self.policy.allow_someobjects: return + + # is the function itself tagged with allow_someobjects? + position_key = where or getattr(self.bookkeeper, 'position_key', None) + if position_key is not None: + graph, block, i = position_key + try: + if graph.func.allow_someobjects: + return + except AttributeError: + pass + msglines = ["annotation of %r degenerated to SomeObject()" % (what,)] - try: - position_key = where or self.bookkeeper.position_key - except AttributeError: - pass - else: + if position_key is not None: msglines.append(".. position: %s" % (self.whereami(position_key),)) if called_from_graph is not None: msglines.append(".. called from %r" % (called_from_graph,)) @@ -427,10 +434,10 @@ assert block in self.annotated self.annotated[block] = False # must re-flow - def bindinputargs(self, graph, block, inputcells, - called_from_graph=None, where=None): + def bindinputargs(self, graph, block, inputcells, called_from_graph=None): # Create the initial bindings for the input args of a block. assert len(block.inputargs) == len(inputcells) + where = (graph, block, None) for a, cell in zip(block.inputargs, inputcells): self.setbinding(a, cell, called_from_graph, where=where) self.annotated[block] = False # must flowin. @@ -442,8 +449,7 @@ unions = [annmodel.unionof(c1,c2) for c1, c2 in zip(oldcells,inputcells)] # if the merged cells changed, we must redo the analysis if unions != oldcells: - self.bindinputargs(graph, block, unions, - called_from_graph, where=(graph, block, None)) + self.bindinputargs(graph, block, unions, called_from_graph) def whereami(self, position_key): graph, block, i = position_key Modified: pypy/dist/pypy/objspace/cpy/ann_policy.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/ann_policy.py (original) +++ pypy/dist/pypy/objspace/cpy/ann_policy.py Tue Apr 25 13:51:27 2006 @@ -1,8 +1,6 @@ from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy class CPyAnnotatorPolicy(PyPyAnnotatorPolicy): - allow_someobjects = True - - # XXX make it more subtle: only allow SomeObjects in - # some specific functions, not all of them. - # Currently only trampoline() in wrapper.py should need them. + """Annotation policy to compile CPython extension modules with + the CPyObjSpace. + """ Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Tue Apr 25 13:51:27 2006 @@ -27,6 +27,7 @@ w_arg = space.W_Object(a) w_result = bltin(space, w_arg) return w_result.value + trampoline.allow_someobjects = True # annotator hint w_result = W_Object(trampoline) From sanxiyn at codespeak.net Tue Apr 25 13:58:58 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Tue, 25 Apr 2006 13:58:58 +0200 (CEST) Subject: [pypy-svn] r26328 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060425115858.3590110084@code0.codespeak.net> Author: sanxiyn Date: Tue Apr 25 13:58:53 2006 New Revision: 26328 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Some corrections to the sprint report Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Tue Apr 25 13:58:53 2006 @@ -59,7 +59,7 @@ - gensqueak/ootypesystem (Nik) * introduction done - * steps towards making genclisp use current infrastructure + * steps towards making gencl use current infrastructure - Implementing Python 2.5 features in PyPy. (Yusei) @@ -95,6 +95,7 @@ * bit running around and working with Nik in the afternoon on ootype + Tuesday, 25th April 2006 ************************ @@ -127,9 +128,7 @@ - reorganize the new socket module in the right project location (Arre, Anders) - - gencl - * compile pystone in gencl - (Valentino, Sanghyeon) + - compile pystone in gencl (Valentino, Sanghyeon) - Coneptual design decisions about ootypesystem (tuples list...) (Samuele, Nik) From arigo at codespeak.net Tue Apr 25 14:20:15 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 14:20:15 +0200 (CEST) Subject: [pypy-svn] r26330 - pypy/dist/pypy/interpreter Message-ID: <20060425122015.2442110087@code0.codespeak.net> Author: arigo Date: Tue Apr 25 14:20:14 2006 New Revision: 26330 Modified: pypy/dist/pypy/interpreter/gateway.py Log: Refactored UnwrapSpecRecipe: every kind of action is in its own subclass instead of having method families on the same class. This way it can be reused from objspace/cpy/wrappable.py too. Modified: pypy/dist/pypy/interpreter/gateway.py ============================================================================== --- pypy/dist/pypy/interpreter/gateway.py (original) +++ pypy/dist/pypy/interpreter/gateway.py Tue Apr 25 14:20:14 2006 @@ -59,33 +59,31 @@ bases_order = [Wrappable, W_Root, ObjSpace, Arguments, object] - def dispatch(self, meth_family, el, orig_sig, new_sig): + def dispatch(self, el, *args): if isinstance(el, str): - getattr(self, "%s_%s" % (meth_family, el))(el, orig_sig, new_sig) + getattr(self, "visit_%s" % (el,))(el, *args) elif isinstance(el, tuple): - getattr(self, "%s_%s" % (meth_family, 'function'))(el, orig_sig, new_sig) + self.visit_function(el, *args) else: for typ in self.bases_order: if issubclass(el, typ): - getattr(self, "%s__%s" % (meth_family, typ.__name__))(el, orig_sig, new_sig) + visit = getattr(self, "visit__%s" % (typ.__name__,)) + visit(el, *args) break else: - assert False, "no match for unwrap_spec element: %s" % el + raise Exception("%s: no match for unwrap_spec element %s" % ( + self.__class__.__name__, el)) - def check(self, el, orig_sig, new_sig): - self.dispatch("check", el, orig_sig, new_sig) - - def emit(self, el, orig_sig, new_sig): - self.dispatch("emit", el, orig_sig, new_sig) +class UnwrapSpec_Check(UnwrapSpecRecipe): # checks for checking interp2app func argument names wrt unwrap_spec # and synthetizing an app-level signature - def check_function(self, (func, cls), orig_sig, app_sig): - self.check(cls, orig_sig, app_sig) + def visit_function(self, (func, cls), orig_sig, app_sig): + self.dispatch(cls, orig_sig, app_sig) - def check__Wrappable(self, el, orig_sig, app_sig): + def visit__Wrappable(self, el, orig_sig, app_sig): name = el.__name__ argname = orig_sig.next_arg() assert not argname.startswith('w_'), ( @@ -93,10 +91,10 @@ "not start with 'w_'" % (name, argname, orig_sig.func)) app_sig.append(argname) - def check__ObjSpace(self, el, orig_sig, app_sig): + def visit__ObjSpace(self, el, orig_sig, app_sig): orig_sig.next_arg() - def check__W_Root(self, el, orig_sig, app_sig): + def visit__W_Root(self, el, orig_sig, app_sig): assert el is W_Root, "oops" argname = orig_sig.next_arg() assert argname.startswith('w_'), ( @@ -104,14 +102,14 @@ "start with 'w_'" % (argname, orig_sig.func)) app_sig.append(argname[2:]) - def check__Arguments(self, el, orig_sig, app_sig): + def visit__Arguments(self, el, orig_sig, app_sig): argname = orig_sig.next_arg() assert app_sig.varargname is None,( "built-in function %r has conflicting rest args specs" % orig_sig.func) app_sig.varargname = 'args' app_sig.kwargname = 'keywords' - def check_starargs(self, el, orig_sig, app_sig): + def visit_starargs(self, el, orig_sig, app_sig): varargname = orig_sig.varargname assert varargname.endswith('_w'), ( "argument *%s of built-in function %r should end in '_w'" % @@ -120,7 +118,7 @@ "built-in function %r has conflicting rest args specs" % orig_sig.func) app_sig.varargname = varargname[:-2] - def check_args_w(self, el, orig_sig, app_sig): + def visit_args_w(self, el, orig_sig, app_sig): argname = orig_sig.next_arg() assert argname.endswith('_w'), ( "rest arguments arg %s of built-in function %r should end in '_w'" % @@ -129,7 +127,7 @@ "built-in function %r has conflicting rest args specs" % orig_sig.func) app_sig.varargname = argname[:-2] - def check_w_args(self, el, orig_sig, app_sig): + def visit_w_args(self, el, orig_sig, app_sig): argname = orig_sig.next_arg() assert argname.startswith('w_'), ( "rest arguments arg %s of built-in function %r should start 'w_'" % @@ -138,7 +136,7 @@ "built-in function %r has conflicting rest args specs" % orig_sig.func) app_sig.varargname = argname[2:] - def check__object(self, el, orig_sig, app_sig): + def visit__object(self, el, orig_sig, app_sig): if el not in (int, str, float): assert False, "unsupported basic type in unwrap_spec" name = el.__name__ @@ -148,9 +146,12 @@ "not start with 'w_'" % (name, argname, orig_sig.func)) app_sig.append(argname) + +class UnwrapSpec_Emit(UnwrapSpecRecipe): + # collect code to emit for interp2app builtin frames based on unwrap_spec - def emit_function(self, (func, cls), orig_sig, emit_sig): + def visit_function(self, (func, cls), orig_sig, emit_sig): name = func.__name__ cur = emit_sig.through_scope_w emit_sig.setfastscope.append( @@ -161,7 +162,7 @@ emit_sig.through_scope_w += 1 emit_sig.run_args.append("self.%s_arg%d" % (name,cur)) - def emit__Wrappable(self, el, orig_sig, emit_sig): + def visit__Wrappable(self, el, orig_sig, emit_sig): name = el.__name__ cur = emit_sig.through_scope_w emit_sig.setfastscope.append( @@ -172,17 +173,17 @@ emit_sig.through_scope_w += 1 emit_sig.run_args.append("self.%s_arg%d" % (name,cur)) - def emit__ObjSpace(self, el, orig_sig, emit_sig): + def visit__ObjSpace(self, el, orig_sig, emit_sig): emit_sig.run_args.append('self.space') - def emit__W_Root(self, el, orig_sig, emit_sig): + def visit__W_Root(self, el, orig_sig, emit_sig): cur = emit_sig.through_scope_w emit_sig.setfastscope.append( "self.w_arg%d = scope_w[%d]" % (cur,cur)) emit_sig.through_scope_w += 1 emit_sig.run_args.append("self.w_arg%d" % cur) - def emit__Arguments(self, el, orig_sig, emit_sig): + def visit__Arguments(self, el, orig_sig, emit_sig): cur = emit_sig.through_scope_w emit_sig.through_scope_w += 2 emit_sig.miniglobals['Arguments'] = Arguments @@ -192,28 +193,28 @@ % (cur, cur+1)) emit_sig.run_args.append("self.arguments_arg") - def emit_starargs(self, el, orig_sig, emit_sig): + def visit_starargs(self, el, orig_sig, emit_sig): emit_sig.setfastscope.append( "self.starargs_arg_w = self.space.unpacktuple(scope_w[%d])" % (emit_sig.through_scope_w)) emit_sig.through_scope_w += 1 emit_sig.run_args.append("*self.starargs_arg_w") - def emit_args_w(self, el, orig_sig, emit_sig): + def visit_args_w(self, el, orig_sig, emit_sig): emit_sig.setfastscope.append( "self.args_w = self.space.unpacktuple(scope_w[%d])" % (emit_sig.through_scope_w)) emit_sig.through_scope_w += 1 emit_sig.run_args.append("self.args_w") - def emit_w_args(self, el, orig_sig, emit_sig): + def visit_w_args(self, el, orig_sig, emit_sig): cur = emit_sig.through_scope_w emit_sig.setfastscope.append( "self.w_args = scope_w[%d]" % cur) emit_sig.through_scope_w += 1 emit_sig.run_args.append("self.w_args") - def emit__object(self, el, orig_sig, emit_sig): + def visit__object(self, el, orig_sig, emit_sig): if el not in (int, str, float): assert False, "unsupported basic type in uwnrap_spec" name = el.__name__ @@ -224,44 +225,42 @@ emit_sig.through_scope_w += 1 emit_sig.run_args.append("self.%s_arg%d" % (name,cur)) - # unwrapping code for fastfunc argument handling - def fastfunc_unwrap(self, el, info): - self.dispatch("fastfunc_unwrap", el, None, info) +class UnwrapSpec_FastFunc_Unwrap(UnwrapSpecRecipe): - def fastfunc_unwrap_function(self, (func, cls), ignore, info): + def visit_function(self, (func, cls), info): raise FastFuncNotSupported - def fastfunc_unwrap__Wrappable(self, el, ignore, info): + def visit__Wrappable(self, el, info): name = el.__name__ cur = info.narg info.unwrap.append("space.interp_w(%s, w%d)" % (name, cur)) info.miniglobals[name] = el info.narg += 1 - def fastfunc_unwrap__ObjSpace(self, el, ignore, info): + def visit__ObjSpace(self, el, info): if info.index != 0: raise FastFuncNotSupported info.unwrap.append("space") - def fastfunc_unwrap__W_Root(self, el, ignore, info): + def visit__W_Root(self, el, info): cur = info.narg info.unwrap.append("w%d" % cur) info.narg += 1 - def fastfunc_unwrap__Arguments(self, el, ignore, info): + def visit__Arguments(self, el, info): raise FastFuncNotSupported - def fastfunc_unwrap_starargs(self, el, ignore, info): + def visit_starargs(self, el, info): raise FastFuncNotSupported - def fastfunc_unwrap_args_w(self, el, ignore, info): + def visit_args_w(self, el, info): raise FastFuncNotSupported - def fastfunc_unwrap_w_args(self, el, ignore, info): + def visit_w_args(self, el, info): raise FastFuncNotSupported - def fastfunc_unwrap__object(self, el, ignore, info): + def visit__object(self, el, info): if el not in (int, str, float): assert False, "unsupported basic type in uwnrap_spec" name = el.__name__ @@ -400,7 +399,7 @@ def make_builtin_frame_factory(func, orig_sig, unwrap_spec): "NOT_RPYTHON" name = (getattr(func, '__module__', None) or '')+'_'+func.__name__ - emit_sig = orig_sig.apply_unwrap_spec(unwrap_spec, UnwrapSpecRecipe().emit, + emit_sig = orig_sig.apply_unwrap_spec(unwrap_spec, UnwrapSpec_Emit().dispatch, BuiltinCodeSignature(name=name, unwrap_spec=unwrap_spec)) return emit_sig.make_frame_factory(func) @@ -416,7 +415,7 @@ def make_fastfunc(func, unwrap_spec): info = FastFuncInfo() - recipe = UnwrapSpecRecipe().fastfunc_unwrap + recipe = UnwrapSpec_FastFunc_Unwrap().dispatch for el in unwrap_spec: recipe(el, info) info.index += 1 @@ -490,7 +489,7 @@ orig_sig = Signature(func, argnames, varargname, kwargname) - app_sig = orig_sig.apply_unwrap_spec(unwrap_spec, UnwrapSpecRecipe().check, + app_sig = orig_sig.apply_unwrap_spec(unwrap_spec, UnwrapSpec_Check().dispatch, Signature(func)) self.sig = argnames, varargname, kwargname = app_sig.signature() From cfbolz at codespeak.net Tue Apr 25 14:24:00 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 25 Apr 2006 14:24:00 +0200 (CEST) Subject: [pypy-svn] r26331 - in pypy/dist/pypy: module/_socket translator/c Message-ID: <20060425122400.A904C1007D@code0.codespeak.net> Author: cfbolz Date: Tue Apr 25 14:23:59 2006 New Revision: 26331 Added: pypy/dist/pypy/module/_socket/conftest.py - copied unchanged from r26303, pypy/dist/pypy/rpython/rctypes/test/conftest.py Modified: pypy/dist/pypy/module/_socket/__init__.py pypy/dist/pypy/translator/c/extfunc.py Log: remove direct import of ctypes to be able to properly skip tests if it is not there Modified: pypy/dist/pypy/module/_socket/__init__.py ============================================================================== --- pypy/dist/pypy/module/_socket/__init__.py (original) +++ pypy/dist/pypy/module/_socket/__init__.py Tue Apr 25 14:23:59 2006 @@ -1,6 +1,5 @@ # Package initialisation from pypy.interpreter.mixedmodule import MixedModule -from pypy.rpython.rctypes.socketmodule import ctypes_socket as _c import sys class Module(MixedModule): @@ -17,18 +16,21 @@ 'socket' : 'interp_socket.getsockettype(space)', } -for name in """ - gethostbyname_ex gethostbyaddr gethostname - getservbyname getservbyport getprotobyname - fromfd socketpair - ntohs ntohl htons htonl inet_aton inet_ntoa inet_pton inet_ntop - getaddrinfo getnameinfo - getdefaulttimeout setdefaulttimeout - """.split(): - - Module.interpleveldefs[name] = 'interp_socket.%s' % (name, ) + def buildloaders(cls): + from pypy.rpython.rctypes.socketmodule import ctypes_socket as _c + for name in """ + gethostbyname_ex gethostbyaddr gethostname + getservbyname getservbyport getprotobyname + fromfd socketpair + ntohs ntohl htons htonl inet_aton inet_ntoa inet_pton inet_ntop + getaddrinfo getnameinfo + getdefaulttimeout setdefaulttimeout + """.split(): + + Module.interpleveldefs[name] = 'interp_socket.%s' % (name, ) -for constant, value in _c.constants.iteritems(): - Module.interpleveldefs[constant] = "space.wrap(%r)" % value + for constant, value in _c.constants.iteritems(): + Module.interpleveldefs[constant] = "space.wrap(%r)" % value + buildloaders = classmethod(buildloaders) #Module.interpleveldefs['has_ipv6'] = "space.wrap(%s)" % _socket.has_ipv6 Modified: pypy/dist/pypy/translator/c/extfunc.py ============================================================================== --- pypy/dist/pypy/translator/c/extfunc.py (original) +++ pypy/dist/pypy/translator/c/extfunc.py Tue Apr 25 14:23:59 2006 @@ -8,7 +8,6 @@ from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack from pypy.module.thread.rpython import ll_thread -from pypy.module._socket.rpython import ll__socket # table of functions hand-written in src/ll_*.h EXTERNALS = { @@ -59,18 +58,6 @@ ll_stackless.ll_stackless_stack_frames_depth: 'LL_stackless_stack_frames_depth', ll_stack.ll_stack_unwind: 'LL_stack_unwind', ll_stack.ll_stack_too_big: 'LL_stack_too_big', - ll__socket.ll__socket_gethostname: 'LL__socket_gethostname', - ll__socket.ll__socket_gethostbyname: 'LL__socket_gethostbyname', - ll__socket.ll__socket_getaddrinfo: 'LL__socket_getaddrinfo', - ll__socket.ll__socket_nextaddrinfo: 'LL__socket_nextaddrinfo', - ll__socket.ll__socket_freeaddrinfo: 'LL__socket_freeaddrinfo', - ll__socket.ll__socket_ntohs: 'LL__socket_ntohs', - ll__socket.ll__socket_htons: 'LL__socket_htons', - ll__socket.ll__socket_htonl: 'LL__socket_htonl', - ll__socket.ll__socket_ntohl: 'LL__socket_htonl', - ll__socket.ll__socket_newsocket: 'LL__socket_newsocket', - ll__socket.ll__socket_connect: 'LL__socket_connect', - ll__socket.ll__socket_getpeername: 'LL__socket_getpeername', } #______________________________________________________ @@ -101,8 +88,6 @@ yield ('RPyFREXP_RESULT', ll_math.FREXP_RESULT) yield ('RPyMODF_RESULT', ll_math.MODF_RESULT) yield ('RPySTAT_RESULT', ll_os.STAT_RESULT) - yield ('RPySOCKET_ADDRINFO', ll__socket.ADDRINFO_RESULT) - yield ('RPySOCKET_SOCKNAME', ll__socket.SOCKNAME) def predeclare_utility_functions(db, rtyper, optimize=True): # Common utility functions @@ -164,17 +149,6 @@ not optimize): r.append(annotate(ll_os.ll_stat_result, *([lltype.Signed] * 10))) - if (ll__socket.ll__socket_nextaddrinfo in db.externalfuncs or - not optimize): - args = [lltype.Signed, lltype.Signed, lltype.Signed, lltype.Ptr(STR), - lltype.Ptr(STR), lltype.Signed, lltype.Signed, lltype.Signed] - r.append(annotate(ll__socket.ll__socket_addrinfo, *args)) - - if (ll__socket.ll__socket_getpeername in db.externalfuncs or - not optimize): - args = [lltype.Ptr(STR), lltype.Signed, lltype.Signed, lltype.Signed] - r.append(annotate(ll__socket.ll__socket_sockname, *args)) - return r def predeclare_extfunc_helpers(db, rtyper, optimize=True): @@ -195,16 +169,6 @@ yield decl(ll_os.ll_stat_result) yield ('LL_NEED_OS_STAT', 1) - if (ll__socket.ll__socket_nextaddrinfo in db.externalfuncs or - not optimize): - yield decl(ll__socket.ll__socket_addrinfo) - yield ('LL_NEED__SOCKET_ADDRINFO', 1) - - if (ll__socket.ll__socket_getpeername in db.externalfuncs or - not optimize): - yield decl(ll__socket.ll__socket_sockname) - yield ('LL_NEED__SOCKET_SOCKNAME', 1) - def predeclare_extfuncs(db, rtyper, optimize=True): modules = {} def module_name(c_name): From arigo at codespeak.net Tue Apr 25 14:50:53 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 14:50:53 +0200 (CEST) Subject: [pypy-svn] r26332 - pypy/dist/pypy/interpreter Message-ID: <20060425125053.32B171007D@code0.codespeak.net> Author: arigo Date: Tue Apr 25 14:50:51 2006 New Revision: 26332 Modified: pypy/dist/pypy/interpreter/baseobjspace.py pypy/dist/pypy/interpreter/typedef.py Log: Comment this out temporarily (it breaks translation of PyPy). Modified: pypy/dist/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/dist/pypy/interpreter/baseobjspace.py (original) +++ pypy/dist/pypy/interpreter/baseobjspace.py Tue Apr 25 14:50:51 2006 @@ -193,7 +193,7 @@ modules.extend(['unicodedata', '_codecs', 'array', 'marshal', 'errno', 'math', '_sre']) - modules.append('_pickle_support') + #modules.append('_pickle_support') if self.options.nofaking: modules.append('posix') Modified: pypy/dist/pypy/interpreter/typedef.py ============================================================================== --- pypy/dist/pypy/interpreter/typedef.py (original) +++ pypy/dist/pypy/interpreter/typedef.py Tue Apr 25 14:50:51 2006 @@ -543,10 +543,10 @@ __eq__ = interp2app(Cell.descr__eq__, unwrap_spec=['self', ObjSpace, W_Root]), __ne__ = descr_generic_ne, - __reduce__ = interp2app(Cell.descr__reduce__, - unwrap_spec=['self', ObjSpace]), - __setstate__ = interp2app(Cell.descr__setstate__, - unwrap_spec=['self', ObjSpace, W_Root]), +## __reduce__ = interp2app(Cell.descr__reduce__, +## unwrap_spec=['self', ObjSpace]), +## __setstate__ = interp2app(Cell.descr__setstate__, +## unwrap_spec=['self', ObjSpace, W_Root]), ) Ellipsis.typedef = TypeDef("Ellipsis", From ac at codespeak.net Tue Apr 25 16:26:11 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 25 Apr 2006 16:26:11 +0200 (CEST) Subject: [pypy-svn] r26335 - pypy/dist/pypy/module/_socket Message-ID: <20060425142611.2EAEC10084@code0.codespeak.net> Author: ac Date: Tue Apr 25 16:26:10 2006 New Revision: 26335 Modified: pypy/dist/pypy/module/_socket/__init__.py Log: Don't forget about buildloader in MixedModule. Modified: pypy/dist/pypy/module/_socket/__init__.py ============================================================================== --- pypy/dist/pypy/module/_socket/__init__.py (original) +++ pypy/dist/pypy/module/_socket/__init__.py Tue Apr 25 16:26:10 2006 @@ -31,6 +31,7 @@ for constant, value in _c.constants.iteritems(): Module.interpleveldefs[constant] = "space.wrap(%r)" % value + super(Module, cls).buildloaders() buildloaders = classmethod(buildloaders) #Module.interpleveldefs['has_ipv6'] = "space.wrap(%s)" % _socket.has_ipv6 From ac at codespeak.net Tue Apr 25 16:41:48 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 25 Apr 2006 16:41:48 +0200 (CEST) Subject: [pypy-svn] r26336 - in pypy/dist/pypy/module/_socket: . test Message-ID: <20060425144148.228FF10088@code0.codespeak.net> Author: ac Date: Tue Apr 25 16:41:47 2006 New Revision: 26336 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/module/_socket/test/test_socket2.py Log: Make w_get_socketerror find the message by itself. Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Tue Apr 25 16:41:47 2006 @@ -80,6 +80,8 @@ w_module = space.getbuiltinmodule('_socket') w_errortype = space.getattr(w_module, space.wrap('error')) if errno > -1: + if message is None: + message = socket_strerror(errno) return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) else: return OperationError(w_errortype, space.wrap(message)) @@ -88,6 +90,8 @@ w_module = space.getbuiltinmodule('_socket') w_errortype = space.getattr(w_module, space.wrap('gaierror')) if errno > -1: + if message is None: + message = _c.gai_strerror(errno) return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) else: return OperationError(w_errortype, space.wrap(message)) @@ -96,6 +100,8 @@ w_module = space.getbuiltinmodule('_socket') w_errortype = space.getattr(w_module, space.wrap('herror')) if errno > -1: + if message is None: + message = _c.hstrerror(errno) return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) else: return OperationError(w_errortype, space.wrap(message)) @@ -117,7 +123,7 @@ res = _c.addr_info_ptr err = _c.getaddrinfo( None, "0", pointer(hints), pointer(res)) if err: - raise w_get_socketgaierror(_c.errno) + raise w_get_socketgaierror(space, None, _c.errno) if res.contents.ai_next: raise OperationError(_socket.error, space.wrap("wildcard resolved to multiple address")) addr = res.contents.ai_addr @@ -142,7 +148,7 @@ namebuff = ctypes.create_string_buffer(BUFFLEN) res = _c.gethostname(namebuff, BUFFLEN - 1) if res < 0: - raise w_get_socketerror(_c.errno.value) + raise w_get_socketerror(space, None, _c.errno.value) return space.wrap(namebuff.value) gethostname.unwrap_spec = [ObjSpace] @@ -153,7 +159,7 @@ """ hostent = _c.gethostbyname(name) if not hostent: - raise w_get_socketherror(_c.h_errno.value) + raise w_get_socketherror(space, None, _c.h_errno.value) return space.wrap(hostent.contents.h_addr) gethostbyname.unwrap_spec = [ObjSpace, str] @@ -171,7 +177,7 @@ def common_gethost(space, hostent): if not hostent: - raise w_get_socketherror(space, _c.hstrerror(_c.h_errno.value), _c.h_errno.value) + raise w_get_socketherror(space, None, _c.h_errno.value) aliases = [] for alias in hostent.contents.h_aliases: if alias is None: @@ -195,7 +201,7 @@ """ hostent = _c.gethostbyname(name) if not hostent: - raise w_get_socketherror(_c.h_errno.value) + raise w_get_socketherror(space, None, _c.h_errno.value) p_addr = hostent.contents.h_addr_list[0] hostent = _c.gethostbyaddr(p_addr, ctypes.sizeof(_c.in_addr), _c.AF_INET) @@ -582,8 +588,8 @@ hints.ai_socktype = socktype hints.ai_protocol = proto retval = _c.getaddrinfo(host, port, ctypes.pointer(hints), ctypes.pointer(res)) - if retval < 0: - raise w_get_socketgaierror(_c.gai_strerror(_c.errno), _c.errno.value) + if retval != 0: + raise w_get_socketgaierror(space, None, retval) try: result = [] @@ -733,7 +739,7 @@ res = _c.close(self.fd) if res < 0: errno = _c.errno.value - raise w_get_socketerror(space, socket_strerror(errno), errno) + raise w_get_socketerror(space, None, errno) self.closed = True close.unwrap_spec = ['self', ObjSpace] Modified: pypy/dist/pypy/module/_socket/test/test_socket2.py ============================================================================== --- pypy/dist/pypy/module/_socket/test/test_socket2.py (original) +++ pypy/dist/pypy/module/_socket/test/test_socket2.py Tue Apr 25 16:41:47 2006 @@ -4,7 +4,7 @@ import py import socket, sys -py.test.skip('In-progress') +#py.test.skip('In-progress') def setup_module(mod): mod.space = StdObjSpace(usemodules=['_socket']) From cfbolz at codespeak.net Tue Apr 25 16:48:25 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 25 Apr 2006 16:48:25 +0200 (CEST) Subject: [pypy-svn] r26337 - pypy/dist/pypy/translator/c/test Message-ID: <20060425144825.9A8AA1008B@code0.codespeak.net> Author: cfbolz Date: Tue Apr 25 16:48:24 2006 New Revision: 26337 Modified: pypy/dist/pypy/translator/c/test/test_ext__socket.py Log: skip socket genc tests Modified: pypy/dist/pypy/translator/c/test/test_ext__socket.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_ext__socket.py (original) +++ pypy/dist/pypy/translator/c/test/test_ext__socket.py Tue Apr 25 16:48:24 2006 @@ -1,5 +1,6 @@ import autopath import py +py.test.skip("in progress") import _socket, thread from pypy.translator.c.test.test_genc import compile from pypy.translator.translator import TranslationContext From mwh at codespeak.net Tue Apr 25 16:51:45 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 25 Apr 2006 16:51:45 +0200 (CEST) Subject: [pypy-svn] r26338 - in pypy/dist/pypy: rpython translator translator/c Message-ID: <20060425145145.1418B1008F@code0.codespeak.net> Author: mwh Date: Tue Apr 25 16:51:42 2006 New Revision: 26338 Modified: pypy/dist/pypy/rpython/extfunctable.py pypy/dist/pypy/translator/c/extfunc.py pypy/dist/pypy/translator/translator.py Log: (mwh, some soothing words from cfbolz when life got too frustrating) some avoidance of annotating helpers for external functions when the said external functions will not be called. this isn't what i wanted to do, what i wanted to do was to annotate the helper's code at the same time as annotating the calling function but that lead to insanity. Modified: pypy/dist/pypy/rpython/extfunctable.py ============================================================================== --- pypy/dist/pypy/rpython/extfunctable.py (original) +++ pypy/dist/pypy/rpython/extfunctable.py Tue Apr 25 16:51:42 2006 @@ -119,7 +119,15 @@ # _____________________________________________________________ - +def record_call(func, args_s, symbol): + from pypy.annotation import bookkeeper + bk = bookkeeper.getbookkeeper() + # this would be nice! + #bk.pbc_call(bk.immutablevalue(func), + # bk.build_args("simple_call", args_s), + # emulated=True) + bk.annotator.translator._implicitly_called_by_externals.append( + (func, args_s, symbol)) def noneannotation(*args): return None @@ -130,14 +138,20 @@ def statannotation(*args): from pypy.annotation.model import SomeInteger, SomeTuple + from pypy.rpython.module.ll_os import ll_stat_result + record_call(ll_stat_result, [SomeInteger()]*10, 'OS_STAT') return SomeTuple((SomeInteger(),)*10) def frexpannotation(*args): from pypy.annotation.model import SomeInteger, SomeTuple, SomeFloat + from pypy.rpython.module.ll_math import ll_frexp_result + record_call(ll_frexp_result, (SomeFloat(), SomeInteger()), 'MATH_FREXP') return SomeTuple((SomeFloat(), SomeInteger())) def modfannotation(*args): from pypy.annotation.model import SomeTuple, SomeFloat + from pypy.rpython.module.ll_math import ll_modf_result + record_call(ll_modf_result, (SomeFloat(), SomeFloat()), 'MATH_MODF') return SomeTuple((SomeFloat(), SomeFloat())) def strnullannotation(*args): Modified: pypy/dist/pypy/translator/c/extfunc.py ============================================================================== --- pypy/dist/pypy/translator/c/extfunc.py (original) +++ pypy/dist/pypy/translator/c/extfunc.py Tue Apr 25 16:51:42 2006 @@ -126,48 +126,22 @@ def get_extfunc_helper_ptrs(db, rtyper, optimize=True): - # XXX need some way of finding out if the externals needing have - # been annotated -- db.externalfuncs gets filled out by - # select_function_code_generator which is called from - # FuncNode.__init__ (probably...) which is after this gets called. - optimize = False - def annotate(func, *argtypes): - fptr = rtyper.annotate_helper(func, argtypes) + + def annotate(func, args): + fptr = rtyper.annotate_helper(func, args) db.helper2ptr[func] = fptr return (func.__name__, fptr) - r = [] - - if ll_math.ll_math_frexp in db.externalfuncs or not optimize: - r.append(annotate(ll_math.ll_frexp_result, lltype.Float, lltype.Signed)) - - if ll_math.ll_math_modf in db.externalfuncs or not optimize: - r.append(annotate(ll_math.ll_modf_result, lltype.Float, lltype.Float)) - - if (ll_os.ll_os_stat in db.externalfuncs or - ll_os.ll_os_fstat in db.externalfuncs or - not optimize): - r.append(annotate(ll_os.ll_stat_result, *([lltype.Signed] * 10))) - - return r + for func, args, symb in db.translator._implicitly_called_by_externals: + yield annotate(func, args) def predeclare_extfunc_helpers(db, rtyper, optimize=True): - def decl(f): - return (f.__name__, db.helper2ptr[f]) - - if ll_math.ll_math_frexp in db.externalfuncs or not optimize: - yield decl(ll_math.ll_frexp_result) - yield ('LL_NEED_MATH_FREXP', 1) - - if ll_math.ll_math_modf in db.externalfuncs or not optimize: - yield decl(ll_math.ll_modf_result) - yield ('LL_NEED_MATH_MODF', 1) - - if (ll_os.ll_os_stat in db.externalfuncs or - ll_os.ll_os_fstat in db.externalfuncs or - not optimize): - yield decl(ll_os.ll_stat_result) - yield ('LL_NEED_OS_STAT', 1) + def decl(func): + return (func.__name__, db.helper2ptr[func]) + + for func, args, symb in db.translator._implicitly_called_by_externals: + yield decl(func) + yield ('LL_NEED_' + symb, 1) def predeclare_extfuncs(db, rtyper, optimize=True): modules = {} Modified: pypy/dist/pypy/translator/translator.py ============================================================================== --- pypy/dist/pypy/translator/translator.py (original) +++ pypy/dist/pypy/translator/translator.py Tue Apr 25 16:51:42 2006 @@ -34,6 +34,8 @@ self.callgraph = {} # {opaque_tag: (caller-graph, callee-graph)} self._prebuilt_graphs = {} # only used by the pygame viewer + self._implicitly_called_by_externals = [] + def buildflowgraph(self, func): """Get the flow graph for a function.""" if not isinstance(func, types.FunctionType): From ac at codespeak.net Tue Apr 25 18:55:28 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 25 Apr 2006 18:55:28 +0200 (CEST) Subject: [pypy-svn] r26342 - pypy/dist/pypy/module/_socket/test Message-ID: <20060425165528.A523610088@code0.codespeak.net> Author: ac Date: Tue Apr 25 18:55:27 2006 New Revision: 26342 Modified: pypy/dist/pypy/module/_socket/test/test_socket2.py Log: Oops, keep socket tests disabled for now. Modified: pypy/dist/pypy/module/_socket/test/test_socket2.py ============================================================================== --- pypy/dist/pypy/module/_socket/test/test_socket2.py (original) +++ pypy/dist/pypy/module/_socket/test/test_socket2.py Tue Apr 25 18:55:27 2006 @@ -4,7 +4,7 @@ import py import socket, sys -#py.test.skip('In-progress') +py.test.skip('In-progress') def setup_module(mod): mod.space = StdObjSpace(usemodules=['_socket']) From antocuni at codespeak.net Tue Apr 25 19:02:13 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 25 Apr 2006 19:02:13 +0200 (CEST) Subject: [pypy-svn] r26343 - in pypy/dist/pypy/rpython: . ootypesystem test Message-ID: <20060425170213.BBA031008A@code0.codespeak.net> Author: antocuni Date: Tue Apr 25 19:02:03 2006 New Revision: 26343 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: Added support for copy(), clear() and update() methods. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Tue Apr 25 19:02:03 2006 @@ -358,6 +358,7 @@ "ll_set": Meth([self.KEYTYPE_T, self.VALUETYPE_T], Void), "ll_remove": Meth([self.KEYTYPE_T], Bool), # return False is key was not present "ll_contains": Meth([self.KEYTYPE_T], Bool), + "ll_clear": Meth([], Void), "ll_get_items_iterator": Meth([], DictItemsIterator(self.KEYTYPE_T, self.VALUETYPE_T)), }) @@ -795,6 +796,9 @@ assert typeOf(key) == self._TYPE._KEYTYPE return key in self._dict + def ll_clear(self): + self._dict.clear() + def ll_get_items_iterator(self): # NOT_RPYTHON ITER = DictItemsIterator(self._TYPE._KEYTYPE, self._TYPE._VALUETYPE) Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Tue Apr 25 19:02:03 2006 @@ -69,6 +69,7 @@ def rtype_len(self, hop): v_dict, = hop.inputargs(self) + hop.exception_cannot_occur() return self.send_message(hop, 'll_length') def rtype_is_true(self, hop): @@ -89,6 +90,17 @@ v_res = hop.gendirectcall(ll_dict_setdefault, v_dict, v_key, v_default) return self.recast_value(hop.llops, v_res) + def rtype_method_copy(self, hop): + v_dict, = hop.inputargs(self) + cDICT = hop.inputconst(ootype.Void, self.lowleveltype) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_dict_copy, cDICT, v_dict) + + def rtype_method_update(self, hop): + v_dict1, v_dict2 = hop.inputargs(self, self) + hop.exception_cannot_occur() + return hop.gendirectcall(ll_dict_update, v_dict1, v_dict2) + def rtype_method_keys(self, hop): return self._rtype_method_kvi(hop, dum_keys) @@ -118,6 +130,10 @@ hop.exception_cannot_occur() return DictIteratorRepr(self, "items").newiter(hop) + def rtype_method_clear(self, hop): + v_dict, = hop.inputargs(self) + hop.exception_cannot_occur() + return self.send_message(hop, 'll_clear') class __extend__(pairtype(DictRepr, rmodel.Repr)): @@ -147,9 +163,12 @@ def rtype_contains((r_dict, r_key), hop): v_dict, v_key = hop.inputargs(r_dict, r_dict.key_repr) + hop.exception_cannot_occur() return r_dict.send_message(hop, 'll_contains') +def rtype_r_dict(hop): + pass # TODO def ll_newdict(DICT): return ootype.new(DICT) @@ -158,6 +177,19 @@ # check if a dict is True, allowing for None return bool(d) and d.ll_length() != 0 +def ll_dict_copy(DICT, d): + res = ootype.new(DICT) + ll_dict_update(res, d) + return res + +def ll_dict_update(d1, d2): + it = d2.ll_get_items_iterator() + while it.ll_go_next(): + key = it.ll_current_key() + value = it.ll_current_value() + d1.ll_set(key, value) + + def ll_dict_getitem(d, key): # TODO: this is inefficient because it does two lookups if d.ll_contains(key): Modified: pypy/dist/pypy/rpython/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/rdict.py (original) +++ pypy/dist/pypy/rpython/rdict.py Tue Apr 25 19:02:03 2006 @@ -81,6 +81,7 @@ class AbstractDictIteratorRepr(rmodel.IteratorRepr): + def newiter(self, hop): v_dict, = hop.inputargs(self.r_dict) citerptr = hop.inputconst(lltype.Void, self.lowleveltype) Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Tue Apr 25 19:02:03 2006 @@ -857,3 +857,38 @@ return tot1, tot2 res = self.ll_to_tuple(self.interpret(func, [42, 13]), 2) assert res == (55, 57) + + def test_copy(self): + def func(x, y): + d = {x: x+1, y: y+1} + d2 = d.copy() + d[x] = 0 + d[y] = 0 + tot1 = 0 + tot2 = 0 + for key, value in d2.iteritems(): + tot1 += key + tot2 += value + return tot1, tot2 + res = self.ll_to_tuple(self.interpret(func, [42, 13]), 2) + assert res == (55, 57) + + def test_update(self): + def func(): + dic = {1:1000, 2:200} + d2 = {2:30, 3:4} + dic.update(d2) + ok = len(dic) == 3 + sum = ok + for key in dic: + sum += dic[key] + return sum + res = self.interpret(func, ()) + assert res == 1035 + + def test_clear(self): + def func(): + dic = {1: 2, 3:4} + dic.clear() + return len(dic) + assert self.interpret(func, ()) == 0 From arigo at codespeak.net Tue Apr 25 19:03:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 19:03:58 +0200 (CEST) Subject: [pypy-svn] r26344 - in pypy/dist/pypy/objspace/cpy: . test Message-ID: <20060425170358.06A3C1008A@code0.codespeak.net> Author: arigo Date: Tue Apr 25 19:03:57 2006 New Revision: 26344 Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/objspace/cpy/wrappable.py Log: UnwrapSpecRecipe magic to support more general unwrap_specs. Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Tue Apr 25 19:03:57 2006 @@ -10,7 +10,7 @@ from pypy import conftest -def test_demo(): +def test_simple_demo(): from pypy.module._demo import demo space = CPyObjSpace() @@ -62,3 +62,24 @@ annotatorpolicy = CPyAnnotatorPolicy()) res = fn(-6) assert res == -42 + +# ____________________________________________________________ + +def makedemotest(): + from pypy.module._demo import demo + space = CPyObjSpace() + func = interp2app(demo.measuretime).__spacebind__(space) + bltin = BuiltinFunction(func) + w_measuretime = space.wrap(bltin) + def entrypoint(n, w_callable): + w_result = space.call_function(w_measuretime, space.wrap(n), + w_callable) + return space.int_w(w_result) + return entrypoint + +def test_compile_demo(): + entrypoint = makedemotest() + fn = compile(entrypoint, [int, CPyObjSpace.W_Object], + annotatorpolicy = CPyAnnotatorPolicy()) + res = fn(10, complex) + assert isinstance(res, int) Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Tue Apr 25 19:03:57 2006 @@ -9,9 +9,47 @@ from pypy.objspace.cpy.objspace import CPyObjSpace from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root +from pypy.interpreter.gateway import UnwrapSpecRecipe, Signature from pypy.rpython import extregistry +class UnwrapSpec_Trampoline(UnwrapSpecRecipe): + + def visit__ObjSpace(self, el, orig_sig, tramp): + argname = orig_sig.next_arg() + assert argname == 'space' + tramp.passedargs.append('___space') + + def visit__W_Root(self, el, orig_sig, tramp): + argname = orig_sig.next_arg() + assert argname.startswith('w_') + basename = 'o_' + argname[2:] + tramp.inputargs.append(basename) + tramp.wrappings.append('%s = ___W_Object(%s)' % (argname, basename)) + tramp.passedargs.append(argname) + + def visit__object(self, el, orig_sig, tramp): + convertermap = {int: '___PyInt_AsLong', + str: 'XXX', + float: 'XXX'} + argname = orig_sig.next_arg() + assert not argname.startswith('w_') + basename = 'o_' + argname + tramp.inputargs.append(basename) + tramp.wrappings.append('%s = %s(%s)' % (argname, + convertermap[el], + basename)) + tramp.passedargs.append(argname) + + +class TrampolineSignature(object): + + def __init__(self): + self.inputargs = [] + self.wrappings = [] + self.passedargs = [] + + class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): def wrap((space, func)): @@ -21,14 +59,35 @@ bltin = factory.behavior unwrap_spec = factory.unwrap_spec - assert unwrap_spec == [ObjSpace, W_Root] # XXX for now + tramp = TrampolineSignature() - def trampoline(a): - w_arg = space.W_Object(a) - w_result = bltin(space, w_arg) - return w_result.value - trampoline.allow_someobjects = True # annotator hint + from pypy.interpreter import pycode + argnames, varargname, kwargname = pycode.cpython_code_signature( + bltin.func_code) + orig_sig = Signature(bltin, argnames, varargname, kwargname) + + orig_sig.apply_unwrap_spec(unwrap_spec, + UnwrapSpec_Trampoline().dispatch, + tramp) + + sourcelines = ['def trampoline(%s):' % (', '.join(tramp.inputargs),)] + for line in tramp.wrappings: + sourcelines.append(' ' + line) + sourcelines.append(' w_result = ___bltin(%s)' % ( + ', '.join(tramp.passedargs),)) + sourcelines.append(' return w_result.value') + sourcelines.append('') + + miniglobals = { + '___space': space, + '___W_Object': CPyObjSpace.W_Object, + '___PyInt_AsLong': PyInt_AsLong, + '___bltin': bltin, + } + exec py.code.Source('\n'.join(sourcelines)).compile() in miniglobals + trampoline = miniglobals['trampoline'] + trampoline.allow_someobjects = True # annotator hint w_result = W_Object(trampoline) # override the annotation behavior of 'w_result' From arigo at codespeak.net Tue Apr 25 19:56:24 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 25 Apr 2006 19:56:24 +0200 (CEST) Subject: [pypy-svn] r26345 - pypy/dist/pypy/module/_pickle_support Message-ID: <20060425175624.DBBF71007D@code0.codespeak.net> Author: arigo Date: Tue Apr 25 19:56:24 2006 New Revision: 26345 Modified: pypy/dist/pypy/module/_pickle_support/ (props changed) Log: fixeol From arigo at codespeak.net Wed Apr 26 09:51:28 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 26 Apr 2006 09:51:28 +0200 (CEST) Subject: [pypy-svn] r26349 - in pypy/dist/pypy: objspace/cpy objspace/cpy/test translator/goal Message-ID: <20060426075128.B656710087@code0.codespeak.net> Author: arigo Date: Wed Apr 26 09:51:25 2006 New Revision: 26349 Modified: pypy/dist/pypy/objspace/cpy/ann_policy.py pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/objspace/cpy/objspace.py pypy/dist/pypy/objspace/cpy/test/test_compile.py pypy/dist/pypy/objspace/cpy/wrappable.py pypy/dist/pypy/translator/goal/targetdemomodule.py Log: * "fastest blinking code of the week": removed the TrampolineEntry extregistry hacks for the second time. Found a robust approach to annotate all the generated trampoline functions. * more CPyObjSpace operations Modified: pypy/dist/pypy/objspace/cpy/ann_policy.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/ann_policy.py (original) +++ pypy/dist/pypy/objspace/cpy/ann_policy.py Wed Apr 26 09:51:25 2006 @@ -1,6 +1,28 @@ from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy +from pypy.annotation.pairtype import pair class CPyAnnotatorPolicy(PyPyAnnotatorPolicy): """Annotation policy to compile CPython extension modules with the CPyObjSpace. """ + + def __init__(self, space): + PyPyAnnotatorPolicy.__init__(self, single_space=space) + self.spaces = {} + + def no_more_blocks_to_annotate(self, annotator): + PyPyAnnotatorPolicy.no_more_blocks_to_annotate(self, annotator) + + # annotate all indirectly reachable call-back functions + space = self.single_space + pending = {} + while True: + nb_done = len(pending) + pending.update(space.wrap_cache) + if len(pending) == nb_done: + break + for obj, w_obj in pending.items(): + pair(space, obj).follow_annotations(annotator.bookkeeper, + w_obj) + # restart this loop: for all we know follow_annotations() + # could have found new objects Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Wed Apr 26 09:51:25 2006 @@ -82,6 +82,14 @@ PyObject_RichCompareBool.argtypes = [W_Object, W_Object, c_int] PyObject_RichCompareBool.restype = c_int +PyObject_GetIter = pythonapi.PyObject_GetIter +PyObject_GetIter.argtypes = [W_Object] +PyObject_GetIter.restype = W_Object + +PyIter_Next = pythonapi.PyIter_Next +PyIter_Next.argtypes = [W_Object] +PyIter_Next.restype = W_Object + ############################################################# # ____________________ Sequence Protocol ____________________ @@ -118,6 +126,10 @@ PyString_InternInPlace.argtypes = [POINTER(W_Object)] PyString_InternInPlace.restype = None +PyString_AsString = pythonapi.PyString_AsString +PyString_AsString.argtypes = [W_Object] +PyString_AsString.restype = c_char_p + ################################################## # ____________________ Tuples ____________________ Modified: pypy/dist/pypy/objspace/cpy/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/objspace.py Wed Apr 26 09:51:25 2006 @@ -1,24 +1,26 @@ from pypy.objspace.cpy.capi import * from pypy.annotation.pairtype import pair from pypy.interpreter import baseobjspace +from pypy.interpreter.error import OperationError -class CPyObjSpace: +class CPyObjSpace(baseobjspace.ObjSpace): from pypy.objspace.cpy.capi import W_Object - def __init__(self): - self.fromcache = baseobjspace.InternalSpaceCache(self).getorbuild + def initialize(self): + self.options.geninterp = True self.w_int = W_Object(int) self.w_None = W_Object(None) self.w_False = W_Object(False) self.w_True = W_Object(True) + self.w_type = W_Object(type) + self.w_Exception = W_Object(Exception) + self.w_StopIteration = W_Object(StopIteration) self.wrap_cache = {} + self.rev_wrap_cache = {} - def enter_cache_building_mode(self): - pass - - def leave_cache_building_mode(self, val): - pass + def _freeze_(self): + return True def getbuiltinmodule(self, name): return PyImport_ImportModule(name) @@ -33,6 +35,7 @@ import pypy.objspace.cpy.wrappable result = pair(self, x).wrap() self.wrap_cache[x] = result + self.rev_wrap_cache[id(result)] = result, x return result if x is None: return self.w_None @@ -47,23 +50,46 @@ assert isinstance(w_obj, W_Object) return w_obj.value + def finditem(self, w_obj, w_key): + try: + return self.getitem(w_obj, w_key) + except KeyError: # XXX think about OperationError + return None + + def interpclass_w(self, w_obj): + try: + return self.rev_wrap_cache[id(w_obj)][1] + except KeyError: + return None + + # __________ operations with a direct CPython equivalent __________ + getattr = staticmethod(PyObject_GetAttr) getitem = staticmethod(PyObject_GetItem) setitem = staticmethod(PyObject_SetItem) int_w = staticmethod(PyInt_AsLong) + str_w = staticmethod(PyString_AsString) + iter = staticmethod(PyObject_GetIter) def call_function(self, w_callable, *args_w): args_w += (None,) return PyObject_CallFunctionObjArgs(w_callable, *args_w) - def _freeze_(self): - return True + def call_args(self, w_callable, args): + args_w, kwds_w = args.unpack() + w_args = self.newtuple(args_w) + w_kwds = self.newdict([(self.wrap(key), w_value) + for key, w_value in kwds_w.items()]) + return PyObject_Call(w_callable, w_args, w_kwds) def new_interned_str(self, s): w_s = self.wrap(s) PyString_InternInPlace(byref(w_s)) return w_s + def newint(self, intval): + return PyInt_FromLong(intval) + def newdict(self, items_w): w_dict = PyDict_New() for w_key, w_value in items_w: @@ -97,3 +123,16 @@ def ne_w(self, w1, w2): return PyObject_RichCompareBool(w1, w2, Py_NE) != 0 def gt_w(self, w1, w2): return PyObject_RichCompareBool(w1, w2, Py_GT) != 0 def ge_w(self, w1, w2): return PyObject_RichCompareBool(w1, w2, Py_GE) != 0 + + def is_w(self, w1, w2): + return w1.value is w2.value # XXX any idea not involving SomeObjects? + is_w.allow_someobjects = True + + def is_(self, w1, w2): + return self.newbool(self.is_w(w1, w2)) + + def next(self, w_obj): + w_res = PyIter_Next(w_obj) + if not w_res: + raise OperationError(self.w_StopIteration, self.w_None) + return w_res Modified: pypy/dist/pypy/objspace/cpy/test/test_compile.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_compile.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_compile.py Wed Apr 26 09:51:25 2006 @@ -18,7 +18,7 @@ return demo.measuretime(space, n, w_callable) fn = compile(entry_point, [int, CPyObjSpace.W_Object], - annotatorpolicy = CPyAnnotatorPolicy()) + annotatorpolicy = CPyAnnotatorPolicy(space)) res = fn(10, long) assert isinstance(res, int) @@ -39,12 +39,12 @@ def entrypoint(n): w_result = space.call_function(w_myfunc, space.wrap(n)) return space.int_w(w_result) - return entrypoint + return space, entrypoint def test_annotate_bltinfunc(): - entrypoint = maketest() + space, entrypoint = maketest() t = TranslationContext() - a = t.buildannotator(policy=CPyAnnotatorPolicy()) + a = t.buildannotator(policy=CPyAnnotatorPolicy(space)) s = a.build_types(entrypoint, [int]) if conftest.option.view: t.view() @@ -56,10 +56,31 @@ s = a.binding(graph.getreturnvar()) assert s.knowntype == CPyObjSpace.W_Object +def test_annotate_indirect(): + space = CPyObjSpace() + func = interp2app(myfunc).__spacebind__(space) + bltin = BuiltinFunction(func) + w_myfunc = space.wrap(bltin) + w_mylist = space.newlist([w_myfunc]) + def entrypoint(): + return w_mylist + t = TranslationContext() + a = t.buildannotator(policy=CPyAnnotatorPolicy(space)) + s = a.build_types(entrypoint, []) + if conftest.option.view: + t.view() + # 'myfunc' should still have been annotated + graph = graphof(t, myfunc) + assert len(graph.getargs()) == 2 + s = a.binding(graph.getargs()[1]) + assert s.knowntype == CPyObjSpace.W_Object + s = a.binding(graph.getreturnvar()) + assert s.knowntype == CPyObjSpace.W_Object + def test_compile_bltinfunc(): - entrypoint = maketest() + space, entrypoint = maketest() fn = compile(entrypoint, [int], - annotatorpolicy = CPyAnnotatorPolicy()) + annotatorpolicy = CPyAnnotatorPolicy(space)) res = fn(-6) assert res == -42 @@ -75,11 +96,11 @@ w_result = space.call_function(w_measuretime, space.wrap(n), w_callable) return space.int_w(w_result) - return entrypoint + return space, entrypoint def test_compile_demo(): - entrypoint = makedemotest() + space, entrypoint = makedemotest() fn = compile(entrypoint, [int, CPyObjSpace.W_Object], - annotatorpolicy = CPyAnnotatorPolicy()) + annotatorpolicy = CPyAnnotatorPolicy(space)) res = fn(10, complex) assert isinstance(res, int) Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Wed Apr 26 09:51:25 2006 @@ -7,10 +7,9 @@ from pypy.annotation.pairtype import pair, pairtype from pypy.objspace.cpy.capi import * from pypy.objspace.cpy.objspace import CPyObjSpace -from pypy.interpreter.function import BuiltinFunction +from pypy.interpreter.function import Function from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root from pypy.interpreter.gateway import UnwrapSpecRecipe, Signature -from pypy.rpython import extregistry class UnwrapSpec_Trampoline(UnwrapSpecRecipe): @@ -50,11 +49,11 @@ self.passedargs = [] -class __extend__(pairtype(CPyObjSpace, BuiltinFunction)): +class __extend__(pairtype(CPyObjSpace, Function)): def wrap((space, func)): # make a built-in function - assert isinstance(func.code, BuiltinCode) + assert isinstance(func.code, BuiltinCode) # XXX factory = func.code.framefactory bltin = factory.behavior unwrap_spec = factory.unwrap_spec @@ -87,28 +86,14 @@ exec py.code.Source('\n'.join(sourcelines)).compile() in miniglobals trampoline = miniglobals['trampoline'] + trampoline.nb_args = len(tramp.inputargs) trampoline.allow_someobjects = True # annotator hint - w_result = W_Object(trampoline) + return W_Object(trampoline) - # override the annotation behavior of 'w_result' - # to emulate a call to the trampoline() function at interp-level - BaseEntry = extregistry._lookup_cls(w_result) + def follow_annotations((space, func), bookkeeper, w_trampoline): + from pypy.annotation import model as annmodel + trampoline = w_trampoline.value + s_trampoline = bookkeeper.immutablevalue(trampoline) + args_s = [annmodel.SomeObject()] * trampoline.nb_args uniquekey = trampoline - nb_args = len(unwrap_spec) - 1 - - class TrampolineEntry(BaseEntry): - _about_ = w_result - - def compute_annotation(self): - from pypy.annotation import model as annmodel - from pypy.annotation.bookkeeper import getbookkeeper - - bookkeeper = getbookkeeper() - if bookkeeper is not None: # else probably already rtyping - s_trampoline = bookkeeper.immutablevalue(trampoline) - args_s = [annmodel.SomeObject()]*nb_args - bookkeeper.emulate_pbc_call(uniquekey, s_trampoline, - args_s) - return super(TrampolineEntry, self).compute_annotation() - - return w_result + bookkeeper.emulate_pbc_call(uniquekey, s_trampoline, args_s) Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Wed Apr 26 09:51:25 2006 @@ -1,18 +1,21 @@ -from pypy.module._demo import demo +from pypy.module._demo import Module, demo from pypy.objspace.cpy.ann_policy import CPyAnnotatorPolicy from pypy.objspace.cpy.objspace import CPyObjSpace import pypy.rpython.rctypes.implementation space = CPyObjSpace() +Module.appleveldefs.clear() # XXX! for now +module = Module(space, space.wrap('_demo')) +w_moduledict = module.getdict() -def entry_point(n, w_callable): - return demo.measuretime(space, n, w_callable) +def getdict(): + return w_moduledict # _____ Define and setup target ___ def target(*args): - return entry_point, [int, CPyObjSpace.W_Object], CPyAnnotatorPolicy() + return getdict, [], CPyAnnotatorPolicy(space) if __name__ == '__main__': @@ -22,4 +25,4 @@ else: N = int(sys.argv[1]) print 'Timing for %d iterations...' % N - print entry_point(N, space.W_Object(int)), 'seconds' + print demo.measuretime(space, N, space.W_Object(int)), 'seconds' From arigo at codespeak.net Wed Apr 26 10:11:37 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 26 Apr 2006 10:11:37 +0200 (CEST) Subject: [pypy-svn] r26350 - in pypy/dist/pypy: doc/discussion jit/timeshifter objspace/constraint objspace/constraint/applevel objspace/constraint/test objspace/std rpython/rctypes/socketmodule/test tool/algo tool/algo/test translator/backendopt translator/backendopt/test translator/cl translator/cl/test translator/goal translator/llvm/llvmcapi translator/llvm/wrapable translator/llvm/wrapable/test translator/microbench translator/microbench/pybench translator/microbench/pybench/package Message-ID: <20060426081137.2B99310087@code0.codespeak.net> Author: arigo Date: Wed Apr 26 10:11:25 2006 New Revision: 26350 Modified: pypy/dist/pypy/doc/discussion/translation-swamp.txt (props changed) pypy/dist/pypy/jit/timeshifter/rcontainer.py (props changed) pypy/dist/pypy/jit/timeshifter/rvalue.py (props changed) pypy/dist/pypy/objspace/constraint/applevel/ (props changed) pypy/dist/pypy/objspace/constraint/applevel/__init__.py (props changed) pypy/dist/pypy/objspace/constraint/applevel/problems.py (props changed) pypy/dist/pypy/objspace/constraint/applevel/solver.py (props changed) pypy/dist/pypy/objspace/constraint/btree.py (props changed) pypy/dist/pypy/objspace/constraint/constraint.py (props changed) pypy/dist/pypy/objspace/constraint/distributor.py (props changed) pypy/dist/pypy/objspace/constraint/domain.py (props changed) pypy/dist/pypy/objspace/constraint/expression_syntax_proposal.py (props changed) pypy/dist/pypy/objspace/constraint/syntax_proposal.py (props changed) pypy/dist/pypy/objspace/constraint/test/targetsmallstuff.py (props changed) pypy/dist/pypy/objspace/constraint/test/test.py (props changed) pypy/dist/pypy/objspace/constraint/test/test_btree.py (props changed) pypy/dist/pypy/objspace/constraint/test/test_distributor.py (props changed) pypy/dist/pypy/objspace/constraint/test/test_fd.py (props changed) pypy/dist/pypy/objspace/constraint/test/test_solver.py (props changed) pypy/dist/pypy/objspace/constraint/util.py (props changed) pypy/dist/pypy/objspace/std/frozensettype.py (props changed) pypy/dist/pypy/rpython/rctypes/socketmodule/test/test_errno.py (props changed) pypy/dist/pypy/tool/algo/multiweakdict.py (props changed) pypy/dist/pypy/tool/algo/test/test_multiweakdict.py (props changed) pypy/dist/pypy/translator/backendopt/graphanalyze.py (props changed) pypy/dist/pypy/translator/backendopt/mallocprediction.py (props changed) pypy/dist/pypy/translator/backendopt/test/test_mallocprediction.py (props changed) pypy/dist/pypy/translator/cl/conftest.py (props changed) pypy/dist/pypy/translator/cl/test/test_buildcl.py (props changed) pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py (props changed) pypy/dist/pypy/translator/goal/targetlogic.py (props changed) pypy/dist/pypy/translator/goal/targetpushpop.py (props changed) pypy/dist/pypy/translator/goal/targetrdicttest.py (props changed) pypy/dist/pypy/translator/goal/targetrdicttest2.py (props changed) pypy/dist/pypy/translator/llvm/llvmcapi/ (props changed) pypy/dist/pypy/translator/llvm/wrapable/ (props changed) pypy/dist/pypy/translator/llvm/wrapable/test/ (props changed) pypy/dist/pypy/translator/microbench/pybench/ (props changed) pypy/dist/pypy/translator/microbench/pybench/Arithmetic.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Calls.py (props changed) pypy/dist/pypy/translator/microbench/pybench/CommandLine.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Constructs.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Dict.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Exceptions.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Imports.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Instances.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Lists.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Lookups.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Numbers.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Setup.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Strings.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Tuples.py (props changed) pypy/dist/pypy/translator/microbench/pybench/Unicode.py (props changed) pypy/dist/pypy/translator/microbench/pybench/package/ (props changed) pypy/dist/pypy/translator/microbench/pybench/package/__init__.py (props changed) pypy/dist/pypy/translator/microbench/pybench/package/submodule.py (props changed) pypy/dist/pypy/translator/microbench/pybench/platform.py (props changed) pypy/dist/pypy/translator/microbench/pybench/pybench.py (props changed) pypy/dist/pypy/translator/microbench/test_dict.py (props changed) Log: fixeol From arigo at codespeak.net Wed Apr 26 10:16:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 26 Apr 2006 10:16:20 +0200 (CEST) Subject: [pypy-svn] r26351 - in pypy/dist/pypy/objspace/cpy: . test Message-ID: <20060426081620.82AAA10087@code0.codespeak.net> Author: arigo Date: Wed Apr 26 10:16:19 2006 New Revision: 26351 Modified: pypy/dist/pypy/objspace/cpy/test/test_wrappable.py pypy/dist/pypy/objspace/cpy/wrappable.py Log: Take more care about argument names, as they are used when called with keywords. Modified: pypy/dist/pypy/objspace/cpy/test/test_wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_wrappable.py Wed Apr 26 10:16:19 2006 @@ -1,19 +1,31 @@ from pypy.objspace.cpy.objspace import CPyObjSpace from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root +from pypy.interpreter.argument import Arguments -def test_builtin_function(): - def entrypoint(space, w_x): - x = space.int_w(w_x) - result = x * 7 - return space.wrap(result) - entrypoint.unwrap_spec = [ObjSpace, W_Root] +def entrypoint1(space, w_x): + x = space.int_w(w_x) + result = x * 7 + return space.wrap(result) +entrypoint1.unwrap_spec = [ObjSpace, W_Root] + +def test_builtin_function(): space = CPyObjSpace() - func = interp2app(entrypoint).__spacebind__(space) + func = interp2app(entrypoint1).__spacebind__(space) bltin = BuiltinFunction(func) w_entrypoint = space.wrap(bltin) w_result = space.call_function(w_entrypoint, space.wrap(-2)) result = space.int_w(w_result) assert result == -14 + +def test_builtin_function_keywords(): + space = CPyObjSpace() + func = interp2app(entrypoint1).__spacebind__(space) + bltin = BuiltinFunction(func) + w_entrypoint = space.wrap(bltin) + args = Arguments(space, [], {'x': space.wrap(-3)}) + w_result = space.call_args(w_entrypoint, args) + result = space.int_w(w_result) + assert result == -21 Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Wed Apr 26 10:16:19 2006 @@ -22,7 +22,7 @@ def visit__W_Root(self, el, orig_sig, tramp): argname = orig_sig.next_arg() assert argname.startswith('w_') - basename = 'o_' + argname[2:] + basename = argname[2:] tramp.inputargs.append(basename) tramp.wrappings.append('%s = ___W_Object(%s)' % (argname, basename)) tramp.passedargs.append(argname) @@ -33,11 +33,10 @@ float: 'XXX'} argname = orig_sig.next_arg() assert not argname.startswith('w_') - basename = 'o_' + argname - tramp.inputargs.append(basename) + tramp.inputargs.append(argname) tramp.wrappings.append('%s = %s(%s)' % (argname, convertermap[el], - basename)) + argname)) tramp.passedargs.append(argname) From auc at codespeak.net Wed Apr 26 11:00:12 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 26 Apr 2006 11:00:12 +0200 (CEST) Subject: [pypy-svn] r26352 - pypy/dist/pypy/doc/discussion Message-ID: <20060426090012.71ADF10087@code0.codespeak.net> Author: auc Date: Wed Apr 26 11:00:10 2006 New Revision: 26352 Added: pypy/dist/pypy/doc/discussion/logic-plan.txt Log: About logic programming in PyPy and what it needs. Added: pypy/dist/pypy/doc/discussion/logic-plan.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/discussion/logic-plan.txt Wed Apr 26 11:00:10 2006 @@ -0,0 +1,193 @@ +==================================================== +Some ideas about implementation of Logic Programming +==================================================== + +The problem +=========== + +Basics +------ + +Logic and contraint programming as envisionned in PyPy draws heavily +on the Computation Space concept present in the Oz language. + +Computation spaces provide an infrastructure and API that allows +seamless integration of concurrent constraint and logic programming in +the language. + +Currently, there is an implementation of computation spaces that is +sufficient enough for constraint solving. It uses : + +ask() +clone() +commit() + +in straightforward ways. This space is merely a constraint store for +finite domain constraint variables. It does not really support +execution of arbitrary computations 'inside' the space ; only +constraint propagation, i.e. a fixed interrpeter-level algorithm, is +triggered on commit() calls. [note: don't look at the current code ...] + +Choice points +------------- + +For logic programming we need more. Basically a 'logic program' is a +standard Python program augmented with the 'choice' operator. The +program's entry point must be a zero arity procedure. Any program +whose execution path contains a 'choice' is a logic, or relational (in +Oz parlance) program. + +For instance : + + def foo(): + choice: + return bar() + or: + from math import sqrt + return sqrt(4) + +def bar(): + choice: return 0 or: return 1 + + def entry_point(): + a = foo() + if a < 2: + fail() + return a + +What should happen when we encounter a choice ? One of the choice +points ought to be chosen 'non-deterministically'. That means that the +decision to choose does not belong to the program istelf (it is not an +'if clause' depending on the internal state of the program) but to +some external entity having interest in the program outcomes depending +on the various choices that can be made in a choice-littered program. + +Such an entity is typically a 'solver' exploring the space of the +program outcomes. The solver can use a variety of search strategies, +for instance depth-first, breadth-first, discrepancy search, +best-search, A* ... It can provide just one solution or some or all of +them. + +Thus the program and the methods to extract information from it are +truly independant, contrarily to Prolog programs which are hard-wired +for depth-first exploration (and do not support concurrency, at least +not easily). + +Choice, continued +----------------- + +The above program contains just one choice point. If given to a depth +first search solver, it will produce three spaces : the two first +spaces will be failed and thus discarded, the third will be a solution +space and ready to be later merged. We leave the semantics of space +merging for another day. + +Pardon the silly ascii art : + +entry_point -> foo : choice + /\ + / \ + / 2 (solution) + bar : choice + /\ + / \ + / \ + 0 1 + (failure)(failure) + +Search and spaces +----------------- + +To allow this de-coupling between program execution and search +strategy, the computation space comes handily. Basically, choices are +made, and program branches are taken, in speculative computations +which are embedded, or encapsulated in the so-called computation +space, and thus do not affect the whole program, nor each other, until +some outcome (failure, values, updated speculative world) is +considered and eventually merged back into the parent world (which +could be the 'top-level' space, aka normal Python world, or another +speculative space). + +For this we need another method of spaces : + +choose() + +The choice operator can be written straightforwardly in terms of +choose : + + def foo(): + choice = choose(3) + if choice == 1: + return 1 + elif choice == 2: + from math import sqrt + return sqrt(4) + else: # choice == 3 + return 3 + +Choose is more general than choice since the number of branches can be +determined at runtime. Conversely, choice is a special case of choose +where the number of branches is statically determined. It is thus +possible to provide syntactic sugar for it. + +It is important to see the relationship between ask(), choose() and +commit(). Ask and commit are used by the search engine running in the +top-level space, in its own thread, as follows : + +ask() wait for the space to be 'stable', which means that the program +running inside is blocked on a choose() call. It returns precisely the +value provided by choose, thus notifying the search engine about the +number of available choice points. The search engine can then, +depending on its strategy, commit() the space to one of its branches, +thus unblocking the choose() call and giving a return value which +represent the branch to be taken. The program encapsulated in the +space can thus run until : + +* it encounters another choice point, + +* it fails (by way of explicitly calling the fail() operator, or + raising an uncatched exception up to its entry point), + +* properly terminating, thus being a candidate 'satisfying' solution + to the problem it stands for. + +Commit and choose allow a disciplined communication between the world +of the search engine (the normal Python world) and the speculative +computation embedded in the space. Of course the search and the +embedded computation run in different threads. We need at least one +thread for the search engine and one per space for this scheme to +work. + +Cloning +------- + +The problematic thing for us is what happens before we commit to a +specific choice point. Since the are many of them, the solver +typically needs to take a snapshot of the current computation space so +as to be able to later try the other choice points (possibly all of +them if it has been asked to enumerate all solutions of a relational +or constraint program). + +Taking a snapshot, aka cloning, is easy when the space is merely a +constraint store. It is more involved when it comes to snapshotting a +whole tree of threads. It is akin to saving and making copies of the +current continuation (up to some thread frame). + + +Solutions, or ideas in the direction of +======================================= + +Copying collector +----------------- + +In Mozart computation space cloning is implemented by leveraging the +copying garbage collector. There is basically a switch on the GC entry +point which tells whether it is asked to really do GC or just clone a +space. + +Thread cloning +-------------- + +It is expected that PyPy coroutines/greenlets/microthreads be at some +point picklable. One could implement clone() using the thread pickling +machinery. From mwh at codespeak.net Wed Apr 26 11:42:22 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 11:42:22 +0200 (CEST) Subject: [pypy-svn] r26354 - pypy/dist/pypy/rpython Message-ID: <20060426094222.77A9310087@code0.codespeak.net> Author: mwh Date: Wed Apr 26 11:42:21 2006 New Revision: 26354 Modified: pypy/dist/pypy/rpython/llinterp.py Log: unsafe_call never returns a pointer, always an address. this is a gross hack, and reflects the gc naive-ness of the stackless transform at the moment. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Wed Apr 26 11:42:21 2006 @@ -450,7 +450,10 @@ for arg in obj.graph.startblock.inputargs: args.append(arg.concretetype._defl()) frame = self.__class__(graph, args, self.llinterpreter, self) - return frame.eval() + result = frame.eval() + if isinstance(lltype.typeOf(result), lltype.Ptr): + result = llmemory.cast_ptr_to_adr(result) + return result def op_malloc(self, obj): if self.llinterpreter.gc is not None: From mwh at codespeak.net Wed Apr 26 11:56:28 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 11:56:28 +0200 (CEST) Subject: [pypy-svn] r26355 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060426095628.449C510087@code0.codespeak.net> Author: mwh Date: Wed Apr 26 11:56:27 2006 New Revision: 26355 Modified: pypy/dist/pypy/translator/c/exceptiontransform.py pypy/dist/pypy/translator/c/test/test_exceptiontransform.py Log: protect against exception transforming the same graph twice. the failure modes otherwise are very, very odd. Modified: pypy/dist/pypy/translator/c/exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/exceptiontransform.py Wed Apr 26 11:56:27 2006 @@ -122,6 +122,8 @@ self.cnulltype = Constant(null_type, self.lltype_of_exception_type) self.lltype_to_classdef = translator.rtyper.lltype_to_classdef_mapping() + + self.seen_graphs = {} def transform_completely(self): for graph in self.translator.graphs: @@ -135,8 +137,11 @@ from the current graph with a special value (False/-1/-1.0/null). Because of the added exitswitch we need an additional block. """ + assert id(graph) not in self.seen_graphs + self.seen_graphs[id(graph)] = True join_blocks(graph) - for block in list(graph.iterblocks()): #collect the blocks before changing them + # collect the blocks before changing them + for block in list(graph.iterblocks()): self.transform_block(graph, block) self.transform_except_block(graph, graph.exceptblock) cleanup_graph(graph) Modified: pypy/dist/pypy/translator/c/test/test_exceptiontransform.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_exceptiontransform.py (original) +++ pypy/dist/pypy/translator/c/test/test_exceptiontransform.py Wed Apr 26 11:56:27 2006 @@ -175,3 +175,14 @@ f = compile_func(foo, [int]) res = f(0) assert res == 43 + +def test_no_multiple_transform(): + def f(x): + return x + 1 + t = TranslationContext() + t.buildannotator().build_types(f, [int]) + t.buildrtyper().specialize() + g = graphof(t, f) + etrafo = exceptiontransform.ExceptionTransformer(t) + etrafo.create_exception_handling(g) + py.test.raises(AssertionError, etrafo.create_exception_handling, g) From mwh at codespeak.net Wed Apr 26 12:32:43 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 12:32:43 +0200 (CEST) Subject: [pypy-svn] r26356 - pypy/dist/pypy/rpython/memory Message-ID: <20060426103243.0856210084@code0.codespeak.net> Author: mwh Date: Wed Apr 26 12:32:43 2006 New Revision: 26356 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: use typesystem.getfunctionptr instead of making function pointers by hand in const_funcptr_fromgraph. this avoids having multiple funtion pointers for the same graph. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Wed Apr 26 12:32:43 2006 @@ -8,7 +8,7 @@ from pypy.translator.backendopt.support import var_needsgc, needs_conservative_livevar_calculation from pypy.translator.backendopt import graphanalyze from pypy.annotation import model as annmodel -from pypy.rpython import rmodel, rptr, annlowlevel +from pypy.rpython import rmodel, rptr, annlowlevel, typesystem from pypy.rpython.memory import gc, lladdress from pypy.rpython.annlowlevel import MixLevelHelperAnnotator import sets, os @@ -528,7 +528,7 @@ FUNC = lltype.FuncType([v.concretetype for v in graph.startblock.inputargs], graph.returnblock.inputargs[0].concretetype) return rmodel.inputconst(lltype.Ptr(FUNC), - lltype.functionptr(FUNC, graph.name, graph=graph)) + typesystem.getfunctionptr(graph)) def find_gc_ptrs_in_type(TYPE): if isinstance(TYPE, lltype.Array): From mwh at codespeak.net Wed Apr 26 12:53:29 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 12:53:29 +0200 (CEST) Subject: [pypy-svn] r26359 - in pypy/dist/pypy/translator: . c stackless stackless/test Message-ID: <20060426105329.C17E710084@code0.codespeak.net> Author: mwh Date: Wed Apr 26 12:53:23 2006 New Revision: 26359 Modified: pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_depth.py pypy/dist/pypy/translator/stackless/test/test_transform.py pypy/dist/pypy/translator/stackless/transform.py pypy/dist/pypy/translator/translator.py Log: integrate the stackless transform into the translation framework more sensibly. i am not at all happy about having to change genc, but it works and the local increase of horribleness is actually fairly low (one could say that genc is already pretty horrible). all functions that do not have a .stackless_explicit = True attribute will get stackless transformed. also: rename StacklessTransfomer to StacklessTransformer (doh!), don't try to insert a UnwindException case if there's one already there and skip a test that i never seemed to check in enough code to make work. Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Wed Apr 26 12:53:23 2006 @@ -645,6 +645,9 @@ from pypy.translator.c.stackless import SlpFunctionCodeGenerator return [SlpFunctionCodeGenerator(fnobj.graph, db, cpython_exc, functionname)] else: + if db.translator and db.translator.stacklesstransformer is not None: + if not hasattr(fnobj, 'isgchelper'): + db.translator.stacklesstransformer.transform_graph(fnobj.graph) return [FunctionCodeGenerator(fnobj.graph, db, cpython_exc, functionname)] elif getattr(fnobj, 'external', None) == 'C': # deprecated case Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Wed Apr 26 12:53:23 2006 @@ -14,6 +14,7 @@ return (currentframe.function, currentframe.retval_type, currentframe.restartstate) +decode_state.stackless_explict = True SWITCH_STATE = lltype.GcStruct('state_switch', ('header', STATE_HEADER), @@ -35,7 +36,7 @@ s = lltype.cast_pointer(lltype.Ptr(SWITCH_STATE), top) global_state.top = s.c return top.f_back - +switch.stackless_explicit = True def stack_frames_depth(): @@ -59,6 +60,7 @@ depth += 1 cur = cur.f_back return depth +stack_frames_depth.stackless_explicit = True class StacklessData: def __init__(self): @@ -90,6 +92,7 @@ elif retval_code == RETVAL_VOID_P: global_state.retval_void_p = lloperation.llop.unsafe_call( llmemory.Address, fn) +call_function.stackless_explicit = True class UnwindException(Exception): def __init__(self): @@ -125,7 +128,7 @@ if global_state.exception is not None: raise global_state.exception - +slp_main_loop.stackless_explicit = True def add_frame_state(u, frame_state): if not u.frame_top: @@ -133,6 +136,7 @@ else: u.frame_bottom.f_back = frame_state u.frame_bottom = frame_state +add_frame_state.stackless_explicit = True def resume_state(): """Return and zero the 'restart_substate', the index of the resume @@ -140,6 +144,7 @@ x = global_state.restart_substate global_state.restart_substate = 0 return x +resume_state.stackless_explicit = True # XXX would like to be able to say #def resume_header(): @@ -152,28 +157,32 @@ def fetch_retval_void(): if global_state.exception: raise global_state.exception +fetch_retval_void.stackless_explicit = True def fetch_retval_long(): if global_state.exception: raise global_state.exception else: return global_state.retval_long +fetch_retval_long.stackless_explicit = True def fetch_retval_longlong(): if global_state.exception: raise global_state.exception else: return global_state.retval_longlong +fetch_retval_longlong.stackless_explicit = True def fetch_retval_float(): if global_state.exception: raise global_state.exception else: return global_state.retval_float +fetch_retval_float.stackless_explicit = True def fetch_retval_void_p(): if global_state.exception: raise global_state.exception else: return global_state.retval_void_p - +fetch_retval_void_p.stackless_explicit = True Modified: pypy/dist/pypy/translator/stackless/test/test_depth.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_depth.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_depth.py Wed Apr 26 12:53:23 2006 @@ -1,6 +1,7 @@ from pypy.translator.stackless.test.test_transform import \ llinterp_stackless_function, run_stackless_function from pypy.translator.stackless import code +import py import os def test_simple(): @@ -23,13 +24,14 @@ count10 = f(10) return count10 - count0 - res = llinterp_stackless_function(fn, fn, f, g2, g1) + res = llinterp_stackless_function(fn) assert res == 10 - res = run_stackless_function(fn, fn, f, g2, g1) + res = run_stackless_function(fn) assert res.strip() == "10" def test_with_ptr(): + py.test.skip("in progress") def f(n): if n > 0: res = f(n-1) @@ -42,8 +44,8 @@ count10, _ = f(10) return count10 - count0 - res = llinterp_stackless_function(fn, fn, f) + res = llinterp_stackless_function(fn) assert res == 10 - res = run_stackless_function(fn, fn, f) + res = run_stackless_function(fn) assert res.strip() == "10" Modified: pypy/dist/pypy/translator/stackless/test/test_transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_transform.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_transform.py Wed Apr 26 12:53:23 2006 @@ -1,6 +1,6 @@ import py import os -from pypy.translator.stackless.transform import StacklessTransfomer +from pypy.translator.stackless.transform import StacklessTransformer from pypy.translator.c.genc import CStandaloneBuilder from pypy.translator.c import gc from pypy.rpython.memory.gctransform import varoftype @@ -38,12 +38,13 @@ def check(x): if x: raise code.UnwindException + check.stackless_explicit = True def g(x): check(x) return x + 1 def example(x): return g(x) + 1 - res = llinterp_stackless_function(example, example, g) + res = llinterp_stackless_function(example) assert res == 3 def test_simple_transform_llinterp_float(): @@ -55,10 +56,10 @@ return x + 0.125 def example(x): return int((g(x) + 1)*1000.0) - res = llinterp_stackless_function(example, example, g) + res = llinterp_stackless_function(example) assert res == 2125 -def test_simple_transform(): +def test_simple_transform_compiled(): def check(x): if x: raise code.UnwindException # XXX or so @@ -67,7 +68,7 @@ return x + 1 def example(x): return g(x) + 1 - res = run_stackless_function(example, example, g) + res = run_stackless_function(example) assert res.strip() == "3" def test_protected_call(): @@ -83,9 +84,9 @@ except Exception: y = -1 return y + 1 - res = llinterp_stackless_function(example, example, g) + res = llinterp_stackless_function(example) assert res == 3 - res = run_stackless_function(example, example, g) + res = run_stackless_function(example) assert res == "3" def test_resume_with_exception(): @@ -105,7 +106,7 @@ return y + 1 info = py.test.raises( llinterp.LLException, - "llinterp_stackless_function(example, example, g, h)") + "llinterp_stackless_function(example)") assert llinterp.type_name(info.value.args[0]) == 'KeyError' def test_resume_with_exception_handling(): @@ -126,14 +127,27 @@ except KeyError: y = -1 return y + 1 - res = llinterp_stackless_function(example, example, g, h) + res = llinterp_stackless_function(example) assert res == 0 -def rtype_stackless_function(fn, *stacklessfuncs): +def test_listcomp(): + def check(x): + if x: + raise code.UnwindException + check.stackless_explicit = True + def f(l): + check(l) + return len([x for x in range(l)]) + res = llinterp_stackless_function(f) + assert res == 1 + + +def rtype_stackless_function(fn): s_list_of_strings = annmodel.SomeList(ListDef(None, annmodel.SomeString())) s_list_of_strings.listdef.resize() t = TranslationContext() annotator = t.buildannotator() + annotator.policy.allow_someobjects = False bk = annotator.bookkeeper # we want to make sure that the annotator knows what # code.UnwindException looks like very early on, because otherwise @@ -151,16 +165,11 @@ raise Exception, "this probably isn't going to work" t.buildrtyper().specialize() - st = StacklessTransfomer(t) - for func in stacklessfuncs: - graph = graphof(t, func) - st.transform_graph(graph) - checkgraph(graph) if conftest.option.view: t.view() return t -def run_stackless_function(fn, *stacklessfuncs): +def run_stackless_function(fn): def entry_point(argv): try: r = fn(len(argv)) @@ -170,15 +179,20 @@ r = code.global_state.retval_long os.write(1, str(r)+'\n') return 0 + entry_point.stackless_explicit = True - t = rtype_stackless_function(entry_point, *stacklessfuncs) + t = rtype_stackless_function(entry_point) + + t.stacklesstransformer = StacklessTransformer(t) cbuilder = CStandaloneBuilder(t, entry_point) cbuilder.generate_source() + if conftest.option.view: + t.view() cbuilder.compile() return cbuilder.cmdexec('').strip() -def llinterp_stackless_function(fn, *stacklessfuncs): +def llinterp_stackless_function(fn): def entry_point(argv): try: r = fn(len(argv)) @@ -187,8 +201,13 @@ code.slp_main_loop() return code.global_state.retval_long return r + entry_point.stackless_explicit = True - t = rtype_stackless_function(entry_point, *stacklessfuncs) + t = rtype_stackless_function(entry_point) + st = StacklessTransformer(t) + st.transform_all() + if conftest.option.view: + t.view() r_list_of_strings = t.rtyper.getrepr( t.annotator.binding(graphof(t, entry_point).startblock.inputargs[0])) Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Wed Apr 26 12:53:23 2006 @@ -74,7 +74,7 @@ self.links_to_resumption = links_to_resumption self.frame_state_type = frame_state_type -class StacklessTransfomer(object): +class StacklessTransformer(object): def __init__(self, translator): self.translator = translator @@ -134,6 +134,7 @@ self.ll_global_state = model.Constant( r_global_state.convert_const(code.global_state), r_global_state.lowleveltype) + self.seen_blocks = set() def frame_type_for_vars(self, vars): @@ -158,14 +159,24 @@ self.frametypes[key] = T return T + def transform_all(self): + for graph in self.translator.graphs: + self.transform_graph(graph) + def transform_graph(self, graph): self.resume_points = [] + if hasattr(graph, 'func'): + if getattr(graph.func, 'stackless_explicit', False): + return + assert self.curr_graph is None self.curr_graph = graph for block in list(graph.iterblocks()): + assert block not in self.seen_blocks self.transform_block(block) + self.seen_blocks.add(block) if self.resume_points: self.insert_resume_handling(graph) @@ -295,9 +306,11 @@ [convertblock.inputargs[returnvarindex]], newvar)) convertblock.exits[0].args[returnvarindex] = newvar - self.translator.rtyper.insert_link_conversions(convertblock) +## self.translator.rtyper.insert_link_conversions( +## convertblock, convertlinks=False) - self.translator.rtyper.insert_link_conversions(newblock) +## self.translator.rtyper.insert_link_conversions( +## newblock, convertlinks=False) resuming_links.append( model.Link([], newblock, resume_point_index+1)) @@ -322,6 +335,9 @@ if i == len(block.operations) - 1 \ and block.exitswitch == model.c_last_exception: link = block.exits[0] + exitcases = dict.fromkeys(l.exitcase for l in block.exits) + if code.UnwindException in exitcases: + return else: link = support.split_block_with_keepalive(block, i+1) # this section deserves a whinge: @@ -335,7 +351,9 @@ # about annotations :( ann = self.translator.annotator for f, t in zip(link.args, link.target.inputargs): - ann.setbinding(t, ann.binding(f)) + nb = ann.binding(f, True) + if nb is not None: + ann.setbinding(t, nb) block.exitswitch = model.c_last_exception link.llexitcase = None var_unwind_exception = varoftype(evalue) @@ -366,7 +384,7 @@ args, var_unwind_exception) self.resume_points.append( - ResumePoint(op.result, args, block.exits, frame_state_type)) + ResumePoint(op.result, args, tuple(block.exits), frame_state_type)) newlink = model.Link(args + [var_unwind_exception], save_block, code.UnwindException) Modified: pypy/dist/pypy/translator/translator.py ============================================================================== --- pypy/dist/pypy/translator/translator.py (original) +++ pypy/dist/pypy/translator/translator.py Wed Apr 26 12:53:23 2006 @@ -30,6 +30,8 @@ raise TypeError("unexpected keyword argument") self.annotator = None self.rtyper = None + print 'NULLING', self + self.stacklesstransformer = None self.graphs = [] # [graph] self.callgraph = {} # {opaque_tag: (caller-graph, callee-graph)} self._prebuilt_graphs = {} # only used by the pygame viewer From cfbolz at codespeak.net Wed Apr 26 13:44:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 26 Apr 2006 13:44:18 +0200 (CEST) Subject: [pypy-svn] r26361 - pypy/dist/pypy/translator/c/test Message-ID: <20060426114418.75D8910084@code0.codespeak.net> Author: cfbolz Date: Wed Apr 26 13:44:17 2006 New Revision: 26361 Modified: pypy/dist/pypy/translator/c/test/test_newgc.py Log: (cfbolz, mwh proofreading): sketch armin's idea of how to make weakrefs work without explicit gc support. Seems to work. Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Wed Apr 26 13:44:17 2006 @@ -185,6 +185,94 @@ res = fn(1) assert res == 1 +from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.annotation import model as annmodel +from pypy.rpython import raddress +from pypy.rpython.lltypesystem.llmemory import NULL +import weakref + +def cast_object_to_address(obj): + pass + +def cast_address_to_object(address, expected_result): + pass + +class Entry(ExtRegistryEntry): + _about_ = cast_object_to_address + + def compute_result_annotation(self, s_obj): + return annmodel.SomeAddress() + + def specialize_call(self, hop): + vlist = hop.inputargs(hop.args_r[0]) + return hop.genop('cast_ptr_to_adr', vlist, + resulttype=hop.r_result.lowleveltype) + +class Entry(ExtRegistryEntry): + _about_ = cast_address_to_object + + def compute_result_annotation(self, s_int, s_clspbc): + assert len(s_clspbc.descriptions) == 1 + desc = s_clspbc.descriptions.keys()[0] + cdef = desc.getuniqueclassdef() + return annmodel.SomeInstance(cdef) + + def specialize_call(self, hop): + assert isinstance(hop.args_r[0], raddress.AddressRepr) + vlist = [hop.inputarg(raddress.address_repr, arg=0)] + return hop.genop('cast_adr_to_ptr', vlist, + resulttype = hop.r_result.lowleveltype) + +class Weakrefable(object): + __lifeline__ = None + +class Weakref(object): + def __init__(self, obj): + self.address = cast_object_to_address(obj) + + def ref(self): + return cast_address_to_object(self.address, Weakrefable) + + def invalidate(self): + self.address = NULL + +class WeakrefLifeline(object): + def __init__(self, obj): + self.ref = Weakref(obj) + + def __del__(self): + self.ref.invalidate() + + def get_weakref(self): + return self.ref + +def get_weakref(obj): + assert isinstance(obj, Weakrefable) + if obj.__lifeline__ is None: + obj.__lifeline__ = WeakrefLifeline(obj) + return obj.__lifeline__.get_weakref() + +def test_weakref_alive(): + def func(): + f = Weakrefable() + f.x = 32 + ref1 = get_weakref(f) + ref2 = get_weakref(f) + return f.x + ref2.ref().x + (ref1 is ref2) + f = compile_func(func, []) + assert f() == 65 + +def test_weakref_dying(): + def g(): + f = Weakrefable() + f.x = 32 + return get_weakref(f) + def func(): + ref = g() + return ref.ref() is None + f = compile_func(func, []) + assert f() + # _______________________________________________________________ # test framework From cfbolz at codespeak.net Wed Apr 26 13:52:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 26 Apr 2006 13:52:17 +0200 (CEST) Subject: [pypy-svn] r26363 - pypy/dist/pypy/translator/c/test Message-ID: <20060426115217.52A8910084@code0.codespeak.net> Author: cfbolz Date: Wed Apr 26 13:52:16 2006 New Revision: 26363 Modified: pypy/dist/pypy/translator/c/test/test_newgc.py Log: (mwh, cfbolz): make the cast operations work on top of CPython too Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Wed Apr 26 13:52:16 2006 @@ -188,14 +188,20 @@ from pypy.rpython.extregistry import ExtRegistryEntry from pypy.annotation import model as annmodel from pypy.rpython import raddress -from pypy.rpython.lltypesystem.llmemory import NULL +from pypy.rpython.lltypesystem.llmemory import NULL, fakeaddress import weakref def cast_object_to_address(obj): - pass + return fakeaddress(weakref.ref(obj)) def cast_address_to_object(address, expected_result): - pass + wref = address.ref().get() + if wref is None: # NULL address + return None + obj = wref() + assert obj is not None + assert isinstance(obj, expected_result) + return obj class Entry(ExtRegistryEntry): _about_ = cast_object_to_address @@ -259,6 +265,7 @@ ref1 = get_weakref(f) ref2 = get_weakref(f) return f.x + ref2.ref().x + (ref1 is ref2) + assert func() == 65 f = compile_func(func, []) assert f() == 65 @@ -270,6 +277,7 @@ def func(): ref = g() return ref.ref() is None + assert func() f = compile_func(func, []) assert f() From sanxiyn at codespeak.net Wed Apr 26 14:47:27 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 14:47:27 +0200 (CEST) Subject: [pypy-svn] r26364 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060426124727.7210410083@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 14:47:21 2006 New Revision: 26364 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: Changes to support passing SomeChar (length-1 string) to and fro Lisp. Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Wed Apr 26 14:47:21 2006 @@ -39,7 +39,7 @@ self.val = val def readlisp(s): - # Return bool/int/str or give up + # Return bool/int/char/str or give up lines = s.splitlines() lines = [ line for line in lines if line and not line.startswith(';') ] assert len(lines) == 1 @@ -49,6 +49,8 @@ return True elif s == "NIL": return False + elif s.startswith("#\\"): + return s[2:] elif s[0] == '"': return s[1:-1] elif s.isdigit(): Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Wed Apr 26 14:47:21 2006 @@ -34,10 +34,13 @@ elif val is None: return "nil" elif isinstance(val, str): - val.replace("\\", "\\\\") - val.replace("\"", "\\\"") - val = '"' + val + '"' - return val + if len(val) == 1: + return "#\%c" % (val,) + else: + val.replace("\\", "\\\\") + val.replace("\"", "\\\"") + val = '"' + val + '"' + return val else: return repr_unknown(val) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Wed Apr 26 14:47:21 2006 @@ -49,10 +49,10 @@ op_char_ne = make_binary_op("char/=") def op_cast_char_to_int(self, result, arg): - yield "(setf %s (char-code (char %s 0)))" % (result, arg) + yield "(setf %s (char-code %s))" % (result, arg) def op_cast_int_to_char(self, result, arg): - yield "(setf %s (string (code-char %s)))" % (result, arg) + yield "(setf %s (code-char %s))" % (result, arg) def op_cast_int_to_float(self, result, arg): yield "(setf %s (float %s))" % (result, arg) Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Wed Apr 26 14:47:21 2006 @@ -2,6 +2,7 @@ import py import os +from pypy.annotation.model import SomeChar from pypy.translator.cl.buildcl import make_cl_func from pypy.translator.cl.buildcl import Literal @@ -13,6 +14,18 @@ cl_return_str = make_cl_func(return_str) assert cl_return_str() == 'test' +def test_chr_ord(): + def chr_ord(num): + char = chr(num) + return ord(char) + cl_chr_ord = make_cl_func(chr_ord, [int]) + assert cl_chr_ord(32) == 32 + def ord_chr(char): + num = ord(char) + return chr(num) + cl_ord_chr = make_cl_func(ord_chr, [SomeChar()]) + assert cl_ord_chr('a') == 'a' + def test_if(): cl_if = make_cl_func(t.if_then_else, [bool, int, int]) assert cl_if(True, 50, 100) == 50 From mwh at codespeak.net Wed Apr 26 15:20:37 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 15:20:37 +0200 (CEST) Subject: [pypy-svn] r26365 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060426132037.63E2F10083@code0.codespeak.net> Author: mwh Date: Wed Apr 26 15:20:36 2006 New Revision: 26365 Modified: pypy/dist/pypy/translator/stackless/test/test_depth.py pypy/dist/pypy/translator/stackless/transform.py Log: (cfbolz, mwh) Make test_with_ptr work reliably. This was all caused by thinking that insert_empty_block produced a block whose .inputargs' and .exits[0].args' ordered matched up with the ordering of the .args of the link being split, which it doesn't. an alternative fix would be to make sure that it does, and that would remove an ouch! from the code. Modified: pypy/dist/pypy/translator/stackless/test/test_depth.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_depth.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_depth.py Wed Apr 26 15:20:36 2006 @@ -31,7 +31,6 @@ assert res.strip() == "10" def test_with_ptr(): - py.test.skip("in progress") def f(n): if n > 0: res = f(n-1) Modified: pypy/dist/pypy/translator/stackless/transform.py ============================================================================== --- pypy/dist/pypy/translator/stackless/transform.py (original) +++ pypy/dist/pypy/translator/stackless/transform.py Wed Apr 26 15:20:36 2006 @@ -297,20 +297,18 @@ newblock.exitswitch = None if need_address_conversion: - convertblock = unsimplify.insert_empty_block( - self.translator, newblock.exits[0]) - returnvarindex = newblock.exits[0].args.index(retval) newvar = unsimplify.copyvar(self.translator, resume_point.var_result) - convertblock.operations.append( - model.SpaceOperation("cast_adr_to_ptr", - [convertblock.inputargs[returnvarindex]], - newvar)) - convertblock.exits[0].args[returnvarindex] = newvar -## self.translator.rtyper.insert_link_conversions( -## convertblock, convertlinks=False) - -## self.translator.rtyper.insert_link_conversions( -## newblock, convertlinks=False) + newops = [model.SpaceOperation("cast_adr_to_ptr", + [retval], + newvar)] + convertblock = unsimplify.insert_empty_block( + self.translator, newblock.exits[0], newops) + # begin ouch! + index = newblock.exits[0].args.index(retval) + var = convertblock.inputargs[index] + index2 = convertblock.exits[0].args.index(var) + convertblock.exits[0].args[index2] = newvar + # end ouch! resuming_links.append( model.Link([], newblock, resume_point_index+1)) From mwh at codespeak.net Wed Apr 26 15:24:19 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 15:24:19 +0200 (CEST) Subject: [pypy-svn] r26366 - pypy/dist/pypy/translator Message-ID: <20060426132419.0A28F10083@code0.codespeak.net> Author: mwh Date: Wed Apr 26 15:24:19 2006 New Revision: 26366 Modified: pypy/dist/pypy/translator/translator.py Log: oops, that wasn't meant to get committed! Modified: pypy/dist/pypy/translator/translator.py ============================================================================== --- pypy/dist/pypy/translator/translator.py (original) +++ pypy/dist/pypy/translator/translator.py Wed Apr 26 15:24:19 2006 @@ -30,7 +30,6 @@ raise TypeError("unexpected keyword argument") self.annotator = None self.rtyper = None - print 'NULLING', self self.stacklesstransformer = None self.graphs = [] # [graph] self.callgraph = {} # {opaque_tag: (caller-graph, callee-graph)} From sanxiyn at codespeak.net Wed Apr 26 15:43:26 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 15:43:26 +0200 (CEST) Subject: [pypy-svn] r26367 - pypy/dist/pypy/translator/cl Message-ID: <20060426134326.31F5710083@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 15:43:22 2006 New Revision: 26367 Modified: pypy/dist/pypy/translator/cl/clrepr.py Log: Typo fix and stylistic changes to clrepr.py Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Wed Apr 26 15:43:22 2006 @@ -24,16 +24,16 @@ if isinstance(val, tuple): val = map(repr_const, val) return "'(%s)" % ' '.join(val) - elif isinstance(val, bool): # should precedes int + if isinstance(val, bool): # should precede int if val: return "t" else: return "nil" - elif isinstance(val, (int, long)): + if isinstance(val, (int, long)): return str(val) - elif val is None: + if val is None: return "nil" - elif isinstance(val, str): + if isinstance(val, str): if len(val) == 1: return "#\%c" % (val,) else: @@ -41,13 +41,11 @@ val.replace("\"", "\\\"") val = '"' + val + '"' return val - else: - return repr_unknown(val) + return repr_unknown(val) def repr_arg(arg): if isinstance(arg, Variable): return repr_var(arg) - elif isinstance(arg, Constant): + if isinstance(arg, Constant): return repr_const(arg.value) - else: - return repr_unknown(arg) + return repr_unknown(arg) From cfbolz at codespeak.net Wed Apr 26 15:45:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 26 Apr 2006 15:45:52 +0200 (CEST) Subject: [pypy-svn] r26368 - in pypy/dist/pypy/rpython: . test Message-ID: <20060426134552.0AA0110084@code0.codespeak.net> Author: cfbolz Date: Wed Apr 26 15:45:51 2006 New Revision: 26368 Modified: pypy/dist/pypy/rpython/objectmodel.py pypy/dist/pypy/rpython/test/test_objectmodel.py Log: (cfbolz, mwh glancing over shoulder): add cast_address_to_object and cast_object_to_address to objectmodel Modified: pypy/dist/pypy/rpython/objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/objectmodel.py (original) +++ pypy/dist/pypy/rpython/objectmodel.py Wed Apr 26 15:45:51 2006 @@ -56,6 +56,52 @@ obj.__dict__ = {} obj.__class__ = FREED_OBJECT +def cast_object_to_address(obj): + import weakref + from pypy.rpython.lltypesystem.llmemory import fakeaddress + return fakeaddress(weakref.ref(obj)) + +def cast_address_to_object(address, expected_result): + wref = address.ref().get() + if wref is None: # NULL address + return None + obj = wref() + assert obj is not None + assert isinstance(obj, expected_result) + return obj + +from pypy.rpython.extregistry import ExtRegistryEntry + +class Entry(ExtRegistryEntry): + _about_ = cast_object_to_address + + def compute_result_annotation(self, s_obj): + from pypy.annotation import model as annmodel + return annmodel.SomeAddress() + + def specialize_call(self, hop): + vlist = hop.inputargs(hop.args_r[0]) + return hop.genop('cast_ptr_to_adr', vlist, + resulttype=hop.r_result.lowleveltype) + +class Entry(ExtRegistryEntry): + _about_ = cast_address_to_object + + def compute_result_annotation(self, s_int, s_clspbc): + from pypy.annotation import model as annmodel + assert len(s_clspbc.descriptions) == 1 + desc = s_clspbc.descriptions.keys()[0] + cdef = desc.getuniqueclassdef() + return annmodel.SomeInstance(cdef) + + def specialize_call(self, hop): + from pypy.rpython import raddress + assert isinstance(hop.args_r[0], raddress.AddressRepr) + vlist = [hop.inputarg(raddress.address_repr, arg=0)] + return hop.genop('cast_adr_to_ptr', vlist, + resulttype = hop.r_result.lowleveltype) + + # __ hlinvoke XXX this doesn't seem completely the right place for this Modified: pypy/dist/pypy/rpython/test/test_objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_objectmodel.py (original) +++ pypy/dist/pypy/rpython/test/test_objectmodel.py Wed Apr 26 15:45:51 2006 @@ -69,6 +69,16 @@ assert d.keys() == [] return True # for the tests below +def test_cast_to_and_from_address(): + class A(object): + pass + class B(object): + pass + a = A() + addr = cast_object_to_address(a) + py.test.raises(AssertionError, "cast_address_to_object(addr, B)") + assert a is cast_address_to_object(addr, A) + def test_recursive_r_dict_repr(): import operator rdic = r_dict(operator.eq, hash) From cfbolz at codespeak.net Wed Apr 26 15:50:06 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 26 Apr 2006 15:50:06 +0200 (CEST) Subject: [pypy-svn] r26369 - pypy/dist/pypy/translator/c/test Message-ID: <20060426135006.E27B010088@code0.codespeak.net> Author: cfbolz Date: Wed Apr 26 15:50:06 2006 New Revision: 26369 Modified: pypy/dist/pypy/translator/c/test/test_newgc.py Log: make the test use the cast* implementations in objectmodel Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Wed Apr 26 15:50:06 2006 @@ -7,7 +7,7 @@ from pypy.translator.backendopt.stat import print_statistics from pypy.translator.c import genc, gc from pypy.rpython.lltypesystem import lltype - +from pypy.rpython.objectmodel import cast_address_to_object, cast_object_to_address from pypy.rpython.memory.gctransform import GCTransformer from pypy import conftest @@ -185,50 +185,6 @@ res = fn(1) assert res == 1 -from pypy.rpython.extregistry import ExtRegistryEntry -from pypy.annotation import model as annmodel -from pypy.rpython import raddress -from pypy.rpython.lltypesystem.llmemory import NULL, fakeaddress -import weakref - -def cast_object_to_address(obj): - return fakeaddress(weakref.ref(obj)) - -def cast_address_to_object(address, expected_result): - wref = address.ref().get() - if wref is None: # NULL address - return None - obj = wref() - assert obj is not None - assert isinstance(obj, expected_result) - return obj - -class Entry(ExtRegistryEntry): - _about_ = cast_object_to_address - - def compute_result_annotation(self, s_obj): - return annmodel.SomeAddress() - - def specialize_call(self, hop): - vlist = hop.inputargs(hop.args_r[0]) - return hop.genop('cast_ptr_to_adr', vlist, - resulttype=hop.r_result.lowleveltype) - -class Entry(ExtRegistryEntry): - _about_ = cast_address_to_object - - def compute_result_annotation(self, s_int, s_clspbc): - assert len(s_clspbc.descriptions) == 1 - desc = s_clspbc.descriptions.keys()[0] - cdef = desc.getuniqueclassdef() - return annmodel.SomeInstance(cdef) - - def specialize_call(self, hop): - assert isinstance(hop.args_r[0], raddress.AddressRepr) - vlist = [hop.inputarg(raddress.address_repr, arg=0)] - return hop.genop('cast_adr_to_ptr', vlist, - resulttype = hop.r_result.lowleveltype) - class Weakrefable(object): __lifeline__ = None @@ -240,6 +196,7 @@ return cast_address_to_object(self.address, Weakrefable) def invalidate(self): + from pypy.rpython.lltypesystem.llmemory import NULL self.address = NULL class WeakrefLifeline(object): From sanxiyn at codespeak.net Wed Apr 26 15:57:41 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 15:57:41 +0200 (CEST) Subject: [pypy-svn] r26370 - pypy/dist/pypy/translator/cl Message-ID: <20060426135741.1E2F41008A@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 15:57:35 2006 New Revision: 26370 Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py Log: Represent Instance as Lisp symbol Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Wed Apr 26 15:57:35 2006 @@ -1,7 +1,7 @@ import types from pypy.objspace.flow.model import Constant, Variable -from pypy.rpython.ootypesystem.ootype import _static_meth +from pypy.rpython.ootypesystem.ootype import Instance, _static_meth def repr_unknown(obj): return '#<%r>' % (obj,) @@ -16,6 +16,8 @@ return name.replace('_', '-') def repr_const(val): + if isinstance(val, Instance): + return "'" + repr_class_name(val._name) if isinstance(val, types.FunctionType): if val.func_name == 'dum_nocheck': # XXX return "'dummy" Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Wed Apr 26 15:57:35 2006 @@ -83,18 +83,16 @@ if class_declaration not in self.gen.declarations: self.gen.declarations.append(class_declaration) - def op_new(self, result, _): + def op_new(self, result, clsname): cls = self.args[0].value if isinstance(cls, List): yield "(setf %s (make-array 0 :adjustable t))" % (result,) else: self.declare_class(cls) - clsname = repr_class_name(cls._name) - yield "(setf %s (make-instance '%s))" % (result, clsname) + yield "(setf %s (make-instance %s))" % (result, clsname) - def op_instanceof(self, result, arg, _): - clsname = repr_class_name(self.args[1].value._name) - yield "(setf %s (typep %s '%s))" % (result, arg, clsname) + def op_instanceof(self, result, arg, clsname): + yield "(setf %s (typep %s %s))" % (result, arg, clsname) def op_oosend(self, result, *ignore): method = self.args[0].value From cfbolz at codespeak.net Wed Apr 26 16:45:05 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 26 Apr 2006 16:45:05 +0200 (CEST) Subject: [pypy-svn] r26371 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060426144505.63DB810087@code0.codespeak.net> Author: cfbolz Date: Wed Apr 26 16:45:04 2006 New Revision: 26371 Added: pypy/dist/pypy/module/_weakref/ (props changed) pypy/dist/pypy/module/_weakref/__init__.py (contents, props changed) pypy/dist/pypy/module/_weakref/app__weakref.py (contents, props changed) pypy/dist/pypy/module/_weakref/interp__weakref.py (contents, props changed) pypy/dist/pypy/module/_weakref/test/ (props changed) pypy/dist/pypy/module/_weakref/test/test_weakref.py (contents, props changed) Log: (cfbolz, mwh): first attempt of writing a _weakref module. it's a bit evil, because it attaches a __lifeline__ = None attribute to W_Root. Probably doesn't translate, but who knows :-). Added: pypy/dist/pypy/module/_weakref/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_weakref/__init__.py Wed Apr 26 16:45:04 2006 @@ -0,0 +1,8 @@ +from pypy.interpreter.mixedmodule import MixedModule + +class Module(MixedModule): + appleveldefs = { + } + interpleveldefs = { + 'ref': 'interp__weakref.W_Weakref' + } Added: pypy/dist/pypy/module/_weakref/app__weakref.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_weakref/app__weakref.py Wed Apr 26 16:45:04 2006 @@ -0,0 +1,2 @@ + + Added: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Wed Apr 26 16:45:04 2006 @@ -0,0 +1,57 @@ +from pypy.interpreter.baseobjspace import Wrappable, W_Root +from pypy.interpreter.argument import Arguments +from pypy.interpreter.error import OperationError +from pypy.interpreter.typedef import GetSetProperty, TypeDef +from pypy.interpreter.gateway import interp2app +from pypy.rpython.objectmodel import cast_address_to_object, cast_object_to_address +class W_Weakref(Wrappable): + pass + +W_Weakrefable = W_Root +W_Weakrefable.__lifeline__ = None + +class W_Weakref(Wrappable): + def __init__(w_self, space, w_obj, w_callable): + w_self.space = space + w_self.address = cast_object_to_address(w_obj) + w_self.w_callable = w_callable + + def descr__call__(self): + return cast_address_to_object(self.address, W_Weakrefable) + + def invalidate(w_self): + from pypy.rpython.lltypesystem.llmemory import NULL + import os + w_self.address = NULL + if not w_self.space.is_w(w_self.w_callable, w_self.space.w_None): + try: + w_self.space.call_function(w_self.w_callable, w_self) + except OperationError, e: + print e + os.write(2, "XXX\n") + +class WeakrefLifeline(object): + def __init__(self): + self.refs_w = [] + + def __del__(self): + for w_ref in self.refs_w: + w_ref.invalidate() + + def get_weakref(self, space, w_subtype, w_obj, w_callable): + w_ref = space.allocate_instance(W_Weakref, w_subtype) + W_Weakref.__init__(w_ref, space, w_obj, w_callable) + self.refs_w.append(w_ref) + return w_ref + +def descr__new__(space, w_subtype, w_obj, w_callable=None): + assert isinstance(w_obj, W_Weakrefable) + if w_obj.__lifeline__ is None: + w_obj.__lifeline__ = WeakrefLifeline() + return w_obj.__lifeline__.get_weakref(space, w_subtype, w_obj, w_callable) + +W_Weakref.typedef = TypeDef("weakref", + __new__ = interp2app(descr__new__), + __call__ = interp2app(W_Weakref.descr__call__, unwrap_spec=['self']) +) + Added: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Wed Apr 26 16:45:04 2006 @@ -0,0 +1,18 @@ +from pypy.conftest import gettestobjspace + +class AppTestWeakref(object): + def setup_class(cls): + space = gettestobjspace(usemodules=('_weakref',)) + cls.space = space + + def test_simple(self): + import _weakref + class A: + pass + a = A() + ref = _weakref.ref(a) + print ref() + assert ref() is a + del a + print ref() + assert ref() is None From sanxiyn at codespeak.net Wed Apr 26 16:52:35 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 16:52:35 +0200 (CEST) Subject: [pypy-svn] r26372 - pypy/dist/pypy/translator/cl Message-ID: <20060426145235.CAC7010083@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 16:52:28 2006 New Revision: 26372 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Translate ootypesystem's Record to Lisp struct Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Wed Apr 26 16:52:28 2006 @@ -2,7 +2,7 @@ from pypy.tool.udir import udir from pypy.translator.translator import graphof -from pypy.rpython.ootypesystem.ootype import Instance, List, _static_meth, _meth, ROOT +from pypy.rpython.ootypesystem.ootype import List, Record, Instance, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name @@ -69,8 +69,18 @@ args = " ".join(args) yield "(setf %s (%s %s))" % (result, fun, args) + def declare_struct(self, cls): + # cls is Record + name = "struct" + str(self.gen.structcount) + field_declaration = cls._fields.keys() + field_declaration = " ".join(field_declaration) + struct_declaration = "(defstruct %s %s)" % (name, field_declaration) + self.gen.declarations.append(struct_declaration) + self.gen.structcount += 1 + return name + def declare_class(self, cls): - # cls is really type of Instance + # cls is Instance name = repr_class_name(cls._name) field_declaration = ['('+field+')' for field in cls._fields] field_declaration = " ".join(field_declaration) @@ -87,9 +97,14 @@ cls = self.args[0].value if isinstance(cls, List): yield "(setf %s (make-array 0 :adjustable t))" % (result,) - else: + elif isinstance(cls, Record): + clsname = self.declare_struct(cls) + yield "(setf %s (make-%s))" % (result, clsname) + elif isinstance(cls, Instance): self.declare_class(cls) yield "(setf %s (make-instance %s))" % (result, clsname) + else: + raise NotImplementedError() def op_instanceof(self, result, arg, clsname): yield "(setf %s (typep %s %s))" % (result, arg, clsname) @@ -162,6 +177,7 @@ self.entry_point = funobj self.pendinggraphs = [funobj] self.declarations = [] + self.structcount = 0 def emitfile(self): name = self.entry_point.func_name From sanxiyn at codespeak.net Wed Apr 26 16:53:40 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 16:53:40 +0200 (CEST) Subject: [pypy-svn] r26373 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060426145340.BCD2B10084@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 16:53:33 2006 New Revision: 26373 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: Implement missing ops and pass the range test Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Wed Apr 26 16:53:33 2006 @@ -35,8 +35,10 @@ return _ op_int_add = make_binary_op("+") + op_int_mul = make_binary_op("*") op_int_eq = make_binary_op("=") op_int_gt = make_binary_op(">") + op_int_ge = make_binary_op(">=") op_int_lt = make_binary_op("<") op_int_le = make_binary_op("<=") op_int_and = make_binary_op("logand") Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Wed Apr 26 16:53:33 2006 @@ -26,6 +26,13 @@ cl_ord_chr = make_cl_func(ord_chr, [SomeChar()]) assert cl_ord_chr('a') == 'a' +def test_range(): + def get_three(): + lst = range(7) + return lst[3] + cl_get_three = make_cl_func(get_three) + assert cl_get_three() == 3 + def test_if(): cl_if = make_cl_func(t.if_then_else, [bool, int, int]) assert cl_if(True, 50, 100) == 50 From sanxiyn at codespeak.net Wed Apr 26 17:22:28 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 17:22:28 +0200 (CEST) Subject: [pypy-svn] r26374 - pypy/dist/pypy/translator/cl Message-ID: <20060426152228.2CF1C10083@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 17:22:25 2006 New Revision: 26374 Modified: pypy/dist/pypy/translator/cl/clrepr.py Log: Representation for List, Record, CLASSTYPE. Now raise error instead of fallback to repr_unknown. Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Wed Apr 26 17:22:25 2006 @@ -1,10 +1,12 @@ import types from pypy.objspace.flow.model import Constant, Variable -from pypy.rpython.ootypesystem.ootype import Instance, _static_meth +from pypy.rpython.ootypesystem.ootype import List, Record, Instance, instance_impl, _static_meth +from pypy.rpython.ootypesystem.rclass import CLASSTYPE def repr_unknown(obj): - return '#<%r>' % (obj,) + name = obj.__class__.__name__ + raise NotImplementedError("cannot represent %s" % (name,)) def repr_var(var): return var.name @@ -16,8 +18,15 @@ return name.replace('_', '-') def repr_const(val): + if isinstance(val, List): + return "'array" + if isinstance(val, Record): + return "'struct" # XXX if isinstance(val, Instance): return "'" + repr_class_name(val._name) + if isinstance(val, instance_impl): + if val._TYPE is CLASSTYPE: + return "'standard-class" if isinstance(val, types.FunctionType): if val.func_name == 'dum_nocheck': # XXX return "'dummy" From sanxiyn at codespeak.net Wed Apr 26 17:26:33 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 17:26:33 +0200 (CEST) Subject: [pypy-svn] r26375 - pypy/dist/pypy/translator/cl/test Message-ID: <20060426152633.A158A10083@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 17:26:30 2006 New Revision: 26375 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py Log: Add a test (function pointer) failing with HalfConcreteWrapper Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py Wed Apr 26 17:26:30 2006 @@ -1,3 +1,4 @@ +import py from pypy.translator.cl.buildcl import make_cl_func def test_call(): @@ -8,3 +9,23 @@ return n + 1 cl_add_one = make_cl_func(add_one, [int]) assert cl_add_one(1) == 2 + +def test_indirect_call(): + py.test.skip("fails with HalfConcreteWrapper") + def id(n): + return n + def square(n): + return n * n + def map_sum(func, n): + sum = 0 + for i in range(1, n+1): + sum += func(i) + return sum + def sum(n): + return map_sum(id, n) + def square_sum(n): + return map_sum(square, n) + cl_sum = make_cl_func(sum, [int]) + assert cl_sum(5) == 15 + cl_square_sum = make_cl_func(square_sum, [int]) + assert cl_square_sum(5) == 55 From sanxiyn at codespeak.net Wed Apr 26 17:44:21 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 17:44:21 +0200 (CEST) Subject: [pypy-svn] r26376 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060426154421.885A610084@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 17:44:15 2006 New Revision: 26376 Added: pypy/dist/pypy/translator/cl/test/test_clrepr.py - copied, changed from r26374, pypy/dist/pypy/translator/cl/test/test_buildcl.py Removed: pypy/dist/pypy/translator/cl/test/test_buildcl.py Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py Log: Get rid of writelisp() and merge with repr_const() Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Wed Apr 26 17:44:15 2006 @@ -58,18 +58,6 @@ else: return Literal(s) -def writelisp(obj): - if isinstance(obj, (bool, int, type(None), str)): - return repr_const(obj) - if isinstance(obj, (tuple, list)): - content = ' '.join([writelisp(elt) for elt in obj]) - content = '(' + content + ')' - if isinstance(obj, list): - content = '#' + content - elif isinstance(obj, tuple): - content = "'" + content # quote Lisp list - return content - def make_cl_func(func, argtypes=[]): global global_cl if global_cl is None: @@ -102,7 +90,7 @@ fp = file(str(fpath), "a") print >>fp, "(write (", repr_fun_name(func.func_name), for arg in args: - print >>fp, writelisp(arg), + print >>fp, repr_const(arg), print >>fp, "))" fp.close() if conftest.option.prettyprint: Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Wed Apr 26 17:44:15 2006 @@ -35,6 +35,9 @@ if isinstance(val, tuple): val = map(repr_const, val) return "'(%s)" % ' '.join(val) + if isinstance(val, list): + val = map(repr_const, val) + return "#(%s)" % ' '.join(val) if isinstance(val, bool): # should precede int if val: return "t" From sanxiyn at codespeak.net Wed Apr 26 17:49:05 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Wed, 26 Apr 2006 17:49:05 +0200 (CEST) Subject: [pypy-svn] r26377 - pypy/dist/pypy/translator/cl/test Message-ID: <20060426154905.6E0CB10084@code0.codespeak.net> Author: sanxiyn Date: Wed Apr 26 17:49:02 2006 New Revision: 26377 Modified: pypy/dist/pypy/translator/cl/test/test_clrepr.py Log: Add a test for SomeChar representation Modified: pypy/dist/pypy/translator/cl/test/test_clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_clrepr.py (original) +++ pypy/dist/pypy/translator/cl/test/test_clrepr.py Wed Apr 26 17:49:02 2006 @@ -5,6 +5,7 @@ assert repr_const(False) == 'nil' assert repr_const(42) == '42' assert repr_const(None) == 'nil' + assert repr_const('a') == '#\\a' assert repr_const('answer') == '"answer"' assert repr_const((2, 3)) == "'(2 3)" assert repr_const([2, 3]) == "#(2 3)" From arigo at codespeak.net Wed Apr 26 19:49:41 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 26 Apr 2006 19:49:41 +0200 (CEST) Subject: [pypy-svn] r26380 - pypy/dist/pypy/translator/cl Message-ID: <20060426174941.E77CA10083@code0.codespeak.net> Author: arigo Date: Wed Apr 26 19:49:41 2006 New Revision: 26380 Modified: pypy/dist/pypy/translator/cl/buildcl.py Log: Import the prettyprint option from the local conftest. Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Wed Apr 26 19:49:41 2006 @@ -6,6 +6,7 @@ from pypy.translator.cl.gencl import GenCL from pypy.translator.cl.clrepr import repr_const, repr_fun_name from pypy import conftest +from pypy.translator.cl import conftest as clconftest global_cl = None @@ -79,7 +80,7 @@ out = generate_cl_func(func, argtypes) fpath = path.join("%s.lisp" % func.func_name) - if conftest.option.prettyprint: + if clconftest.option.prettyprint: script = path.join(".printer.lisp") fp = file(str(script), "w") fp.write(pretty_printer % (fpath,)) @@ -93,7 +94,7 @@ print >>fp, repr_const(arg), print >>fp, "))" fp.close() - if conftest.option.prettyprint: + if clconftest.option.prettyprint: py.process.cmdexec("%s %s" % (cl, str(script))) output = py.process.cmdexec("%s %s" % (cl, str(fpath))) return readlisp(output) From arigo at codespeak.net Wed Apr 26 21:31:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 26 Apr 2006 21:31:05 +0200 (CEST) Subject: [pypy-svn] r26383 - in pypy/dist/pypy/translator: . backendopt/test Message-ID: <20060426193105.0B70F1008B@code0.codespeak.net> Author: arigo Date: Wed Apr 26 21:31:03 2006 New Revision: 26383 Modified: pypy/dist/pypy/translator/backendopt/test/test_all.py pypy/dist/pypy/translator/simplify.py Log: Bugfix in the caller of replace_exitswitch_by_constant(). Caused inlining to produce broken code in some cases. Modified: pypy/dist/pypy/translator/backendopt/test/test_all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_all.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_all.py Wed Apr 26 21:31:03 2006 @@ -150,3 +150,20 @@ digest3 = md5digest(t) assert digest1 == digest3 + +def test_bug_inlined_if(): + def f(x, flag): + if flag: + y = x + else: + y = x+1 + return y*5 + def myfunc(x): + return f(x, False) - f(x, True) + + assert myfunc(10) == 5 + + t = translateopt(myfunc, [int], inline_threshold=100) + interp = LLInterpreter(t.rtyper) + res = interp.eval_graph(graphof(t, myfunc), [10]) + assert res == 5 Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Wed Apr 26 21:31:03 2006 @@ -48,6 +48,7 @@ newexits[0].llexitcase = None block.exitswitch = None block.recloseblock(*newexits) + return newexits # ____________________________________________________________ @@ -341,7 +342,8 @@ link.prevblock.exitswitch = newexitswitch link.prevblock.recloseblock(*exits) if isinstance(newexitswitch, Constant) and newexitswitch != c_last_exception: - replace_exitswitch_by_constant(link.prevblock, newexitswitch) + exits = replace_exitswitch_by_constant(link.prevblock, + newexitswitch) stack.extend(exits) else: if link.target not in seen: From cfbolz at codespeak.net Wed Apr 26 22:23:42 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 26 Apr 2006 22:23:42 +0200 (CEST) Subject: [pypy-svn] r26385 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060426202342.D005710089@code0.codespeak.net> Author: cfbolz Date: Wed Apr 26 22:23:40 2006 New Revision: 26385 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: test weakref with callbacks. call callbacks from newest to oldest. Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Wed Apr 26 22:23:40 2006 @@ -27,7 +27,6 @@ try: w_self.space.call_function(w_self.w_callable, w_self) except OperationError, e: - print e os.write(2, "XXX\n") class WeakrefLifeline(object): @@ -35,7 +34,8 @@ self.refs_w = [] def __del__(self): - for w_ref in self.refs_w: + for i in range(len(self.refs_w) - 1, -1, -1): + w_ref = self.refs_w[i] w_ref.invalidate() def get_weakref(self, space, w_subtype, w_obj, w_callable): Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Wed Apr 26 22:23:40 2006 @@ -11,8 +11,36 @@ pass a = A() ref = _weakref.ref(a) - print ref() assert ref() is a del a - print ref() assert ref() is None + + def test_callback(self): + import _weakref + class A: + pass + a1 = A() + a2 = A() + def callback(ref): + a2.ref = ref() + ref1 = _weakref.ref(a1, callback) + ref2 = _weakref.ref(a1) + del a1 + assert ref1() is None + assert a2.ref is None + + def test_callback_order(self): + import _weakref + class A: + pass + a1 = A() + a2 = A() + def callback1(ref): + a2.x = 42 + def callback2(ref): + a2.x = 43 + ref1 = _weakref.ref(a1, callback1) + ref2 = _weakref.ref(a1, callback2) + del a1 + assert a2.x == 42 + From mwh at codespeak.net Wed Apr 26 22:48:17 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 26 Apr 2006 22:48:17 +0200 (CEST) Subject: [pypy-svn] r26386 - in pypy/dist/pypy/translator/stackless: . test Message-ID: <20060426204817.E082A10089@code0.codespeak.net> Author: mwh Date: Wed Apr 26 22:48:16 2006 New Revision: 26386 Modified: pypy/dist/pypy/translator/stackless/code.py pypy/dist/pypy/translator/stackless/test/test_depth.py Log: two more stackless tests ported from c/test/test_stackless and one consequent bugfix to slp_main_loop. Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Wed Apr 26 22:48:16 2006 @@ -118,7 +118,8 @@ try: call_function(fn, signature) except UnwindException, u: #XXX annotation support needed - nextframe = u.frame_top + u.frame_bottom.f_back = nextframe + nextframe = u.frame_top except Exception, e: global_state.exception = e else: Modified: pypy/dist/pypy/translator/stackless/test/test_depth.py ============================================================================== --- pypy/dist/pypy/translator/stackless/test/test_depth.py (original) +++ pypy/dist/pypy/translator/stackless/test/test_depth.py Wed Apr 26 22:48:16 2006 @@ -48,3 +48,43 @@ res = run_stackless_function(fn) assert res.strip() == "10" + +def test_manytimes(): + def f(n): + if n > 0: + res = f(n-1) + else: + res = code.stack_frames_depth(), 1 + return res + + def fn(ignored): + count0, _ = f(0) + count10, _ = f(100) + return count10 - count0 + + res = llinterp_stackless_function(fn) + assert res == 100 + + res = run_stackless_function(fn) + assert res.strip() == "100" + +def test_arguments(): + def f(n, d, t): + if n > 0: + res = f(n-1, d, t) + else: + res = code.stack_frames_depth(), d, t + return res + + def fn(ignored): + count0, d, t = f(0, 5.5, (1, 2)) + count10, d, t = f(10, 5.5, (1, 2)) + return count10 - count0 + int(d) + + res = llinterp_stackless_function(fn) + assert res == 15 + + res = run_stackless_function(fn) + assert res.strip() == "15" + + From dialtone at codespeak.net Thu Apr 27 03:37:54 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Thu, 27 Apr 2006 03:37:54 +0200 (CEST) Subject: [pypy-svn] r26387 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060427013754.2A0A71008B@code0.codespeak.net> Author: dialtone Date: Thu Apr 27 03:37:50 2006 New Revision: 26387 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Planning for today 27th Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Thu Apr 27 03:37:50 2006 @@ -130,5 +130,47 @@ - compile pystone in gencl (Valentino, Sanghyeon) - - Coneptual design decisions about ootypesystem (tuples list...) (Samuele, Nik) + - Conceptual design decisions about ootypesystem (tuples list...) (Samuele, Nik) +Thursday, 27th April 2006 +************************ + +Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Nik, Anders + +Status after day 3: + + - gencl (Sanghyeon, Valentino) + * refactored another bit + * superclasses and inheritance and isinstance and range + * Record type + + - Pickle support for cell objects (Eric) + * works but breaks translation + + - OOTypeSystem (Nik, Samuele) + * Improved forward references and made the List type lazily inittable. + * Introduced the Record type + * Introduced Range support in the OOTypeSystem. + + - ctypes _socket (and select ;)) (Arre, Anders) + * Passing 14 tests for the translatable module + * It should be translatable now (although not tested) + +Pairings today: + + - Pickling cells (Eric) + * fix translation + * add support for additional objects like code objects. + + - ctypes _socket (Arre, Anders) + * Pass all previous tests. + * start proper translation work. + + - OOTypeSystem (Nik, Sanghyeon) + * Dictionaries need to be lazy too. + * Record type for iterator and tuples and make it generic + * String type + + - progress further on rpystone gencl translation (Valentino) + + - general wizardry (Samuele) From ericvrp at codespeak.net Thu Apr 27 04:48:55 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 27 Apr 2006 04:48:55 +0200 (CEST) Subject: [pypy-svn] r26388 - pypy/dist/pypy/interpreter Message-ID: <20060427024855.0CDE11008F@code0.codespeak.net> Author: ericvrp Date: Thu Apr 27 04:48:54 2006 New Revision: 26388 Modified: pypy/dist/pypy/interpreter/baseobjspace.py pypy/dist/pypy/interpreter/nestedscope.py pypy/dist/pypy/interpreter/typedef.py Log: (pedronis, ericvrp) Fix for annotation problem we had with _pickle_support mixedmodule. Modified: pypy/dist/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/dist/pypy/interpreter/baseobjspace.py (original) +++ pypy/dist/pypy/interpreter/baseobjspace.py Thu Apr 27 04:48:54 2006 @@ -193,7 +193,7 @@ modules.extend(['unicodedata', '_codecs', 'array', 'marshal', 'errno', 'math', '_sre']) - #modules.append('_pickle_support') + modules.append('_pickle_support') if self.options.nofaking: modules.append('posix') Modified: pypy/dist/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/dist/pypy/interpreter/nestedscope.py (original) +++ pypy/dist/pypy/interpreter/nestedscope.py Thu Apr 27 04:48:54 2006 @@ -2,6 +2,7 @@ from pypy.interpreter.pyopcode import PyInterpFrame from pypy.interpreter import function, pycode, pyframe from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.mixedmodule import MixedModule class Cell(Wrappable): "A simple container for a wrapped value." @@ -35,7 +36,9 @@ return space.eq(self.w_value, other.w_value) def descr__reduce__(self, space): - cell_new = space.getbuiltinmodule('_pickle_support').get('cell_new') + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + cell_new = mod.get('cell_new') if self.w_value is None: #when would this happen? return space.newtuple([cell_new, space.newtuple([])]) return space.newtuple([cell_new, space.newtuple([]), Modified: pypy/dist/pypy/interpreter/typedef.py ============================================================================== --- pypy/dist/pypy/interpreter/typedef.py (original) +++ pypy/dist/pypy/interpreter/typedef.py Thu Apr 27 04:48:54 2006 @@ -543,10 +543,10 @@ __eq__ = interp2app(Cell.descr__eq__, unwrap_spec=['self', ObjSpace, W_Root]), __ne__ = descr_generic_ne, -## __reduce__ = interp2app(Cell.descr__reduce__, -## unwrap_spec=['self', ObjSpace]), -## __setstate__ = interp2app(Cell.descr__setstate__, -## unwrap_spec=['self', ObjSpace, W_Root]), + __reduce__ = interp2app(Cell.descr__reduce__, + unwrap_spec=['self', ObjSpace]), + __setstate__ = interp2app(Cell.descr__setstate__, + unwrap_spec=['self', ObjSpace, W_Root]), ) Ellipsis.typedef = TypeDef("Ellipsis", From sanxiyn at codespeak.net Thu Apr 27 04:49:47 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 04:49:47 +0200 (CEST) Subject: [pypy-svn] r26389 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060427024947.AD9451008F@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 04:49:43 2006 New Revision: 26389 Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py Log: Handle HalfConcreteWrapper and Atom. Needs exception handling to pass sqaure_sum test... Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Thu Apr 27 04:49:43 2006 @@ -1,6 +1,7 @@ import types -from pypy.objspace.flow.model import Constant, Variable +from pypy.objspace.flow.model import Constant, Variable, Atom +from pypy.rpython.rmodel import HalfConcreteWrapper from pypy.rpython.ootypesystem.ootype import List, Record, Instance, instance_impl, _static_meth from pypy.rpython.ootypesystem.rclass import CLASSTYPE @@ -10,7 +11,10 @@ def repr_var(var): return var.name - + +def repr_atom(atom): + return "'" + str(atom) + def repr_class_name(name): return name.replace('_', '-') @@ -18,6 +22,11 @@ return name.replace('_', '-') def repr_const(val): + if isinstance(val, HalfConcreteWrapper): + val = val.concretize().value + return repr_const(val) + if isinstance(val, Atom): + return repr_atom(val) if isinstance(val, List): return "'array" if isinstance(val, Record): Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py Thu Apr 27 04:49:43 2006 @@ -11,7 +11,7 @@ assert cl_add_one(1) == 2 def test_indirect_call(): - py.test.skip("fails with HalfConcreteWrapper") + py.test.skip("needs exception handling") def id(n): return n def square(n): From sanxiyn at codespeak.net Thu Apr 27 04:51:57 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 04:51:57 +0200 (CEST) Subject: [pypy-svn] r26390 - pypy/dist/pypy/translator/cl/test Message-ID: <20060427025157.7062810091@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 04:51:54 2006 New Revision: 26390 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (nik, dialtone, sanxiyn) Add a failing class variable test Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Thu Apr 27 04:51:54 2006 @@ -1,3 +1,4 @@ +import py from pypy.translator.cl.buildcl import make_cl_func, generate_cl_func def test_simple(): @@ -52,6 +53,26 @@ cl_check_isinstance = make_cl_func(check_isinstance, [bool]) assert cl_check_isinstance(True) == True +def test_class(): + py.test.skip("TODO") + class Foo: + value = 0 + class Bar(Foo): + value = 1 + class Baz(Foo): + value = 2 + def pick_class(flag): + if flag: + return Bar + else: + return Baz + def dynamic_class(flag): + cls = pick_class(flag) + return cls.value + cl_dynamic_class = make_cl_func(dynamic_class, [bool]) + assert cl_dynamic_class(True) == 1 + assert cl_dynamic_class(False) == 2 + def test_list_length(): def list_length_one(number): lst = [number] From nik at codespeak.net Thu Apr 27 05:02:02 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 05:02:02 +0200 (CEST) Subject: [pypy-svn] r26391 - in pypy/dist/pypy/translator: cl/test test Message-ID: <20060427030202.3FCC41009D@code0.codespeak.net> Author: nik Date: Thu Apr 27 05:01:57 2006 New Revision: 26391 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py pypy/dist/pypy/translator/test/snippet.py Log: (sanxiyn, nik) fixed test snippet is_one_or_two to use lists instead of tuples. contains on tuples doesn't work right not with ootypesystem because rdict is incomplete. Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Thu Apr 27 05:01:57 2006 @@ -58,7 +58,6 @@ assert cl_bool(True) == True def test_contains(): - py.test.skip("fails for obscure dict-related reasons") def contains_int(num): return t.my_contains([1,2,3], num) my_contains = make_cl_func(contains_int, [int]) Modified: pypy/dist/pypy/translator/test/snippet.py ============================================================================== --- pypy/dist/pypy/translator/test/snippet.py (original) +++ pypy/dist/pypy/translator/test/snippet.py Thu Apr 27 05:01:57 2006 @@ -41,7 +41,7 @@ return elem in seq def is_one_or_two(n=int): - return n in (1, 2) + return n in [1, 2] def two_plus_two(): """Array test""" From nik at codespeak.net Thu Apr 27 05:29:46 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 05:29:46 +0200 (CEST) Subject: [pypy-svn] r26392 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427032946.BCE071009D@code0.codespeak.net> Author: nik Date: Thu Apr 27 05:29:42 2006 New Revision: 26392 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: have a more informative string representation for Records. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Thu Apr 27 05:29:42 2006 @@ -225,6 +225,11 @@ def _lookup(self, meth_name): return self, None + def __str__(self): + item_str = ["%s: %s" % (str(name), str(ITEMTYPE)) + for name, (ITEMTYPE, _) in self._fields.items()] + return '%s(%s)' % (self.__class__.__name__, ", ".join(item_str)) + class BuiltinADTType(BuiltinType): def _setup_methods(self, generic_types): From ac at codespeak.net Thu Apr 27 05:34:55 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 27 Apr 2006 05:34:55 +0200 (CEST) Subject: [pypy-svn] r26393 - in pypy/dist/pypy: module/_socket module/_socket/test rpython/rctypes/socketmodule Message-ID: <20060427033455.C5F56100A0@code0.codespeak.net> Author: ac Date: Thu Apr 27 05:34:55 2006 New Revision: 26393 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/module/_socket/test/test_socket2.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (aleale, arre) The tests of module/_socket now pass again (if you don't enable _socket). Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Thu Apr 27 05:34:55 2006 @@ -215,13 +215,13 @@ The optional protocol name, if given, should be 'tcp' or 'udp', otherwise any protocol will match. """ - try: - if w_proto is None: - return space.wrap(socket.getservbyname(name)) - else: - return space.wrap(socket.getservbyname(name, space.str_w(w_proto))) - except socket.error, e: - raise wrap_socketerror(space, e) + proto = None + if w_proto: + proto = space.str_w(w_proto) + servent_ptr = _c.getservbyname(name, proto) + if not servent_ptr: + raise w_get_socketerror("service/port not found") + return space.wrap(_c.ntohs(servent_ptr.contents.s_port)) getservbyname.unwrap_spec = [ObjSpace, str, W_Root] def getservbyport(space, port, w_proto=NoneNotWrapped): @@ -231,13 +231,13 @@ The optional protocol name, if given, should be 'tcp' or 'udp', otherwise any protocol will match. """ - try: - if w_proto is None: - return space.wrap(socket.getservbyport(port)) - else: - return space.wrap(socket.getservbyport(port, space.str_w(w_proto))) - except socket.error, e: - raise wrap_socketerror(space, e) + proto = None + if w_proto: + proto = space.str_w(w_proto) + servent_ptr = _c.getservbyport(_c.htons(port), proto) + if not servent_ptr: + raise w_get_socketerror("port/proto not found") + return space.wrap(servent_ptr.contents.s_name) getservbyport.unwrap_spec = [ObjSpace, int, W_Root] def getprotobyname(space, name): @@ -245,10 +245,11 @@ Return the protocol number for the named protocol. (Rarely used.) """ - try: - return space.wrap(socket.getprotobyname(name)) - except socket.error, e: - raise wrap_socketerror(space, e) + protoent_ptr = _c.getprotobyname(name) + if not protoent_ptr: + raise w_get_socketerror('protocol not found') + return space.wrap(protoent_ptr.contents.p_proto) + getprotobyname.unwrap_spec = [ObjSpace, str] def fromfd(space, fd, family, type, w_proto=NoneNotWrapped): @@ -257,13 +258,15 @@ Create a socket object from the given file descriptor. The remaining arguments are the same as for socket(). """ - try: - if w_proto is None: - return space.wrap(socket.fromfd(fd, family, type)) - else: - return space.wrap(socket.fromfd(fd, family, type, space.int_w(w_proto))) - except socket.error, e: - raise wrap_socketerror(space, e) + + newfd = _c.dup(fd) + if newfd < 0: + raise w_get_socketerror(None, _c.errno) + if w_proto is None: + return space.wrap(Socket(space, newfd, family, type)) + else: + proto = space.int_w(w_proto) + return space.wrap(Socket(space, newfd, family, type, proto)) fromfd.unwrap_spec = [ObjSpace, int, int, int, W_Root] def socketpair(space, w_family=NoneNotWrapped, w_type=NoneNotWrapped, w_proto=NoneNotWrapped): @@ -617,11 +620,54 @@ """getnameinfo(sockaddr, flags) --> (host, port) Get host and port for a sockaddr.""" - sockaddr = space.unwrap(w_sockaddr) - try: - return space.wrap(_c.getnameinfo(sockaddr, flags)) - except _c.error, e: - raise wrap_socketerror(space, e) + w_flowinfo = w_scope_id = space.wrap(0) + sockaddr_len = space.int_w(space.len(w_sockaddr)) + if sockaddr_len == 2: + w_host, w_port = space.unpackiterable(w_sockaddr, 2) + elif sockaddr_len == 3: + w_host, w_port, w_flowinfo = space.unpackiterable(w_sockaddr, 3) + elif sockaddr_len == 4: + w_host, w_port, w_flowinfo, w_scope_id = space.unpackiterable(w_sockaddr, 4) + else: + raise OperationError(space.w_TypeError, + space.wrap('argument 1 should be 2-4 items (%d given)' % sockaddr_len)) + host = space.str_w(w_host) + port = space.int_w(w_port) + flowinfo = space.int_w(w_flowinfo) + scope_id = space.int_w(w_scope_id) + + res = _c.addrinfo_ptr() + hints = _c.addrinfo() + hints.ai_family = _c.AF_UNSPEC + hints.ai_socktype = _c.SOCK_DGRAM + retval = _c.getaddrinfo(host, str(port), ctypes.pointer(hints), ctypes.pointer(res)) + if retval != 0: + raise w_get_socketgaierror(space, None, retval) + family = res.contents.ai_family + if family == _c.AF_INET: + if sockaddr_len != 2: + if res: + _c.freeaddrinfo(res) + raise OperationError(space.w_TypeError, + space.wrap('argument 1 should be 2 items (%d given)' % sockaddr_len)) + + elif family == _c.AF_INET6: + sin6_ptr = ctypes.cast(res.contents.ai_addr, POINTER(_c.sockaddr_in6)) + sin6_ptr.contents.sin6_flowinfo = flowinfo + sin6_ptr.contents.sin6_scope_id = scope_id + + hostbuf = ctypes.create_string_buffer(_c.NI_MAXHOST) + portbuf = ctypes.create_string_buffer(_c.NI_MAXSERV) + error = _c.getnameinfo(res.contents.ai_addr, res.contents.ai_addrlen, + hostbuf, ctypes.sizeof(hostbuf), + portbuf, ctypes.sizeof(portbuf), flags) + + if res: + _c.freeaddrinfo(res) + if error: + raise w_get_socketgaierror(None, error) + return space.newtuple([space.wrap(hostbuf.value), + space.wrap(portbuf.value)]) getnameinfo.unwrap_spec = [ObjSpace, W_Root, int] # _____________________________________________________________ @@ -667,13 +713,6 @@ def newsocket(space, w_subtype, family=_c.AF_INET, type=_c.SOCK_STREAM, proto=0): - # sets the timeout for the CPython implementation - timeout = getstate(space).defaulttimeout - if timeout < 0.0: - _c.setdefaulttimeout(None) - else: - _c.setdefaulttimeout(timeout) - try: fd = _c.socket(family, type, proto) except _c.error, e: # On untranslated PyPy @@ -690,16 +729,26 @@ descr_socket_new = interp2app(newsocket, unwrap_spec=[ObjSpace, W_Root, int, int, int]) +def setblocking(fd, block): + delay_flag = _c.fcntl(fd, _c.F_GETFL, 0) + if block: + delay_flag &= ~_c.O_NONBLOCK + else: + delay_flag |= _c.O_NONBLOCK + _c.fcntl(fd, _c.F_SETFL, delay_flag) + class Socket(Wrappable): "A wrappable box around an interp-level socket object." - def __init__(self, space, fd, family, type, proto): + def __init__(self, space, fd, family, type, proto=0): self.fd = fd self.family = family self.type = type self.proto = proto - self.timeout = getstate(space).defaulttimeout self.closed = False + self.timeout = getstate(space).defaulttimeout + if self.timeout >= 0.0: + setblocking(self.fd, False) def accept(self, space): """accept() -> (socket object, address info) Modified: pypy/dist/pypy/module/_socket/test/test_socket2.py ============================================================================== --- pypy/dist/pypy/module/_socket/test/test_socket2.py (original) +++ pypy/dist/pypy/module/_socket/test/test_socket2.py Thu Apr 27 05:34:55 2006 @@ -4,8 +4,6 @@ import py import socket, sys -py.test.skip('In-progress') - def setup_module(mod): mod.space = StdObjSpace(usemodules=['_socket']) mod.w_socket = space.appexec([], "(): import _socket as m; return m") @@ -92,13 +90,13 @@ """(_socket, fd, family, type, proto): return _socket.fromfd(fd, family, type, proto)""") - assert space.unwrap(fd).fileno() + assert space.unwrap(space.call_method(fd, 'fileno')) fd = space.appexec([w_socket, space.wrap(orig_fd.fileno()), space.wrap(socket.AF_INET), space.wrap(socket.SOCK_STREAM)], """(_socket, fd, family, type): return _socket.fromfd(fd, family, type)""") - assert space.unwrap(fd).fileno() + assert space.unwrap(space.call_method(fd, 'fileno')) def test_ntohs(): w_n = space.appexec([w_socket, space.wrap(125)], Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Thu Apr 27 05:34:55 2006 @@ -9,6 +9,7 @@ 'netinet/in.h', 'netinet/tcp.h', 'unistd.h', + 'fcntl.h', 'stdio.h', 'netdb.h', 'arpa/inet.h' @@ -19,6 +20,9 @@ class CConfig: _header_ = HEADER # constants + O_NONBLOCK = ctypes_platform.ConstantInteger('O_NONBLOCK') + F_GETFL = ctypes_platform.ConstantInteger('F_GETFL') + F_SETFL = ctypes_platform.ConstantInteger('F_SETFL') constant_names = ['AF_APPLETALK', 'AF_ASH', 'AF_ATMPVC', 'AF_ATMSVC', 'AF_AX25', 'AF_BLUETOOTH', 'AF_BRIDGE', 'AF_ECONET', 'AF_INET', 'AF_INET6', @@ -102,6 +106,11 @@ [('sin_family', c_int), ('sin_port', c_ushort), ('sin_addr', CConfig.in_addr)]) + +CConfig.sockaddr_in6 = ctypes_platform.Struct('struct sockaddr_in6', + [('sin6_flowinfo', c_int), + ('sin6_scope_id', c_int), + ]) addrinfo_ptr = POINTER("addrinfo") CConfig.addrinfo = ctypes_platform.Struct('struct addrinfo', [('ai_flags', c_int), @@ -122,6 +131,13 @@ ]) +CConfig.servent = ctypes_platform.Struct('struct servent', + [('s_name', c_char_p), + ('s_port', c_int)]) + +CConfig.protoent = ctypes_platform.Struct('struct protoent', + [('p_proto', c_int), + ]) class cConfig: pass @@ -138,8 +154,14 @@ else: constants[name] = default +constants['has_ipv6'] = True # This is a configuration option in CPython + locals().update(constants) +O_NONBLOCK = cConfig.O_NONBLOCK +F_GETFL = cConfig.F_GETFL +F_SETFL = cConfig.F_SETFL + uint16_t = cConfig.uint16_t uint32_t = cConfig.uint32_t size_t = cConfig.size_t @@ -158,6 +180,10 @@ assert dllname is not None socketdll = cdll.LoadLibrary(dllname) +dup = socketdll.dup +dup.argtypes = [c_int] +dup.restype = c_int + errno = c_int.in_dll(socketdll, 'errno') strerror = socketdll.strerror @@ -295,3 +321,23 @@ gethostbyname = socketdll.gethostbyname gethostbyname.argtypes = [POINTER(c_char)] gethostbyname.restype = POINTER(cConfig.hostent) + +gethostbyaddr = socketdll.gethostbyaddr +gethostbyaddr.argtypes = [POINTER(c_char), c_int, c_int] +gethostbyaddr.restype = POINTER(cConfig.hostent) + +getservbyname = socketdll.getservbyname +getservbyname.argtypes = [c_char_p, c_char_p] +getservbyname.restype = POINTER(cConfig.servent) + +getservbyport = socketdll.getservbyport +getservbyport.argtypes = [c_int, c_char_p] +getservbyport.restype = POINTER(cConfig.servent) + +getprotobyname = socketdll.getprotobyname +getprotobyname.argtypes = [c_char_p] +getprotobyname.restype = POINTER(cConfig.protoent) + +fcntl = socketdll.fcntl +fcntl.argtypes = [c_int] * 3 +fcntl.restype = c_int From ericvrp at codespeak.net Thu Apr 27 06:34:37 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 27 Apr 2006 06:34:37 +0200 (CEST) Subject: [pypy-svn] r26394 - in pypy/dist/pypy/interpreter: . test Message-ID: <20060427043437.CD4911009F@code0.codespeak.net> Author: ericvrp Date: Thu Apr 27 06:34:31 2006 New Revision: 26394 Modified: pypy/dist/pypy/interpreter/nestedscope.py pypy/dist/pypy/interpreter/pycode.py pypy/dist/pypy/interpreter/test/test_pickle.py pypy/dist/pypy/interpreter/typedef.py Log: (pedronis, ericvrp) Added support for pickling code objects. (commiting to test translation on faster machine) Modified: pypy/dist/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/dist/pypy/interpreter/nestedscope.py (original) +++ pypy/dist/pypy/interpreter/nestedscope.py Thu Apr 27 06:34:31 2006 @@ -38,11 +38,12 @@ def descr__reduce__(self, space): w_mod = space.getbuiltinmodule('_pickle_support') mod = space.interp_w(MixedModule, w_mod) - cell_new = mod.get('cell_new') + new_inst = mod.get('cell_new') if self.w_value is None: #when would this happen? - return space.newtuple([cell_new, space.newtuple([])]) - return space.newtuple([cell_new, space.newtuple([]), - space.newtuple([self.w_value])]) + return space.newtuple([new_inst, space.newtuple([])]) + tup = [self.w_value] + return space.newtuple([new_inst, space.newtuple([]), + space.newtuple(tup)]) def descr__setstate__(self, space, w_state): self.w_value = space.getitem(w_state, space.wrap(0)) Modified: pypy/dist/pypy/interpreter/pycode.py ============================================================================== --- pypy/dist/pypy/interpreter/pycode.py (original) +++ pypy/dist/pypy/interpreter/pycode.py Thu Apr 27 06:34:31 2006 @@ -10,6 +10,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import NoneNotWrapped from pypy.interpreter.baseobjspace import ObjSpace, W_Root +from pypy.interpreter.mixedmodule import MixedModule # helper @@ -380,3 +381,26 @@ return space.wrap(code) descr_code__new__.unwrap_spec = unwrap_spec + def descr__reduce__(self, space): + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + new_inst = mod.get('code_new') + w = space.wrap + tup = [ + w(self.co_argcount), + w(self.co_nlocals), + w(self.co_stacksize), + w(self.co_flags), + w(self.co_code), + space.newtuple(self.co_consts_w), + space.newtuple(self.co_names_w), + space.newtuple([w(v) for v in self.co_varnames]), + w(self.co_filename), + w(self.co_name), + w(self.co_firstlineno), + w(self.co_lnotab), + space.newtuple([w(v) for v in self.co_freevars]), + space.newtuple([w(v) for v in self.co_cellvars]), + #hidden_applevel=False, magic = 62061 | 0x0a0d0000 + ] + return space.newtuple([new_inst, space.newtuple(tup)]) \ No newline at end of file Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Thu Apr 27 06:34:31 2006 @@ -1,5 +1,17 @@ class AppTestInterpObjectPickling: - + + def test_pickle_code(self): + import pickle + def f(): + return 42 + code = f.func_code + pckl = pickle.dumps(code) + result = pickle.loads(pckl) + assert code == result + + def DONTtest_pickle_func(self): + pass + def test_pickle_cell(self): import pickle def g(): @@ -14,6 +26,42 @@ assert cell == result assert not (cell != result) + def DONTtest_pickle_frame(self): + pass + + def DONTtest_pickle_traceback(self): + pass + + def DONTtest_pickle_module(self): + pass + + def DONTtest_pickle_moduledict(self): + pass + + def DONTtest_pickle_iter(self): + pass + + def DONTtest_pickle_method(self): + pass + + def DONTtest_pickle_dictiter(self): + pass + + def DONTtest_pickle_enum(self): + pass + + def DONTtest_pickle_enumfactory(self): + pass + + def DONTtest_pickle_listiter(self): + pass + + def DONTtest_pickle_rangeiter(self): + pass + + def DONTtest_pickle_tupleiter(self): + pass + #def test_pickle_generator(self): # import pickle # def giveme(n): @@ -22,28 +70,3 @@ # yield x # generator = giveme(10) # print pickle.dumps(generator) - -#TODO: test pickling of code objects -#TODO: test pickling of function objects -#TODO: test pickling of frame objects -#TODO: test pickling of tracebacks -#TODO: test pickling of modules - -''' -etc. etc. etc. -init_codetype() -init_functype() -init_celltype() -init_frametype() -init_tracebacktype() -init_moduletype() -init_moduledicttype() -init_itertype() -init_methodtype() -init_dictitertype() -init_enumtype() -init_enumfactorytype() -init_listitertype() -init_rangeitertype() -init_tupleitertype() -''' Modified: pypy/dist/pypy/interpreter/typedef.py ============================================================================== --- pypy/dist/pypy/interpreter/typedef.py (original) +++ pypy/dist/pypy/interpreter/typedef.py Thu Apr 27 06:34:31 2006 @@ -415,6 +415,8 @@ __new__ = interp2app(PyCode.descr_code__new__.im_func), __eq__ = interp2app(PyCode.descr_code__eq__), __ne__ = descr_generic_ne, + __reduce__ = interp2app(PyCode.descr__reduce__, + unwrap_spec=['self', ObjSpace]), co_argcount = interp_attrproperty('co_argcount', cls=PyCode), co_nlocals = interp_attrproperty('co_nlocals', cls=PyCode), co_stacksize = interp_attrproperty('co_stacksize', cls=PyCode), From ericvrp at codespeak.net Thu Apr 27 06:37:23 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 27 Apr 2006 06:37:23 +0200 (CEST) Subject: [pypy-svn] r26395 - pypy/dist/pypy/module/_pickle_support Message-ID: <20060427043723.8E712100A4@code0.codespeak.net> Author: ericvrp Date: Thu Apr 27 06:37:20 2006 New Revision: 26395 Modified: pypy/dist/pypy/module/_pickle_support/__init__.py pypy/dist/pypy/module/_pickle_support/maker.py Log: (pedronis, ericvrp) forgot to commit this with the previous checkin Modified: pypy/dist/pypy/module/_pickle_support/__init__.py ============================================================================== --- pypy/dist/pypy/module/_pickle_support/__init__.py (original) +++ pypy/dist/pypy/module/_pickle_support/__init__.py Thu Apr 27 06:37:20 2006 @@ -7,5 +7,6 @@ } interpleveldefs = { - 'cell_new': 'maker.cell_new' + 'cell_new': 'maker.cell_new', + 'code_new': 'maker.code_new', } Modified: pypy/dist/pypy/module/_pickle_support/maker.py ============================================================================== --- pypy/dist/pypy/module/_pickle_support/maker.py (original) +++ pypy/dist/pypy/module/_pickle_support/maker.py Thu Apr 27 06:37:20 2006 @@ -1,9 +1,19 @@ from pypy.interpreter.nestedscope import Cell +from pypy.interpreter.pycode import PyCode +from pypy.rpython.objectmodel import instantiate +from pypy.interpreter.argument import Arguments +from pypy.interpreter.baseobjspace import ObjSpace + #note: for now we don't use the actual value when creating the Cell. # (i.e. we assume it will be handled by __setstate__) # Stackless does use this so it might be needed here as well. def cell_new(space): - return space.wrap(Cell()) + return space.wrap(instantiate(Cell)) #cell_new.unwrap_spec = [...] + +def code_new(space, __args__): + w_codetype = space.gettypeobject(PyCode.typedef) + return space.call_args(w_codetype, __args__) +code_new.unwrap_spec = [ObjSpace, Arguments] \ No newline at end of file From sanxiyn at codespeak.net Thu Apr 27 06:41:07 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 06:41:07 +0200 (CEST) Subject: [pypy-svn] r26396 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060427044107.66124100A7@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 06:41:03 2006 New Revision: 26396 Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (nik, dialtone, sanxiyn) Support for constant instances. This enables us to support metaclass hierarchy. *That* enables us to support class variables. Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Thu Apr 27 06:41:03 2006 @@ -2,7 +2,7 @@ from pypy.objspace.flow.model import Constant, Variable, Atom from pypy.rpython.rmodel import HalfConcreteWrapper -from pypy.rpython.ootypesystem.ootype import List, Record, Instance, instance_impl, _static_meth +from pypy.rpython.ootypesystem.ootype import List, Record, Instance, instance_impl, _class, _static_meth from pypy.rpython.ootypesystem.rclass import CLASSTYPE def repr_unknown(obj): @@ -33,9 +33,8 @@ return "'struct" # XXX if isinstance(val, Instance): return "'" + repr_class_name(val._name) - if isinstance(val, instance_impl): - if val._TYPE is CLASSTYPE: - return "'standard-class" + if isinstance(val, _class): + return "'" + repr_class_name(val._INSTANCE._name) if isinstance(val, types.FunctionType): if val.func_name == 'dum_nocheck': # XXX return "'dummy" Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Thu Apr 27 06:41:03 2006 @@ -1,8 +1,9 @@ import types from pypy.tool.udir import udir +from pypy.objspace.flow.model import Constant from pypy.translator.translator import graphof -from pypy.rpython.ootypesystem.ootype import List, Record, Instance, _static_meth, _meth, ROOT +from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name @@ -19,7 +20,7 @@ def __iter__(self): method = getattr(self, "op_" + self.opname) result = repr_arg(self.result) - args = map(repr_arg, self.args) + args = map(self.gen.repr_arg, self.args) for line in method(result, *args): yield line @@ -28,6 +29,7 @@ op_same_as = nop op_ooupcast = nop + op_oodowncast = nop def make_binary_op(cl_op): def _(self, result, arg1, arg2): @@ -71,39 +73,15 @@ args = " ".join(args) yield "(setf %s (%s %s))" % (result, fun, args) - def declare_struct(self, cls): - # cls is Record - name = "struct" + str(self.gen.structcount) - field_declaration = cls._fields.keys() - field_declaration = " ".join(field_declaration) - struct_declaration = "(defstruct %s %s)" % (name, field_declaration) - self.gen.declarations.append(struct_declaration) - self.gen.structcount += 1 - return name - - def declare_class(self, cls): - # cls is Instance - name = repr_class_name(cls._name) - field_declaration = ['('+field+')' for field in cls._fields] - field_declaration = " ".join(field_declaration) - if cls._superclass in (OBJECT, ROOT): - class_declaration = "(defclass %s () (%s))" % (name, field_declaration) - else: - self.declare_class(cls._superclass) - supername = repr_class_name(cls._superclass._name) - class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) - if class_declaration not in self.gen.declarations: - self.gen.declarations.append(class_declaration) - def op_new(self, result, clsname): cls = self.args[0].value if isinstance(cls, List): yield "(setf %s (make-array 0 :adjustable t))" % (result,) elif isinstance(cls, Record): - clsname = self.declare_struct(cls) + clsname = self.gen.declare_struct(cls) yield "(setf %s (make-%s))" % (result, clsname) elif isinstance(cls, Instance): - self.declare_class(cls) + self.gen.declare_class(cls) yield "(setf %s (make-instance %s))" % (result, clsname) else: raise NotImplementedError() @@ -125,8 +103,8 @@ methodobj._method_name = method # XXX self.gen.pendinggraphs.append(methodobj) name = repr_fun_name(method) - selfvar = repr_arg(receiver) - args = map(repr_arg, args) + selfvar = repr_var(receiver) + args = map(self.gen.repr_arg, args) args = " ".join(args) if args: yield "(setf %s (%s %s %s))" % (result, name, selfvar, args) @@ -139,8 +117,6 @@ def op_oosetfield(self, result, obj, _, value): fieldname = self.args[1].value - if fieldname == "meta": # XXX - raise StopIteration yield "(setf (slot-value %s '%s) %s)" % (obj, fieldname, value) def op_ooidentityhash(self, result, arg): @@ -153,7 +129,7 @@ class ListImpl: def __init__(self, receiver): - self.obj = repr_arg(receiver) + self.obj = repr_var(receiver) def ll_length(self): return "(length %s)" % (self.obj,) @@ -179,8 +155,57 @@ self.entry_point = funobj self.pendinggraphs = [funobj] self.declarations = [] + self.constcount = 0 self.structcount = 0 + def repr_arg(self, arg): + if isinstance(arg, Constant): + if isinstance(arg.concretetype, Instance): + return self.declare_constant_instance(arg) + return repr_arg(arg) + + def declare_struct(self, cls): + # cls is Record + name = "struct" + str(self.structcount) + field_declaration = cls._fields.keys() + field_declaration = " ".join(field_declaration) + struct_declaration = "(defstruct %s %s)" % (name, field_declaration) + self.declarations.append(struct_declaration) + self.structcount += 1 + return name + + def declare_class(self, cls): + # cls is Instance + name = repr_class_name(cls._name) + field_declaration = ['('+field+')' for field in cls._fields] + field_declaration = " ".join(field_declaration) + if cls._superclass is ROOT: + class_declaration = "(defclass %s () (%s))" % (name, field_declaration) + else: + self.declare_class(cls._superclass) + supername = repr_class_name(cls._superclass._name) + class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) + if class_declaration not in self.declarations: + self.declarations.append(class_declaration) + + def declare_constant_instance(self, const): + # const.concretetype is Instance + name = "const" + str(self.constcount) + INST = dynamicType(const.value) + self.declare_class(INST) + inst = oodowncast(INST, const.value) + cls = repr_const(INST) + const_declaration = [] + const_declaration.append("(setf %s (make-instance %s))" % (name, cls)) + fields = INST._allfields() + for fieldname in fields: + fieldvalue = repr_const(getattr(inst, fieldname)) + const_declaration.append("(setf (slot-value %s '%s) %s)" % (name, fieldname, fieldvalue)) + const_declaration = "\n".join(const_declaration) + self.declarations.append(const_declaration) + self.constcount += 1 + return name + def emitfile(self): name = self.entry_point.func_name path = udir.join("%s.lisp" % (name,)) @@ -314,7 +339,7 @@ return "(go tag" + str(tag) + ")" def emit_link(self, link): - source = map(repr_arg, link.args) + source = map(self.repr_arg, link.args) target = map(repr_var, link.target.inputargs) couples = [ "%s %s" % (t, s) for (s, t) in zip(source, target)] couples = " ".join(couples) Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Thu Apr 27 06:41:03 2006 @@ -32,10 +32,14 @@ class Bar(Foo): pass def check_inheritance(): - obj = Bar() + Bar() code = generate_cl_func(check_inheritance) print code - assert code.count("defclass") == 2 + classcount = code.count("defclass") + # Divide by two to get rid of meta hierarchy + # Minus one to get rid of Object + realcount = (classcount / 2) - 1 + assert realcount == 2 def test_isinstance(): class Foo: @@ -54,7 +58,6 @@ assert cl_check_isinstance(True) == True def test_class(): - py.test.skip("TODO") class Foo: value = 0 class Bar(Foo): From sanxiyn at codespeak.net Thu Apr 27 06:44:20 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 06:44:20 +0200 (CEST) Subject: [pypy-svn] r26397 - pypy/dist/pypy/translator/cl/test Message-ID: <20060427044420.DDFCA100A7@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 06:44:18 2006 New Revision: 26397 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: Add a simpler failing test for exception handling Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Thu Apr 27 06:44:18 2006 @@ -33,6 +33,16 @@ cl_get_three = make_cl_func(get_three) assert cl_get_three() == 3 +def test_iteration(): + py.test.skip("needs exception handling") + def get_last(num): + last = 0 + for i in range(num): + last = i + return last + cl_get_last = make_cl_func(get_last, [int]) + assert cl_get_last(5) == 4 + def test_if(): cl_if = make_cl_func(t.if_then_else, [bool, int, int]) assert cl_if(True, 50, 100) == 50 From ac at codespeak.net Thu Apr 27 08:02:34 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 27 Apr 2006 08:02:34 +0200 (CEST) Subject: [pypy-svn] r26400 - in pypy/dist/pypy: module/_socket rpython/rctypes/socketmodule Message-ID: <20060427060234.3BD5110091@code0.codespeak.net> Author: ac Date: Thu Apr 27 08:02:33 2006 New Revision: 26400 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (aleale, arre) Now the app-level tests pass too. Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Thu Apr 27 08:02:33 2006 @@ -1,11 +1,12 @@ import sys from pypy.interpreter.typedef import TypeDef -from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.baseobjspace import Wrappable, UnpackValueError from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import W_Root, NoneNotWrapped from pypy.interpreter.gateway import ObjSpace, interp2app from pypy.rpython.rctypes.socketmodule import ctypes_socket as _c import ctypes +import errno IPV4_ADDRESS_SIZE = 4 IPV6_ADDRESS_SIZE = 16 @@ -82,7 +83,7 @@ if errno > -1: if message is None: message = socket_strerror(errno) - return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) + return OperationError(w_errortype, space.newtuple([space.wrap(errno), space.wrap(message)])) else: return OperationError(w_errortype, space.wrap(message)) @@ -92,7 +93,7 @@ if errno > -1: if message is None: message = _c.gai_strerror(errno) - return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) + return OperationError(w_errortype, space.newtuple([space.wrap(errno), space.wrap(message)])) else: return OperationError(w_errortype, space.wrap(message)) @@ -102,7 +103,7 @@ if errno > -1: if message is None: message = _c.hstrerror(errno) - return OperationError(w_errortype, space.wrap(errno), space.wrap(message)) + return OperationError(w_errortype, space.newtuple([space.wrap(errno), space.wrap(message)])) else: return OperationError(w_errortype, space.wrap(message)) @@ -123,7 +124,7 @@ res = _c.addr_info_ptr err = _c.getaddrinfo( None, "0", pointer(hints), pointer(res)) if err: - raise w_get_socketgaierror(space, None, _c.errno) + raise w_get_socketgaierror(space, None, err) if res.contents.ai_next: raise OperationError(_socket.error, space.wrap("wildcard resolved to multiple address")) addr = res.contents.ai_addr @@ -261,7 +262,7 @@ newfd = _c.dup(fd) if newfd < 0: - raise w_get_socketerror(None, _c.errno) + raise w_get_socketerror(None, _c.errno.value) if w_proto is None: return space.wrap(Socket(space, newfd, family, type)) else: @@ -713,12 +714,9 @@ def newsocket(space, w_subtype, family=_c.AF_INET, type=_c.SOCK_STREAM, proto=0): - try: - fd = _c.socket(family, type, proto) - except _c.error, e: # On untranslated PyPy - raise wrap_socketerror(space, e) - except OSError, e: # On translated PyPy - raise w_get_socketerror(space, e.strerror, e.errno) + fd = _c.socket(family, type, proto) + if fd < 0: + raise w_get_socketerror(space, None, _c.errno.value) # XXX If we want to support subclassing the socket type we will need # something along these lines. But allocate_instance is only defined # on the standard object space, so this is not really correct. @@ -750,6 +748,36 @@ if self.timeout >= 0.0: setblocking(self.fd, False) + def _getsockaddr(self, space, w_addr): + """Returns a pointer to a sockaddr""" + if self.family == _c.AF_INET: + try: + w_host, w_port = space.unpackiterable(w_addr, 2) + except UnpackValueError: + e_msg = space.wrap("getsockaddrarg: AF_INET address must be a tuple of two elements") + raise OperationError(space.w_TypeError, e_msg) + + port = space.int_w(w_port) + host = space.str_w(w_host) + res = _c.addrinfo_ptr() + hints = _c.addrinfo() + hints.ai_family = self.family + hints.ai_socktype = self.type + hints.ai_protocol = self.proto + retval = _c.getaddrinfo(host, str(port), ctypes.pointer(hints), ctypes.pointer(res)) + if retval != 0: + raise w_get_socketgaierror(space, None, retval) + addrinfo = res.contents + addrlen = addrinfo.ai_addrlen + caddr_ptr = ctypes.create_string_buffer(addrlen) + _c.memcpy(caddr_ptr, addrinfo.ai_addr, addrlen) + + sockaddr_ptr = ctypes.cast(caddr_ptr, _c.sockaddr_ptr) + return sockaddr_ptr, addrlen + + else: + raise NotImplementedError('Unsupported address family') # XXX + def accept(self, space): """accept() -> (socket object, address info) @@ -798,31 +826,14 @@ Connect the socket to a remote address. For IP sockets, the address is a pair (host, port). """ - if self.family == socket.AF_INET: - if not (space.is_true(space.isinstance(w_addr, space.w_tuple)) and - space.int_w(space.len(w_addr)) == 2): - raise OperationError(space.w_TypeError, - space.wrap("AF_INET address must be tuple of length 2")) - addr_w = space.unpackiterable(w_addr) - if not (space.is_true(space.isinstance(addr_w[0], space.w_str)) - and space.is_true(space.isinstance(addr_w[1], space.w_int))): - raise OperationError(space.w_TypeError, - space.wrap("tuple of a string and an int required")) - host = space.str_w(addr_w[0]) - port = space.int_w(addr_w[1]) - sockname = (host, port, 0, 0) - else: - # XXX IPv6 and Unix sockets missing here - pass - try: - rsocket.connect(self.fd, sockname, self.family) - except OSError, ex: - raise w_get_socketerror(space, e.strerror, e.errno) - # XXX timeout doesn't really work at the moment - except socket.timeout: - raise wrap_timeouterror(space) - except socket.error, e: - raise wrap_socketerror(space, e) + sockaddr_ptr, sockaddr_len = self._getsockaddr(space, w_addr) + err = _c.socketconnect(self.fd, sockaddr_ptr, sockaddr_len) + if err: + errno = _c.errno.value + if self.timeout > 0.0: + # XXX timeout doesn't really work at the moment + pass + raise w_get_socketerror(space, None, errno) connect.unwrap_spec = ['self', ObjSpace, W_Root] def connect_ex(self, space, w_addr): @@ -868,12 +879,14 @@ Return the address of the remote endpoint. For IP sockets, the address info is a pair (hostaddr, port). """ - try: - name = rsocket.getpeername(self.fd) - # XXX IPv4 only - return space.newtuple([space.wrap(name[0]), space.wrap(name[1])]) - except socket.error, e: - raise wrap_socketerror(space, e) + def getpeername(self, space): + peeraddr = ctypes.pointer(_c.sockaddr()) + peeraddrlen = _c.socklen_t(ctypes.sizeof(_c.sockaddr)) + res = _c.socketgetpeername(self.fd, peeraddr, + ctypes.pointer(peeraddrlen)) + if res < 0: + raise w_get_socketerror(None, _c.errno.value) + return w_makesockaddr(space, peeraddr, peeraddrlen, self.proto) getpeername.unwrap_spec = ['self', ObjSpace] def getsockname(self, space): Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Thu Apr 27 08:02:33 2006 @@ -341,3 +341,7 @@ fcntl = socketdll.fcntl fcntl.argtypes = [c_int] * 3 fcntl.restype = c_int + +memcpy = socketdll.memcpy +memcpy.argtypes = [c_void_p, c_void_p, size_t] +memcpy.restype = c_void_p From sanxiyn at codespeak.net Thu Apr 27 08:25:17 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 08:25:17 +0200 (CEST) Subject: [pypy-svn] r26401 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060427062517.3704910091@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 08:25:13 2006 New Revision: 26401 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: (nik, dialtone, sanxiyn) Add a failing test, implement runtimenew, declare class before representing it as symbol, and... Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Thu Apr 27 08:25:13 2006 @@ -3,7 +3,7 @@ from pypy.tool.udir import udir from pypy.objspace.flow.model import Constant from pypy.translator.translator import graphof -from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _static_meth, _meth, ROOT +from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _class, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name @@ -86,6 +86,9 @@ else: raise NotImplementedError() + def op_runtimenew(self, result, arg): + yield "(setf %s (make-instance %s))" % (result, arg) + def op_instanceof(self, result, arg, clsname): yield "(setf %s (typep %s %s))" % (result, arg, clsname) @@ -199,8 +202,11 @@ const_declaration.append("(setf %s (make-instance %s))" % (name, cls)) fields = INST._allfields() for fieldname in fields: - fieldvalue = repr_const(getattr(inst, fieldname)) - const_declaration.append("(setf (slot-value %s '%s) %s)" % (name, fieldname, fieldvalue)) + fieldvalue = getattr(inst, fieldname) + if isinstance(fieldvalue, _class): + self.declare_class(fieldvalue._INSTANCE) + fieldvaluerepr = repr_const(getattr(inst, fieldname)) + const_declaration.append("(setf (slot-value %s '%s) %s)" % (name, fieldname, fieldvaluerepr)) const_declaration = "\n".join(const_declaration) self.declarations.append(const_declaration) self.constcount += 1 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Thu Apr 27 08:25:13 2006 @@ -76,6 +76,27 @@ assert cl_dynamic_class(True) == 1 assert cl_dynamic_class(False) == 2 +def test_instance(): + py.test.skip("TODO") + class Foo: + value = 0 + class Bar(Foo): + value = 1 + class Baz(Foo): + value = 2 + def pick_class(flag): + if flag: + return Bar + else: + return Baz + def dynamic_instance(flag): + cls = pick_class(flag) + obj = cls() + return obj.value + cl_dynamic_instance = make_cl_func(dynamic_instance, [bool]) + assert cl_dynamic_instance(True) == 1 + assert cl_dynamic_instance(False) == 2 + def test_list_length(): def list_length_one(number): lst = [number] From nik at codespeak.net Thu Apr 27 08:28:09 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 08:28:09 +0200 (CEST) Subject: [pypy-svn] r26402 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427062809.999BE10091@code0.codespeak.net> Author: nik Date: Thu Apr 27 08:28:06 2006 New Revision: 26402 Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py Log: really use Record now for tuples of ootypesystem. Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Thu Apr 27 08:28:06 2006 @@ -8,8 +8,7 @@ def __init__(self, rtyper, items_r): AbstractTupleRepr.__init__(self, rtyper, items_r) - #self.lowleveltype = ootype.Record(dict(zip(self.fieldnames, self.lltypes))) - self.lowleveltype = tuple_type(self.fieldnames, self.lltypes) + self.lowleveltype = ootype.Record(dict(zip(self.fieldnames, self.lltypes))) def newtuple(cls, llops, r_tuple, items_v): # items_v should have the lowleveltype of the internal reprs From sanxiyn at codespeak.net Thu Apr 27 08:31:03 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 08:31:03 +0200 (CEST) Subject: [pypy-svn] r26403 - pypy/dist/pypy/translator/cl/test Message-ID: <20060427063103.9147410095@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 08:30:59 2006 New Revision: 26403 Added: pypy/dist/pypy/translator/cl/test/test_list.py Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: Split list tests to the separate file Modified: pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Thu Apr 27 08:30:59 2006 @@ -96,17 +96,3 @@ cl_dynamic_instance = make_cl_func(dynamic_instance, [bool]) assert cl_dynamic_instance(True) == 1 assert cl_dynamic_instance(False) == 2 - -def test_list_length(): - def list_length_one(number): - lst = [number] - return len(lst) - cl_list_length_one = make_cl_func(list_length_one, [int]) - assert cl_list_length_one(0) == 1 - -def test_list_get(): - def list_and_get(number): - lst = [number] - return lst[0] - cl_list_and_get = make_cl_func(list_and_get, [int]) - assert cl_list_and_get(1985) == 1985 Added: pypy/dist/pypy/translator/cl/test/test_list.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/test/test_list.py Thu Apr 27 08:30:59 2006 @@ -0,0 +1,15 @@ +from pypy.translator.cl.buildcl import make_cl_func + +def test_list_length(): + def list_length_one(number): + lst = [number] + return len(lst) + cl_list_length_one = make_cl_func(list_length_one, [int]) + assert cl_list_length_one(0) == 1 + +def test_list_get(): + def list_and_get(number): + lst = [number] + return lst[0] + cl_list_and_get = make_cl_func(list_and_get, [int]) + assert cl_list_and_get(1985) == 1985 From sanxiyn at codespeak.net Thu Apr 27 08:32:54 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 08:32:54 +0200 (CEST) Subject: [pypy-svn] r26404 - pypy/dist/pypy/translator/cl/test Message-ID: <20060427063254.C33A71009F@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 08:32:49 2006 New Revision: 26404 Added: pypy/dist/pypy/translator/cl/test/test_call.py - copied unchanged from r26400, pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py pypy/dist/pypy/translator/cl/test/test_oo.py - copied unchanged from r26403, pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Removed: pypy/dist/pypy/translator/cl/test/test_cltrans_multi.py pypy/dist/pypy/translator/cl/test/test_cltrans_oo.py Log: Rename test files From nik at codespeak.net Thu Apr 27 08:37:56 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 08:37:56 +0200 (CEST) Subject: [pypy-svn] r26405 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427063756.AECCE100A4@code0.codespeak.net> Author: nik Date: Thu Apr 27 08:37:53 2006 New Revision: 26405 Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py Log: remove unused tuple type caching code. Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Thu Apr 27 08:37:53 2006 @@ -55,18 +55,6 @@ hop.genop('oosend', [c_setitem, v_list, c_index, v_item], resulttype=ootype.Void) return v_list -_tuple_types = {} - -def tuple_type(fieldnames, fieldtypes): - key = tuple(fieldtypes) - if _tuple_types.has_key(key): - return _tuple_types[key] - else: - fields = dict(zip(fieldnames, fieldtypes)) - INST = ootype.Instance("Tuple%s" % len(fieldnames), ootype.ROOT, fields) - _tuple_types[key] = INST - return INST - def rtype_newtuple(hop): return TupleRepr._rtype_newtuple(hop) From sanxiyn at codespeak.net Thu Apr 27 08:55:47 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Thu, 27 Apr 2006 08:55:47 +0200 (CEST) Subject: [pypy-svn] r26406 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060427065547.0AA1F1009B@code0.codespeak.net> Author: sanxiyn Date: Thu Apr 27 08:55:37 2006 New Revision: 26406 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_clrepr.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: (dialtone, sanxiyn) Get rid of Literal in readlisp() and raise error instead. Support for passing float to and fro Lisp. Test. Refactor unary operations. Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Thu Apr 27 08:55:37 2006 @@ -35,12 +35,8 @@ return "sbclinvoke.sh" return None -class Literal: - def __init__(self, val): - self.val = val - def readlisp(s): - # Return bool/int/char/str or give up + # Return bool/char/str/int/float or give up lines = s.splitlines() lines = [ line for line in lines if line and not line.startswith(';') ] assert len(lines) == 1 @@ -56,8 +52,11 @@ return s[1:-1] elif s.isdigit(): return int(s) - else: - return Literal(s) + try: + return float(s) + except ValueError: + pass + raise NotImplementedError("cannot read %s" % (s,)) def make_cl_func(func, argtypes=[]): global global_cl Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Thu Apr 27 08:55:37 2006 @@ -53,6 +53,8 @@ return "nil" if isinstance(val, (int, long)): return str(val) + if isinstance(val, float): + return str(val) if val is None: return "nil" if isinstance(val, str): Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Thu Apr 27 08:55:37 2006 @@ -31,6 +31,17 @@ op_ooupcast = nop op_oodowncast = nop + def make_unary_op(cl_op): + def _(self, result, arg): + yield "(setf %s (%s %s))" % (result, cl_op, arg) + return _ + + op_bool_not = make_unary_op("not") + op_cast_char_to_int = make_unary_op("char-code") + op_cast_int_to_char = make_unary_op("code-char") + op_cast_float_to_int = make_unary_op("truncate") + op_cast_int_to_float = make_unary_op("float") + def make_binary_op(cl_op): def _(self, result, arg1, arg2): yield "(setf %s (%s %s %s))" % (result, cl_op, arg1, arg2) @@ -52,18 +63,6 @@ op_char_le = make_binary_op("char<=") op_char_ne = make_binary_op("char/=") - def op_cast_char_to_int(self, result, arg): - yield "(setf %s (char-code %s))" % (result, arg) - - def op_cast_int_to_char(self, result, arg): - yield "(setf %s (code-char %s))" % (result, arg) - - def op_cast_int_to_float(self, result, arg): - yield "(setf %s (float %s))" % (result, arg) - - def op_bool_not(self, result, arg): - yield "(setf %s (not %s))" % (result, arg) - def op_int_is_true(self, result, arg): yield "(setf %s (not (zerop %s)))" % (result, arg) Modified: pypy/dist/pypy/translator/cl/test/test_clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_clrepr.py (original) +++ pypy/dist/pypy/translator/cl/test/test_clrepr.py Thu Apr 27 08:55:37 2006 @@ -4,6 +4,7 @@ assert repr_const(True) == 't' assert repr_const(False) == 'nil' assert repr_const(42) == '42' + assert repr_const(1.5) == '1.5' assert repr_const(None) == 'nil' assert repr_const('a') == '#\\a' assert repr_const('answer') == '"answer"' Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Thu Apr 27 08:55:37 2006 @@ -4,7 +4,6 @@ from pypy.annotation.model import SomeChar from pypy.translator.cl.buildcl import make_cl_func -from pypy.translator.cl.buildcl import Literal from pypy.translator.test import snippet as t @@ -26,6 +25,18 @@ cl_ord_chr = make_cl_func(ord_chr, [SomeChar()]) assert cl_ord_chr('a') == 'a' +def test_float_int(): + def cast_float(num): + return float(num) + cl_cast_float = make_cl_func(cast_float, [int]) + assert cl_cast_float(1) == 1.0 + def cast_int(num): + return int(num) + cl_cast_int = make_cl_func(cast_int, [float]) + assert cl_cast_int(1.0) == 1 + assert cl_cast_int(1.5) == 1 + assert cl_cast_int(-1.5) == -1 + def test_range(): def get_three(): lst = range(7) From hpk at codespeak.net Thu Apr 27 09:23:09 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 27 Apr 2006 09:23:09 +0200 (CEST) Subject: [pypy-svn] r26408 - pypy/extradoc/minute Message-ID: <20060427072309.79192100A4@code0.codespeak.net> Author: hpk Date: Thu Apr 27 09:23:05 2006 New Revision: 26408 Added: pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt (contents, props changed) Log: draft of minutes from tuesday evening's pypy-sync triggered 0.9 EU work synchronisation meeting. Feel free to amend or correct the minutes until friday. Added: pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt ============================================================================== --- (empty file) +++ pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt Thu Apr 27 09:23:05 2006 @@ -0,0 +1,605 @@ + +EU pypy-sync 25th April 2006, 05:10-06:30 PM (UTC+2) +===================================================== + +attendants: Michael Hudson, Armin Rigo, Christian Tismer, Carl Friedrich Bolz + Samuele Pedroni, Aurelien Campeas, Anders Chrigstroem, + Anders Lehmann, Jan Balster, Stephan Diehl, Gerald Klix + Holger Krekel (moderation, minutes) + +The meeting's purpose was to synchronise work efforts related to the +PyPy EU project and more specifically to structure and plan work for +the 0.9 release. The meeting invitation resulted from +discussions on the 20th April pypy-sync meeting. Further +below is the IRC log of the roughly 1.5 hour long session. +Holger suggested a list of 0.9 topics which was accepted and +then discussed one by one: + +1. stackless support / WP07 +------------------------------ + +Christian has produced a partial draft on his stackless +and "thread pickling" ideas in pypy/doc/discussion/howtoimplementstackless.txt. +This got discussed especially in relation to what we currently +have in our pypy translation tool chain. It was noted that +the current PyPy stackless approach does not directly map +to the new description. Some partial agreements and +open questions were identified and the final discussion and decision +about the 0.9 stackless approach was delegated to Christian Tismer, +Michael Hudson, Samuele Pedroni and Armin Rigo (Armin will care about +inviting and making sure that a proper description about the approach +and the tasks is provided and agreed upon). The decision should be +taken in consensus with a deadline of end april. Christian plans to +spend his time after 1st May fully working on this support. + +Holger additionally notes that WP07 has more tasks than +tasklet pickling and all those tasks are promised to be +finished until June and raises the question if this is +still feasible. + +Aurelien notes that WP09 (constraint engine) depends on the ability +to clone threads. Christian thinks that the upcoming stackless +support will allow the required usages. Aurelien promises to write up +documentation about his implementation approaches and goals +and the needed features but also notes that the need for tasklet +cloning was mentioned two months ago. + + +2. ext compiler being able to produce extensions for CPython and PyPy +------------------------------------------------------------------------ + +Armin has done a lot of progress on the revised WP03 approach +which we communicated to the EU with the fourth amendment +of our contract. The rctypes approach is already working quite well +and he is currently working on a cpyobjspace which is then used +for writing and testing mixed modules. We are going to +require explicit wrapping, i.e. users will need to care explicitely +for the interp/applevel distinction and cannot expect +automatic wrapping. However, it is envisioned that for past +0.9 development we might investigate "automatic" approaches on which +Christian has done some work already. + +For 0.9 everyone agrees and settles on the explicit approach. + +Armin will take care to bring the code to the point where we +have a command line tool which can produce CPython or PyPy +extension modules from a single (mixed module) source. + +3. GC framework integration + __del__ and maybe weakref +------------------------------------------------------------------------ + +For 0.9 Carl will continue his GC framework integration work +and also try to implement __del__ and a new approach for weakref +support to work with all GCs. Especially the weakref support +needs to be discussed and experimented with further. Some +approaches lead to rather large performance penalties so +this remains a matter of experimentation. + +4. high-level backend snapshots + documentation? +------------------------------------------------------------------------ + +We have a number of sub projects going on to implement high level +backends on top of ootype model: .NET, Squeak and recently a LISP +one. After short discussions everybody welcomed the idea and +would be glad to include a snapshot of the high-level backend +works. For 0.9 this requires writing documentation. Holger will care +to ask Antonio and Samuele will ask Seo about it. Niklaus +already mentioned at the meeting that he will try to write +some documentation about ootypes. + + +5. logic/constraint related goals? +------------------------------------------------------------------------ + +Aurelien states that basic constraint coding work has been done +though he is not sure that it can be translated. He sees a +correlation to WP10 work and also mentions that he will need +some help. Holger notes that the D9.1 report is thought +to have an intermediate draft ready in June (as the original +deliverable "initial constraint satisfaction engine" was scheduled +for that time). Aurelien agreed to document their current approaches +so that they can be advertised and snapshotted with 0.9 and +also state missing/blocking features. + + +6. _some_ amount of release polishing and updating documentation, fixing issues +--------------------------------------------------------------------------------------- + +Holger reminded everyone that we have some 20-30 open issues for 0.9 +in the tracker at http://codespeak.net/issue/pypy-dev/ . +It was commonly agreed that everyone goes through the list and +takes responsibility for some issues. + + +last +-------- + +we are going to to have a follow up eu sync meeting mid may or maybe +at the Iceland sprint to assess ongoing and remaining 0.9 work +as well as develop plans for the remainder of the EU project. +This is connected to an assessment of resource and time usage +at consortium-level. + +IRC log +-------------------------- + +:: + + Apr 25 17:02:28 arre: hi arre, is samuele going to join soon? + Apr 25 17:02:51 +I don't know. We have separate rooms. + Apr 25 17:03:05 i wouldn't think he sleeps already :) + Apr 25 17:03:24 --> nikh (n=nikh at 222-151-092-147.jp.fiberbit.net) has joined #pypy-sync + Apr 25 17:04:09 +we need to wait for Chsritian, I suspect + Apr 25 17:04:14 +Christian sorry + Apr 25 17:04:18 and samuele + Apr 25 17:04:25 +yes + Apr 25 17:04:30 otoh we can at least start and see how we want to go about things + Apr 25 17:04:43 without deciding any content topic + Apr 25 17:04:50 because i think that part of the problem is sorting issues and topics out + Apr 25 17:04:52 +hello + Apr 25 17:05:09 hi michael + Apr 25 17:05:47 can everyone here state already how long you have time or at which time you would like to limit this meeting? + Apr 25 17:06:02 +i have an hour, easily + Apr 25 17:06:20 +no limit ... + Apr 25 17:06:35 +I have no real limit, except that it might be hard to get home at one point :-) + Apr 25 17:07:12 +there's a #europython meeting at 6, but i don't know who is going to turn up + Apr 25 17:07:18 +and they can be overlapped + Apr 25 17:07:34 --> aleale (n=aleale at 222-151-094-211.jp.fiberbit.net) has joined #pypy-sync + Apr 25 17:07:34 --> stakkars (n=tismer at 66.151.59.5) has joined #pypy-sync + Apr 25 17:07:38 +Since its 0:07 here, shorter meeting => more sleep. + Apr 25 17:07:47 +:-) + Apr 25 17:07:51 stakkars stedi67 Apr 25 17:07:51 +arre: :) + Apr 25 17:08:00 stakkars: morning christian! + Apr 25 17:08:16 +sorry for being late - my battery is broken, so I had to move to the office + Apr 25 17:08:19 +Thiugh 1 hour wont hurt too much. + Apr 25 17:08:24 +sorry I am late - networktrouble + Apr 25 17:08:33 +and this was not a nice night at all - nevermind + Apr 25 17:08:34 ok, i'll try to keep the meeting at around 1 hour + Apr 25 17:08:53 up front, i'd like to not try to resolve every topic that turns up + Apr 25 17:09:09 but rather resolve to delegation and organising who cares for which area in separate chats + Apr 25 17:09:17 --> pedronis (n=Samuele_ at 222-151-092-147.jp.fiberbit.net) has joined #pypy-sync + Apr 25 17:09:29 +that's the way to go! + Apr 25 17:09:51 ok, then let me suggest a bit of structure + Apr 25 17:09:59 pedronis: evening samuele! + Apr 25 17:10:18 the main issue is organising work for 0.9 + Apr 25 17:10:26 which is scheduled to happen first half june + Apr 25 17:10:50 +ACTION nods and grabs coffee + Apr 25 17:11:12 regarding having a deeper look into the remainder of the EU project + Apr 25 17:11:24 we might not cover it during this meeting although it plays a role in the background + Apr 25 17:11:59 for the latter i suggest that we work on getting a full picture (until december) in May + Apr 25 17:12:18 which also co-incides with the consortium-level assessment of resources + Apr 25 17:12:27 resource usage and time + Apr 25 17:12:40 +i guess by may you mean "in iceland" ? + Apr 25 17:12:49 maybe, yes, that might make sense + Apr 25 17:13:14 but logilab won't be there, i think, for xample + Apr 25 17:13:27 +yup + Apr 25 17:13:37 +hrm + Apr 25 17:14:11 if there are no objections i suggest to discuss 0.9 now + Apr 25 17:14:13 + still, it's going to be the largest in person gathering for a while + Apr 25 17:14:17 +hpk: yes + Apr 25 17:14:26 mwh: yes, we will certainly make some use of that + Apr 25 17:14:55 here is a list of topics that i suggest to discuss one after another and deciding what we do about it regaridng 0.9 + Apr 25 17:15:08 +mwh: my babelfish is sleeping. what do you mean by "still, it's going to be the largest in person gathering for a while" + Apr 25 17:15:21 +lots of people ? + Apr 25 17:15:27 +(will be there) + Apr 25 17:15:29 stakkars stedi67 Apr 25 17:15:42 +yes + Apr 25 17:15:45 stakkars: 8-9 EU p?roject related people are there + Apr 25 17:16:09 ok, here is the list: + Apr 25 17:16:13 1. stackless support + Apr 25 17:16:13 2. ext compiler being able to produce extensions for CPython and PyPy + Apr 25 17:16:13 3. GC framework integration + __del__ and maybe weakref + Apr 25 17:16:13 4. high-level backend snapshots + documentation? + Apr 25 17:16:13 5. logic/constraint related goals? + Apr 25 17:16:13 6. _some_ amount of release polishing and updating documentation, fixing issues + Apr 25 17:16:33 are you fine with going through them one by one and deciding about their 0.9 fate? + Apr 25 17:16:42 +sure + Apr 25 17:16:58 +yes + Apr 25 17:17:05 is there anything more you'd like to add as a major topic? + Apr 25 17:17:14 +hpk: ... + Apr 25 17:17:25 +what do you mean by logic/constraint "related goals" ? + Apr 25 17:17:41 the question what (if anything) 0.9 should include + Apr 25 17:17:51 regarding wp09/wp10 + Apr 25 17:17:51 +constraint : yes + Apr 25 17:17:58 +logic : probably not + Apr 25 17:18:03 auc: please not now, i'd like to try to keep some structure :) + Apr 25 17:18:10 but thanks anyway + Apr 25 17:18:19 ok, first topic then is stackless support + Apr 25 17:18:29 which also has a report due in June IIRC + Apr 25 17:19:03 +well, i've now read stakkars's document + Apr 25 17:19:16 +I did some work on structuring this, and it turns out to be doable in rather short time. + Apr 25 17:19:29 +pickling ? + Apr 25 17:19:33 +you think so? + Apr 25 17:19:38 +auc: yes + Apr 25 17:19:53 stakkars: you mean the full wp07? + Apr 25 17:20:06 +given that - yes - we are able to identify and remove blocks in tail-position, this is the blocker, if any. + Apr 25 17:20:20 +stakkars: you would seem to be approaching things from rather the other end of things than the current "stackless" code in pypy does + Apr 25 17:21:05 +no, the stackless code in PyPy approaches things from the other side that Stgackless does, and this is the reference :-) + Apr 25 17:21:19 +well, ok + Apr 25 17:21:30 either way, there is not a common approach yet + Apr 25 17:21:37 +but which side do you think tasklet pickling in pypy should come from? + Apr 25 17:21:38 +sure there is + Apr 25 17:21:50 +can be make the precise goals more explicit? + Apr 25 17:22:03 +s/be/we + Apr 25 17:22:03 +(there is also other stackless stuff: channels, greenlets, maybe) + Apr 25 17:22:07 +sorry that I could not finish my document last nite. + Apr 25 17:22:25 +please let's stick with one topic at the time + Apr 25 17:22:44 +ok + Apr 25 17:23:24 +the missing spot with stackless PyPy is that we create very many activation records which are not needed. 95 percent of them can be removed. + Apr 25 17:23:52 +and my approach is to exactly allow pickling in a context where this removal is possible. + Apr 25 17:24:00 +stakkars: I'd still like to state the goals we are seeking more precisely + Apr 25 17:24:19 +was I not precise? + Apr 25 17:24:19 -well, is rather unclear how to do that, without changing the interpreter + Apr 25 17:24:35 +I don't think so. + Apr 25 17:24:41 -arigo: I think stakkars goals is to pickle as mostly a chain of python-level frames + Apr 25 17:25:18 +yes. going further would drive us into deeper problems right now, since we have no interface at all. + Apr 25 17:25:30 +the onl?y valid interface IMHO is the python frame. + Apr 25 17:25:36 +but more precisely: is our first goal to pickle only chains of Python-level frames and give up in other cases, or cover the general case inefficiently and then progress to make it more efficient for the common case? + Apr 25 17:26:12 +arigo: what would be "other cases" ? + Apr 25 17:26:33 +from a POV about what we promised, I see no direct reason to go any further than Stackless did in the past. + Apr 25 17:26:35 +auc: any situation where a Python function is not called just by a direct function call from another Python function + Apr 25 17:27:03 +arigo: actually, these cases do not differ so much. + Apr 25 17:27:21 +there is a hand-ful of real cases, where extra information is needed. + Apr 25 17:27:51 +this is when a series of calles cannot unwind without keepin state. + Apr 25 17:27:59 +I somehow disagree with that, but that answers my question + Apr 25 17:28:03 +for instance: think about map() + Apr 25 17:28:57 -well, but some builtin are app-level defined + Apr 25 17:29:01 +there are ways to handle such cases, if it is worth the effort. + Apr 25 17:29:24 +one way is to move the problem to app-level, exactly. + Apr 25 17:29:51 +I just wanted to make it clear that there was a very different approach of pickling at RPython level, which is general but full of other problems, which is not the approach you want + Apr 25 17:29:57 +another one is to create special frames or structures which take care. + Apr 25 17:31:04 +so the work here is: implement normal pickling for various things, and provide an interface for RPython code to at least check the RPython call chain + Apr 25 17:31:09 +I understand that. In PyPy's current state, I can't see how to do this in a generally useful way. + Apr 25 17:31:17 +...to check that we're in a simple case. + Apr 25 17:31:42 +this seems to be the simplest possible approach for now, yes. + Apr 25 17:31:44 +stakkars: yes, I agree that pickling RPython frames is full of problems and somehow very fragile + Apr 25 17:31:45 +ok + Apr 25 17:31:56 +yes + Apr 25 17:32:13 +you could do full-image dumping with only moderately extreme amounts of effort + Apr 25 17:32:26 +but, well, who cares + Apr 25 17:32:46 +and I agree that I did not tell the full story, since my paper isn't finished. There are a handfulof different cases which need to be handled. + Apr 25 17:32:47 +that's a feature I tend to consider mostly useless... + Apr 25 17:32:54 +it's not that interesting a solution imho + Apr 25 17:32:59 +indeed + Apr 25 17:33:06 -arigo: but even in the simple case you need ways to recreate the c stack or avoid it to be there + Apr 25 17:33:16 +mwh: I would agree with this if we had a flexible backend, where programs are data. + Apr 25 17:33:18 -??s unclear to me how the intepreter needs to change to get that + Apr 25 17:33:28 +pedronis: ah yes, we need to throw it away or rebuild it + Apr 25 17:33:40 +mwh, what do you mean by full image dump? + Apr 25 17:33:58 +well, avoiding callbacks from external libraries, we can recreate the c stack + Apr 25 17:34:11 +because this is how the stackless-in-pypy works + Apr 25 17:34:12 +Gromit: being able to safe the _whole_ program + Apr 25 17:34:12 +Gromit: maybe something a la smalltalk/lisp + Apr 25 17:34:21 +mwh: the problem is how to know which RPython-level values to put back into the RPython frames + Apr 25 17:34:36 +so i understood it right, well I would care + Apr 25 17:34:39 -arigo: yes + Apr 25 17:34:42 +mwh: I consider dumping the binary a fake solution. + Apr 25 17:34:49 +i don't see why this is sooooooo hard + Apr 25 17:34:51 +but also! + Apr 25 17:34:54 +it's a dead topic + Apr 25 17:34:54 -unless we make the intepreter itself sort of stackless + Apr 25 17:35:01 +Gromit: nobody of them has tried smalltalk :) + Apr 25 17:35:04 +feel free to enlighten me later + Apr 25 17:35:09 anyway + Apr 25 17:35:21 i don't think we can fully resolve this without sorting out some more details in written form + Apr 25 17:35:25 +going this road would kill parts of wp9 + Apr 25 17:35:33 +(as we see it) + Apr 25 17:35:35 so here is a suggestion + Apr 25 17:35:36 stakkars stedi67 Apr 25 17:36:01 stakkars should complete his document some more until end of the week + Apr 25 17:36:15 and others may want to amend it or describe a different approach + Apr 25 17:36:20 +better today, or it won't happen :-) + Apr 25 17:36:46 and then at least stakkars, arigo, mwh and pedronis get together and agree on an approach (and anyone can join who wants to in this discussion) + Apr 25 17:36:55 +I will try to describe how to identify removable code paths. + Apr 25 17:36:57 is that agreeable? + Apr 25 17:37:00 +stakkars: are you able to devote serious time between now and iceland on implementing this stuff? + Apr 25 17:37:08 +hpk: yes + Apr 25 17:37:29 +sure! I want all this stuff off my table until then. + Apr 25 17:38:04 +cool + Apr 25 17:38:30 +on the other question: the missing stackless interface will be done by an expert on this. + Apr 25 17:38:32 we need a common approach and i want at least the four people i mentioned to agree on it, so arigo and pedronis, are you fine with that as well? + Apr 25 17:38:33 +btw + Apr 25 17:38:37 +i am on holiday next week + Apr 25 17:38:53 +don't expect me to comment when i'm backpacking in the highlands :) + Apr 25 17:39:19 well, you should try to resolve it until end this week, i'd say + Apr 25 17:39:22 +hpk: yes, though I now see pedronis' objections as important + Apr 25 17:39:39 +but that should not be discussed here + Apr 25 17:39:48 +which objectioms? did I forget something? + Apr 25 17:40:37 arigo: i wasn't implying a consensus or that everyone agrees already + Apr 25 17:41:01 the fact is that tasklet pickling is just _one_ issue we are dealing with + Apr 25 17:41:06 +stakkars: no, I think you've mostly thought it out, but we'll need to state it clearly: how to rebuild the interp-level frames when restoring (not now, put it in the docs) + Apr 25 17:41:27 +hpk: I know, I wanted to see if there was a critical problem, but not necessarily + Apr 25 17:41:35 +ok ok, I'll do the fine-print :-) + Apr 25 17:41:36 +hpk: so we can move on as far as I'm concerned + Apr 25 17:41:41 pedronis: are you ok with being part of the decision group? + Apr 25 17:41:52 +stakkars: :-) + Apr 25 17:41:57 -yes + Apr 25 17:42:03 ok, then we are settled for that + Apr 25 17:42:15 -(tomorrow is break day here btw) + Apr 25 17:42:19 +what was auc's concern, was there a contradiction? + Apr 25 17:42:34 mwh: can you care to co-ordinate this decision and make an appointment / posting it to pypy-dev? + Apr 25 17:42:56 +pedronis: did you just invite me for discussion, or the opposite? + Apr 25 17:42:57 otherwise let me note that WP07 contains more issue than just tasklet pickling + Apr 25 17:43:02 +stakkars: we think we depend on stackless pickling mecanisms to make part of wp9 work + Apr 25 17:43:02 stakkars stedi67 Apr 25 17:43:12 +given the amount of rushing around i'm doing, i'm not sure i'm the best man for the job + Apr 25 17:43:21 +stakkars: note that i don't understand anything about the issues involved + Apr 25 17:43:23 i see, arigo, can you care? + Apr 25 17:43:25 +this is a thing I wished you had told me some months ago. + Apr 25 17:43:27 +stakkars, auc: the conclusion was that full-program dumping is not useful for wp9, which is fine + Apr 25 17:43:36 +and possible you were talking about some mecanisms which do not conern us + Apr 25 17:43:40 +until now, nobody every relied on this feature. + Apr 25 17:43:48 * hpk ?would really like to have for all the issues exact people caring for things and not just leave things in the air + Apr 25 17:44:08 stakkars stedi67 Apr 25 17:44:18 +stakkars: we stated, months ago that thread pickling was a dependency for us + Apr 25 17:44:23 +yes + Apr 25 17:44:27 +but no problem + Apr 25 17:44:29 -stakkars: no, that tomorrow I'm not going to be around + Apr 25 17:44:43 +hpk: I'll send a note with a meeting time on Thursday then + Apr 25 17:44:51 +you'll get it, but probably not more than Stackless can give you + Apr 25 17:44:58 +auc: is it documented? :) + Apr 25 17:44:58 +if that's fine with all of you + Apr 25 17:45:16 +xorAxAx: not published, but i can do something about that + Apr 25 17:45:24 ok, so the tasklet pickling issue is delegated to arigo, stakkars, pedronis and mwh to sort out and agree on + Apr 25 17:45:42 +hpk: +1 + Apr 25 17:45:44 +aye aye + Apr 25 17:45:48 +we'de like to know more in details the feature set ... + Apr 25 17:46:06 auc: you may also write down some features you need, if you like, ok? + Apr 25 17:46:14 +ok + Apr 25 17:46:15 +i'll post them soon + Apr 25 17:46:19 good, thanks + Apr 25 17:46:43 +I guess they want to pickle from interpreter-level, which I think is doable as well with similar restrictions + Apr 25 17:47:03 just so we don't forget it: wp07 contains more (pointer tagging, pre-emptive scheduler, study memory<->speed tradeoffs, object layout, multimethod optimisations) + Apr 25 17:47:16 +it is just necessary to tame myriads of activation records, this is all I'm saying. + Apr 25 17:47:29 and i'd dare to say it's doubtful we'll get all that until June including an EU report, or does anyone disagree? :) + Apr 25 17:48:01 +I can take the pre-emptive scheduler as well. + Apr 25 17:48:20 +having that ready before June. + Apr 25 17:48:26 i'd like to not discuss that now, but we should not think that WP07 is done by caring/resolving tasklet pickling + Apr 25 17:48:29 +well, we can argue we've done _some_ of those things + Apr 25 17:48:33 yes, i know + Apr 25 17:48:53 still we need to look into things a bit more there + Apr 25 17:49:07 latest in May (when we plan for the rest of the project) + Apr 25 17:49:19 ok, so topic 1. for release 0.9 is so fr handled + Apr 25 17:49:21 let's head on + Apr 25 17:49:28 2. ext compiler being able to produce extensions for CPython and PyPy + Apr 25 17:50:06 <-- auc has quit (Remote closed the connection) + Apr 25 17:50:06 here again, we need agreement on how we go about it for 0.9, including the actual approach + Apr 25 17:50:25 i presume that everyone is familiar with the pypy-dev posts related to the topic + Apr 25 17:50:27 +what is the actual approach + Apr 25 17:50:54 --> auc (n=auc at logilab.net2.nerim.net) has joined #pypy-sync + Apr 25 17:51:03 +at the moment I'm progressing on the cpyobjspace + Apr 25 17:51:04 +you don't mean the half-way that I've taken, but Armin's new approach, I guess? + Apr 25 17:51:20 +the two approaches can converge at some point + Apr 25 17:51:24 +(which I like very much) + Apr 25 17:51:32 i was just refering to the pypy-dev posts and indeed armin did describe his ideas there to some detail + Apr 25 17:51:35 +for now the first goal would be to translate a simple MixedModule to CPython + Apr 25 17:51:45 +with no SomeObjects :-) + Apr 25 17:51:58 +well, progress is good, basically + Apr 25 17:52:11 +I wanted, but then didn't try to defend mine, since I agree that Armin's is the way to go. + Apr 25 17:52:50 ok, so we'll base the ext-compiler on the rctypes+cpyobjspace+explicit wrapping approach. + Apr 25 17:53:00 +stakkars: keep your code in a corner, I know that removing some of this space.xyz() boilerplate will be nice later :-) + Apr 25 17:53:19 +on the other hand, I would appreciate to send SomeObjects into never-never-land + Apr 25 17:53:28 +i had a vague thought about this a few days ago, but this probably isn't the place... + Apr 25 17:53:53 +(for a SomeGenericObject as well as the "i don't know" SomeObject) + Apr 25 17:54:01 ok, i don't hear any disagreements so far + Apr 25 17:54:03 a question: + Apr 25 17:54:20 how exactly is a user supposed to use the "ext-compiler"? + Apr 25 17:54:30 +me too, I'd like to keep the annotation instead of loosing it. Well, wrong place... + Apr 25 17:54:49 +mwh, stakkars: seems like we all thought about this + Apr 25 17:55:15 +you bet why I can't sleep since weeks + Apr 25 17:55:16 +hpk: for now, the goal is: write a MixedModule for PyPy and run a command-line utility + Apr 25 17:55:21 +arigo: ok + Apr 25 17:55:22 +later! + Apr 25 17:55:37 arigo: which can produce a cpython ext + Apr 25 17:55:44 +hpk: yes, you get a .so for CPython with no extra effort + Apr 25 17:56:02 ok + Apr 25 17:56:15 +there will also be a simple way to import and test the MixedModule directly on top of CPython + Apr 25 17:56:27 +by running it with the CPyObjSpace, which works well already in simple tests + Apr 25 17:56:28 +ok, modulo a non-existent space variable, and if you use the interp-level part, only, then I have something like that ready + Apr 25 17:56:29 arigo: are you going to care for that up to the point that this works reasonably? (i know you are far but still :)) + Apr 25 17:56:52 arigo arre Apr 25 17:56:57 arigo: the testing part sounds great :) + Apr 25 17:57:00 +actually I don't think I'm that far any more, so yes :-) + Apr 25 17:57:01 +and I'm of course happy to move things to CPyObjSpace + Apr 25 17:57:20 +hehe + Apr 25 17:57:34 ok, so arigo heads off and finishes this and maybe should then post to pypy-dev (well ahead of 0.9) so that people can feedback and try it out on some things (maybe also in iceland) + Apr 25 17:57:34 +it all falls into place magically also thanks to all the pyobj.py magic that creates the correct wrappers in genc + Apr 25 17:57:59 everybody fine with this? + Apr 25 17:58:06 +ACTION agrees + Apr 25 17:58:19 +ACTION wonders but of course agrees + Apr 25 17:58:28 -+1 + Apr 25 17:59:24 +I need a short break -- my stomach... + Apr 25 18:00:11 ok, i take it that the others at least don't disagree, but wouldn't mind to hear explicit opinions + Apr 25 18:00:20 +I agree, fwiw + Apr 25 18:00:34 +i agree + Apr 25 18:00:36 +no opinion + Apr 25 18:00:37 ++1 + Apr 25 18:00:45 +tis++ + Apr 25 18:01:00 thanks :) + Apr 25 18:01:08 next topic then + Apr 25 18:01:11 3. GC framework integration + __del__ and maybe weakref + Apr 25 18:01:36 cfbolz: following up on your gc integration work with samuele, you are now heading for __del__, right? + Apr 25 18:01:54 +I am currently working on testing the gcs differently, which is a mess, but basically yes + Apr 25 18:02:12 +I didn't understand that one. There is __del__, isn't it? Or dependent from which gc it is? + Apr 25 18:02:21 +stakkars: yes, not for all gcs yet + Apr 25 18:02:37 --> Gromit_ (n=gek at codespeak.net) has joined #pypy-sync + Apr 25 18:02:56 +stakkars: it works for boehm and refcounting + Apr 25 18:03:01 +but not for mark and sweep + Apr 25 18:03:05 question is if we shouldn't at least try to get a weakref module working + Apr 25 18:03:20 it may not be crucial for 0.9 exactly but it's a long standing issue and it's good to have that resolved at some point + Apr 25 18:03:31 <-- Gromit has quit (Nick collision from services.) + Apr 25 18:03:51 --- Gromit_ is now known as Gromit + Apr 25 18:03:52 any opinions on that? + Apr 25 18:03:55 +what is the status of the idea of implementing weakrefs purely at interp-level, by playing with __del__? + Apr 25 18:03:56 +I'd like to understand why weakref is a problem. wrong list, I know + Apr 25 18:04:08 +arigo: it works, but is extremely slow for boehm + Apr 25 18:04:12 +arigo: wasn't it that it murdered performance? + Apr 25 18:04:23 +what about using extra objects? + Apr 25 18:04:25 +well, I am (hust) using weakrefs. :-> + Apr 25 18:04:36 stakkars stedi67 Apr 25 18:04:40 +as in, we have a field that is usually NULL but can point to an object with a __del__... + Apr 25 18:04:41 +arigo: ? + Apr 25 18:04:50 +ah + Apr 25 18:04:58 +then we pay the __del__ penalty only if there is a weakref actually taken to that object + Apr 25 18:05:10 +yes, could possibly work + Apr 25 18:05:20 +then it's a quick hack to try + Apr 25 18:05:21 +and could be tried reasonably easy, I guess + Apr 25 18:05:25 +:-) + Apr 25 18:05:28 ok, let's do that then :) + Apr 25 18:05:38 +right now we are missing an operation that would handle this field correctly + Apr 25 18:05:49 +stakkars: no, I don't think so + Apr 25 18:05:59 +the cool thing about this approach would be that it wporks for all gcs that support __del__ + Apr 25 18:06:03 +it's just a plain RPython trick + Apr 25 18:06:06 +yes + Apr 25 18:06:40 +I'm always happy to get enlightened (again not here of course) + Apr 25 18:06:46 me too + Apr 25 18:06:58 +I guess I could write that done until end this week + Apr 25 18:07:05 +or maybe even just try it + Apr 25 18:07:18 ok, try it out then. + Apr 25 18:07:41 +ACTION sorry have to leave for 3 minutes + Apr 25 18:07:44 i think we should find out how we can get to supporting weakref some time not too far away + Apr 25 18:08:02 +ok, that all sounds good :) + Apr 25 18:08:06 so i think we can close the topic for now + Apr 25 18:08:07 +indeedy + Apr 25 18:08:10 +think so + Apr 25 18:08:29 4. high-level backend snapshots + documentation? + Apr 25 18:09:10 i suggest we ask Antonio (and maybe Seo) to document in May the status and usability of their ootype-based backends and that we mention it in the release + Apr 25 18:09:50 it's not strictly EU related but it's part of the project and we should thus think of it accordingly + Apr 25 18:10:02 +yes, I think it will attract people too to see things compiled for very different platforms + Apr 25 18:10:10 +maybe Jython people too + Apr 25 18:10:27 +(there was a Parrot guy the other day on #pypy) + Apr 25 18:10:34 yes, in some sense "0.9" is a "community release" :) + Apr 25 18:10:38 <-- mwh has quit (Remote closed the connection) + Apr 25 18:11:07 --> mwh (n=mwh at fwstups.cs.uni-duesseldorf.de) has joined #pypy-sync + Apr 25 18:11:18 if everyone is fine with it, i'll ask Antonio (who needs to write up stuff for his thesis early July anyway :) + Apr 25 18:12:07 +yes, also if other people want to step in they are welcome (nikh, ericvrp... hint hint) + Apr 25 18:12:07 pedronis, arre: isn't Seo heading into a CL backend based on ootypes as well? + Apr 25 18:12:11 +hehe + Apr 25 18:12:41 -hpk: yes, the work happening here on the CL backend is ootype based + Apr 25 18:13:10 pedronis: can you see at the end of the sprint where things are and if Seo or someone else wants to work further on it and document it for 0.9? + Apr 25 18:13:41 -will ask + Apr 25 18:13:43 +ACTION wakes up + Apr 25 18:13:58 nikh: hey nikh, indeed i forgot to ask you about gensqueak :) + Apr 25 18:14:05 +unfortunately i won't have much time in may/june for pypy ... but maybe i can help a bit with ootypesystem documentation. + Apr 25 18:14:26 +i'd happily hack some bits of gencl + Apr 25 18:14:41 sounds all good, ok then, i think that settles this topic + Apr 25 18:14:53 5. logic/constraint related goals? + Apr 25 18:15:21 origianlly we promised to have an "initial constraint satisfaction engine" ready in May i think + Apr 25 18:15:31 +basic constraint stuff is there + Apr 25 18:15:35 +without doubt + Apr 25 18:15:45 +ah, but does it translate ? + Apr 25 18:16:05 +i postponed this when i reached eval() + Apr 25 18:16:26 +which should disappear with the help of wp10 (maybe) + Apr 25 18:16:32 +s/should/might + Apr 25 18:16:36 ok + Apr 25 18:17:10 +logic programming needs help + Apr 25 18:17:11 +re + Apr 25 18:17:21 auc: i think d09.1 is a good candidate for an intermediate EU report, btw + Apr 25 18:17:27 +and we decided it would come from tasklet *cloning* + Apr 25 18:17:33 because originally the deadline for an initial engine was May or June + Apr 25 18:18:06 +it's ok + Apr 25 18:18:15 +well it depends + Apr 25 18:18:17 +cloning will be possible. Not supporting continuations in the first place, but virtually + Apr 25 18:18:20 +it does not translate + Apr 25 18:18:28 +by unpicklijng. I will elaborate on that. + Apr 25 18:18:49 +cloning by pickling, yes + Apr 25 18:18:56 +sounds heavy and performance killing + Apr 25 18:18:58 +but ... + Apr 25 18:19:02 +we want features first + Apr 25 18:19:11 +gee, not at all. + Apr 25 18:19:23 +ah ? + Apr 25 18:19:26 +nice, then + Apr 25 18:19:55 +pickling can be made very cheap, especially if you avoid to create the pickle at all. + Apr 25 18:20:00 +stakkars: because when doing search we might want to clone a lot ... + Apr 25 18:20:12 +stakkars: absolutely we're not intersted in the pickle + Apr 25 18:20:37 +so that's good news + Apr 25 18:20:41 +you just use the interface to reak the object into its spare parts, then re-assemble it. No strings attached :-) + Apr 25 18:20:48 +break + Apr 25 18:20:49 auc: ok, but regarding 0.9: can you write up your status and begin to document your approaches so they can be advertised and snapshotted with 0.9? Together with missing/blocking features from your perspectives? + Apr 25 18:21:04 +hpk: yes + Apr 25 18:21:16 thanks + Apr 25 18:21:30 then i think that we can head to the last topic for 0.9 + Apr 25 18:21:42 6. _some_ amount of release polishing and updating documentation, fixing issues + Apr 25 18:22:07 +auc: the tiny difference to continuations is that I can produce a clone for you, which is not identical. The latter would be an order of magnitude harder, see Stackless 10 + Apr 25 18:22:35 +what do you mean by 'not identical' ? + Apr 25 18:22:44 here i note that not many people signed up for any issues currently marked as 0.9 in the tracker + Apr 25 18:22:49 +stakkars, auc: next topic :-) + Apr 25 18:23:13 +a different object. continuations keep object identity, re-startable many times. + Apr 25 18:23:17 +oki + Apr 25 18:23:31 +hm, we'll see + Apr 25 18:23:40 +it'l be fine + Apr 25 18:24:19 +ACTION hopes so, since everything requiring full continuations has turned out to be insane during the last 7 years + Apr 25 18:25:00 +stakkars: don't worry + Apr 25 18:25:24 so are there any suggestions? + Apr 25 18:25:33 +uh ? + Apr 25 18:25:36 i personally don't believe that 0.9 will just magically ship :) + Apr 25 18:25:45 +about what? sorry I lost track + Apr 25 18:25:51 stakkars stedi67 Apr 25 18:25:58 stakkars: i called for the last topic and discussing it + Apr 25 18:26:04 6. _some_ amount of release polishing and updating documentation, fixing issues + Apr 25 18:26:05 +ah, strategy about how to go for 0.9? + Apr 25 18:26:23 we have some 20-30 issues for 0.9, not all of them crucial so can be postponed if neccessary + Apr 25 18:26:29 however, postponing all of them is probably not a good idea + Apr 25 18:26:43 and we need everybody to go through the list and sign up for some + Apr 25 18:26:47 in order to distribute the work a bit + Apr 25 18:26:52 i think it's sensible to ask this of everyone + Apr 25 18:26:54 +would it make sense to do a round of everybody grabing issues that he feels like doing in the next two days and we see what's left + Apr 25 18:26:59 +? + Apr 25 18:27:31 yes, that makes sense IMO + Apr 25 18:27:56 +ACTION nods. + Apr 25 18:27:57 +yes + Apr 25 18:28:15 +but I also think we should devote some time to nice docs + Apr 25 18:28:21 +to make the release accessible + Apr 25 18:28:34 +otherwise there is not much sense in releasing :-) + Apr 25 18:28:36 which mainly - i think - refers to updating the translation documentation + Apr 25 18:28:39 +note that thanks to holiday i won't grab any in the next days + Apr 25 18:28:43 and writing some about the ext-compiler and stackless support + Apr 25 18:28:48 +hpk: yes + Apr 25 18:28:53 +and will see what remains ungrabbed when i get back + Apr 25 18:29:03 +:-) + Apr 25 18:29:05 ok, but please take this seriously + Apr 25 18:29:13 if we all work on it a bit it's much easier to get something nice + Apr 25 18:29:34 +I'd love to help with this - will do! + Apr 25 18:29:59 ok, then i thank everybody for coming and for discussing constructively + Apr 25 18:30:14 +bussies from LA + Apr 25 18:30:33 and suggest that we get together around mid may latest in this constellation to assess how far we are and what remains + Apr 25 18:30:52 +yes + Apr 25 18:30:54 +ACTION -> bed + Apr 25 18:30:55 <-- nikh has quit ("zzzz") + Apr 25 18:30:57 +are we doing a meeting on Thursday? + Apr 25 18:31:05 +stakkars: probably + Apr 25 18:31:06 +0.9 is aimed for ~9 june? + Apr 25 18:31:11 mwh: yes, around that + Apr 25 18:31:27 +so work like crazy for a few weeks, assess again in iceland, then work to the release? + Apr 25 18:31:30 +ok + Apr 25 18:31:35 +yup + Apr 25 18:31:36 yes + Apr 25 18:31:48 stakkars stedi67 Apr 25 18:32:14 i'd rather do a pypy-sync the next week and would send a note to pypy-dev accordingly + Apr 25 18:32:24 +I think meeting time could be more convenient for people if we let it happen at midnight LA time? + Apr 25 18:32:50 unless anyone thinks we have some immediate pressing topics for this thursday + Apr 25 18:32:54 +stakkars: well, the tokyo sprint is only this week, so the problem does not really occur again + Apr 25 18:33:29 +it would, for next Thursday + Apr 25 18:33:37 ? + Apr 25 18:33:39 right + Apr 25 18:33:47 +yes, sorry + Apr 25 18:34:02 that's why (and because of today's meeting and the ongoing tokyo sprint) i propose to go for the next meeting next week + Apr 25 18:34:32 mwh, cfbolz: please remind everyone regarding SoC btw :) + Apr 25 18:34:33 +stakkars: but LA midnight is quite early in the morning for Europe I fear + Apr 25 18:34:47 +hpk: ah, yes + Apr 25 18:34:51 +and my idea is to try to get the largest time gap to where most people are sleeping. + Apr 25 18:34:52 +arigo, 9am AFAIK + Apr 25 18:35:07 +no, it is nine'o clock, which is an advantage + Apr 25 18:35:15 +which is not a realistic time for me at _any_ weekday (lectures) + Apr 25 18:35:47 <-- arre has quit (Remote closed the connection) + Apr 25 18:36:01 can we go for 06:00 pm (UTC+2) next thursday 4th May? + Apr 25 18:36:25 +fine for me + Apr 25 18:36:53 +ok here + Apr 25 18:37:05 +fine with me. Why this late, is anybody travelling, then? + Apr 25 18:37:12 +probably not + Apr 25 18:37:20 stakkars: so that you don't have to get up too early :) + Apr 25 18:37:26 and can grab a coffee and such + Apr 25 18:37:30 +I'm back on the 1st or 2nd of May + Apr 25 18:37:39 aha, sorry i missed this + Apr 25 18:37:54 then it's all a bit less critical + Apr 25 18:38:16 +you're nice. No, I have to go back and save my son's ass. + Apr 25 18:38:27 yeah, heard about that ... not easy + Apr 25 18:38:50 fwiw: the meeting is closed from my side (if it is not closed already) + Apr 25 18:39:10 +thanks all + Apr 25 18:39:16 -ok, going to bed here + Apr 25 18:39:19 -bye + Apr 25 18:39:23 good night + Apr 25 18:39:25 +pedronis: night! + Apr 25 18:39:44 +bye From nik at codespeak.net Thu Apr 27 09:26:00 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 09:26:00 +0200 (CEST) Subject: [pypy-svn] r26409 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427072600.25C681009D@code0.codespeak.net> Author: nik Date: Thu Apr 27 09:25:57 2006 New Revision: 26409 Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py Log: use Records for tuple iterators. Modified: pypy/dist/pypy/rpython/ootypesystem/rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rtuple.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rtuple.py Thu Apr 27 09:25:57 2006 @@ -1,6 +1,5 @@ from pypy.rpython.rmodel import inputconst from pypy.rpython.rtuple import AbstractTupleRepr, AbstractTupleIteratorRepr -from pypy.rpython.ootypesystem.riterable import iterator_type from pypy.rpython.ootypesystem import ootype @@ -67,7 +66,8 @@ def __init__(self, r_tuple): self.r_tuple = r_tuple - self.lowleveltype = iterator_type(r_tuple) + self.lowleveltype = ootype.Record( + {"iterable": r_tuple.lowleveltype, "index": ootype.Signed}) self.ll_tupleiter = ll_tupleiter self.ll_tuplenext = ll_tuplenext From tismer at codespeak.net Thu Apr 27 09:55:21 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 27 Apr 2006 09:55:21 +0200 (CEST) Subject: [pypy-svn] r26412 - in pypy/dist/pypy/translator/c: . src test winproj/extension Message-ID: <20060427075521.D8EA5100A8@code0.codespeak.net> Author: tismer Date: Thu Apr 27 09:55:16 2006 New Revision: 26412 Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/src/module.h pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj pypy/dist/pypy/translator/c/wrapper.py Log: added support for creation of real PyCFunction based methods, for speed Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Thu Apr 27 09:55:16 2006 @@ -33,7 +33,7 @@ self.libraries = libraries self.exports = {} - def build_database(self, exports=[]): + def build_database(self, exports=[], pyobj_options=None): translator = self.translator db = LowLevelDatabase(translator, standalone=self.standalone, gcpolicy=self.gcpolicy, thread_enabled=self.thread_enabled) @@ -43,6 +43,11 @@ db.stacklessdata = StacklessData(db) db.use_stackless_transformation = self.use_stackless_transformation + # pass extra options into pyobjmaker + if pyobj_options: + for key, value in pyobj_options.items(): + setattr(db.pyobjmaker, key, value) + # we need a concrete gcpolicy to do this self.libraries += db.gcpolicy.gc_libraries() @@ -720,6 +725,7 @@ pyobjnode = database.containernodes[pyobjptr._obj] print >> f, '\tPyModule_AddObject(m, "%s", %s);' % (publicname, pyobjnode.name) + print >> f, '\tcall_postsetup(m);' print >> f, '}' f.close() Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Thu Apr 27 09:55:16 2006 @@ -39,6 +39,8 @@ self.import_hints = {} # I don't seem to need it any longer. # leaving the import support intact, doesn't hurt. self.name_for_meth = {} # get nicer wrapper names + self.is_method = {} + self.use_true_methods = False # may be overridden def nameof(self, obj, debug=None): if debug: @@ -212,7 +214,8 @@ return self.skipped_function(func) fwrapper = gen_wrapper(func, self.translator, - newname=self.name_for_meth.get(func, func.__name__)) + newname=self.name_for_meth.get(func, func.__name__), + as_method=func in self.is_method) pycfunctionobj = self.uniquename('gfunc_' + func.__name__) self.wrappers[pycfunctionobj] = func.__name__, self.getvalue(fwrapper), func.__doc__ return pycfunctionobj @@ -574,6 +577,7 @@ if ann.binding(graph.getargs()[0]).classdef is not clsdef: value = new_method_graph(graph, clsdef, fname, self.translator) self.name_for_meth[value] = fname + self.is_method[value] = self.use_true_methods yield '%s.%s = %s' % (name, key, self.nameof(value)) if not init_seen: log.WARNING('No __init__ found for %s - you cannot build instances' % Modified: pypy/dist/pypy/translator/c/src/module.h ============================================================================== --- pypy/dist/pypy/translator/c/src/module.h (original) +++ pypy/dist/pypy/translator/c/src/module.h Thu Apr 27 09:55:16 2006 @@ -60,8 +60,10 @@ /* helper-hook for post-setup */ static globalfunctiondef_t *globalfunctiondefsptr; -static PyObject *postsetup_get_type_dict(PyObject *tp); +static PyObject *postsetup_get_typedict(PyObject *tp); +static PyObject *postsetup_get_methodname(int funcidx); static PyObject *postsetup_build_method(int funcidx, PyObject *type); +int call_postsetup(PyObject *m); /* implementations */ @@ -145,7 +147,7 @@ return 0; } -static PyObject *postsetup_get_type_dict(PyObject *tp) +static PyObject *postsetup_get_typedict(PyObject *tp) { PyTypeObject *type = (PyTypeObject *)tp; PyObject *ret; @@ -155,11 +157,39 @@ return ret; } +static PyObject *postsetup_get_methodname(int funcidx) +{ + globalfunctiondef_t *gfuncdef = &globalfunctiondefsptr[funcidx]; + + if (gfuncdef->p) + return PyString_FromString(gfuncdef->gfunc_name); + Py_INCREF(Py_None); + return Py_None; +} + static PyObject *postsetup_build_method(int funcidx, PyObject *type) { globalfunctiondef_t *gfuncdef = &globalfunctiondefsptr[funcidx]; - return PyDescr_NewMethod((PyTypeObject *)type, &gfuncdef->ml); + if (gfuncdef->p) + return PyDescr_NewMethod((PyTypeObject *)type, &gfuncdef->ml); + Py_INCREF(Py_None); + return Py_None; +} + +int call_postsetup(PyObject *m) +{ + PyObject *init, *ret; + + init = PyDict_GetItemString(this_module_globals, "__init__"); + if (init == NULL) { + PyErr_Clear(); + return 0; + } + ret = PyObject_CallFunction(init, "O", m); + if (ret == NULL) + return -1; + return 0; } #endif /* PYPY_NOT_MAIN_FILE */ Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Thu Apr 27 09:55:16 2006 @@ -9,7 +9,7 @@ import sys, types -P = not False # debug printing +P = False # debug printing def get_annotation(func): argstypelist = [] @@ -68,7 +68,10 @@ cbuilder = CExtModuleBuilder(t, func, gcpolicy=gcpolicy) # explicit build of database - db = cbuilder.build_database(exports=exports) + pyobj_options = { + 'use_true_methods': '__init__' in exports + } + db = cbuilder.build_database(exports=exports, pyobj_options=pyobj_options) cbuilder.generate_source(db) if view: t.viewcg() @@ -238,38 +241,70 @@ exec src del __builtin__, name, obj, src -def setup_new_module(mod, modname): - # note the name clash with py.test on setup_module + +def get_methodname(funcidx): + pass + +class Entry(ExtRegistryEntry): + _about_ = get_methodname + s_result_annotation = annmodel.SomeObject() + + def specialize_call(self, hop): + v_idx, = hop.inputargs(*hop.args_r) + v_res = hop.llops.gencapicall('postsetup_get_methodname', [v_idx], + resulttype=robject.pyobj_repr) + return v_res + +def build_method(funcidx): + pass + +class Entry(ExtRegistryEntry): + _about_ = build_method + s_result_annotation = annmodel.SomeObject() + + def specialize_call(self, hop): + v_idx, v_type = hop.inputargs(*hop.args_r) + v_res = hop.llops.gencapicall('postsetup_build_method', [v_idx, v_type], + resulttype=robject.pyobj_repr) + return v_res + +def get_typedict(cls): + pass + +class Entry(ExtRegistryEntry): + _about_ = get_typedict + s_result_annotation = annmodel.SomeObject() + + def specialize_call(self, hop): + v_type, = hop.inputargs(*hop.args_r) + v_res = hop.llops.gencapicall('postsetup_get_typedict', [v_type], + resulttype=robject.pyobj_repr) + return v_res + +def __init__(mod): import types - m = types.ModuleType(modname) - allobjs = mod.__dict__.values() - funcs = eval('[]') # or import list from __builtin__ - # one alternative - #bltn = BuiltinHelper() - # less code: import __builtin__ as bltn - if P:print bltn.list('hallo') - #from twisted.internet import reactor - #print dir(reactor) - #whow this works - isinstance = eval('isinstance') - # above is possible, this is probably a better compromise: + hasattr = bltn.hasattr isinstance = bltn.isinstance - for obj in allobjs: - if isinstance(obj, types.BuiltinFunctionType): - funcs.append( (obj.__name__, obj) ) - if P:print 'funcs=', funcs - if P:print funcs[3:] - #funcs += [2, 3, 5] - # not yet - stuff = bltn.range(10) - if P:print stuff[3:] - if P:print stuff[:3] - if P:print stuff[3:7] - if P:print stuff[:-1] - funcs.sort() - return m - + classes = [] + x = 0 + for name in mod.__all__: + obj = getattr(mod, name) + if isinstance(obj, type) and hasattr(obj, '__self__'): + classes.append(obj) + idx = 0 + while 1: + name = get_methodname(idx) + if not name: + break + func = getattr(mod, name) + for cls in classes: + dic = get_typedict(cls) + for methname, value in dic.items(): + if func is value: + meth = build_method(idx, cls) + dic[methname] = meth + idx += 1 # creating an object, wrapping, unwrapping, call function, check whether __del__ is called def test_wrap_call_dtor(): @@ -281,7 +316,7 @@ # exposing and using classes from a generasted extension module def test_expose_classes(): m = get_compiled_module(democlass_helper2, use_boehm=not True, exports=[ - DemoClass, DemoSubclass, DemoNotAnnotated, setup_new_module]) + DemoClass, DemoSubclass, DemoNotAnnotated, __init__]) obj = m.DemoClass(2, 3) res = obj.demo() assert res == DemoClass(2, 3).demo() Modified: pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj ============================================================================== --- pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj (original) +++ pypy/dist/pypy/translator/c/winproj/extension/extension.vcproj Thu Apr 27 09:55:16 2006 @@ -208,7 +208,7 @@ + RelativePath="..\..\..\..\..\..\..\Documents and Settings\ctismer\Local Settings\Temp\usession-2434\testing_1\testing_1.c"> Modified: pypy/dist/pypy/translator/c/wrapper.py ============================================================================== --- pypy/dist/pypy/translator/c/wrapper.py (original) +++ pypy/dist/pypy/translator/c/wrapper.py Thu Apr 27 09:55:16 2006 @@ -13,7 +13,7 @@ ALWAYS_INLINE = False -def gen_wrapper(func, translator, newname=None): +def gen_wrapper(func, translator, newname=None, as_method=False): """generate a wrapper function for 'func' that can be put in a PyCFunction object. The wrapper has signature @@ -63,6 +63,9 @@ varguments = [] varnames = func.func_code.co_varnames func_defaults = func.func_defaults or () + if as_method: + nb_positional_args -= 1 + varnames = varnames[1:] for i in range(nb_positional_args): # "argument_i = decode_arg(fname, i, name, vargs, vkwds)" or # "argument_i = decode_arg_def(fname, i, name, vargs, vkwds, default)" @@ -102,6 +105,8 @@ # use the rtyper to produce the conversions inputargs = f._obj.graph.getargs() + if as_method: + varguments.insert(0, vself) for i in range(len(varguments)): if FUNCTYPE.ARGS[i] != PyObjPtr: # "argument_i = type_conversion_operations(argument_i)" From nik at codespeak.net Thu Apr 27 09:57:02 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 09:57:02 +0200 (CEST) Subject: [pypy-svn] r26413 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427075702.17588100A8@code0.codespeak.net> Author: nik Date: Thu Apr 27 09:56:58 2006 New Revision: 26413 Removed: pypy/dist/pypy/rpython/ootypesystem/riterable.py Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py Log: use Record for list iterators. remove old iterator type caching code. Modified: pypy/dist/pypy/rpython/ootypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rlist.py Thu Apr 27 09:56:58 2006 @@ -5,7 +5,6 @@ from pypy.rpython.rmodel import inputconst, externalvsinternal from pypy.rpython.lltypesystem.lltype import Signed, Void from pypy.rpython.ootypesystem import ootype -from pypy.rpython.ootypesystem.riterable import iterator_type from pypy.rpython.ootypesystem.rslice import SliceRepr, \ startstop_slice_repr, startonly_slice_repr, minusone_slice_repr @@ -98,7 +97,8 @@ def __init__(self, r_list): self.r_list = r_list - self.lowleveltype = iterator_type(r_list, r_list.item_repr) + self.lowleveltype = ootype.Record( + {"iterable": r_list.lowleveltype, "index": ootype.Signed}) self.ll_listiter = ll_listiter self.ll_listnext = ll_listnext From dialtone at codespeak.net Thu Apr 27 09:57:57 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Thu, 27 Apr 2006 09:57:57 +0200 (CEST) Subject: [pypy-svn] r26414 - pypy/dist/pypy/translator/cl Message-ID: <20060427075757.842EB100AA@code0.codespeak.net> Author: dialtone Date: Thu Apr 27 09:57:53 2006 New Revision: 26414 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: use a dictionary to store declarations, change repr_arg to check_declaration before completely refactor clrepr.py Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Thu Apr 27 09:57:53 2006 @@ -7,6 +7,18 @@ from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name +class InsertionOrderedDict(dict): + def __init__(self): + super(InsertionOrderedDict, self).__init__() + self.ordered_keys = [] + + def __setitem__(self, key, val): + super(InsertionOrderedDict, self).__setitem__(key, val) + if key not in self.ordered_keys: + self.ordered_keys.append(key) + + def values(self): + return [self[key] for key in self.ordered_keys] class Op: @@ -20,7 +32,7 @@ def __iter__(self): method = getattr(self, "op_" + self.opname) result = repr_arg(self.result) - args = map(self.gen.repr_arg, self.args) + args = map(self.gen.check_declaration, self.args) for line in method(result, *args): yield line @@ -106,7 +118,7 @@ self.gen.pendinggraphs.append(methodobj) name = repr_fun_name(method) selfvar = repr_var(receiver) - args = map(self.gen.repr_arg, args) + args = map(self.gen.check_declaration, args) args = " ".join(args) if args: yield "(setf %s (%s %s %s))" % (result, name, selfvar, args) @@ -156,11 +168,11 @@ self.context = context self.entry_point = funobj self.pendinggraphs = [funobj] - self.declarations = [] + self.declarations = InsertionOrderedDict() self.constcount = 0 self.structcount = 0 - def repr_arg(self, arg): + def check_declaration(self, arg): if isinstance(arg, Constant): if isinstance(arg.concretetype, Instance): return self.declare_constant_instance(arg) @@ -172,7 +184,7 @@ field_declaration = cls._fields.keys() field_declaration = " ".join(field_declaration) struct_declaration = "(defstruct %s %s)" % (name, field_declaration) - self.declarations.append(struct_declaration) + self.declarations[name] = struct_declaration self.structcount += 1 return name @@ -187,8 +199,7 @@ self.declare_class(cls._superclass) supername = repr_class_name(cls._superclass._name) class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) - if class_declaration not in self.declarations: - self.declarations.append(class_declaration) + self.declarations[name] = class_declaration def declare_constant_instance(self, const): # const.concretetype is Instance @@ -207,7 +218,7 @@ fieldvaluerepr = repr_const(getattr(inst, fieldname)) const_declaration.append("(setf (slot-value %s '%s) %s)" % (name, fieldname, fieldvaluerepr)) const_declaration = "\n".join(const_declaration) - self.declarations.append(const_declaration) + self.declarations[const] = const_declaration self.constcount += 1 return name @@ -220,7 +231,7 @@ def emitcode(self): lines = list(self.emit()) - declarations = "\n".join(self.declarations) + declarations = "\n".join(self.declarations.values()) code = "\n".join(lines) if declarations: return declarations + "\n" + code + "\n" @@ -344,7 +355,7 @@ return "(go tag" + str(tag) + ")" def emit_link(self, link): - source = map(self.repr_arg, link.args) + source = map(self.check_declaration, link.args) target = map(repr_var, link.target.inputargs) couples = [ "%s %s" % (t, s) for (s, t) in zip(source, target)] couples = " ".join(couples) From nik at codespeak.net Thu Apr 27 10:12:10 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 10:12:10 +0200 (CEST) Subject: [pypy-svn] r26415 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427081210.5425E100A8@code0.codespeak.net> Author: nik Date: Thu Apr 27 10:12:07 2006 New Revision: 26415 Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py Log: add a convert_const to ootype DictRepr. this fixes the test failures in test_rtuple that have existed for a few days. but this convert_const implementation is not complete, it ignores r_dict (see the lltype DictRepr.convert_const) which i still don't completely understand even after a lecture about it by Samuele. ;) Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Thu Apr 27 10:12:07 2006 @@ -135,6 +135,28 @@ hop.exception_cannot_occur() return self.send_message(hop, 'll_clear') + def convert_const(self, dictobj): + if dictobj is None: + return lltype.nullptr(self.DICT) + if not isinstance(dictobj, dict): + raise TyperError("expected a dict: %r" % (dictobj,)) + try: + key = Constant(dictobj) + return self.dict_cache[key] + except KeyError: + self.setup() + l_dict = ll_newdict(self.DICT) + self.dict_cache[key] = l_dict + r_key = self.key_repr + r_value = self.value_repr + + # XXX need handling of r_dict here + for dictkey, dictvalue in dictobj.items(): + llkey = r_key.convert_const(dictkey) + llvalue = r_value.convert_const(dictvalue) + l_dict.ll_set(llkey, llvalue) + return l_dict + class __extend__(pairtype(DictRepr, rmodel.Repr)): def rtype_getitem((r_dict, r_key), hop): From arigo at codespeak.net Thu Apr 27 10:22:00 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 10:22:00 +0200 (CEST) Subject: [pypy-svn] r26417 - pypy/dist/pypy/translator/goal Message-ID: <20060427082200.E55E710095@code0.codespeak.net> Author: arigo Date: Thu Apr 27 10:22:00 2006 New Revision: 26417 Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py Log: Set up the module's dict automatically, by using r26412 (tismer) which directly calls the compiled function at import time if it is called __init__. Modified: pypy/dist/pypy/translator/goal/targetdemomodule.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetdemomodule.py (original) +++ pypy/dist/pypy/translator/goal/targetdemomodule.py Thu Apr 27 10:22:00 2006 @@ -9,13 +9,16 @@ module = Module(space, space.wrap('_demo')) w_moduledict = module.getdict() -def getdict(): - return w_moduledict +def __init__(mod): + w_mod = CPyObjSpace.W_Object(mod) + w_dict = space.getattr(w_mod, space.wrap('__dict__')) + space.call_method(w_dict, 'update', w_moduledict) +__init__.allow_someobjects = True # _____ Define and setup target ___ def target(*args): - return getdict, [], CPyAnnotatorPolicy(space) + return __init__, [object], CPyAnnotatorPolicy(space) if __name__ == '__main__': From auc at codespeak.net Thu Apr 27 10:33:45 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 10:33:45 +0200 (CEST) Subject: [pypy-svn] r26419 - pypy/extradoc/soc-2006 Message-ID: <20060427083345.739F61009D@code0.codespeak.net> Author: auc Date: Thu Apr 27 10:33:43 2006 New Revision: 26419 Modified: pypy/extradoc/soc-2006/wp09_10_ideas.txt Log: minor adjustment Modified: pypy/extradoc/soc-2006/wp09_10_ideas.txt ============================================================================== --- pypy/extradoc/soc-2006/wp09_10_ideas.txt (original) +++ pypy/extradoc/soc-2006/wp09_10_ideas.txt Thu Apr 27 10:33:43 2006 @@ -12,17 +12,16 @@ with possibly subotpimal runtime performance but no external dependency, -* wrapping Gecode : first finish the c-wrapper around Gecode, then use - rctypes to provide interpreter access to low-level Gecode - functionality, and finally integrate it into the existing - interpreter-level constraint solver. +* wrapping Gecode : first finish the c-wrapper around Gecode (unless + the wrapper for C++ becomes available), then use rctypes to provide + interpreter access to low-level Gecode functionality, and finally + integrate it into the existing interpreter-level constraint solver. Rctypes is what PyPy offers, mimicking python ctypes, to interface the -interpreter with C code. The machinery is still very young and the -advancement of the project might be hindered by defects or -incompleteness of rctypes. The point is that, even if the actual -wrapping of gecode is incomplete, the effort to build it at least -should yield a better rctypes. +interpreter with C code. The machinery is still young and the +advancement of the project might be hindered by bugs of rctypes. The +point is that, even if the actual wrapping of gecode is incomplete, +the effort to build it at least should yield a better rctypes. One modest, reasonable target could be running the simple send-more-money problem with very few actual search steps due to the From auc at codespeak.net Thu Apr 27 10:34:29 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 10:34:29 +0200 (CEST) Subject: [pypy-svn] r26420 - pypy/extradoc/minute Message-ID: <20060427083429.BDA4B100BA@code0.codespeak.net> Author: auc Date: Thu Apr 27 10:34:27 2006 New Revision: 26420 Modified: pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt Log: some precisions Modified: pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt ============================================================================== --- pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt (original) +++ pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt Thu Apr 27 10:34:27 2006 @@ -37,12 +37,12 @@ finished until June and raises the question if this is still feasible. -Aurelien notes that WP09 (constraint engine) depends on the ability -to clone threads. Christian thinks that the upcoming stackless -support will allow the required usages. Aurelien promises to write up -documentation about his implementation approaches and goals -and the needed features but also notes that the need for tasklet -cloning was mentioned two months ago. +Aurelien notes that WP09 (logic engine) depends on the ability to +clone threads. Christian thinks that the upcoming stackless support +will allow the required usages. Aurelien promises to write up +documentation about his implementation approaches and goals and the +needed features but also notes that the need for tasklet cloning was +mentioned two months ago. 2. ext compiler being able to produce extensions for CPython and PyPy @@ -78,28 +78,28 @@ 4. high-level backend snapshots + documentation? ------------------------------------------------------------------------ -We have a number of sub projects going on to implement high level -backends on top of ootype model: .NET, Squeak and recently a LISP -one. After short discussions everybody welcomed the idea and -would be glad to include a snapshot of the high-level backend -works. For 0.9 this requires writing documentation. Holger will care -to ask Antonio and Samuele will ask Seo about it. Niklaus -already mentioned at the meeting that he will try to write -some documentation about ootypes. +We have a number of sub projects going on to implement high level +backends on top of ootype model: .NET, Squeak and recently a Common +Lisp one. After short discussions everybody welcomed the idea and +would be glad to include a snapshot of the high-level backend works. +For 0.9 this requires writing documentation. Holger will care to ask +Antonio and Samuele will ask Seo about it. Niklaus already mentioned +at the meeting that he will try to write some documentation about +ootypes. 5. logic/constraint related goals? ------------------------------------------------------------------------ -Aurelien states that basic constraint coding work has been done -though he is not sure that it can be translated. He sees a -correlation to WP10 work and also mentions that he will need -some help. Holger notes that the D9.1 report is thought -to have an intermediate draft ready in June (as the original -deliverable "initial constraint satisfaction engine" was scheduled -for that time). Aurelien agreed to document their current approaches -so that they can be advertised and snapshotted with 0.9 and -also state missing/blocking features. +Aurelien states that basic constraint coding work has been done though +he asserts that it is not currently translatable. He sees a +(non-vital) dependency on WP10 work and also mentions that he will +need some help wrt translatability. Holger notes that the D9.1 report +is thought to have an intermediate draft ready in June (as the +original deliverable "initial constraint satisfaction engine" was +scheduled for that time). Aurelien agreed to document their current +approaches so that they can be advertised and snapshotted with 0.9 and +also state missing/blocking features. 6. _some_ amount of release polishing and updating documentation, fixing issues From ericvrp at codespeak.net Thu Apr 27 10:44:45 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 27 Apr 2006 10:44:45 +0200 (CEST) Subject: [pypy-svn] r26421 - in pypy/dist/pypy: interpreter interpreter/test module/_pickle_support Message-ID: <20060427084445.A39DA100C6@code0.codespeak.net> Author: ericvrp Date: Thu Apr 27 10:44:36 2006 New Revision: 26421 Modified: pypy/dist/pypy/interpreter/function.py pypy/dist/pypy/interpreter/test/test_pickle.py pypy/dist/pypy/interpreter/typedef.py pypy/dist/pypy/module/_pickle_support/__init__.py pypy/dist/pypy/module/_pickle_support/maker.py Log: More pickle tests (they are currently skipped) Modified: pypy/dist/pypy/interpreter/function.py ============================================================================== --- pypy/dist/pypy/interpreter/function.py (original) +++ pypy/dist/pypy/interpreter/function.py Thu Apr 27 10:44:36 2006 @@ -169,6 +169,33 @@ def descr_function_repr(self): return self.getrepr(self.space, 'function %s' % (self.name,)) + def descr__reduce__(self, space): + ''' + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + new_inst = mod.get('code_new') + w = space.wrap + tup = [ + w(self.co_argcount), + w(self.co_nlocals), + w(self.co_stacksize), + w(self.co_flags), + w(self.co_code), + space.newtuple(self.co_consts_w), + space.newtuple(self.co_names_w), + space.newtuple([w(v) for v in self.co_varnames]), + w(self.co_filename), + w(self.co_name), + w(self.co_firstlineno), + w(self.co_lnotab), + space.newtuple([w(v) for v in self.co_freevars]), + space.newtuple([w(v) for v in self.co_cellvars]), + #hidden_applevel=False, magic = 62061 | 0x0a0d0000 + ] + return space.newtuple([new_inst, space.newtuple(tup)]) + ''' + raise 'Function.desc__reduce__ here' + def fget_func_defaults(space, self): values_w = self.defs_w if not values_w: Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Thu Apr 27 10:44:36 2006 @@ -1,72 +1,142 @@ class AppTestInterpObjectPickling: def test_pickle_code(self): - import pickle def f(): return 42 + import pickle code = f.func_code pckl = pickle.dumps(code) result = pickle.loads(pckl) assert code == result - def DONTtest_pickle_func(self): - pass - + def test_pickle_func(self): + skip("work in progress") + def func(): + return 42 + import pickle + pckl = pickle.dumps(func) + result = pickle.loads(pckl) + assert func == result + def test_pickle_cell(self): - import pickle def g(): x = [42] def f(): x[0] += 1 return x return f.func_closure[0] + import pickle cell = g() - pckl = pickle.dumps(g()) + pckl = pickle.dumps(cell) result = pickle.loads(pckl) assert cell == result assert not (cell != result) - def DONTtest_pickle_frame(self): - pass - - def DONTtest_pickle_traceback(self): - pass - - def DONTtest_pickle_module(self): - pass - - def DONTtest_pickle_moduledict(self): - pass - - def DONTtest_pickle_iter(self): - pass - - def DONTtest_pickle_method(self): - pass - - def DONTtest_pickle_dictiter(self): - pass + def test_pickle_frame(self): + skip("work in progress") + from sys import exc_info + def f(): + try: + raise Exception() + except: + exc_type, exc, tb = exc_info() + return tb.tb_frame + import pickle + frame = f() + pckl = pickle.dumps(frame) + result = pickle.loads(pckl) + assert frame == result + + def test_pickle_traceback(self): + skip("work in progress") + def f(): + try: + raise Exception() + except: + from sys import exc_info + exc_type, exc, tb = exc_info() + return tb + import pickle + tb = f() + pckl = pickle.dumps(tb) + result = pickle.loads(pckl) + assert tb == result - def DONTtest_pickle_enum(self): - pass + def test_pickle_module(self): #XXX this passes for the wrong reason! + skip("work in progress") + def f(): + pass + import pickle + mod = f.__module__ #XXX returns a string? + pckl = pickle.dumps(mod) + result = pickle.loads(pckl) + assert mod == result - def DONTtest_pickle_enumfactory(self): - pass + def test_pickle_moduledict(self): #XXX this test is not correct! + skip("work in progress") + def f(): + pass + import pickle + modedict = f.__module__.__dict__ + pckl = pickle.dumps(moddict) + result = pickle.loads(pckl) + assert mod == result + + def test_pickle_iter(self): + skip("work in progress") + + def test_pickle_method(self): + skip("work in progress") + class C(object): + def f(self): + pass + import pickle + method = C.f + pckl = pickle.dumps(method) + result = pickle.loads(pckl) + assert method == result + + def test_pickle_dictiter(self): + skip("work in progress") + import pickle + diter = iter({}) + pckl = pickle.dumps(diter) + result = pickle.loads(pckl) + assert diter == result - def DONTtest_pickle_listiter(self): - pass + def test_pickle_enum(self): + skip("work in progress") - def DONTtest_pickle_rangeiter(self): - pass + def test_pickle_enumfactory(self): + skip("work in progress") + + def test_pickle_sequenceiter(self): + ''' + In PyPy there is no distinction here between listiterator and + tupleiterator that is why you will find no test_pickle_listiter nor + test_pickle_tupleiter here, just this test. + ''' + skip("work in progress") + import pickle + liter = iter([]) + pckl = pickle.dumps(liter) + result = pickle.loads(pckl) + assert liter == result - def DONTtest_pickle_tupleiter(self): - pass + def test_pickle_rangeiter(self): + skip("work in progress") + import pickle + riter = iter(xrange(5)) + pckl = pickle.dumps(riter) + result = pickle.loads(pckl) + assert riter == result - #def test_pickle_generator(self): - # import pickle - # def giveme(n): - # x = 0 - # while x < n: - # yield x - # generator = giveme(10) - # print pickle.dumps(generator) + def test_pickle_generator(self): + skip("work in progress") + import pickle + def giveme(n): + x = 0 + while x < n: + yield x + generator = giveme(10) + print pickle.dumps(generator) Modified: pypy/dist/pypy/interpreter/typedef.py ============================================================================== --- pypy/dist/pypy/interpreter/typedef.py (original) +++ pypy/dist/pypy/interpreter/typedef.py Thu Apr 27 10:44:36 2006 @@ -483,6 +483,8 @@ unwrap_spec=['self', Arguments]), __get__ = interp2app(descr_function_get), __repr__ = interp2app(Function.descr_function_repr), + __reduce__ = interp2app(Function.descr__reduce__, + unwrap_spec=['self', ObjSpace]), func_code = getset_func_code, func_doc = getset_func_doc, func_name = getset_func_name, Modified: pypy/dist/pypy/module/_pickle_support/__init__.py ============================================================================== --- pypy/dist/pypy/module/_pickle_support/__init__.py (original) +++ pypy/dist/pypy/module/_pickle_support/__init__.py Thu Apr 27 10:44:36 2006 @@ -9,4 +9,5 @@ interpleveldefs = { 'cell_new': 'maker.cell_new', 'code_new': 'maker.code_new', + 'func_new': 'maker.func_new', } Modified: pypy/dist/pypy/module/_pickle_support/maker.py ============================================================================== --- pypy/dist/pypy/module/_pickle_support/maker.py (original) +++ pypy/dist/pypy/module/_pickle_support/maker.py Thu Apr 27 10:44:36 2006 @@ -1,5 +1,6 @@ from pypy.interpreter.nestedscope import Cell from pypy.interpreter.pycode import PyCode +from pypy.interpreter.function import Function from pypy.rpython.objectmodel import instantiate from pypy.interpreter.argument import Arguments from pypy.interpreter.baseobjspace import ObjSpace @@ -11,9 +12,13 @@ def cell_new(space): return space.wrap(instantiate(Cell)) -#cell_new.unwrap_spec = [...] def code_new(space, __args__): - w_codetype = space.gettypeobject(PyCode.typedef) - return space.call_args(w_codetype, __args__) -code_new.unwrap_spec = [ObjSpace, Arguments] \ No newline at end of file + w_type = space.gettypeobject(PyCode.typedef) + return space.call_args(w_type, __args__) +code_new.unwrap_spec = [ObjSpace, Arguments] + +def func_new(space, __args__): + w_type = space.gettypeobject(Function.typedef) + return space.call_args(w_type, __args__) +func_new.unwrap_spec = [ObjSpace, Arguments] From arigo at codespeak.net Thu Apr 27 10:59:26 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 10:59:26 +0200 (CEST) Subject: [pypy-svn] r26422 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060427085926.EF5D8100B0@code0.codespeak.net> Author: arigo Date: Thu Apr 27 10:59:24 2006 New Revision: 26422 Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Log: ctypes.sizeof() support in rctypes. Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/astringbuf.py Thu Apr 27 10:59:24 2006 @@ -1,8 +1,8 @@ from pypy.rpython.extregistry import ExtRegistryEntry from pypy.rpython.rctypes.implementation import CTypesObjEntry -from pypy.annotation.model import SomeCTypesObject, SomeString +from pypy.annotation.model import SomeCTypesObject, SomeString, SomeInteger -from ctypes import create_string_buffer, c_char +from ctypes import create_string_buffer, c_char, sizeof class StringBufferType(object): @@ -46,3 +46,29 @@ from pypy.rpython.rctypes import rstringbuf return rstringbuf.StringBufRepr(rtyper, s_stringbuf, rstringbuf.STRBUFTYPE) + + +class SizeOfFnEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to ctypes.sizeof()" + _about_ = sizeof + + def compute_result_annotation(self, s_arg): + return SomeInteger(nonneg=True) + + def specialize_call(self, hop): + from pypy.rpython.lltypesystem import lltype, llmemory + from pypy.rpython.error import TyperError + [s_arg] = hop.args_s + [r_arg] = hop.args_r + if isinstance(s_arg, SomeCTypesObject): + if s_arg.knowntype is StringBufferType: + # sizeof(string_buffer) == len(string_buffer) + return r_arg.rtype_len(hop) + else: + if not s_arg.is_constant(): + raise TyperError("ctypes.sizeof(non_ctypes_object)") + # XXX check that s_arg.const is really a ctypes type + ctype = s_arg.const + s_arg = SomeCTypesObject(ctype, SomeCTypesObject.OWNSMEMORY) + r_arg = hop.rtyper.getrepr(s_arg) + return hop.inputconst(lltype.Signed, llmemory.sizeof(r_arg.ll_type)) Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Thu Apr 27 10:59:24 2006 @@ -191,3 +191,9 @@ pythonapi.PyInt_FromLong.argtypes = [c_long] pythonapi.PyInt_FromLong.restype = W_Object assert isinstance(pythonapi.PyInt_FromLong(17), W_Object) + +def test_sizeof(): + x = create_string_buffer(117) + assert sizeof(x) == 117 # assumes that chars are one byte each + x = (c_int * 42)() + assert sizeof(x) == 42 * sizeof(c_int) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Thu Apr 27 10:59:24 2006 @@ -12,7 +12,7 @@ import sys from pypy.rpython.test.test_llinterp import interpret -from ctypes import create_string_buffer +from ctypes import create_string_buffer, sizeof, c_int from pypy.rpython.rctypes.astringbuf import StringBufferType @@ -86,7 +86,7 @@ res = interpret(func, [0]) assert res == 0 - def test_annotate_value(self): + def test_specialize_value(self): def func(n): buf = create_string_buffer(n) buf[0] = 'x' @@ -95,3 +95,25 @@ res = interpret(func, [12]) assert ''.join(res.chars) == "xy" + + def test_specialize_sizeof(self): + def func(n): + buf = create_string_buffer(n) + return sizeof(buf) + res = interpret(func, [117]) + assert res == 117 + + +class Test_compilation: + def test_compile_const_sizeof(self): + A = c_int * 42 + def func(): + x = c_int() + a = A() + return sizeof(x), sizeof(a), sizeof(c_int), sizeof(A) + fn = compile(func, []) + res = fn() + assert res[0] == sizeof(c_int) + assert res[1] == sizeof(c_int) * 42 + assert res[2] == sizeof(c_int) + assert res[3] == sizeof(c_int) * 42 From nik at codespeak.net Thu Apr 27 10:59:50 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 10:59:50 +0200 (CEST) Subject: [pypy-svn] r26423 - pypy/dist/pypy/translator/squeak/test Message-ID: <20060427085950.6EBFA100C9@code0.codespeak.net> Author: nik Date: Thu Apr 27 10:59:44 2006 New Revision: 26423 Modified: pypy/dist/pypy/translator/squeak/test/runtest.py Log: cleaner checking of squeak-specific py.test options, as done in the lisp backend now. Modified: pypy/dist/pypy/translator/squeak/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/squeak/test/runtest.py (original) +++ pypy/dist/pypy/translator/squeak/test/runtest.py Thu Apr 27 10:59:44 2006 @@ -4,6 +4,7 @@ from pypy.translator.squeak.gensqueak import GenSqueak, camel_case from pypy.translator.translator import TranslationContext from pypy import conftest +from pypy.translator.squeak import conftest as sqconftest def compile_function(func, annotation=[], graph=None): return SqueakFunction(func, annotation, graph) @@ -93,7 +94,7 @@ # NB: only integers arguments are supported currently startup_st = self._write_startup() options = "-headless" - if hasattr(conftest, "showsqueak") and conftest.option.showsqueak: + if sqconftest.option.showsqueak: options = "" cmd = 'squeak %s -- %s %s "%s" %s' \ % (options, startup_st, udir.join(self._gen.filename), From nik at codespeak.net Thu Apr 27 11:00:31 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 11:00:31 +0200 (CEST) Subject: [pypy-svn] r26424 - pypy/dist/pypy/translator/squeak Message-ID: <20060427090031.53EBE100AD@code0.codespeak.net> Author: nik Date: Thu Apr 27 11:00:25 2006 New Revision: 26424 Modified: pypy/dist/pypy/translator/squeak/codeformatter.py pypy/dist/pypy/translator/squeak/gensqueak.py pypy/dist/pypy/translator/squeak/node.py Log: added Record support to gensqueak with surprisingly little code. Modified: pypy/dist/pypy/translator/squeak/codeformatter.py ============================================================================== --- pypy/dist/pypy/translator/squeak/codeformatter.py (original) +++ pypy/dist/pypy/translator/squeak/codeformatter.py Thu Apr 27 11:00:25 2006 @@ -96,7 +96,7 @@ def name_constant(self, value): if isinstance(value, bool): return str(value).lower() - elif isinstance(value, ootype.Instance): + elif isinstance(value, (ootype.Instance, ootype.Record)): return self.format_Instance(value) elif value is None: return "nil" Modified: pypy/dist/pypy/translator/squeak/gensqueak.py ============================================================================== --- pypy/dist/pypy/translator/squeak/gensqueak.py (original) +++ pypy/dist/pypy/translator/squeak/gensqueak.py Thu Apr 27 11:00:25 2006 @@ -1,6 +1,7 @@ from pypy.translator.gensupp import NameManager from pypy.translator.squeak.node import FunctionNode, ClassNode, SetupNode from pypy.translator.squeak.node import MethodNode, SetterNode, GetterNode +from pypy.rpython.ootypesystem.ootype import Record try: set except NameError: @@ -71,8 +72,11 @@ return squeak_method_name def unique_class_name(self, INSTANCE): - self.schedule_node(ClassNode(self, INSTANCE)) - class_name = INSTANCE._name.split(".")[-1] + class_node = self.schedule_node(ClassNode(self, INSTANCE)) + if isinstance(INSTANCE, Record): # XXX quick hack + class_name = "Record" + else: + class_name = INSTANCE._name.split(".")[-1] squeak_class_name = self.unique_name(INSTANCE, class_name) return "Py%s" % squeak_class_name Modified: pypy/dist/pypy/translator/squeak/node.py ============================================================================== --- pypy/dist/pypy/translator/squeak/node.py (original) +++ pypy/dist/pypy/translator/squeak/node.py Thu Apr 27 11:00:25 2006 @@ -4,7 +4,7 @@ from pypy.translator.squeak.opformatter import OpFormatter from pypy.translator.squeak.codeformatter import CodeFormatter, Message from pypy.translator.squeak.codeformatter import Field, Assignment, CustomVariable -from pypy.rpython.ootypesystem.ootype import Instance, Class, ROOT, _view +from pypy.rpython.ootypesystem.ootype import Instance, Class, Record, ROOT, _view from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast class CodeNode: @@ -31,11 +31,15 @@ self.class_vars = class_vars self.host_base = host_base self.hash_key = INSTANCE + # We can treat Instances and Records uniformly, this looks + # slightly hackish but just works. + if isinstance(INSTANCE, Record): + self.host_base = "Object" def dependencies(self): deps = [] - if self.INSTANCE._superclass is not None \ - and self.host_base is None: # not root + if self.host_base is None \ + and self.INSTANCE._superclass is not None: # not root or record deps.append(ClassNode(self.gen, self.INSTANCE._superclass)) return deps From cfbolz at codespeak.net Thu Apr 27 11:08:43 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 11:08:43 +0200 (CEST) Subject: [pypy-svn] r26425 - pypy/extradoc/soc-2006 Message-ID: <20060427090843.AD728100BA@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 11:08:42 2006 New Revision: 26425 Modified: pypy/extradoc/soc-2006/planning.txt Log: I am registered by now Modified: pypy/extradoc/soc-2006/planning.txt ============================================================================== --- pypy/extradoc/soc-2006/planning.txt (original) +++ pypy/extradoc/soc-2006/planning.txt Thu Apr 27 11:08:42 2006 @@ -6,7 +6,6 @@ Armin Rigo Michael Hudson -Carl Friedrich Bolz Christian Tismer Samuele Pedroni Anders Chrigstroem @@ -16,6 +15,7 @@ People already registered as a PSF mentor: Michael Hudson +Carl Friedrich Bolz On the 27th of April or so, mwh will send a list of all not-yet-registered mentors to Neal Norwitz. From cfbolz at codespeak.net Thu Apr 27 11:35:45 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 11:35:45 +0200 (CEST) Subject: [pypy-svn] r26426 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427093545.65636100C6@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 11:35:44 2006 New Revision: 26426 Modified: pypy/dist/pypy/module/_weakref/__init__.py pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: fix some problems in weakref, add getweakrefs and getweakrefcount Modified: pypy/dist/pypy/module/_weakref/__init__.py ============================================================================== --- pypy/dist/pypy/module/_weakref/__init__.py (original) +++ pypy/dist/pypy/module/_weakref/__init__.py Thu Apr 27 11:35:44 2006 @@ -4,5 +4,8 @@ appleveldefs = { } interpleveldefs = { - 'ref': 'interp__weakref.W_Weakref' + 'ref': 'interp__weakref.W_Weakref', + 'getweakrefcount': 'interp__weakref.getweakrefcount', + 'getweakrefs': 'interp__weakref.getweakrefs', + 'ReferenceType': 'interp__weakref.W_Weakref', } Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 11:35:44 2006 @@ -4,46 +4,92 @@ from pypy.interpreter.typedef import GetSetProperty, TypeDef from pypy.interpreter.gateway import interp2app from pypy.rpython.objectmodel import cast_address_to_object, cast_object_to_address -class W_Weakref(Wrappable): - pass +from pypy.rpython.lltypesystem.llmemory import NULL W_Weakrefable = W_Root W_Weakrefable.__lifeline__ = None class W_Weakref(Wrappable): - def __init__(w_self, space, w_obj, w_callable): + def __init__(w_self, space, lifeline, index, w_obj, w_callable): w_self.space = space w_self.address = cast_object_to_address(w_obj) w_self.w_callable = w_callable + w_self.addr_lifeline = cast_object_to_address(lifeline) + w_self.index = index def descr__call__(self): return cast_address_to_object(self.address, W_Weakrefable) def invalidate(w_self): - from pypy.rpython.lltypesystem.llmemory import NULL - import os w_self.address = NULL + + def activate_callback(w_self): + import os if not w_self.space.is_w(w_self.w_callable, w_self.space.w_None): try: w_self.space.call_function(w_self.w_callable, w_self) except OperationError, e: os.write(2, "XXX\n") + def __del__(w_self): + if w_self.address != NULL: + lifeline = cast_address_to_object(w_self.addr_lifeline, + WeakrefLifeline) + lifeline.ref_is_dead(w_self.index) + class WeakrefLifeline(object): def __init__(self): - self.refs_w = [] + self.addr_refs = [] def __del__(self): - for i in range(len(self.refs_w) - 1, -1, -1): - w_ref = self.refs_w[i] - w_ref.invalidate() + for i in range(len(self.addr_refs) - 1, -1, -1): + addr_ref = self.addr_refs[i] + if addr_ref != NULL: + w_ref = cast_address_to_object(addr_ref, W_Weakref) + w_ref.invalidate() + for i in range(len(self.addr_refs) - 1, -1, -1): + addr_ref = self.addr_refs[i] + if addr_ref != NULL: + w_ref = cast_address_to_object(addr_ref, W_Weakref) + w_ref.activate_callback() def get_weakref(self, space, w_subtype, w_obj, w_callable): w_ref = space.allocate_instance(W_Weakref, w_subtype) - W_Weakref.__init__(w_ref, space, w_obj, w_callable) - self.refs_w.append(w_ref) + W_Weakref.__init__(w_ref, space, self, len(self.addr_refs), + w_obj, w_callable) + self.addr_refs.append(cast_object_to_address(w_ref)) return w_ref + def ref_is_dead(self, index): + self.addr_refs[index] = NULL + +def getweakrefcount(space, w_obj): + if not isinstance(w_obj, W_Weakrefable): + return space.wrap(0) + if w_obj.__lifeline__ is None: + return space.wrap(0) + else: + lifeline = w_obj.__lifeline__ + result = 0 + for i in range(len(lifeline.addr_refs)): + if lifeline.addr_refs[i] != NULL: + result += 1 + return space.wrap(result) + +def getweakrefs(space, w_obj): + if not isinstance(w_obj, W_Weakrefable): + return space.newlist() + if w_obj.__lifeline__ is None: + return space.newlist() + else: + lifeline = w_obj.__lifeline__ + result = [] + for i in range(len(lifeline.addr_refs)): + addr = lifeline.addr_refs[i] + if addr != NULL: + result.append(cast_address_to_object(addr, W_Weakref)) + return space.newlist(result) + def descr__new__(space, w_subtype, w_obj, w_callable=None): assert isinstance(w_obj, W_Weakrefable) if w_obj.__lifeline__ is None: Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 11:35:44 2006 @@ -10,8 +10,10 @@ class A: pass a = A() + assert _weakref.getweakrefcount(a) == 0 ref = _weakref.ref(a) assert ref() is a + assert _weakref.getweakrefcount(a) == 1 del a assert ref() is None @@ -25,6 +27,7 @@ a2.ref = ref() ref1 = _weakref.ref(a1, callback) ref2 = _weakref.ref(a1) + assert _weakref.getweakrefcount(a1) == 2 del a1 assert ref1() is None assert a2.ref is None @@ -44,3 +47,33 @@ del a1 assert a2.x == 42 + def test_dont_callback_if_weakref_dead(self): + import _weakref + class A: + pass + a1 = A() + a1.x = 40 + a2 = A() + def callback(ref): + a1.x = 42 + assert _weakref.getweakrefcount(a2) == 0 + ref = _weakref.ref(a2, callback) + assert _weakref.getweakrefcount(a2) == 1 + ref = None + assert _weakref.getweakrefcount(a2) == 0 + a2 = None + assert a1.x == 40 + + def test_callback_cannot_ressurect(self): + import _weakref + class A: + pass + a = A() + alive = A() + alive.a = 1 + def callback(ref2): + alive.a = ref1() + ref1 = _weakref.ref(a, callback) + ref2 = _weakref.ref(a, callback) + del a + assert alive.a is None From auc at codespeak.net Thu Apr 27 11:44:54 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 11:44:54 +0200 (CEST) Subject: [pypy-svn] r26427 - pypy/extradoc/soc-2006 Message-ID: <20060427094454.C0DBC100D2@code0.codespeak.net> Author: auc Date: Thu Apr 27 11:44:52 2006 New Revision: 26427 Modified: pypy/extradoc/soc-2006/planning.txt Log: registered, added one-liners Modified: pypy/extradoc/soc-2006/planning.txt ============================================================================== --- pypy/extradoc/soc-2006/planning.txt (original) +++ pypy/extradoc/soc-2006/planning.txt Thu Apr 27 11:44:52 2006 @@ -10,12 +10,12 @@ Samuele Pedroni Anders Chrigstroem Holger Krekel -Aurelien Campeas People already registered as a PSF mentor: Michael Hudson Carl Friedrich Bolz +Aurelien Campeas (pending) On the 27th of April or so, mwh will send a list of all not-yet-registered mentors to Neal Norwitz. @@ -30,5 +30,8 @@ - writing extension modules - various optimization approaches - extending py.test to support distributed testing +- specialized propagators for the constraint solver [1] +- efficient builtin numeric arrays [1] +- a template system for python code [1] - +[1] see wp09_10_ideas.txt for more complete descriptions. From nik at codespeak.net Thu Apr 27 11:56:39 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 11:56:39 +0200 (CEST) Subject: [pypy-svn] r26429 - pypy/extradoc/minute Message-ID: <20060427095639.95EF6100D8@code0.codespeak.net> Author: nik Date: Thu Apr 27 11:56:35 2006 New Revision: 26429 Modified: pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt Log: i was there too. ;) Modified: pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt ============================================================================== --- pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt (original) +++ pypy/extradoc/minute/pypy-eu-sync-2006-04-25.txt Thu Apr 27 11:56:35 2006 @@ -4,8 +4,8 @@ attendants: Michael Hudson, Armin Rigo, Christian Tismer, Carl Friedrich Bolz Samuele Pedroni, Aurelien Campeas, Anders Chrigstroem, - Anders Lehmann, Jan Balster, Stephan Diehl, Gerald Klix - Holger Krekel (moderation, minutes) + Anders Lehmann, Jan Balster, Stephan Diehl, Gerald Klix, + Niklaus Haldimann, Holger Krekel (moderation, minutes) The meeting's purpose was to synchronise work efforts related to the PyPy EU project and more specifically to structure and plan work for From nik at codespeak.net Thu Apr 27 11:59:06 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Thu, 27 Apr 2006 11:59:06 +0200 (CEST) Subject: [pypy-svn] r26430 - pypy/dist/pypy/interpreter Message-ID: <20060427095906.C9C2C100DF@code0.codespeak.net> Author: nik Date: Thu Apr 27 11:59:03 2006 New Revision: 26430 Modified: pypy/dist/pypy/interpreter/function.py Log: don't raise a string exception in rpython code. this makes translation work again. Modified: pypy/dist/pypy/interpreter/function.py ============================================================================== --- pypy/dist/pypy/interpreter/function.py (original) +++ pypy/dist/pypy/interpreter/function.py Thu Apr 27 11:59:03 2006 @@ -194,7 +194,7 @@ ] return space.newtuple([new_inst, space.newtuple(tup)]) ''' - raise 'Function.desc__reduce__ here' + raise Exception('Function.desc__reduce__ here') def fget_func_defaults(space, self): values_w = self.defs_w From mwh at codespeak.net Thu Apr 27 12:03:33 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 27 Apr 2006 12:03:33 +0200 (CEST) Subject: [pypy-svn] r26431 - in pypy/extradoc/talk: . accu2006 Message-ID: <20060427100333.B433A100DB@code0.codespeak.net> Author: mwh Date: Thu Apr 27 12:03:19 2006 New Revision: 26431 Added: pypy/extradoc/talk/accu2006/accu-2006.ppt (contents, props changed) pypy/extradoc/talk/sprint-introduction.ppt (contents, props changed) Log: add ppt versions of two recent talks. Added: pypy/extradoc/talk/accu2006/accu-2006.ppt ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/sprint-introduction.ppt ============================================================================== Binary file. No diff available. From arigo at codespeak.net Thu Apr 27 12:25:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 12:25:46 +0200 (CEST) Subject: [pypy-svn] r26432 - pypy/dist/pypy/doc Message-ID: <20060427102546.BEC55100B6@code0.codespeak.net> Author: arigo Date: Thu Apr 27 12:25:46 2006 New Revision: 26432 Added: pypy/dist/pypy/doc/independent-project-ideas.txt Log: (mwh, cfbolz, arigo) A first list of Summer of Code project ideas. Note that the goal of this page is also to list general ideas for interested people. Added: pypy/dist/pypy/doc/independent-project-ideas.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/independent-project-ideas.txt Thu Apr 27 12:25:46 2006 @@ -0,0 +1,61 @@ +Independent project ideas in PyPy +================================= + +PyPy allows experimentation in a lot of directions. This page is meant to +collect some ideas of things to try. We have built it with Google's Summer of +Code in mind, which means that we try to describe ideas and projects that +should fit in two months of work including getting started with PyPy. +(This might not actually be true, if you want to finish it all; two months +should get some major sub-goals done.) + +In no particular order: + +* Rewrite one or several CPython extension modules to be based on **ctypes** + (newly integrated in Python 2.5): this is generally useful for Python + developpers, and it is now the best path to write extension modules that are + compatible with both CPython and PyPy. See for example + http://wiki.python.org/moin/CodingProjectIdeas/PygameOnCtypes . A related + idea is to provide efficient numeric arrays (as in numeric/numpy/numarray) + in this way. + +* Start a back-end for a new target platform, e.g. for the **Java** virtual + machine. This would be a path to bring PyPy and Jython closer and help + support Jython's effort to keep up-to-date, in particular with extension + modules. + +* A research project: experiment with optimizations in PyPy, e.g. by trying + various data structures for dicts or other built-in objects, by switching + between data structures at run-time, comparing results for different + applications, etc. This should produce experimental results (mostly + timings, memory usage trade-offs) as much as code. + +* Revive **rexec** : implement security checks, sandboxing, or + some similar model within PyPy (which, if I may venture an opinion, makes + more sense and is more robust than trying to do it in CPython). + +* Write **new object spaces** adding features like transparent or + semi-transparent distribution of a program across several machines, and/or + persistence (pickling of program state). + +* Write an interpreter for **another dynamic language** in the PyPy framework. + For example, a Javascript interpreter would be suitable. Ruby too (though + the latter is probably more than two months of work). Or Scheme, or... etc. + +* A different kind of project: work on **py.test** to add features like + distributed testing, automatic reports, etc. + +* Constraint programming: `Specialized propagators for specialized finite + domains`_. + +...or whatever else interests you! + +Note: currently it is a bit difficult for us to identify nice independent +sub-tasks in the context of the JIT compiler of PyPy... + +Feel free to mention your interest and discuss these ideas on the `pypy-dev +mailing list`_. + + +.. _`Specialized propagators for specialized finite domains`: http://codespeak.net/svn/pypy/extradoc/soc-2006/wp09_10_ideas.txt +.. _`pypy-dev mailing list`: http://codespeak.net/mailman/listinfo/pypy-dev + From pedronis at codespeak.net Thu Apr 27 12:29:32 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 27 Apr 2006 12:29:32 +0200 (CEST) Subject: [pypy-svn] r26433 - pypy/extradoc/soc-2006 Message-ID: <20060427102932.97139100BF@code0.codespeak.net> Author: pedronis Date: Thu Apr 27 12:29:29 2006 New Revision: 26433 Modified: pypy/extradoc/soc-2006/planning.txt Log: I'm registered Modified: pypy/extradoc/soc-2006/planning.txt ============================================================================== --- pypy/extradoc/soc-2006/planning.txt (original) +++ pypy/extradoc/soc-2006/planning.txt Thu Apr 27 12:29:29 2006 @@ -7,7 +7,6 @@ Armin Rigo Michael Hudson Christian Tismer -Samuele Pedroni Anders Chrigstroem Holger Krekel @@ -16,6 +15,7 @@ Michael Hudson Carl Friedrich Bolz Aurelien Campeas (pending) +Samuele Pedroni On the 27th of April or so, mwh will send a list of all not-yet-registered mentors to Neal Norwitz. From arigo at codespeak.net Thu Apr 27 12:31:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 12:31:55 +0200 (CEST) Subject: [pypy-svn] r26434 - pypy/dist/pypy/doc Message-ID: <20060427103155.8CB06100C0@code0.codespeak.net> Author: arigo Date: Thu Apr 27 12:31:54 2006 New Revision: 26434 Modified: pypy/dist/pypy/doc/independent-project-ideas.txt Log: Put a bold keyword in each idea. Modified: pypy/dist/pypy/doc/independent-project-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/independent-project-ideas.txt (original) +++ pypy/dist/pypy/doc/independent-project-ideas.txt Thu Apr 27 12:31:54 2006 @@ -24,7 +24,7 @@ modules. * A research project: experiment with optimizations in PyPy, e.g. by trying - various data structures for dicts or other built-in objects, by switching + various **data structures** for dicts or other built-in objects, by switching between data structures at run-time, comparing results for different applications, etc. This should produce experimental results (mostly timings, memory usage trade-offs) as much as code. From dialtone at codespeak.net Thu Apr 27 12:46:21 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Thu, 27 Apr 2006 12:46:21 +0200 (CEST) Subject: [pypy-svn] r26435 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060427104621.F31C81006B@code0.codespeak.net> Author: dialtone Date: Thu Apr 27 12:46:16 2006 New Revision: 26435 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_clrepr.py Log: (nik, sanxiyn, dialtone) refactored clrepr.py finally Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Thu Apr 27 12:46:16 2006 @@ -4,7 +4,7 @@ from pypy.tool.udir import udir from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL -from pypy.translator.cl.clrepr import repr_const, repr_fun_name +from pypy.translator.cl.clrepr import clrepr from pypy import conftest from pypy.translator.cl import conftest as clconftest @@ -88,9 +88,9 @@ def _(*args): fpath.write(out) fp = file(str(fpath), "a") - print >>fp, "(write (", repr_fun_name(func.func_name), + print >>fp, "(write (", clrepr(func.func_name), for arg in args: - print >>fp, repr_const(arg), + print >>fp, clrepr(arg), print >>fp, "))" fp.close() if clconftest.option.prettyprint: Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Thu Apr 27 12:46:16 2006 @@ -1,10 +1,35 @@ -import types - from pypy.objspace.flow.model import Constant, Variable, Atom from pypy.rpython.rmodel import HalfConcreteWrapper -from pypy.rpython.ootypesystem.ootype import List, Record, Instance, instance_impl, _class, _static_meth +from pypy.rpython.ootypesystem.ootype import List, Record, Instance +from pypy.rpython.ootypesystem.ootype import Signed, Unsigned, Float, Char +from pypy.rpython.ootypesystem.ootype import Bool, Void, UniChar, Class +from pypy.rpython.ootypesystem.ootype import StaticMethod, Meth, typeOf from pypy.rpython.ootypesystem.rclass import CLASSTYPE +def clrepr(item): + if isinstance(item, str): + if len(item) == 1: + return "#\\" + item + return repr_fun_name(item) + if isinstance(item, bool): + if item: + return "t" + else: + return "nil" + if isinstance(item, (int, long, float)): + return str(item) + if isinstance(item, (list, tuple)): + return "'(" + ' '.join(item) + ")" + if isinstance(item, Variable): + return repr_var(item) + if isinstance(item, Constant): + return repr_const(item) + if isinstance(item, Instance): + return "'" + repr_class_name(item._name) + if typeOf(item) is Class: + return "'" + item._INSTANCE._name + return repr_unknown(item) + def repr_unknown(obj): name = obj.__class__.__name__ raise NotImplementedError("cannot represent %s" % (name,)) @@ -21,55 +46,47 @@ def repr_fun_name(name): return name.replace('_', '-') -def repr_const(val): - if isinstance(val, HalfConcreteWrapper): - val = val.concretize().value - return repr_const(val) - if isinstance(val, Atom): +def repr_const(item): + if isinstance(item.value, HalfConcreteWrapper): + item = item.concretize() + + if isinstance(item.concretetype, Atom): return repr_atom(val) - if isinstance(val, List): - return "'array" - if isinstance(val, Record): - return "'struct" # XXX - if isinstance(val, Instance): - return "'" + repr_class_name(val._name) - if isinstance(val, _class): - return "'" + repr_class_name(val._INSTANCE._name) - if isinstance(val, types.FunctionType): - if val.func_name == 'dum_nocheck': # XXX - return "'dummy" - if isinstance(val, _static_meth): - return repr_fun_name(val._name) # XXX make sure function names are unique - if isinstance(val, tuple): - val = map(repr_const, val) - return "'(%s)" % ' '.join(val) - if isinstance(val, list): - val = map(repr_const, val) + + if isinstance(item.concretetype, List): + val = map(repr_const, item.value) return "#(%s)" % ' '.join(val) - if isinstance(val, bool): # should precede int - if val: + + if isinstance(item.concretetype, Record): + val = map(repr_const, item.value) + return "'(%s)" % ' '.join(val) + + if isinstance(item.concretetype, Instance): + return "'" + repr_class_name(item.value._name) + + if item.concretetype is Class: + return "'" + repr_class_name(item.value._INSTANCE._name) + + if item.concretetype is Void: + return "nil" + + if isinstance(item.concretetype, StaticMethod): + return repr_fun_name(item.value._name) + + if item.concretetype is Bool: # should precede int + if item.value: return "t" else: return "nil" - if isinstance(val, (int, long)): - return str(val) - if isinstance(val, float): - return str(val) - if val is None: + + if item.concretetype is Signed or item.concretetype is Unsigned: + #, long)): Not yet real longs + return str(item.value) + + if item.concretetype is Float: + return str(item.value) + + if item.value is None: return "nil" - if isinstance(val, str): - if len(val) == 1: - return "#\%c" % (val,) - else: - val.replace("\\", "\\\\") - val.replace("\"", "\\\"") - val = '"' + val + '"' - return val - return repr_unknown(val) - -def repr_arg(arg): - if isinstance(arg, Variable): - return repr_var(arg) - if isinstance(arg, Constant): - return repr_const(arg.value) - return repr_unknown(arg) + + return repr_unknown(item) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Thu Apr 27 12:46:16 2006 @@ -5,7 +5,7 @@ from pypy.translator.translator import graphof from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _class, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT -from pypy.translator.cl.clrepr import repr_arg, repr_var, repr_const, repr_fun_name, repr_class_name +from pypy.translator.cl.clrepr import clrepr, repr_fun_name, repr_class_name class InsertionOrderedDict(dict): def __init__(self): @@ -31,7 +31,7 @@ def __iter__(self): method = getattr(self, "op_" + self.opname) - result = repr_arg(self.result) + result = clrepr(self.result) args = map(self.gen.check_declaration, self.args) for line in method(result, *args): yield line @@ -93,6 +93,7 @@ yield "(setf %s (make-%s))" % (result, clsname) elif isinstance(cls, Instance): self.gen.declare_class(cls) + clsname = clrepr(cls) yield "(setf %s (make-instance %s))" % (result, clsname) else: raise NotImplementedError() @@ -101,6 +102,7 @@ yield "(setf %s (make-instance %s))" % (result, arg) def op_instanceof(self, result, arg, clsname): + clsname = clrepr(self.args[1].value) yield "(setf %s (typep %s %s))" % (result, arg, clsname) def op_oosend(self, result, *ignore): @@ -117,7 +119,7 @@ methodobj._method_name = method # XXX self.gen.pendinggraphs.append(methodobj) name = repr_fun_name(method) - selfvar = repr_var(receiver) + selfvar = clrepr(receiver) args = map(self.gen.check_declaration, args) args = " ".join(args) if args: @@ -143,22 +145,22 @@ class ListImpl: def __init__(self, receiver): - self.obj = repr_var(receiver) + self.obj = clrepr(receiver) def ll_length(self): return "(length %s)" % (self.obj,) def ll_getitem_fast(self, index): - index = repr_arg(index) + index = clrepr(index) return "(aref %s %s)" % (self.obj, index) def ll_setitem_fast(self, index, value): - index = repr_arg(index) - value = repr_arg(value) + index = clrepr(index) + value = clrepr(value) return "(setf (aref %s %s) %s)" % (self.obj, index, value) def _ll_resize(self, size): - size = repr_arg(size) + size = clrepr(size) return "(adjust-array %s %s)" % (self.obj, size) @@ -176,7 +178,7 @@ if isinstance(arg, Constant): if isinstance(arg.concretetype, Instance): return self.declare_constant_instance(arg) - return repr_arg(arg) + return clrepr(arg) def declare_struct(self, cls): # cls is Record @@ -207,7 +209,7 @@ INST = dynamicType(const.value) self.declare_class(INST) inst = oodowncast(INST, const.value) - cls = repr_const(INST) + cls = clrepr(INST) const_declaration = [] const_declaration.append("(setf %s (make-instance %s))" % (name, cls)) fields = INST._allfields() @@ -215,7 +217,7 @@ fieldvalue = getattr(inst, fieldname) if isinstance(fieldvalue, _class): self.declare_class(fieldvalue._INSTANCE) - fieldvaluerepr = repr_const(getattr(inst, fieldname)) + fieldvaluerepr = clrepr(getattr(inst, fieldname)) const_declaration.append("(setf (slot-value %s '%s) %s)" % (name, fieldname, fieldvaluerepr)) const_declaration = "\n".join(const_declaration) self.declarations[const] = const_declaration @@ -258,7 +260,7 @@ def emit_defun(self, fun): yield "(defun " + repr_fun_name(fun.name) arglist = fun.getargs() - args = " ".join(map(repr_var, arglist)) + args = " ".join(map(clrepr, arglist)) yield "(%s)" % (args,) for line in self.emit_body(fun, arglist): yield line @@ -266,9 +268,9 @@ def emit_defmethod(self, fun, name): yield "(defmethod %s" % (repr_fun_name(name)) arglist = fun.getargs() - selfvar = repr_var(arglist[0]) + selfvar = clrepr(arglist[0]) clsname = repr_class_name(arglist[0].concretetype._name) - args = " ".join(map(repr_var, arglist[1:])) + args = " ".join(map(clrepr, arglist[1:])) if args: yield "((%s %s) %s)" % (selfvar, clsname, args) else: @@ -289,7 +291,7 @@ vardict[var] = None varnames = [] for var in vardict: - varname = repr_var(var) + varname = clrepr(var) if var in arglist: varnames.append("(%s %s)" % (varname, varname)) else: @@ -318,7 +320,7 @@ if (len(exits) == 2 and exits[0].exitcase == False and exits[1].exitcase == True): - yield "(if " + repr_arg(block.exitswitch) + yield "(if " + clrepr(block.exitswitch) yield "(progn" for line in self.emit_link(exits[1]): yield line @@ -332,22 +334,22 @@ # shouldn't be needed but in Python 2.2 we can't tell apart # 0 vs nil and 1 vs t :-( for exit in exits[:-1]: - yield "(if (equalp " + repr_arg(block.exitswitch) - yield repr_const(exit.exitcase) + ')' + yield "(if (equalp " + clrepr(block.exitswitch) + yield clrepr(exit.exitcase) + ')' yield "(progn" for line in self.emit_link(exit): yield line yield ")" - yield "(progn ; else should be %s" % repr_const(exits[-1].exitcase) + yield "(progn ; else should be %s" % clrepr(exits[-1].exitcase) for line in self.emit_link(exits[-1]): yield line yield ")" * len(exits) elif len(block.inputargs) == 2: # exc_cls, exc_value - exc_cls = repr_var(block.inputargs[0]) - exc_value = repr_var(block.inputargs[1]) + exc_cls = clrepr(block.inputargs[0]) + exc_value = clrepr(block.inputargs[1]) yield "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) else: - retval = repr_var(block.inputargs[0]) + retval = clrepr(block.inputargs[0]) yield "(return %s)" % retval def format_jump(self, block): @@ -356,7 +358,7 @@ def emit_link(self, link): source = map(self.check_declaration, link.args) - target = map(repr_var, link.target.inputargs) + target = map(clrepr, link.target.inputargs) couples = [ "%s %s" % (t, s) for (s, t) in zip(source, target)] couples = " ".join(couples) yield "(setf %s)" % (couples,) Modified: pypy/dist/pypy/translator/cl/test/test_clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_clrepr.py (original) +++ pypy/dist/pypy/translator/cl/test/test_clrepr.py Thu Apr 27 12:46:16 2006 @@ -1,6 +1,8 @@ -from pypy.translator.cl.clrepr import repr_const +import py +from pypy.translator.cl.clrepr import clrepr def test_const(): + py.test.skip('changed') assert repr_const(True) == 't' assert repr_const(False) == 'nil' assert repr_const(42) == '42' From ac at codespeak.net Thu Apr 27 12:59:03 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 27 Apr 2006 12:59:03 +0200 (CEST) Subject: [pypy-svn] r26436 - in pypy/dist/pypy: module/_socket rpython/rctypes/socketmodule Message-ID: <20060427105903.7862F1006B@code0.codespeak.net> Author: ac Date: Thu Apr 27 12:59:02 2006 New Revision: 26436 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (aleale, arre) All old implementation has been replaced by ctypes implementations. Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Thu Apr 27 12:59:02 2006 @@ -4,8 +4,10 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import W_Root, NoneNotWrapped from pypy.interpreter.gateway import ObjSpace, interp2app +from pypy.interpreter import gateway from pypy.rpython.rctypes.socketmodule import ctypes_socket as _c import ctypes +from pypy.rpython.rctypes import implementation as rctypes_implemetation import errno IPV4_ADDRESS_SIZE = 4 @@ -122,7 +124,7 @@ hints.ai_socktype = _c.SOCK_DGRAM hints.ai_flags = _c.AI_PASSIVE res = _c.addr_info_ptr - err = _c.getaddrinfo( None, "0", pointer(hints), pointer(res)) + err = _c.getaddrinfo( None, "0", _c.pointer(hints), _c.pointer(res)) if err: raise w_get_socketgaierror(space, None, err) if res.contents.ai_next: @@ -205,7 +207,7 @@ raise w_get_socketherror(space, None, _c.h_errno.value) p_addr = hostent.contents.h_addr_list[0] - hostent = _c.gethostbyaddr(p_addr, ctypes.sizeof(_c.in_addr), _c.AF_INET) + hostent = _c.gethostbyaddr(p_addr, _c.in_addr, _c.AF_INET) return common_gethost(space, hostent) gethostbyaddr.unwrap_spec = [ObjSpace, str] @@ -221,7 +223,7 @@ proto = space.str_w(w_proto) servent_ptr = _c.getservbyname(name, proto) if not servent_ptr: - raise w_get_socketerror("service/port not found") + raise w_get_socketerror(space, "service/port not found") return space.wrap(_c.ntohs(servent_ptr.contents.s_port)) getservbyname.unwrap_spec = [ObjSpace, str, W_Root] @@ -237,7 +239,7 @@ proto = space.str_w(w_proto) servent_ptr = _c.getservbyport(_c.htons(port), proto) if not servent_ptr: - raise w_get_socketerror("port/proto not found") + raise w_get_socketerror(space, "port/proto not found") return space.wrap(servent_ptr.contents.s_name) getservbyport.unwrap_spec = [ObjSpace, int, W_Root] @@ -248,7 +250,7 @@ """ protoent_ptr = _c.getprotobyname(name) if not protoent_ptr: - raise w_get_socketerror('protocol not found') + raise w_get_socketerror(space, 'protocol not found') return space.wrap(protoent_ptr.contents.p_proto) getprotobyname.unwrap_spec = [ObjSpace, str] @@ -262,7 +264,7 @@ newfd = _c.dup(fd) if newfd < 0: - raise w_get_socketerror(None, _c.errno.value) + raise w_get_socketerror(space, None, _c.errno.value) if w_proto is None: return space.wrap(Socket(space, newfd, family, type)) else: @@ -270,7 +272,7 @@ return space.wrap(Socket(space, newfd, family, type, proto)) fromfd.unwrap_spec = [ObjSpace, int, int, int, W_Root] -def socketpair(space, w_family=NoneNotWrapped, w_type=NoneNotWrapped, w_proto=NoneNotWrapped): +def socketpair(space, w_family=NoneNotWrapped, w_sock_type=NoneNotWrapped, w_proto=NoneNotWrapped): """socketpair([family[, type[, proto]]]) -> (socket object, socket object) Create a pair of socket objects from the sockets returned by the platform @@ -278,20 +280,24 @@ The arguments are the same as for socket() except the default family is AF_UNIX if defined on the platform; otherwise, the default is AF_INET. """ - try: - if w_family is None: - return space.wrap(socket.socketpair()) - elif w_type is None: - return space.wrap(socket.socketpair(space.int_w(w_family))) - elif w_proto is None: - return space.wrap(socket.socketpair(space.int_w(w_family), - space.int_w(w_type))) - else: - return space.wrap(socket.socketpair(space.int_w(w_family), - space.int_w(w_type), - space.int_w(w_proto))) - except socket.error, e: - raise wrap_socketerror(space, e) + family = _c.AF_UNIX + sock_type = _c.SOCK_STREAM + proto = 0 + if w_family is not None: + family = space.int_w(w_family) + + if w_sock_type is not None: + sock_type = space.int_w(w_sock_type) + if w_proto is not None: + proto = space.int_w(w_proto) + result = ctypes.ARRAY(ctypes.c_int, 2) + error = _c.socketpair(family, sock_type, proto, result) + if error < 0: + raise w_get_socketerror(space, None, _c.errno.value) + s0 = Socket(space, result[0], family, sock_type, proto) + s1 = Socket(space, result[1], family, sock_type, proto) + return space.newtuple([s0, s1]) + socketpair.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root] def ntohs(space, x): @@ -591,7 +597,7 @@ hints.ai_family = family hints.ai_socktype = socktype hints.ai_protocol = proto - retval = _c.getaddrinfo(host, port, ctypes.pointer(hints), ctypes.pointer(res)) + retval = _c.getaddrinfo(host, port, _c.pointer(hints), _c.pointer(res)) if retval != 0: raise w_get_socketgaierror(space, None, retval) @@ -608,8 +614,9 @@ w_canonname = space.wrap(info.ai_canonname) else: w_canonname = space.wrap('') - w_addr = w_makesockaddr(space, info.ai_addr, info.ai_addrlen, - info.ai_protocol) + w_addr = w_makesockaddr(space, + _c.cast(info.ai_addr, _c.sockaddr_ptr), + info.ai_addrlen, info.ai_protocol) result.append(space.newtuple([w_family, w_socktype, w_proto, w_canonname, w_addr])) return space.newlist(result) @@ -653,15 +660,15 @@ space.wrap('argument 1 should be 2 items (%d given)' % sockaddr_len)) elif family == _c.AF_INET6: - sin6_ptr = ctypes.cast(res.contents.ai_addr, POINTER(_c.sockaddr_in6)) + sin6_ptr = ctypes.cast(res.contents.ai_addr, ctypes.POINTER(_c.sockaddr_in6)) sin6_ptr.contents.sin6_flowinfo = flowinfo sin6_ptr.contents.sin6_scope_id = scope_id hostbuf = ctypes.create_string_buffer(_c.NI_MAXHOST) portbuf = ctypes.create_string_buffer(_c.NI_MAXSERV) error = _c.getnameinfo(res.contents.ai_addr, res.contents.ai_addrlen, - hostbuf, ctypes.sizeof(hostbuf), - portbuf, ctypes.sizeof(portbuf), flags) + hostbuf, _c.NI_MAXHOST, + portbuf, _c.NI_MAXSERV, flags) if res: _c.freeaddrinfo(res) @@ -785,12 +792,14 @@ connection, and the address of the client. For IP sockets, the address info is a pair (hostaddr, port). """ - try: - newfd, address = self.fd.accept() - except socket.error, e: - raise wrap_socketerror(space, e) - newsock = Socket(newfd, self.family, self.type, self.proto) - return space.wrap((newsock, address)) + peeraddr = _c.pointer(_c.sockaddr()) + peeraddrlen = _c.socklen_t(_c.sockaddr_size) + newfd = _c.socketaccept(self._fd, peeraddr, + _c.pointer(peeraddrlen)) + if newfd < 0: + raise w_get_socketerror(_c.errno.value) + newsocket = Socket(space, newfd, self.family, self.type, self.proto) + return space.newtuple([newsocket, w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto)]) accept.unwrap_spec = ['self', ObjSpace] def bind(self, space, w_addr): @@ -800,11 +809,10 @@ pair (host, port); the host must refer to the local host. For raw packet sockets the address is a tuple (ifname, proto [,pkttype [,hatype]]) """ - addr = space.unwrap(w_addr) - try: - self.fd.bind(addr) - except socket.error, e: - raise wrap_socketerror(space, e) + caddr_ptr, caddr_len = self._getsockaddr(space, w_addr) + res = _c.socketbind(self._fd, caddr_ptr, caddr_len) + if res < 0: + raise w_get_socketerror(space, None, _c.errno.value) bind.unwrap_spec = ['self', ObjSpace, W_Root] def close(self, space): @@ -826,13 +834,8 @@ Connect the socket to a remote address. For IP sockets, the address is a pair (host, port). """ - sockaddr_ptr, sockaddr_len = self._getsockaddr(space, w_addr) - err = _c.socketconnect(self.fd, sockaddr_ptr, sockaddr_len) - if err: - errno = _c.errno.value - if self.timeout > 0.0: - # XXX timeout doesn't really work at the moment - pass + errno = self.connect_ex(space, w_addr) + if errno: raise w_get_socketerror(space, None, errno) connect.unwrap_spec = ['self', ObjSpace, W_Root] @@ -842,11 +845,15 @@ This is like connect(address), but returns an error code (the errno value) instead of raising an exception when an error occurs. """ - addr = space.unwrap(w_addr) - try: - self.fd.connect(addr) - except socket.error, e: - return space.wrap(e.errno) + sockaddr_ptr, sockaddr_len = self._getsockaddr(space, w_addr) + err = _c.socketconnect(self.fd, sockaddr_ptr, sockaddr_len) + if err: + errno = _c.errno.value + if self.timeout > 0.0: + # XXX timeout doesn't really work at the moment + pass + return errno + return 0 connect_ex.unwrap_spec = ['self', ObjSpace, W_Root] def dup(self, space): @@ -854,12 +861,11 @@ Return a new socket object connected to the same system resource. """ - try: - newfd = self.fd.dup() - except socket.error, e: - raise wrap_socketerror(space, e) - newsock = Socket(newfd, self.family, self.type, self.proto) - return space.wrap(newsock) + newfd = _c.dup(self.fd) + if newfd < 0: + raise w_get_socketerror(space, None, _c.errno.value) + return Socket(space, newfd, self.family, self.type, self.proto) + dup.unwrap_spec = ['self', ObjSpace] def fileno(self, space): @@ -881,12 +887,12 @@ """ def getpeername(self, space): peeraddr = ctypes.pointer(_c.sockaddr()) - peeraddrlen = _c.socklen_t(ctypes.sizeof(_c.sockaddr)) + peeraddrlen = _c.socklen_t(_c.sockaddr_size) res = _c.socketgetpeername(self.fd, peeraddr, ctypes.pointer(peeraddrlen)) if res < 0: - raise w_get_socketerror(None, _c.errno.value) - return w_makesockaddr(space, peeraddr, peeraddrlen, self.proto) + raise w_get_socketerror(space, None, _c.errno.value) + return w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto) getpeername.unwrap_spec = ['self', ObjSpace] def getsockname(self, space): @@ -895,10 +901,13 @@ Return the address of the local endpoint. For IP sockets, the address info is a pair (hostaddr, port). """ - try: - return space.wrap(self.fd.getsockname()) - except socket.error, e: - raise wrap_socketerror(space, e) + peeraddr = ctypes.pointer(_c.sockaddr()) + peeraddrlen = _c.socklen_t(_c.sockaddr_size) + res = _c.socketgetsockname(self.fd, peeraddr, + ctypes.pointer(peeraddrlen)) + if res < 0: + raise w_get_socketerror(space, None, _c.errno.value) + return w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto) getsockname.unwrap_spec = ['self', ObjSpace] def getsockopt(self, space, level, option, w_buffersize=NoneNotWrapped): @@ -908,14 +917,21 @@ If a nonzero buffersize argument is given, the return value is a string of that length; otherwise it is an integer. """ - try: - if w_buffersize is None: - return space.wrap(self.fd.getsockopt(level, option)) - else: - buffersize = space.int_w(w_buffersize) - return space.wrap(self.fd.getsockopt(level, option, buffersize)) - except socket.error, e: - raise wrap_socketerror(space, e) + if w_buffersize is not None: + c_buffersize = _c.c_int(space.int_w(w_buffersize)) + buffer = ctypes.create_string_buffer(c_buffersize.value) + err = _c.socketgetsockopt(self.fd, level, option, buffer, + ctypes.POINTER(c_buffersize)) + if err: + raise w_get_socketerror(space, None, _c.errno.value) + return space.wrap(buffer[:c_buffersize.value]) + # Assume integer option + optval = _c.c_int() + err = _c.socketgetsockopt(self.fd, level, option, _c.pointer(optval), + _c.c_int_size) + if err: + raise w_get_socketerror(space, None, _c.errno.value) + return space.wrap(optval.value) getsockopt.unwrap_spec = ['self', ObjSpace, int, int, W_Root] def listen(self, space, backlog): @@ -925,24 +941,16 @@ least 1; it specifies the number of unaccepted connection that the system will allow before refusing new connections. """ - try: - self.fd.listen(backlog) - except socket.error, e: - raise wrap_socketerror(space, e) + if backlog < 1: + backlog = 1 + res = _c.socketlisten(self.fd, backlog) + if res == -1: + raise w_get_socketerror(space, None, _c.errno.value) listen.unwrap_spec = ['self', ObjSpace, int] - def makefile(self, space, mode="r", buffersize=-1): - """makefile([mode[, buffersize]]) -> file object - - Return a regular file object corresponding to the socket. - The mode and buffersize arguments are as for the built-in open() function. - """ - try: - f = self.fd.makefile(mode, buffersize) - except socket.error, e: - raise wrap_socketerror(space, e) - return f - makefile.unwrap_spec = ['self', ObjSpace, str, int] + def makefile(self, space, w_mode='r', w_buffsize=-1): + return app_makefile(space, self, w_mode, w_buffsize) + makefile.unwrap_spec = ['self', ObjSpace, W_Root, W_Root] def recv(self, space, buffersize, flags=0): """recv(buffersize[, flags]) -> data @@ -952,10 +960,12 @@ at least one byte is available or until the remote end is closed. When the remote end is closed and all data is read, return the empty string. """ - try: - return space.wrap(self.fd.recv(buffersize, flags)) - except socket.error, e: - raise wrap_socketerror(space, e) + buf = _c.create_string_buffer(buffersize) + read_bytes = _c.socketrecv(self.fd, buf, buffersize, flags) + if read_bytes < 0: + raise w_get_socketerror(space, None, _c.errno.value) + return space.wrap(buf[:read_bytes]) + recv.unwrap_spec = ['self', ObjSpace, int, int] def recvfrom(self, space, buffersize, flags=0): @@ -963,10 +973,14 @@ Like recv(buffersize, flags) but also return the sender's address info. """ - try: - return space.wrap(self.fd.recvfrom(buffersize, flags)) - except socket.error, e: - raise wrap_socketerror(space, e) + buf = _c.create_string_buffer(buffersize) + sockaddr = _c.sockaddr() + read_bytes = _c.recvfrom(self.fd, buf, buffersize, flags, + _c.pointer(sockaddr), _c.sockaddr_size) + w_addr = w_makesockaddr(space, _c.pointer(sockaddr), _c.sockaddr_size, self.proto) + if read_bytes < 0: + raise w_get_socketerror(space, None, _c.errno.value) + return space.newtuple([space.wrap(buf[:read_bytes]), w_addr]) recvfrom.unwrap_spec = ['self', ObjSpace, int, int] def send(self, space, data, flags=0): @@ -976,10 +990,10 @@ argument, see the Unix manual. Return the number of bytes sent; this may be less than len(data) if the network is busy. """ - try: - return space.wrap(self.fd.send(data, flags)) - except socket.error, e: - raise wrap_socketerror(space, e) + res = _c.send(self.fd, data, len(data), flags) + if res < 0: + raise w_get_socketerror(space, None, _c.errno.value) + return space.wrap(res) send.unwrap_spec = ['self', ObjSpace, str, int] def sendall(self, space, data, flags=0): @@ -990,10 +1004,11 @@ until all data is sent. If an error occurs, it's impossible to tell how much data has been sent. """ - try: - self.fd.sendall(data, flags) - except socket.error, e: - raise wrap_socketerror(space, e) + while data: + res = _c.send(self.fd, data, len(data), flags) + if res < 0: + raise w_get_socketerror(space, None, _c.errno.value) + data = data[res:] sendall.unwrap_spec = ['self', ObjSpace, str, int] def sendto(self, space, data, w_param2, w_param3=NoneNotWrapped): @@ -1005,15 +1020,15 @@ if w_param3 is None: # 2 args version flags = 0 - addr = space.str_w(w_param2) + addr, addr_len = self._getsockaddr(space, w_param2) else: # 3 args version flags = space.int_w(w_param2) - addr = space.str_w(w_param3) - try: - self.fd.sendto(data, flags, addr) - except socket.error, e: - raise wrap_socketerror(space, e) + addr, addr_len = self._getsockaddr(space, w_param3) + res = _c.sendto(self.fd, data, len(data), flags, addr, addr_len) + if res < 0: + raise w_get_socketerror(space, None, _c.errno.value) + return space.wrap(res) sendto.unwrap_spec = ['self', ObjSpace, str, W_Root, W_Root] def setblocking(self, space, flag): @@ -1035,13 +1050,14 @@ Set a socket option. See the Unix manual for level and option. The value argument can either be an integer or a string. """ - if space.is_true(space.isinstance(w_value, space.w_str)): strvalue = space.str_w(w_value) - self.fd.setsockopt(level, option, strvalue) + _c.socketsetsockopt(self.fd, level, option, strvalue, + len(strvalue)) else: - intvalue = space.int_w(w_value) - self.fd.setsockopt(level, option, intvalue) + intvalue = ctypes.c_int(space.int_w(w_value)) + _c.socketsetsockopt(self.fd, level, option, _c.pointer(intvalue), + _c.c_int_size) setsockopt.unwrap_spec = ['self', ObjSpace, int, int, W_Root] def gettimeout(self, space): @@ -1072,9 +1088,8 @@ if timeout < 0.0: raise OperationError(space.w_ValueError, space.wrap("Timeout value out of range")) - self.timeout = timeout - self.fd.settimeout(timeout) + self.setblocking(space, timeout < 0.0) settimeout.unwrap_spec = ['self', ObjSpace, W_Root] def shutdown(self, space, how): @@ -1084,9 +1099,24 @@ writing side of the socket (flag == SHUT_WR), or both ends (flag == SHUT_RDWR). """ - self.fd.shutdown(how) + err = _c.shutdown(self.fd, how) + if err: + raise w_get_socketerror(space, None, _c.errno.value) + shutdown.unwrap_spec = ['self', ObjSpace, int] +app_makefile = gateway.applevel(r''' +def makefile(self, mode="r", buffersize=-1): + """makefile([mode[, buffersize]]) -> file object + + Return a regular file object corresponding to the socket. + The mode and buffersize arguments are as for the built-in open() function. + """ + import os + newfd = os.dup(self.fileno()) + return os.fdopen(newfd, mode, buffersize) +''', filename =__file__).interphook('makefile') + socketmethodnames = """ accept bind close connect connect_ex dup fileno getpeername getsockname getsockopt listen makefile recv Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Thu Apr 27 12:59:02 2006 @@ -71,7 +71,6 @@ return (newsocket, makesockaddr(peeraddr, peeraddrlen, self.proto)) def connect_ex(self, addr): - host, port = addr caddr = self._getsockaddr(addr) paddr = cast(pointer(caddr), _c.sockaddr_ptr) result = _c.socketconnect(self._fd, paddr, Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Thu Apr 27 12:59:02 2006 @@ -168,10 +168,14 @@ ssize_t = cConfig.ssize_t socklen_t = cConfig.socklen_t sockaddr = cConfig.sockaddr +sockaddr_size = sizeof(sockaddr) sockaddr_in = cConfig.sockaddr_in +sockaddr_in6 = cConfig.sockaddr_in6 in_addr = cConfig.in_addr +in_addr_size = sizeof(in_addr) addrinfo = cConfig.addrinfo +c_int_size = sizeof(c_int) SetPointerType(addrinfo_ptr, addrinfo) SetPointerType(sockaddr_ptr, sockaddr) @@ -288,22 +292,22 @@ socketrecv.argtypes = [c_int, c_void_p, c_int, c_int] socketrecv.recv = ssize_t -socketrecvfrom = socketdll.recvfrom -socketrecvfrom.argtypes = [c_int, c_void_p, size_t, - c_int, sockaddr_ptr, POINTER(socklen_t)] -socketrecvfrom.restype = ssize_t +recvfrom = socketdll.recvfrom +recvfrom.argtypes = [c_int, c_void_p, size_t, + c_int, sockaddr_ptr, POINTER(socklen_t)] +recvfrom.restype = ssize_t -socketsend = socketdll.send -socketsend.argtypes = [c_int, +send = socketdll.send +send.argtypes = [c_int, c_void_p, #this should be constant size_t, c_int] -socketsend.restype = ssize_t +send.restype = ssize_t -socketsendto = socketdll.sendto -socketsendto.argtypes = [c_int, c_void_p, #this should be constant +sendto = socketdll.sendto +sendto.argtypes = [c_int, c_void_p, #this should be constant size_t, c_int, sockaddr_ptr, #this should be const socklen_t] -socketsendto.restype = ssize_t +sendto.restype = ssize_t socketshutdown = socketdll.shutdown socketshutdown.argtypes = [c_int, c_int] @@ -345,3 +349,11 @@ memcpy = socketdll.memcpy memcpy.argtypes = [c_void_p, c_void_p, size_t] memcpy.restype = c_void_p + +socketpair = socketdll.socketpair +socketpair.argtypes = [c_int, c_int, c_int, POINTER(c_int)] +socketpair.restype = c_int + +shutdown = socketdll.shutdown +shutdown.argtypes = [c_int, c_int] +shutdown.restype = c_int From ac at codespeak.net Thu Apr 27 13:02:25 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 27 Apr 2006 13:02:25 +0200 (CEST) Subject: [pypy-svn] r26437 - pypy/dist/pypy/module/_socket Message-ID: <20060427110225.4912710061@code0.codespeak.net> Author: ac Date: Thu Apr 27 13:02:24 2006 New Revision: 26437 Modified: pypy/dist/pypy/module/_socket/interp_socket.py Log: (aleale, arre) Typo. Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Thu Apr 27 13:02:24 2006 @@ -207,7 +207,7 @@ raise w_get_socketherror(space, None, _c.h_errno.value) p_addr = hostent.contents.h_addr_list[0] - hostent = _c.gethostbyaddr(p_addr, _c.in_addr, _c.AF_INET) + hostent = _c.gethostbyaddr(p_addr, _c.in_addr_size, _c.AF_INET) return common_gethost(space, hostent) gethostbyaddr.unwrap_spec = [ObjSpace, str] From ericvrp at codespeak.net Thu Apr 27 13:10:23 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 27 Apr 2006 13:10:23 +0200 (CEST) Subject: [pypy-svn] r26438 - in pypy/dist: lib-python/modified-2.4.1 pypy/interpreter pypy/interpreter/test Message-ID: <20060427111023.30B45100B9@code0.codespeak.net> Author: ericvrp Date: Thu Apr 27 13:10:11 2006 New Revision: 26438 Added: pypy/dist/lib-python/modified-2.4.1/pickle.py - copied, changed from r26388, pypy/dist/lib-python/2.4.1/pickle.py Modified: pypy/dist/pypy/interpreter/function.py pypy/dist/pypy/interpreter/test/test_pickle.py Log: (pedronis, ericvrp) More support for making function pickling work. We need to check and port some more code from stackless' pickle.py to make this work. Modified: pypy/dist/pypy/interpreter/function.py ============================================================================== --- pypy/dist/pypy/interpreter/function.py (original) +++ pypy/dist/pypy/interpreter/function.py Thu Apr 27 13:10:11 2006 @@ -170,32 +170,21 @@ return self.getrepr(self.space, 'function %s' % (self.name,)) def descr__reduce__(self, space): - ''' + from pypy.interpreter.mixedmodule import MixedModule w_mod = space.getbuiltinmodule('_pickle_support') mod = space.interp_w(MixedModule, w_mod) - new_inst = mod.get('code_new') + new_inst = mod.get('func_new') w = space.wrap tup = [ - w(self.co_argcount), - w(self.co_nlocals), - w(self.co_stacksize), - w(self.co_flags), - w(self.co_code), - space.newtuple(self.co_consts_w), - space.newtuple(self.co_names_w), - space.newtuple([w(v) for v in self.co_varnames]), - w(self.co_filename), - w(self.co_name), - w(self.co_firstlineno), - w(self.co_lnotab), - space.newtuple([w(v) for v in self.co_freevars]), - space.newtuple([w(v) for v in self.co_cellvars]), - #hidden_applevel=False, magic = 62061 | 0x0a0d0000 + w(self.code), + #space.newdict([]), #XXX because pickle.py has no _pickle_moduledict yet... + self.w_func_globals, + w(self.name), + space.newtuple(self.defs_w), + w(self.closure), ] return space.newtuple([new_inst, space.newtuple(tup)]) - ''' - raise Exception('Function.desc__reduce__ here') - + def fget_func_defaults(space, self): values_w = self.defs_w if not values_w: Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Thu Apr 27 13:10:11 2006 @@ -9,13 +9,34 @@ result = pickle.loads(pckl) assert code == result - def test_pickle_func(self): - skip("work in progress") + def test_pickle_global_func(self): + import new + mod = new.module('mod') + import sys + sys.modules['mod'] = mod def func(): return 42 + mod.__dict__['func'] = func + func.__module__ = 'mod' import pickle pckl = pickle.dumps(func) result = pickle.loads(pckl) + assert func is result + del sys.modules['mod'] + + def test_pickle_builtin_func(self): + import pickle + pckl = pickle.dumps(map) + result = pickle.loads(pckl) + assert map is result + + def test_pickle_nested_func(self): + skip("work in progress") + def func(): + return 42 + import pickle + pckl = pickle.dumps(func) + result = pickle.loads(pckl) assert func == result def test_pickle_cell(self): From antocuni at codespeak.net Thu Apr 27 13:59:14 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 27 Apr 2006 13:59:14 +0200 (CEST) Subject: [pypy-svn] r26440 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060427115914.4C33310091@code0.codespeak.net> Author: antocuni Date: Thu Apr 27 13:59:05 2006 New Revision: 26440 Modified: pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/function.py pypy/dist/pypy/translator/cli/gencli.py pypy/dist/pypy/translator/cli/metavm.py pypy/dist/pypy/translator/cli/test/compile.py Log: Fixed a bug that prevented the code from being compiled when there are two distinct graphs with the very same name. Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Thu Apr 27 13:59:05 2006 @@ -38,7 +38,7 @@ def record_class(self, classdef, name): self.classes[classdef] = name - def function_name(self, graph): + def graph_name(self, graph): return self.functions.get(graph, None) def class_name(self, classdef): Modified: pypy/dist/pypy/translator/cli/function.py ============================================================================== --- pypy/dist/pypy/translator/cli/function.py (original) +++ pypy/dist/pypy/translator/cli/function.py Thu Apr 27 13:59:05 2006 @@ -47,7 +47,7 @@ return (not block.exits) and len(block.inputargs) == 2 def render(self, ilasm): - if self.db.function_name(self.graph) is not None and not self.is_method: + if self.db.graph_name(self.graph) is not None and not self.is_method: return # already rendered self.ilasm = ilasm @@ -255,9 +255,10 @@ def emit(self, instr, *args): self.ilasm.opcode(instr, *args) - def call(self, graph, func_name): + def call_graph(self, graph): self.db.pending_function(graph) - self.ilasm.call(func_name) + func_sig = self.function_signature(graph) + self.ilasm.call(func_sig) def call_signature(self, signature): self.ilasm.call(signature) Modified: pypy/dist/pypy/translator/cli/gencli.py ============================================================================== --- pypy/dist/pypy/translator/cli/gencli.py (original) +++ pypy/dist/pypy/translator/cli/gencli.py Thu Apr 27 13:59:05 2006 @@ -45,7 +45,8 @@ # TODO: instance methods that are also called as unbound # methods are rendered twice, once within the class and once - # as an external function. Fix this. + # as an external function. Fix this. + self.fix_names() self.gen_entrypoint() self.gen_pendings() self.db.gen_constants(self.ilasm) @@ -65,4 +66,11 @@ node = self.db._pending_nodes.pop() node.render(self.ilasm) - + def fix_names(self): + # it could happen that two distinct graph have the same name; + # here we assign an unique name to each graph. + names = set() + for graph in self.translator.graphs: + while graph.name in names: + graph.name += '_' + names.add(graph.name) Modified: pypy/dist/pypy/translator/cli/metavm.py ============================================================================== --- pypy/dist/pypy/translator/cli/metavm.py (original) +++ pypy/dist/pypy/translator/cli/metavm.py Thu Apr 27 13:59:05 2006 @@ -63,10 +63,10 @@ self._render_method(generator, method_name, op.args[1:]) def _render_function(self, generator, graph, args): - func_sig = generator.function_signature(graph) + #func_sig = generator.function_signature(graph) for func_arg in args[1:]: # push parameters generator.load(func_arg) - generator.call(graph, func_sig) + generator.call_graph(graph) def _render_method(self, generator, method_name, args): this = args[0] Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Thu Apr 27 13:59:05 2006 @@ -26,8 +26,8 @@ def bar(x, y): - d = {x: y} - return d[x] + a = [x] + [y] + return a[0] f = compile_function(bar, [int, int]) From auc at codespeak.net Thu Apr 27 14:24:19 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 14:24:19 +0200 (CEST) Subject: [pypy-svn] r26441 - pypy/extradoc/soc-2006 Message-ID: <20060427122419.20D74100B2@code0.codespeak.net> Author: auc Date: Thu Apr 27 14:24:15 2006 New Revision: 26441 Added: pypy/extradoc/soc-2006/code-templating.txt pypy/extradoc/soc-2006/constraints.txt pypy/extradoc/soc-2006/numeric-arrays.txt Removed: pypy/extradoc/soc-2006/wp09_10_ideas.txt Log: exploded proposals in three files Added: pypy/extradoc/soc-2006/code-templating.txt ============================================================================== --- (empty file) +++ pypy/extradoc/soc-2006/code-templating.txt Thu Apr 27 14:24:15 2006 @@ -0,0 +1,167 @@ +A template system for Python code +================================= + +Related mainly to WP10 (aspects ...), but of WP9 interest. + +Use case +-------- + +We might need a way to extend python with macro-like features, so +as to be able to provide the choice/or operator (and others) without +touching the grammar file (it means in short : making available the new +operator at runtime by way of importing some module). + +Examples of choice usage : + + def soft_color(): + choice: + return 'beige' + or: + return 'coral' + + def hard_color(): + choice: + return 'mauve' + or: + return 'ochre' + + def contrast_colors(C1, C2): + choice: + C1 :=: soft_color() + C2 :=: hard_color() + or: + C1 :=: hard_color() + C2 :=: soft_color() + + def suit(): + Shirt, Pants, Socks = ?, ?, ? + contrast_colors(Shirt, Pants) + contrast_colors(Pants, Socks) + if Shirt == Socks: + fail() + return (Shirt, Pants, Socks) + + +If we give the suit entry point to the solver built for WP9, we can +(lazily) get an enumeration of all suits respecting our ruleset. + +Some explanation for choice/or, ?, :=: and fail() : + +* choice/or : this construct places a so-called non-deterministic + choice point into a program. It is different from an 'if' in the + sense that for one complete run of the program, only one choice + point may be chosen. The solver is practically responsible for + exploring the space of 'worlds' in which the program go through the + different choice points. Some of these worlds will fail (they don't + belong to a solution), some other will yield a valid solution of our + program. Each choice point can be made of arbitrary Python code. + + The important thing, wrt WP10, is to understand that choice is + merely syntactic sugar over the primitive choose() operator. Any + choice/or construct can be rewritten in terms of choose, as in : + + choice: or: or: ... or: + + which has to be rewritten as : + + choice = choose(N) + if choice == 1: + + elif choice == 2: + + ... + else: # choice == N + + + This is possible because all choice points are known at compile + time. + + See the Annex for an example on how (shortly) this could be done in + Lisp. + +* ? denotes a logic variable, i.e a variable which has no value at + creation-time (not even None) and can be bound only once. Currently, + the newvar() builtin is used instead of it in PyPy + +* :=: is a short-cut notation we might want instead of merely calling + unify, as in : unify(Term1, Term2) + +* fail() makes the current computation space fail: it means that the + current computation reached an inconsistent (from a logic point of + view) state and cannot yield a solution. + +Rationale +--------- + +Macros are functions that execute at compile-time, so as to provide +language extensions usable at run-time, by way of source code (or AST) +transformations. Tens of years of work in the field have make it clear +that many macros are infinitely easier to write provided one has means +to express source code as templates in which compile-time computed +information can be injected. + +Some easy example for Python : let's say we want an 'unless operator', +which writes as : + + unless : + ... op sequence ... + +and should be translated back to the following legal CPython code : + + if not : + ... op sequence ... + +One could define unless as a macro which takes the test and operation +sequence as parameters and returns an AST which conforms to plain Python. + + def test(test, op_seq): + templ = `if not ~test: ~op_seq` + return templ + +Here we use a set of new operators, which provide a functionality +sometimes called quasiquotation. + +The `...` (backquote) syntax allows to embed literal pieces of Python +code ; the ~ (called unquote, and others, to be defined) allows to +inject (eventually computed) data into the template. + +This is extremely rough and basic but can give an idea. Mature ways +to do it are exposed in the programming languages Dylan +(http://people.csail.mit.edu/jrb/Projects/dexprs.pdf) and, maybe more +in touch with the Python world, logix (http://livelogix.net/logix/). + +Todo +---- + +Investigate ways to provide a quasiquotation/templating mechanism +suitable for a language like Python. + + +Annex +----- + + A Common Lisp programmer would have a 'choice' operator like this : + + (choice ) + + ... for instance : + + (defun contrast (C1 C2) + (choice + ((unify C1 (soft-color)) + (unify C2 (hard-color))) + ((unify C1 (hard-color)) + (unify C2 (soft-color))))) + + He would define choice as a macro, as follows : + + (defmacro choice (&body choice-points) + (let ((choices (length choice-points)) + (choice (gensym "choice"))) + `(let ((,choice (choose ,choices))) + (cond + ,(loop for alternative in choice-points + for i from 1 upto choices + collect `((= ,i ,choice) + (progn ,alternative))))))) + Added: pypy/extradoc/soc-2006/constraints.txt ============================================================================== --- (empty file) +++ pypy/extradoc/soc-2006/constraints.txt Thu Apr 27 14:24:15 2006 @@ -0,0 +1,30 @@ +Specialized propagators for specialized finite domains +====================================================== + +Concerns WP9 (concurrent constraint & logic programming workpackage). + +We need specialized propagators for specialized finite domains (over +numbers, sets, intervals, etc.). There are two ways to provide them : + +* RPython : which means implementing and debugging a lot of stuff, + with possibly subotpimal runtime performance but no external + dependency, and a lot of fun, + +* wrapping Gecode[1] : first finish the c-wrapper around Gecode (unless + interfacing PyPy with C++ becomes available), then use rctypes to + provide interpreter access to low-level Gecode functionality, and + finally integrate it into the existing interpreter-level constraint + solver. + +Rctypes is what PyPy offers, mimicking python ctypes, to interface the +interpreter with C code. The machinery is still young and the +advancement of the project might be hindered by bugs of rctypes. The +point is that, even if the actual wrapping of gecode is incomplete, +the effort to build it at least should yield a better rctypes. + +One modest, reasonable target could be running the simple +send-more-money problem with very few actual search steps due to the +efficient propagation happening on integer finite domains. + + +[1] http://www.gecode.org/ \ No newline at end of file Added: pypy/extradoc/soc-2006/numeric-arrays.txt ============================================================================== --- (empty file) +++ pypy/extradoc/soc-2006/numeric-arrays.txt Thu Apr 27 14:24:15 2006 @@ -0,0 +1,6 @@ +Provide efficient builtin numeric arrays +======================================== + +Implements numeric arrays (as in numeric/numpy/numarray) as builtin +types of pypy so that numeric operations on arrays are recognized by +the annotator, rtyper and backends and translated optimally. From auc at codespeak.net Thu Apr 27 14:24:47 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 14:24:47 +0200 (CEST) Subject: [pypy-svn] r26442 - pypy/dist/pypy/doc Message-ID: <20060427122447.98E55100B2@code0.codespeak.net> Author: auc Date: Thu Apr 27 14:24:44 2006 New Revision: 26442 Modified: pypy/dist/pypy/doc/independent-project-ideas.txt Log: Modified: pypy/dist/pypy/doc/independent-project-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/independent-project-ideas.txt (original) +++ pypy/dist/pypy/doc/independent-project-ideas.txt Thu Apr 27 14:24:44 2006 @@ -14,9 +14,10 @@ (newly integrated in Python 2.5): this is generally useful for Python developpers, and it is now the best path to write extension modules that are compatible with both CPython and PyPy. See for example - http://wiki.python.org/moin/CodingProjectIdeas/PygameOnCtypes . A related - idea is to provide efficient numeric arrays (as in numeric/numpy/numarray) - in this way. + http://wiki.python.org/moin/CodingProjectIdeas/PygameOnCtypes . + +* A related idea is to provide `efficient numeric arrays`_ (as in + numeric/numpy/numarray) in this way. * Start a back-end for a new target platform, e.g. for the **Java** virtual machine. This would be a path to bring PyPy and Jython closer and help @@ -44,8 +45,11 @@ * A different kind of project: work on **py.test** to add features like distributed testing, automatic reports, etc. -* Constraint programming: `Specialized propagators for specialized finite - domains`_. +* Constraint programming: `efficient propagators for specialized + finite domains`_ (like numbers, sets, intervals). + +* A `code templating solution`_ for Python code, allowing to extend + the language syntax, control flow operators, etc. ...or whatever else interests you! @@ -56,6 +60,8 @@ mailing list`_. -.. _`Specialized propagators for specialized finite domains`: http://codespeak.net/svn/pypy/extradoc/soc-2006/wp09_10_ideas.txt +.. _`efficient propagators for specialized finite domains`: http://codespeak.net/svn/pypy/extradoc/soc-2006/constraints.txt +.. _`efficient numeric arrays`: http://codespeak.net/svn/pypy/extradoc/soc-2006/numeric-arrays.txt +.. _`code templating solution`: http://codespeak.net/svn/pypy/extradoc/soc-2006/code-templating.txt .. _`pypy-dev mailing list`: http://codespeak.net/mailman/listinfo/pypy-dev From antocuni at codespeak.net Thu Apr 27 14:34:44 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 27 Apr 2006 14:34:44 +0200 (CEST) Subject: [pypy-svn] r26443 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060427123444.77BEF100B3@code0.codespeak.net> Author: antocuni Date: Thu Apr 27 14:34:39 2006 New Revision: 26443 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: Added _lookup_field to Record, so that its interface is more similar to the Instance's one. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Thu Apr 27 14:34:39 2006 @@ -225,6 +225,12 @@ def _lookup(self, meth_name): return self, None + def _lookup_field(self, name): + try: + return self, self._field_type(name) + except TypeError: + return self, None + def __str__(self): item_str = ["%s: %s" % (str(name), str(ITEMTYPE)) for name, (ITEMTYPE, _) in self._fields.items()] From antocuni at codespeak.net Thu Apr 27 14:47:34 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 27 Apr 2006 14:47:34 +0200 (CEST) Subject: [pypy-svn] r26444 - in pypy/dist/pypy/translator/cli: . test Message-ID: <20060427124734.BC460100B5@code0.codespeak.net> Author: antocuni Date: Thu Apr 27 14:47:24 2006 New Revision: 26444 Added: pypy/dist/pypy/translator/cli/record.py (contents, props changed) pypy/dist/pypy/translator/cli/test/test_tuple.py (contents, props changed) Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/database.py pypy/dist/pypy/translator/cli/test/compile.py Log: Added support for Record. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Thu Apr 27 14:47:24 2006 @@ -6,7 +6,7 @@ from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong -from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod, List +from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod, List, Record from pypy.translator.cli.option import getoption from pypy.translator.cli import oopspec @@ -62,6 +62,9 @@ if isinstance(t, Instance): self.db.pending_class(t) return self.__class(t._name, include_class) + elif isinstance(t, Record): + name = self.db.pending_record(t) + return self.__class(name, include_class) elif isinstance(t, StaticMethod): return 'void' # TODO: is it correct to ignore StaticMethod? elif isinstance(t, List): Modified: pypy/dist/pypy/translator/cli/database.py ============================================================================== --- pypy/dist/pypy/translator/cli/database.py (original) +++ pypy/dist/pypy/translator/cli/database.py Thu Apr 27 14:47:24 2006 @@ -1,6 +1,7 @@ from pypy.translator.cli.cts import CTS from pypy.translator.cli.function import Function from pypy.translator.cli.class_ import Class +from pypy.translator.cli.record import Record from pypy.rpython.ootypesystem import ootype try: @@ -27,6 +28,11 @@ def pending_class(self, classdef): self.pending_node(Class(self, classdef)) + def pending_record(self, record): + r = Record(self, record) + self.pending_node(r) + return r.get_name() + def pending_node(self, node): if node in self._pending_nodes or node in self._rendered_nodes: return Added: pypy/dist/pypy/translator/cli/record.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/record.py Thu Apr 27 14:47:24 2006 @@ -0,0 +1,55 @@ +import string + +from pypy.translator.cli.node import Node +from pypy.translator.cli.cts import CTS + +class Record(Node): + def __init__(self, db, record): + self.db = db + self.cts = CTS(db) + self.record = record + + trans = string.maketrans('(),', '___') + name = ['Record'] + for f_name, (f_type, f_default) in record._fields.iteritems(): + type_name = f_type._short_name().translate(trans) + name.append(type_name) + self.name = '__'.join(name) + record._name = self.name + + def __hash__(self): + return hash(self.record) + + def __eq__(self, other): + return self.record == other.record + + def get_name(self): + return self.name + + def get_base_class(self): + return '[mscorlib]System.Object' + + def render(self, ilasm): + if self.db.class_name(self.record) is not None: + return # already rendered + + self.ilasm = ilasm + + ilasm.begin_class(self.name, self.get_base_class()) + for f_name, (f_type, f_default) in self.record._fields.iteritems(): + cts_type = self.cts.lltype_to_cts(f_type) + if cts_type != 'void': + ilasm.field(f_name, cts_type) + + self._ctor() + ilasm.end_class() + + self.db.record_class(self.record, self.name) + + def _ctor(self): + self.ilasm.begin_function('.ctor', [], 'void', False, 'specialname', 'rtspecialname', 'instance') + self.ilasm.opcode('ldarg.0') + self.ilasm.call('instance void %s::.ctor()' % self.get_base_class()) + self.ilasm.opcode('ret') + self.ilasm.end_function() + Modified: pypy/dist/pypy/translator/cli/test/compile.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/compile.py (original) +++ pypy/dist/pypy/translator/cli/test/compile.py Thu Apr 27 14:47:24 2006 @@ -24,10 +24,15 @@ print 'OK' +class Foo: + def __init__(self, x): + self.x = x def bar(x, y): - a = [x] + [y] - return a[0] + t = (x,y) + return t[0] +# f = Foo(3) +# return f.x f = compile_function(bar, [int, int]) Added: pypy/dist/pypy/translator/cli/test/test_tuple.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/test/test_tuple.py Thu Apr 27 14:47:24 2006 @@ -0,0 +1,17 @@ +from pypy.translator.cli.test.runtest import check + +def create_tuple(x, y): + return x, y + +def test_tuple(): + def func(x, y): + t = create_tuple(x, y) + return t[0] + t[1] + check(func, [int, int], (42, 13)) + +def test_list_item(): + def func(x, y): + t = ([x, y], x) + return t[0][0] + check(func, [int, int], (42, 13)) + From auc at codespeak.net Thu Apr 27 15:17:53 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 15:17:53 +0200 (CEST) Subject: [pypy-svn] r26446 - pypy/dist/pypy/doc Message-ID: <20060427131753.47FF6100B7@code0.codespeak.net> Author: auc Date: Thu Apr 27 15:17:51 2006 New Revision: 26446 Modified: pypy/dist/pypy/doc/independent-project-ideas.txt Log: oops Modified: pypy/dist/pypy/doc/independent-project-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/independent-project-ideas.txt (original) +++ pypy/dist/pypy/doc/independent-project-ideas.txt Thu Apr 27 15:17:51 2006 @@ -16,8 +16,8 @@ compatible with both CPython and PyPy. See for example http://wiki.python.org/moin/CodingProjectIdeas/PygameOnCtypes . -* A related idea is to provide `efficient numeric arrays`_ (as in - numeric/numpy/numarray) in this way. +* A related idea is to provide efficient numeric arrays (as in + numeric/numpy/numarray) in this way, or `recreate them as builtins`_. * Start a back-end for a new target platform, e.g. for the **Java** virtual machine. This would be a path to bring PyPy and Jython closer and help @@ -61,7 +61,7 @@ .. _`efficient propagators for specialized finite domains`: http://codespeak.net/svn/pypy/extradoc/soc-2006/constraints.txt -.. _`efficient numeric arrays`: http://codespeak.net/svn/pypy/extradoc/soc-2006/numeric-arrays.txt +.. _`recreate them as builtins`: http://codespeak.net/svn/pypy/extradoc/soc-2006/numeric-arrays.txt .. _`code templating solution`: http://codespeak.net/svn/pypy/extradoc/soc-2006/code-templating.txt .. _`pypy-dev mailing list`: http://codespeak.net/mailman/listinfo/pypy-dev From cfbolz at codespeak.net Thu Apr 27 16:06:26 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 16:06:26 +0200 (CEST) Subject: [pypy-svn] r26447 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427140626.BA106100B5@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 16:06:19 2006 New Revision: 26447 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: test + fix for getweakrefs. use proper way to eat exceptions caused by a callback Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 16:06:19 2006 @@ -29,7 +29,7 @@ try: w_self.space.call_function(w_self.w_callable, w_self) except OperationError, e: - os.write(2, "XXX\n") + e.write_unraisable(w_self.space, 'function', w_self.w_callable) def __del__(w_self): if w_self.address != NULL: @@ -78,9 +78,9 @@ def getweakrefs(space, w_obj): if not isinstance(w_obj, W_Weakrefable): - return space.newlist() + return space.newlist([]) if w_obj.__lifeline__ is None: - return space.newlist() + return space.newlist([]) else: lifeline = w_obj.__lifeline__ result = [] Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 16:06:19 2006 @@ -77,3 +77,13 @@ ref2 = _weakref.ref(a, callback) del a assert alive.a is None + + def test_getweakrefs(self): + import _weakref + class A: + pass + a = A() + assert _weakref.getweakrefs(a) == [] + assert _weakref.getweakrefs(None) == [] + ref1 = _weakref.ref(a) + assert _weakref.getweakrefs(a) == [ref1] From auc at codespeak.net Thu Apr 27 16:19:22 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 27 Apr 2006 16:19:22 +0200 (CEST) Subject: [pypy-svn] r26448 - pypy/extradoc/soc-2006 Message-ID: <20060427141922.89451100C2@code0.codespeak.net> Author: auc Date: Thu Apr 27 16:19:06 2006 New Revision: 26448 Modified: pypy/extradoc/soc-2006/constraints.txt Log: Modified: pypy/extradoc/soc-2006/constraints.txt ============================================================================== --- pypy/extradoc/soc-2006/constraints.txt (original) +++ pypy/extradoc/soc-2006/constraints.txt Thu Apr 27 16:19:06 2006 @@ -10,11 +10,11 @@ with possibly subotpimal runtime performance but no external dependency, and a lot of fun, -* wrapping Gecode[1] : first finish the c-wrapper around Gecode (unless - interfacing PyPy with C++ becomes available), then use rctypes to - provide interpreter access to low-level Gecode functionality, and - finally integrate it into the existing interpreter-level constraint - solver. +* wrapping Gecode[1] : first finish the c-wrapper around Gecode + (unless interfacing PyPy with C++ becomes available), then use + rctypes to provide interpreter access to low-level Gecode + functionality, and finally integrate it into the existing + interpreter-level constraint solver. Rctypes is what PyPy offers, mimicking python ctypes, to interface the interpreter with C code. The machinery is still young and the From cfbolz at codespeak.net Thu Apr 27 17:04:16 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 17:04:16 +0200 (CEST) Subject: [pypy-svn] r26449 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427150416.B3242100B5@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 17:04:14 2006 New Revision: 26449 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: enable reusing of the weakref that has no callbacks. a bit annoying to get right with subclasses of weakref.ref Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 17:04:14 2006 @@ -24,7 +24,6 @@ w_self.address = NULL def activate_callback(w_self): - import os if not w_self.space.is_w(w_self.w_callable, w_self.space.w_None): try: w_self.space.call_function(w_self.w_callable, w_self) @@ -40,6 +39,7 @@ class WeakrefLifeline(object): def __init__(self): self.addr_refs = [] + self.w_cached_weakref = None def __del__(self): for i in range(len(self.addr_refs) - 1, -1, -1): @@ -54,10 +54,18 @@ w_ref.activate_callback() def get_weakref(self, space, w_subtype, w_obj, w_callable): + w_weakreftype = space.gettypeobject(W_Weakref.typedef) + is_weakreftype = space.is_w(w_weakreftype, w_subtype) + can_reuse = space.is_w(w_callable, space.w_None) + if is_weakreftype and can_reuse and self.w_cached_weakref is not None: + return self.w_cached_weakref w_ref = space.allocate_instance(W_Weakref, w_subtype) - W_Weakref.__init__(w_ref, space, self, len(self.addr_refs), + index = len(self.addr_refs) + W_Weakref.__init__(w_ref, space, self, index, w_obj, w_callable) self.addr_refs.append(cast_object_to_address(w_ref)) + if is_weakreftype and can_reuse: + self.w_cached_weakref = w_ref return w_ref def ref_is_dead(self, index): Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 17:04:14 2006 @@ -78,6 +78,19 @@ del a assert alive.a is None + def test_weakref_reusing(self): + import _weakref + class A: + pass + a = A() + ref1 = _weakref.ref(a) + ref2 = _weakref.ref(a) + assert ref1 is ref2 + class wref(_weakref.ref): + pass + wref1 = wref(a) + assert isinstance(wref1, wref) + def test_getweakrefs(self): import _weakref class A: From cfbolz at codespeak.net Thu Apr 27 17:07:48 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 17:07:48 +0200 (CEST) Subject: [pypy-svn] r26450 - pypy/dist/pypy/module/_weakref Message-ID: <20060427150748.C53C6100B7@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 17:07:45 2006 New Revision: 26450 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py Log: code reordering Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 17:07:45 2006 @@ -9,32 +9,6 @@ W_Weakrefable = W_Root W_Weakrefable.__lifeline__ = None -class W_Weakref(Wrappable): - def __init__(w_self, space, lifeline, index, w_obj, w_callable): - w_self.space = space - w_self.address = cast_object_to_address(w_obj) - w_self.w_callable = w_callable - w_self.addr_lifeline = cast_object_to_address(lifeline) - w_self.index = index - - def descr__call__(self): - return cast_address_to_object(self.address, W_Weakrefable) - - def invalidate(w_self): - w_self.address = NULL - - def activate_callback(w_self): - if not w_self.space.is_w(w_self.w_callable, w_self.space.w_None): - try: - w_self.space.call_function(w_self.w_callable, w_self) - except OperationError, e: - e.write_unraisable(w_self.space, 'function', w_self.w_callable) - - def __del__(w_self): - if w_self.address != NULL: - lifeline = cast_address_to_object(w_self.addr_lifeline, - WeakrefLifeline) - lifeline.ref_is_dead(w_self.index) class WeakrefLifeline(object): def __init__(self): @@ -71,6 +45,48 @@ def ref_is_dead(self, index): self.addr_refs[index] = NULL + +class W_Weakref(Wrappable): + def __init__(w_self, space, lifeline, index, w_obj, w_callable): + w_self.space = space + w_self.address = cast_object_to_address(w_obj) + w_self.w_callable = w_callable + w_self.addr_lifeline = cast_object_to_address(lifeline) + w_self.index = index + + def descr__call__(self): + return cast_address_to_object(self.address, W_Weakrefable) + + def invalidate(w_self): + w_self.address = NULL + + def activate_callback(w_self): + if not w_self.space.is_w(w_self.w_callable, w_self.space.w_None): + try: + w_self.space.call_function(w_self.w_callable, w_self) + except OperationError, e: + e.write_unraisable(w_self.space, 'function', w_self.w_callable) + + def __del__(w_self): + if w_self.address != NULL: + lifeline = cast_address_to_object(w_self.addr_lifeline, + WeakrefLifeline) + lifeline.ref_is_dead(w_self.index) + + +def descr__new__(space, w_subtype, w_obj, w_callable=None): + assert isinstance(w_obj, W_Weakrefable) + if w_obj.__lifeline__ is None: + w_obj.__lifeline__ = WeakrefLifeline() + return w_obj.__lifeline__.get_weakref(space, w_subtype, w_obj, w_callable) + + +W_Weakref.typedef = TypeDef("weakref", + __new__ = interp2app(descr__new__), + __call__ = interp2app(W_Weakref.descr__call__, unwrap_spec=['self']) +) + + def getweakrefcount(space, w_obj): if not isinstance(w_obj, W_Weakrefable): return space.wrap(0) @@ -98,14 +114,4 @@ result.append(cast_address_to_object(addr, W_Weakref)) return space.newlist(result) -def descr__new__(space, w_subtype, w_obj, w_callable=None): - assert isinstance(w_obj, W_Weakrefable) - if w_obj.__lifeline__ is None: - w_obj.__lifeline__ = WeakrefLifeline() - return w_obj.__lifeline__.get_weakref(space, w_subtype, w_obj, w_callable) - -W_Weakref.typedef = TypeDef("weakref", - __new__ = interp2app(descr__new__), - __call__ = interp2app(W_Weakref.descr__call__, unwrap_spec=['self']) -) From cfbolz at codespeak.net Thu Apr 27 17:35:07 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 17:35:07 +0200 (CEST) Subject: [pypy-svn] r26451 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427153507.7D7D8100B0@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 17:35:05 2006 New Revision: 26451 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: add equality of weak references Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 17:35:05 2006 @@ -2,7 +2,7 @@ from pypy.interpreter.argument import Arguments from pypy.interpreter.error import OperationError from pypy.interpreter.typedef import GetSetProperty, TypeDef -from pypy.interpreter.gateway import interp2app +from pypy.interpreter.gateway import interp2app, ObjSpace from pypy.rpython.objectmodel import cast_address_to_object, cast_object_to_address from pypy.rpython.lltypesystem.llmemory import NULL @@ -80,9 +80,20 @@ w_obj.__lifeline__ = WeakrefLifeline() return w_obj.__lifeline__.get_weakref(space, w_subtype, w_obj, w_callable) +def descr__eq__(space, ref1, ref2): + if ref1.address == NULL or ref2.address == NULL: + return space.is_(ref1, ref2) + return space.eq(ref1.descr__call__(), ref2.descr__call__()) + +def descr__ne__(space, ref1, ref2): + return space.not_(space.eq(ref1, ref2)) W_Weakref.typedef = TypeDef("weakref", __new__ = interp2app(descr__new__), + __eq__ = interp2app(descr__eq__, + unwrap_spec=[ObjSpace, W_Weakref, W_Weakref]), + __ne__ = interp2app(descr__ne__, + unwrap_spec=[ObjSpace, W_Weakref, W_Weakref]), __call__ = interp2app(W_Weakref.descr__call__, unwrap_spec=['self']) ) Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 17:35:05 2006 @@ -91,6 +91,23 @@ wref1 = wref(a) assert isinstance(wref1, wref) + def test_weakref_equality(self): + import _weakref + class A: + def __eq__(self, other): + return True + a1 = A() + a2 = A() + ref1 = _weakref.ref(a1) + ref2 = _weakref.ref(a2) + assert ref1 == ref2 + del a1 + assert not ref1 == ref2 + assert ref1 != ref2 + del a2 + assert not ref1 == ref2 + assert ref1 != ref2 + def test_getweakrefs(self): import _weakref class A: From mwh at codespeak.net Thu Apr 27 18:17:01 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 27 Apr 2006 18:17:01 +0200 (CEST) Subject: [pypy-svn] r26452 - pypy/extradoc/sprintinfo/iceland Message-ID: <20060427161701.47073100AD@code0.codespeak.net> Author: mwh Date: Thu Apr 27 18:17:00 2006 New Revision: 26452 Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Log: add armin's and my dates. Modified: pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (original) +++ pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt Thu Apr 27 18:17:00 2006 @@ -122,3 +122,17 @@ organisation: comments: every departure date it's fine to me. + +Armin Rigo +--------------------- + +arrival date: 21 +departure date 31 + +Michael Hudson +------------------------ + +arrival date: 21 +departure date: 28 + +(Armin and Michael are being paid for by their employer, HHU) From arigo at codespeak.net Thu Apr 27 19:29:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 19:29:49 +0200 (CEST) Subject: [pypy-svn] r26455 - pypy/dist/pypy/interpreter Message-ID: <20060427172949.34F1D100A5@code0.codespeak.net> Author: arigo Date: Thu Apr 27 19:29:48 2006 New Revision: 26455 Modified: pypy/dist/pypy/interpreter/function.py Log: wrap(list) crashes the annotator, and is not correct here because at app-level we need a wrapped tuple. Modified: pypy/dist/pypy/interpreter/function.py ============================================================================== --- pypy/dist/pypy/interpreter/function.py (original) +++ pypy/dist/pypy/interpreter/function.py Thu Apr 27 19:29:48 2006 @@ -175,13 +175,17 @@ mod = space.interp_w(MixedModule, w_mod) new_inst = mod.get('func_new') w = space.wrap + if self.closure is None: + w_closure = space.w_None + else: + w_closure = space.newtuple([w(cell) for cell in self.closure]) tup = [ w(self.code), #space.newdict([]), #XXX because pickle.py has no _pickle_moduledict yet... self.w_func_globals, w(self.name), space.newtuple(self.defs_w), - w(self.closure), + w_closure, ] return space.newtuple([new_inst, space.newtuple(tup)]) From tismer at codespeak.net Thu Apr 27 20:18:02 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 27 Apr 2006 20:18:02 +0200 (CEST) Subject: [pypy-svn] r26456 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060427181802.F1CF41009D@code0.codespeak.net> Author: tismer Date: Thu Apr 27 20:18:00 2006 New Revision: 26456 Modified: pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/test/test_wrapping.py Log: oops, tiny logic error Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Thu Apr 27 20:18:00 2006 @@ -577,7 +577,8 @@ if ann.binding(graph.getargs()[0]).classdef is not clsdef: value = new_method_graph(graph, clsdef, fname, self.translator) self.name_for_meth[value] = fname - self.is_method[value] = self.use_true_methods + if self.use_true_methods: + self.is_method[value] = True yield '%s.%s = %s' % (name, key, self.nameof(value)) if not init_seen: log.WARNING('No __init__ found for %s - you cannot build instances' % Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Thu Apr 27 20:18:00 2006 @@ -37,11 +37,13 @@ bk = rtyper.annotator.bookkeeper if not exports: exports = [] - all = [obj.__name__ for obj in exports] + all = [obj.__name__ for obj in exports if obj.__name__ != '__init__'] exports = exports + [('__all__', all)] ann.build_types(func, get_annotation(func)) + pyobj_options = {} + for obj in exports: if isinstance(obj, type): clsdef = bk.getuniqueclassdef(obj) @@ -50,6 +52,8 @@ if not ann.bookkeeper.getdesc(obj).querycallfamily(): # not annotated, so enforce it ann.build_types(obj, get_annotation(obj)) + if obj.__name__ == '__init__': + pyobj_options['use_true_methods'] = True if view: t.viewcg() @@ -68,9 +72,6 @@ cbuilder = CExtModuleBuilder(t, func, gcpolicy=gcpolicy) # explicit build of database - pyobj_options = { - 'use_true_methods': '__init__' in exports - } db = cbuilder.build_database(exports=exports, pyobj_options=pyobj_options) cbuilder.generate_source(db) if view: From arigo at codespeak.net Thu Apr 27 20:35:32 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 20:35:32 +0200 (CEST) Subject: [pypy-svn] r26457 - pypy/dist/pypy/doc Message-ID: <20060427183532.BFBEC10092@code0.codespeak.net> Author: arigo Date: Thu Apr 27 20:35:31 2006 New Revision: 26457 Modified: pypy/dist/pypy/doc/independent-project-ideas.txt Log: Added links for the topics where I can think of one. Modified: pypy/dist/pypy/doc/independent-project-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/independent-project-ideas.txt (original) +++ pypy/dist/pypy/doc/independent-project-ideas.txt Thu Apr 27 20:35:31 2006 @@ -15,6 +15,9 @@ developpers, and it is now the best path to write extension modules that are compatible with both CPython and PyPy. See for example http://wiki.python.org/moin/CodingProjectIdeas/PygameOnCtypes . + (This is not documented so far; for an example on which we are working now, + look at the socket module at + http://codespeak.net/svn/pypy/dist/pypy/module/_socket/). * A related idea is to provide efficient numeric arrays (as in numeric/numpy/numarray) in this way, or `recreate them as builtins`_. @@ -22,7 +25,9 @@ * Start a back-end for a new target platform, e.g. for the **Java** virtual machine. This would be a path to bring PyPy and Jython closer and help support Jython's effort to keep up-to-date, in particular with extension - modules. + modules. Other back-ends that we are already working on are for Squeak, + Microsoft's CLI, and Common Lisp; look for ``squeak``, ``cli`` and ``cl`` + in http://codespeak.net/svn/pypy/dist/pypy/translator. * A research project: experiment with optimizations in PyPy, e.g. by trying various **data structures** for dicts or other built-in objects, by switching @@ -37,13 +42,15 @@ * Write **new object spaces** adding features like transparent or semi-transparent distribution of a program across several machines, and/or persistence (pickling of program state). + Read more about `object spaces`_... * Write an interpreter for **another dynamic language** in the PyPy framework. For example, a Javascript interpreter would be suitable. Ruby too (though the latter is probably more than two months of work). Or Scheme, or... etc. * A different kind of project: work on **py.test** to add features like - distributed testing, automatic reports, etc. + distributed testing, automatic reports, etc. The py.test tool is part of + the `py lib`_, on which many PyPy developers work. * Constraint programming: `efficient propagators for specialized finite domains`_ (like numbers, sets, intervals). @@ -57,11 +64,13 @@ sub-tasks in the context of the JIT compiler of PyPy... Feel free to mention your interest and discuss these ideas on the `pypy-dev -mailing list`_. +mailing list`_. You can also have a look around our documentation_. .. _`efficient propagators for specialized finite domains`: http://codespeak.net/svn/pypy/extradoc/soc-2006/constraints.txt .. _`recreate them as builtins`: http://codespeak.net/svn/pypy/extradoc/soc-2006/numeric-arrays.txt .. _`code templating solution`: http://codespeak.net/svn/pypy/extradoc/soc-2006/code-templating.txt .. _`pypy-dev mailing list`: http://codespeak.net/mailman/listinfo/pypy-dev - +.. _`object spaces`: objspace.html +.. _`py lib`: http://codespeak.net/py/ +.. _documentation: index.html From arigo at codespeak.net Thu Apr 27 20:37:04 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 20:37:04 +0200 (CEST) Subject: [pypy-svn] r26458 - pypy/dist/pypy/doc/discussion Message-ID: <20060427183704.6B5AA10092@code0.codespeak.net> Author: arigo Date: Thu Apr 27 20:37:03 2006 New Revision: 26458 Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt Log: Restification. Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/logic-plan.txt (original) +++ pypy/dist/pypy/doc/discussion/logic-plan.txt Thu Apr 27 20:37:03 2006 @@ -37,7 +37,7 @@ whose execution path contains a 'choice' is a logic, or relational (in Oz parlance) program. -For instance : +For instance:: def foo(): choice: @@ -46,7 +46,7 @@ from math import sqrt return sqrt(4) -def bar(): + def bar(): choice: return 0 or: return 1 def entry_point(): @@ -82,9 +82,9 @@ space and ready to be later merged. We leave the semantics of space merging for another day. -Pardon the silly ascii art : +Pardon the silly ascii art:: -entry_point -> foo : choice + entry_point -> foo : choice /\ / \ / 2 (solution) @@ -113,7 +113,7 @@ choose() The choice operator can be written straightforwardly in terms of -choose : +choose:: def foo(): choice = choose(3) From arigo at codespeak.net Thu Apr 27 20:44:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 27 Apr 2006 20:44:05 +0200 (CEST) Subject: [pypy-svn] r26459 - pypy/dist/pypy/module/__builtin__ Message-ID: <20060427184405.36E541009B@code0.codespeak.net> Author: arigo Date: Thu Apr 27 20:44:03 2006 New Revision: 26459 Modified: pypy/dist/pypy/module/__builtin__/__init__.py pypy/dist/pypy/module/__builtin__/app_inspect.py pypy/dist/pypy/module/__builtin__/operation.py Log: Commented out the app-level version of callable(), now that we have an interp-level equivalent. Modified: pypy/dist/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/__init__.py (original) +++ pypy/dist/pypy/module/__builtin__/__init__.py Thu Apr 27 20:44:03 2006 @@ -37,7 +37,6 @@ 'issubclass' : 'app_inspect.issubclass', 'isinstance' : 'app_inspect.isinstance', 'hasattr' : 'app_inspect.hasattr', - 'callable' : 'app_inspect.callable', 'globals' : 'app_inspect.globals', 'locals' : 'app_inspect.locals', 'vars' : 'app_inspect.vars', @@ -105,6 +104,7 @@ 'id' : 'operation.id', '_seqiter' : 'operation._seqiter', 'intern' : 'operation.intern', + 'callable' : 'operation.callable', 'compile' : 'compiling.compile', 'eval' : 'compiling.eval', Modified: pypy/dist/pypy/module/__builtin__/app_inspect.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/app_inspect.py (original) +++ pypy/dist/pypy/module/__builtin__/app_inspect.py Thu Apr 27 20:44:03 2006 @@ -78,15 +78,16 @@ except AttributeError: return False -def callable(ob): - import __builtin__ # XXX this is insane but required for now for geninterp - for c in type(ob).__mro__: - if '__call__' in c.__dict__: - if isinstance(ob, __builtin__._instance): # old style instance! - return getattr(ob, '__call__', None) is not None - return True - else: - return False +# Replaced by the interp-level helper space.callable(): +##def callable(ob): +## import __builtin__ # XXX this is insane but required for now for geninterp +## for c in type(ob).__mro__: +## if '__call__' in c.__dict__: +## if isinstance(ob, __builtin__._instance): # old style instance! +## return getattr(ob, '__call__', None) is not None +## return True +## else: +## return False def dir(*args): """dir([object]) -> list of strings Modified: pypy/dist/pypy/module/__builtin__/operation.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/operation.py (original) +++ pypy/dist/pypy/module/__builtin__/operation.py Thu Apr 27 20:44:03 2006 @@ -164,3 +164,6 @@ if space.is_w(space.type(w_str), space.w_str): return space.new_interned_str(space.str_w(w_str)) raise OperationError(space.w_TypeError, space.wrap("intern() argument 1 must be string.")) + +def callable(space, w_object): + return space.callable(w_object) From antocuni at codespeak.net Thu Apr 27 21:30:15 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 27 Apr 2006 21:30:15 +0200 (CEST) Subject: [pypy-svn] r26460 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060427193015.566DB100AD@code0.codespeak.net> Author: antocuni Date: Thu Apr 27 21:30:05 2006 New Revision: 26460 Added: pypy/dist/pypy/translator/cli/test/test_dict.py (contents, props changed) Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/src/pypylib.cs Log: Added some support for dicts. By now all operation are supported except for iteration. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Thu Apr 27 21:30:05 2006 @@ -4,9 +4,10 @@ import exceptions -from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float +#from pypy.rpython.lltypesystem.lltype import Signed, Unsigned, Void, Bool, Float from pypy.rpython.lltypesystem.lltype import SignedLongLong, UnsignedLongLong -from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod, List, Record +#from pypy.rpython.ootypesystem.ootype import Instance, Class, StaticMethod, List, Record, Dict +from pypy.rpython.ootypesystem import ootype from pypy.translator.cli.option import getoption from pypy.translator.cli import oopspec @@ -16,20 +17,24 @@ py.log.setconsumer("cli", ansi_log) PYPY_LIST = '[pypylib]pypy.runtime.List`1<%s>' +PYPY_DICT = '[pypylib]pypy.runtime.Dict`2<%s, %s>' _lltype_to_cts = { - Void: 'void', - Signed: 'int32', - Unsigned: 'unsigned int32', + ootype.Void: 'void', + ootype.Signed: 'int32', + ootype.Unsigned: 'unsigned int32', SignedLongLong: 'int64', UnsignedLongLong: 'unsigned int64', - Bool: 'bool', - Float: 'float64', - Class: 'class [mscorlib]System.Type', - - # TODO: it seems a hack - List.SELFTYPE_T: 'class ' + (PYPY_LIST % '!0'), - List.ITEMTYPE_T: '!0', + ootype.Bool: 'bool', + ootype.Float: 'float64', + ootype.Class: 'class [mscorlib]System.Type', + + # maps generic types to their ordinal + ootype.List.SELFTYPE_T: 'class ' + (PYPY_LIST % '!0'), + ootype.List.ITEMTYPE_T: '!0', + ootype.Dict.SELFTYPE_T: 'class ' + (PYPY_DICT % ('!0', '!1')), + ootype.Dict.KEYTYPE_T: '!0', + ootype.Dict.VALUETYPE_T: '!1', } _pyexception_to_cts = { @@ -59,17 +64,21 @@ return result def lltype_to_cts(self, t, include_class=True): - if isinstance(t, Instance): + if isinstance(t, ootype.Instance): self.db.pending_class(t) return self.__class(t._name, include_class) - elif isinstance(t, Record): + elif isinstance(t, ootype.Record): name = self.db.pending_record(t) return self.__class(name, include_class) - elif isinstance(t, StaticMethod): + elif isinstance(t, ootype.StaticMethod): return 'void' # TODO: is it correct to ignore StaticMethod? - elif isinstance(t, List): + elif isinstance(t, ootype.List): item_type = self.lltype_to_cts(t._ITEMTYPE) return self.__class(PYPY_LIST % item_type, include_class) + elif isinstance(t, ootype.Dict): + key_type = self.lltype_to_cts(t._KEYTYPE) + value_type = self.lltype_to_cts(t._VALUETYPE) + return self.__class(PYPY_DICT % (key_type, value_type), include_class) return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) @@ -86,7 +95,7 @@ ret_type, ret_var = self.llvar_to_cts(graph.getreturnvar()) func_name = func_name or graph.name - args = [arg for arg in graph.getargs() if arg.concretetype is not Void] + args = [arg for arg in graph.getargs() if arg.concretetype is not ootype.Void] if is_method: args = args[1:] @@ -97,13 +106,13 @@ def method_signature(self, obj, name): # TODO: use callvirt only when strictly necessary - if isinstance(obj, Instance): + if isinstance(obj, ootype.Instance): owner, meth = obj._lookup(name) class_name = obj._name full_name = 'class %s::%s' % (class_name, name) return self.graph_to_signature(meth.graph, True, full_name), True - elif isinstance(obj, List): + elif isinstance(obj, ootype.BuiltinType): meth = oopspec.get_method(obj, name) class_name = self.lltype_to_cts(obj) ret_type = self.lltype_to_cts(meth.RESULT) Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Thu Apr 27 21:30:05 2006 @@ -80,4 +80,37 @@ } */ } + + public class Dict: System.Collections.Generic.Dictionary + { + public int ll_length() + { + return this.Count; + } + + public TValue ll_get(TKey key) + { + return this[key]; + } + + public void ll_set(TKey key, TValue value) + { + this[key] = value; + } + + public bool ll_remove(TKey key) + { + return this.Remove(key); + } + + public bool ll_contains(TKey key) + { + return this.ContainsKey(key); + } + + public void ll_clear() + { + this.Clear(); + } + } } Added: pypy/dist/pypy/translator/cli/test/test_dict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cli/test/test_dict.py Thu Apr 27 21:30:05 2006 @@ -0,0 +1,8 @@ +from pypy.translator.cli.test.runtest import check + + +def test_dict(): + def func(x, y): + d = {x: x+1, y: y+1} + return d[x] + check(func, [int, int], (42, 13)) From antocuni at codespeak.net Thu Apr 27 21:50:34 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 27 Apr 2006 21:50:34 +0200 (CEST) Subject: [pypy-svn] r26461 - in pypy/dist/pypy/translator/cli: . src test Message-ID: <20060427195034.C9D671009F@code0.codespeak.net> Author: antocuni Date: Thu Apr 27 21:50:24 2006 New Revision: 26461 Modified: pypy/dist/pypy/translator/cli/cts.py pypy/dist/pypy/translator/cli/record.py pypy/dist/pypy/translator/cli/src/pypylib.cs pypy/dist/pypy/translator/cli/test/test_dict.py Log: Added support for dict iteration. Modified: pypy/dist/pypy/translator/cli/cts.py ============================================================================== --- pypy/dist/pypy/translator/cli/cts.py (original) +++ pypy/dist/pypy/translator/cli/cts.py Thu Apr 27 21:50:24 2006 @@ -18,6 +18,7 @@ PYPY_LIST = '[pypylib]pypy.runtime.List`1<%s>' PYPY_DICT = '[pypylib]pypy.runtime.Dict`2<%s, %s>' +PYPY_DICT_ITEMS_ITERATOR = '[pypylib]pypy.runtime.DictItemsIterator`2<%s, %s>' _lltype_to_cts = { ootype.Void: 'void', @@ -35,6 +36,9 @@ ootype.Dict.SELFTYPE_T: 'class ' + (PYPY_DICT % ('!0', '!1')), ootype.Dict.KEYTYPE_T: '!0', ootype.Dict.VALUETYPE_T: '!1', + ootype.DictItemsIterator.SELFTYPE_T: 'class ' + (PYPY_DICT_ITEMS_ITERATOR % ('!0', '!1')), + ootype.DictItemsIterator.KEYTYPE_T: '!0', + ootype.DictItemsIterator.VALUETYPE_T: '!1', } _pyexception_to_cts = { @@ -79,6 +83,10 @@ key_type = self.lltype_to_cts(t._KEYTYPE) value_type = self.lltype_to_cts(t._VALUETYPE) return self.__class(PYPY_DICT % (key_type, value_type), include_class) + elif isinstance(t, ootype.DictItemsIterator): + key_type = self.lltype_to_cts(t._KEYTYPE) + value_type = self.lltype_to_cts(t._VALUETYPE) + return self.__class(PYPY_DICT_ITEMS_ITERATOR % (key_type, value_type), include_class) return _get_from_dict(_lltype_to_cts, t, 'Unknown type %s' % t) Modified: pypy/dist/pypy/translator/cli/record.py ============================================================================== --- pypy/dist/pypy/translator/cli/record.py (original) +++ pypy/dist/pypy/translator/cli/record.py Thu Apr 27 21:50:24 2006 @@ -9,7 +9,7 @@ self.cts = CTS(db) self.record = record - trans = string.maketrans('(),', '___') + trans = string.maketrans('<>(), ', '______') name = ['Record'] for f_name, (f_type, f_default) in record._fields.iteritems(): type_name = f_type._short_name().translate(trans) Modified: pypy/dist/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/dist/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/dist/pypy/translator/cli/src/pypylib.cs Thu Apr 27 21:50:24 2006 @@ -1,4 +1,5 @@ using System; +using System.Collections.Generic; namespace pypy.runtime { @@ -112,5 +113,35 @@ { this.Clear(); } + + public DictItemsIterator ll_get_items_iterator() + { + return new DictItemsIterator(this.GetEnumerator()); + } + } + + public class DictItemsIterator + { + IEnumerator> it; + + public DictItemsIterator(IEnumerator> it) + { + this.it = it; + } + + public bool ll_go_next() + { + return it.MoveNext(); + } + + public TKey ll_current_key() + { + return it.Current.Key; + } + + public TValue ll_current_value() + { + return it.Current.Value; + } } } Modified: pypy/dist/pypy/translator/cli/test/test_dict.py ============================================================================== --- pypy/dist/pypy/translator/cli/test/test_dict.py (original) +++ pypy/dist/pypy/translator/cli/test/test_dict.py Thu Apr 27 21:50:24 2006 @@ -6,3 +6,12 @@ d = {x: x+1, y: y+1} return d[x] check(func, [int, int], (42, 13)) + +def test_iteration(): + def func(x, y): + d = {x: x+1, y: y+1} + tot = 0 + for value in d.itervalues(): + tot += value + return tot + check(func, [int, int], (42, 13)) From cfbolz at codespeak.net Thu Apr 27 22:01:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 22:01:49 +0200 (CEST) Subject: [pypy-svn] r26462 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427200149.EB617100B7@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 22:01:47 2006 New Revision: 26462 Modified: pypy/dist/pypy/module/_weakref/__init__.py pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: first cut at implementing proxies Modified: pypy/dist/pypy/module/_weakref/__init__.py ============================================================================== --- pypy/dist/pypy/module/_weakref/__init__.py (original) +++ pypy/dist/pypy/module/_weakref/__init__.py Thu Apr 27 22:01:47 2006 @@ -8,4 +8,7 @@ 'getweakrefcount': 'interp__weakref.getweakrefcount', 'getweakrefs': 'interp__weakref.getweakrefs', 'ReferenceType': 'interp__weakref.W_Weakref', + 'ProxyType': 'interp__weakref.W_Proxy', + 'CallableProxyType': 'interp__weakref.W_Proxy', + 'proxy': 'interp__weakref.proxy' } Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 22:01:47 2006 @@ -1,3 +1,4 @@ +import py from pypy.interpreter.baseobjspace import Wrappable, W_Root from pypy.interpreter.argument import Arguments from pypy.interpreter.error import OperationError @@ -14,6 +15,7 @@ def __init__(self): self.addr_refs = [] self.w_cached_weakref = None + self.w_cached_proxy = None def __del__(self): for i in range(len(self.addr_refs) - 1, -1, -1): @@ -42,6 +44,20 @@ self.w_cached_weakref = w_ref return w_ref + def get_proxy(self, space, w_obj, w_callable): + can_reuse = space.is_w(w_callable, space.w_None) + if can_reuse and self.w_cached_proxy is not None: + return self.w_cached_proxy + index = len(self.addr_refs) + if space.is_true(space.callable(w_obj)): + w_proxy = W_CallableProxy(space, self, index, w_obj, w_callable) + else: + w_proxy = W_Proxy(space, self, index, w_obj, w_callable) + self.addr_refs.append(cast_object_to_address(w_proxy)) + if can_reuse: + self.w_cached_proxy = w_proxy + return w_proxy + def ref_is_dead(self, index): self.addr_refs[index] = NULL @@ -53,10 +69,10 @@ w_self.w_callable = w_callable w_self.addr_lifeline = cast_object_to_address(lifeline) w_self.index = index - - def descr__call__(self): - return cast_address_to_object(self.address, W_Weakrefable) + def dereference(self): + return cast_address_to_object(self.address, W_Weakrefable) + def invalidate(w_self): w_self.address = NULL @@ -74,7 +90,7 @@ lifeline.ref_is_dead(w_self.index) -def descr__new__(space, w_subtype, w_obj, w_callable=None): +def descr__new__weakref(space, w_subtype, w_obj, w_callable=None): assert isinstance(w_obj, W_Weakrefable) if w_obj.__lifeline__ is None: w_obj.__lifeline__ = WeakrefLifeline() @@ -83,18 +99,18 @@ def descr__eq__(space, ref1, ref2): if ref1.address == NULL or ref2.address == NULL: return space.is_(ref1, ref2) - return space.eq(ref1.descr__call__(), ref2.descr__call__()) + return space.eq(ref1.dereference(), ref2.dereference()) def descr__ne__(space, ref1, ref2): return space.not_(space.eq(ref1, ref2)) W_Weakref.typedef = TypeDef("weakref", - __new__ = interp2app(descr__new__), + __new__ = interp2app(descr__new__weakref), __eq__ = interp2app(descr__eq__, unwrap_spec=[ObjSpace, W_Weakref, W_Weakref]), __ne__ = interp2app(descr__ne__, unwrap_spec=[ObjSpace, W_Weakref, W_Weakref]), - __call__ = interp2app(W_Weakref.descr__call__, unwrap_spec=['self']) + __call__ = interp2app(W_Weakref.dereference, unwrap_spec=['self']) ) @@ -125,4 +141,68 @@ result.append(cast_address_to_object(addr, W_Weakref)) return space.newlist(result) +#_________________________________________________________________ +# Proxy + +class W_Proxy(W_Weakref): + pass + +class W_CallableProxy(W_Proxy): + def descr__call__(self, space, __args__): + w_obj = force(space, self) + return space.call_args(w_obj, __args__) + +def proxy(space, w_obj, w_callable=None): + assert isinstance(w_obj, W_Weakrefable) + if w_obj.__lifeline__ is None: + w_obj.__lifeline__ = WeakrefLifeline() + return w_obj.__lifeline__.get_proxy(space, w_obj, w_callable) + +def descr__new__proxy(space, w_subtype, w_obj, w_callable=None): + assert isinstance(w_obj, W_Weakrefable) + if w_obj.__lifeline__ is None: + w_obj.__lifeline__ = WeakrefLifeline() + return w_obj.__lifeline__.get_proxy(space, w_subtype, w_obj, w_callable) + +def force(space, proxy): + if not isinstance(proxy, W_Proxy): + return proxy + w_obj = proxy.dereference() + if space.is_w(w_obj, space.w_None): + raise OperationError( + space.w_ReferenceError, + space.wrap("weakly referenced object no longer exists")) + return w_obj + +proxy_typedef_dict = {} +callable_proxy_typedef_dict = {} +special_ops = {'repr': True} + +for opname, _, arity, special_methods in ObjSpace.MethodTable: + if opname in special_ops: + continue + nonspaceargs = ", ".join(["w_obj%s" % i for i in range(arity)]) + code = "def func(space, %s):\n" % (nonspaceargs, ) + for i in range(arity): + code += " w_obj%s = force(space, w_obj%s)\n" % (i, i) + code += " return space.%s(%s)" % (opname, nonspaceargs) + exec py.code.Source(code).compile() + for special_method in special_methods: + proxy_typedef_dict[special_method] = interp2app( + func, unwrap_spec=[ObjSpace] + [W_Root] * arity) + callable_proxy_typedef_dict[special_method] = interp2app( + func, unwrap_spec=[ObjSpace] + [W_Root] * arity) + + +W_Proxy.typedef = TypeDef("weakproxy", + __new__ = interp2app(descr__new__proxy), + **proxy_typedef_dict) +W_Proxy.typedef.accepable_as_base_class = False + +W_CallableProxy.typedef = TypeDef("callableweakproxy", + __new__ = interp2app(descr__new__proxy), + __call__ = interp2app(W_CallableProxy.descr__call__, + unwrap_spec=['self', ObjSpace, Arguments]), + **callable_proxy_typedef_dict) +W_CallableProxy.typedef.accepable_as_base_class = False Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 22:01:47 2006 @@ -117,3 +117,39 @@ assert _weakref.getweakrefs(None) == [] ref1 = _weakref.ref(a) assert _weakref.getweakrefs(a) == [ref1] + +class AppTestProxy(object): + def setup_class(cls): + space = gettestobjspace(usemodules=('_weakref',)) + cls.space = space + + def test_simple(self): + import _weakref + class A(object): + def __init__(self, x): + self.x = x + a = A(1) + p = _weakref.proxy(a) + assert p.x == 1 + assert str(p) == str(a) + raises(TypeError, p) + + def test_caching(self): + import _weakref + class A(object): pass + a = A() + assert _weakref.proxy(a) is _weakref.proxy(a) + + def test_callable_proxy(self): + import _weakref + class A(object): + def __call__(self): + global_a.x = 1 + global_a = A() + global_a.x = 41 + A_ = _weakref.proxy(A) + a = A_() + assert isinstance(a, A) + a_ = _weakref.proxy(a) + a_() + assert global_a.x == 1 From cfbolz at codespeak.net Thu Apr 27 22:11:06 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 22:11:06 +0200 (CEST) Subject: [pypy-svn] r26463 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427201106.E461F100B7@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 22:11:00 2006 New Revision: 26463 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: prevent direct creation of instances Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 22:11:00 2006 @@ -159,10 +159,15 @@ return w_obj.__lifeline__.get_proxy(space, w_obj, w_callable) def descr__new__proxy(space, w_subtype, w_obj, w_callable=None): - assert isinstance(w_obj, W_Weakrefable) - if w_obj.__lifeline__ is None: - w_obj.__lifeline__ = WeakrefLifeline() - return w_obj.__lifeline__.get_proxy(space, w_subtype, w_obj, w_callable) + raise OperationError( + space.w_TypeError, + space.wrap("cannot create 'weakproxy' instances")) + +def descr__new__callableproxy(space, w_subtype, w_obj, w_callable=None): + raise OperationError( + space.w_TypeError, + space.wrap("cannot create 'weakcallableproxy' instances")) + def force(space, proxy): if not isinstance(proxy, W_Proxy): @@ -199,8 +204,8 @@ **proxy_typedef_dict) W_Proxy.typedef.accepable_as_base_class = False -W_CallableProxy.typedef = TypeDef("callableweakproxy", - __new__ = interp2app(descr__new__proxy), +W_CallableProxy.typedef = TypeDef("weakcallableproxy", + __new__ = interp2app(descr__new__callableproxy), __call__ = interp2app(W_CallableProxy.descr__call__, unwrap_spec=['self', ObjSpace, Arguments]), **callable_proxy_typedef_dict) Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 22:11:00 2006 @@ -153,3 +153,9 @@ a_ = _weakref.proxy(a) a_() assert global_a.x == 1 + + def test_dont_create_directly(self): + import _weakref + raises(TypeError, _weakref.ProxyType, []) + raises(TypeError, _weakref.CallableProxyType, []) + From cfbolz at codespeak.net Thu Apr 27 22:43:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 22:43:18 +0200 (CEST) Subject: [pypy-svn] r26464 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427204318.5F6F2100B7@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 22:43:11 2006 New Revision: 26464 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: make proxy objects unhashable Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 22:43:11 2006 @@ -71,6 +71,8 @@ w_self.index = index def dereference(self): + if self.address == NULL: + return self.space.w_None return cast_address_to_object(self.address, W_Weakrefable) def invalidate(w_self): @@ -145,7 +147,9 @@ # Proxy class W_Proxy(W_Weakref): - pass + def descr__hash__(self, space): + raise OperationError(space.w_TypeError, + space.wrap("unhashable type")) class W_CallableProxy(W_Proxy): def descr__call__(self, space, __args__): @@ -173,6 +177,7 @@ if not isinstance(proxy, W_Proxy): return proxy w_obj = proxy.dereference() + assert w_obj is not None if space.is_w(w_obj, space.w_None): raise OperationError( space.w_ReferenceError, @@ -181,13 +186,13 @@ proxy_typedef_dict = {} callable_proxy_typedef_dict = {} -special_ops = {'repr': True} +special_ops = {'repr': True, 'userdel': True, 'hash': True} for opname, _, arity, special_methods in ObjSpace.MethodTable: if opname in special_ops: continue nonspaceargs = ", ".join(["w_obj%s" % i for i in range(arity)]) - code = "def func(space, %s):\n" % (nonspaceargs, ) + code = "def func(space, %s):\n '''%s'''\n" % (nonspaceargs, opname) for i in range(arity): code += " w_obj%s = force(space, w_obj%s)\n" % (i, i) code += " return space.%s(%s)" % (opname, nonspaceargs) @@ -201,11 +206,13 @@ W_Proxy.typedef = TypeDef("weakproxy", __new__ = interp2app(descr__new__proxy), + __hash__ = interp2app(W_Proxy.descr__hash__, unwrap_spec=['self', ObjSpace]), **proxy_typedef_dict) W_Proxy.typedef.accepable_as_base_class = False W_CallableProxy.typedef = TypeDef("weakcallableproxy", __new__ = interp2app(descr__new__callableproxy), + __hash__ = interp2app(W_Proxy.descr__hash__, unwrap_spec=['self', ObjSpace]), __call__ = interp2app(W_CallableProxy.descr__call__, unwrap_spec=['self', ObjSpace, Arguments]), **callable_proxy_typedef_dict) Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 22:43:11 2006 @@ -159,3 +159,10 @@ raises(TypeError, _weakref.ProxyType, []) raises(TypeError, _weakref.CallableProxyType, []) + def test_dont_hash(self): + import _weakref + class A(object): + pass + a = A() + p = _weakref.proxy(a) + raises(TypeError, hash, p) From cfbolz at codespeak.net Thu Apr 27 22:45:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 22:45:52 +0200 (CEST) Subject: [pypy-svn] r26465 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427204552.D4797100A4@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 22:45:51 2006 New Revision: 26465 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: _really_ prevent subclassing + tests Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 22:45:51 2006 @@ -208,7 +208,7 @@ __new__ = interp2app(descr__new__proxy), __hash__ = interp2app(W_Proxy.descr__hash__, unwrap_spec=['self', ObjSpace]), **proxy_typedef_dict) -W_Proxy.typedef.accepable_as_base_class = False +W_Proxy.typedef.acceptable_as_base_class = False W_CallableProxy.typedef = TypeDef("weakcallableproxy", __new__ = interp2app(descr__new__callableproxy), @@ -216,5 +216,5 @@ __call__ = interp2app(W_CallableProxy.descr__call__, unwrap_spec=['self', ObjSpace, Arguments]), **callable_proxy_typedef_dict) -W_CallableProxy.typedef.accepable_as_base_class = False +W_CallableProxy.typedef.acceptable_as_base_class = False Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 22:45:51 2006 @@ -166,3 +166,11 @@ a = A() p = _weakref.proxy(a) raises(TypeError, hash, p) + + def test_subclassing_not_allowed(self): + import _weakref + def tryit(): + class A(_weakref.ProxyType): + pass + return A + raises(TypeError, tryit) From cfbolz at codespeak.net Thu Apr 27 23:12:14 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 27 Apr 2006 23:12:14 +0200 (CEST) Subject: [pypy-svn] r26467 - in pypy/dist/pypy/module/_weakref: . test Message-ID: <20060427211214.78E7A100C0@code0.codespeak.net> Author: cfbolz Date: Thu Apr 27 23:12:11 2006 New Revision: 26467 Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py pypy/dist/pypy/module/_weakref/test/test_weakref.py Log: make weakrefs properly hashable Modified: pypy/dist/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/dist/pypy/module/_weakref/interp__weakref.py Thu Apr 27 23:12:11 2006 @@ -21,12 +21,12 @@ for i in range(len(self.addr_refs) - 1, -1, -1): addr_ref = self.addr_refs[i] if addr_ref != NULL: - w_ref = cast_address_to_object(addr_ref, W_Weakref) + w_ref = cast_address_to_object(addr_ref, W_WeakrefBase) w_ref.invalidate() for i in range(len(self.addr_refs) - 1, -1, -1): addr_ref = self.addr_refs[i] if addr_ref != NULL: - w_ref = cast_address_to_object(addr_ref, W_Weakref) + w_ref = cast_address_to_object(addr_ref, W_WeakrefBase) w_ref.activate_callback() def get_weakref(self, space, w_subtype, w_obj, w_callable): @@ -62,7 +62,7 @@ self.addr_refs[index] = NULL -class W_Weakref(Wrappable): +class W_WeakrefBase(Wrappable): def __init__(w_self, space, lifeline, index, w_obj, w_callable): w_self.space = space w_self.address = cast_object_to_address(w_obj) @@ -91,6 +91,20 @@ WeakrefLifeline) lifeline.ref_is_dead(w_self.index) +class W_Weakref(W_WeakrefBase): + def __init__(w_self, space, lifeline, index, w_obj, w_callable): + W_WeakrefBase.__init__(w_self, space, lifeline, index, w_obj, w_callable) + w_self.w_hash = None + + def descr_hash(self): + if self.w_hash is not None: + return self.w_hash + w_obj = self.dereference() + if self.space.is_w(w_obj, self.space.w_None): + raise OperationError(self.space.w_TypeError, + self.space.wrap("weak object has gone away")) + self.w_hash = self.space.hash(w_obj) + return self.w_hash def descr__new__weakref(space, w_subtype, w_obj, w_callable=None): assert isinstance(w_obj, W_Weakrefable) @@ -112,6 +126,7 @@ unwrap_spec=[ObjSpace, W_Weakref, W_Weakref]), __ne__ = interp2app(descr__ne__, unwrap_spec=[ObjSpace, W_Weakref, W_Weakref]), + __hash__ = interp2app(W_Weakref.descr_hash, unwrap_spec=['self']), __call__ = interp2app(W_Weakref.dereference, unwrap_spec=['self']) ) @@ -140,13 +155,13 @@ for i in range(len(lifeline.addr_refs)): addr = lifeline.addr_refs[i] if addr != NULL: - result.append(cast_address_to_object(addr, W_Weakref)) + result.append(cast_address_to_object(addr, W_WeakrefBase)) return space.newlist(result) #_________________________________________________________________ # Proxy -class W_Proxy(W_Weakref): +class W_Proxy(W_WeakrefBase): def descr__hash__(self, space): raise OperationError(space.w_TypeError, space.wrap("unhashable type")) Modified: pypy/dist/pypy/module/_weakref/test/test_weakref.py ============================================================================== --- pypy/dist/pypy/module/_weakref/test/test_weakref.py (original) +++ pypy/dist/pypy/module/_weakref/test/test_weakref.py Thu Apr 27 23:12:11 2006 @@ -118,6 +118,19 @@ ref1 = _weakref.ref(a) assert _weakref.getweakrefs(a) == [ref1] + def test_hashing(self): + import _weakref + class A(object): + def __hash__(self): + return 42 + a = A() + w = _weakref.ref(a) + assert hash(a) == hash(w) + del a + assert hash(w) == 42 + w = _weakref.ref(A()) + raises(TypeError, hash, w) + class AppTestProxy(object): def setup_class(cls): space = gettestobjspace(usemodules=('_weakref',)) From sanxiyn at codespeak.net Fri Apr 28 02:34:25 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 02:34:25 +0200 (CEST) Subject: [pypy-svn] r26468 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428003425.8385A10095@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 02:34:18 2006 New Revision: 26468 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py Log: Add a test demonstrating duplicate method generation Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 02:34:18 2006 @@ -26,6 +26,22 @@ cl_inc = make_cl_func(inc, [int]) assert cl_inc(5) == 6 +def test_twice(): + class IntHolder: + def __init__(self, number): + self.number = number + def inc(self): + self.number += 1 + def get(self): + return self.number + def inc_two(number): + obj = IntHolder(number) + obj.inc() + obj.inc() + return obj.get() + cl_inc_two = make_cl_func(inc_two, [int]) + assert cl_inc_two(5) == 7 + def test_inherit(): class Foo: pass From sanxiyn at codespeak.net Fri Apr 28 02:42:45 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 02:42:45 +0200 (CEST) Subject: [pypy-svn] r26469 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060428004245.712CA1009D@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 02:42:36 2006 New Revision: 26469 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/test/test_clrepr.py Log: More clrepr Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Fri Apr 28 02:42:36 2006 @@ -4,7 +4,7 @@ from pypy.tool.udir import udir from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL -from pypy.translator.cl.clrepr import clrepr +from pypy.translator.cl.clrepr import clrepr, repr_fun_name from pypy import conftest from pypy.translator.cl import conftest as clconftest @@ -88,7 +88,7 @@ def _(*args): fpath.write(out) fp = file(str(fpath), "a") - print >>fp, "(write (", clrepr(func.func_name), + print >>fp, "(write (", repr_fun_name(func.func_name), for arg in args: print >>fp, clrepr(arg), print >>fp, "))" Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Fri Apr 28 02:42:36 2006 @@ -9,8 +9,8 @@ def clrepr(item): if isinstance(item, str): if len(item) == 1: - return "#\\" + item - return repr_fun_name(item) + return "#\\%c" % (item,) + return '"%s"' % (item,) if isinstance(item, bool): if item: return "t" @@ -18,6 +18,8 @@ return "nil" if isinstance(item, (int, long, float)): return str(item) + if item is None: + return "nil" if isinstance(item, (list, tuple)): return "'(" + ' '.join(item) + ")" if isinstance(item, Variable): Modified: pypy/dist/pypy/translator/cl/test/test_clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_clrepr.py (original) +++ pypy/dist/pypy/translator/cl/test/test_clrepr.py Fri Apr 28 02:42:36 2006 @@ -2,13 +2,10 @@ from pypy.translator.cl.clrepr import clrepr def test_const(): - py.test.skip('changed') - assert repr_const(True) == 't' - assert repr_const(False) == 'nil' - assert repr_const(42) == '42' - assert repr_const(1.5) == '1.5' - assert repr_const(None) == 'nil' - assert repr_const('a') == '#\\a' - assert repr_const('answer') == '"answer"' - assert repr_const((2, 3)) == "'(2 3)" - assert repr_const([2, 3]) == "#(2 3)" + assert clrepr(True) == 't' + assert clrepr(False) == 'nil' + assert clrepr(42) == '42' + assert clrepr(1.5) == '1.5' + assert clrepr(None) == 'nil' + assert clrepr('a') == '#\\a' + assert clrepr('answer') == '"answer"' From sanxiyn at codespeak.net Fri Apr 28 02:48:43 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 02:48:43 +0200 (CEST) Subject: [pypy-svn] r26470 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428004843.0AAD0100AB@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 02:48:38 2006 New Revision: 26470 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py Log: Add a test failing with null instance Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 02:48:38 2006 @@ -112,3 +112,22 @@ cl_dynamic_instance = make_cl_func(dynamic_instance, [bool]) assert cl_dynamic_instance(True) == 1 assert cl_dynamic_instance(False) == 2 + +def test_nullable(): + py.test.skip("fails with null inst") + class Foo: + def __init__(self, value): + self.value = value + def maybe_foo(flag): + if flag: + return Foo(1) + else: + return None + def maybe_one(flag): + obj = maybe_foo(flag) + if obj is None: + obj = Foo(2) + return obj.value + cl_maybe_one = make_cl_func(maybe_one, [bool]) + assert cl_maybe_one(True) == 1 + assert cl_maybe_one(False) == 2 From nik at codespeak.net Fri Apr 28 03:30:08 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 03:30:08 +0200 (CEST) Subject: [pypy-svn] r26471 - pypy/dist/pypy/rpython/test Message-ID: <20060428013008.CC202100B5@code0.codespeak.net> Author: nik Date: Fri Apr 28 03:29:56 2006 New Revision: 26471 Modified: pypy/dist/pypy/rpython/test/test_rdict.py Log: add a failing test about recursive dicts. the plan is to remove ForwardReferences form the Dict type. this test fails for an unrelated reason, due to recursive string representation. Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Fri Apr 28 03:29:56 2006 @@ -892,3 +892,13 @@ dic.clear() return len(dic) assert self.interpret(func, ()) == 0 + + def test_recursive(self): + py.test.skip("work-in-progress") + def func(i): + dic = {i: {}} + dic[i] = dic + return dic[i] + res = self.interpret(func, [5]) + assert res.ll_get(5) is res + From dialtone at codespeak.net Fri Apr 28 03:36:25 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Fri, 28 Apr 2006 03:36:25 +0200 (CEST) Subject: [pypy-svn] r26472 - pypy/dist/pypy/translator/cl Message-ID: <20060428013625.1F003100B3@code0.codespeak.net> Author: dialtone Date: Fri Apr 28 03:36:17 2006 New Revision: 26472 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py Log: further cleanup of clrepr Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Fri Apr 28 03:36:17 2006 @@ -4,7 +4,7 @@ from pypy.tool.udir import udir from pypy.translator.translator import TranslationContext from pypy.translator.cl.gencl import GenCL -from pypy.translator.cl.clrepr import clrepr, repr_fun_name +from pypy.translator.cl.clrepr import clrepr from pypy import conftest from pypy.translator.cl import conftest as clconftest @@ -88,7 +88,7 @@ def _(*args): fpath.write(out) fp = file(str(fpath), "a") - print >>fp, "(write (", repr_fun_name(func.func_name), + print >>fp, "(write (", clrepr(func.func_name, symbol=True), for arg in args: print >>fp, clrepr(arg), print >>fp, "))" Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Fri Apr 28 03:36:17 2006 @@ -4,91 +4,129 @@ from pypy.rpython.ootypesystem.ootype import Signed, Unsigned, Float, Char from pypy.rpython.ootypesystem.ootype import Bool, Void, UniChar, Class from pypy.rpython.ootypesystem.ootype import StaticMethod, Meth, typeOf -from pypy.rpython.ootypesystem.rclass import CLASSTYPE -def clrepr(item): - if isinstance(item, str): - if len(item) == 1: - return "#\\%c" % (item,) - return '"%s"' % (item,) - if isinstance(item, bool): - if item: - return "t" - else: - return "nil" - if isinstance(item, (int, long, float)): - return str(item) +def clrepr(item, symbol=False): + """ This is the main repr function and is the only one that should be + used to represent python values in lisp. + """ if item is None: return "nil" - if isinstance(item, (list, tuple)): - return "'(" + ' '.join(item) + ")" - if isinstance(item, Variable): - return repr_var(item) - if isinstance(item, Constant): - return repr_const(item) - if isinstance(item, Instance): - return "'" + repr_class_name(item._name) + + fun = bltn_dispatch.get(type(item), None) + if fun is not None: + return fun(item, symbol) + if typeOf(item) is Class: return "'" + item._INSTANCE._name return repr_unknown(item) -def repr_unknown(obj): - name = obj.__class__.__name__ - raise NotImplementedError("cannot represent %s" % (name,)) - -def repr_var(var): - return var.name - -def repr_atom(atom): - return "'" + str(atom) - -def repr_class_name(name): - return name.replace('_', '-') - -def repr_fun_name(name): - return name.replace('_', '-') - def repr_const(item): if isinstance(item.value, HalfConcreteWrapper): item = item.concretize() - if isinstance(item.concretetype, Atom): - return repr_atom(val) + fun = dispatch.get(type(item.concretetype), None) + if fun is not None: + return fun(item) + + fun = dispatch.get(item.concretetype, None) + if fun is not None: + return fun(item) - if isinstance(item.concretetype, List): - val = map(repr_const, item.value) - return "#(%s)" % ' '.join(val) + if item.value is None: + return "nil" - if isinstance(item.concretetype, Record): - val = map(repr_const, item.value) - return "'(%s)" % ' '.join(val) + return repr_unknown(item) - if isinstance(item.concretetype, Instance): - return "'" + repr_class_name(item.value._name) +def repr_bltn_str(item, symbol): + if symbol: + return item.replace('_', '-') + if len(item) == 1: + return "#\\%c" % (item,) + return '"%s"' % (item,) + +def repr_bltn_bool(item, _): + if item: + return "t" + else: + return "nil" - if item.concretetype is Class: - return "'" + repr_class_name(item.value._INSTANCE._name) +def repr_bltn_number(item, _): + return str(item) - if item.concretetype is Void: - return "nil" +def repr_bltn_seq(item, _): + return "'(" + ' '.join(item) + ")" - if isinstance(item.concretetype, StaticMethod): - return repr_fun_name(item.value._name) +def repr_Variable(item, _): + return clrepr(item.name, symbol=True) - if item.concretetype is Bool: # should precede int - if item.value: - return "t" - else: - return "nil" - - if item.concretetype is Signed or item.concretetype is Unsigned: - #, long)): Not yet real longs - return str(item.value) +def repr_Constant(item, _): + return repr_const(item) + +def repr_Instance(item, _): + return "'" + clrepr(item._name, symbol=True) + +bltn_dispatch = { + str: repr_bltn_str, + bool: repr_bltn_bool, + int: repr_bltn_number, + long: repr_bltn_number, + float: repr_bltn_number, + list: repr_bltn_seq, + tuple: repr_bltn_seq, + Variable: repr_Variable, + Constant: repr_Constant, + Instance: repr_Instance +} - if item.concretetype is Float: - return str(item.value) +def repr_atom(atom): + return "'" + clrepr(str(atom), symbol=True) - if item.value is None: +def repr_class(item): + return clrepr(item.value._INSTANCE._name, symbol=True) + +def repr_void(item): + return "nil" + +def repr_bool(item): + if item.value: + return "t" + else: return "nil" - return repr_unknown(item) +def repr_int(item): + return str(item.value) + +def repr_float(item): + return str(item.value) + +def repr_list(item): + val = map(clrepr, item.value) + return "'(%s)" % ' '.join(val) + +def repr_record(item): + val = map(clrepr, item.value) + return "#(%s)" % ' '.join(val) + +def repr_instance(item): + return "'" + repr_class(item) + +def repr_static_method(item): + return clrepr(item.value._name, symbol=True) + +dispatch = { + Class: repr_class, + Void: repr_void, + Bool: repr_bool, + Signed: repr_int, + Unsigned: repr_int, + Float: repr_float, + Atom: repr_atom, + List: repr_list, + Record: repr_record, + Instance: repr_instance, + StaticMethod: repr_static_method +} + +def repr_unknown(obj): + name = obj.__class__.__name__ + raise NotImplementedError("cannot represent %s" % (name,)) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 03:36:17 2006 @@ -5,7 +5,7 @@ from pypy.translator.translator import graphof from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _class, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT -from pypy.translator.cl.clrepr import clrepr, repr_fun_name, repr_class_name +from pypy.translator.cl.clrepr import clrepr class InsertionOrderedDict(dict): def __init__(self): @@ -118,7 +118,7 @@ methodobj = cls._methods[method] methodobj._method_name = method # XXX self.gen.pendinggraphs.append(methodobj) - name = repr_fun_name(method) + name = clrepr(method, symbol=True) selfvar = clrepr(receiver) args = map(self.gen.check_declaration, args) args = " ".join(args) @@ -192,14 +192,14 @@ def declare_class(self, cls): # cls is Instance - name = repr_class_name(cls._name) + name = clrepr(cls._name, symbol=True) field_declaration = ['('+field+')' for field in cls._fields] field_declaration = " ".join(field_declaration) if cls._superclass is ROOT: class_declaration = "(defclass %s () (%s))" % (name, field_declaration) else: self.declare_class(cls._superclass) - supername = repr_class_name(cls._superclass._name) + supername = clrepr(cls._superclass._name, symbol=True) class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) self.declarations[name] = class_declaration @@ -258,7 +258,7 @@ yield line def emit_defun(self, fun): - yield "(defun " + repr_fun_name(fun.name) + yield "(defun " + clrepr(fun.name, symbol=True) arglist = fun.getargs() args = " ".join(map(clrepr, arglist)) yield "(%s)" % (args,) @@ -266,10 +266,10 @@ yield line def emit_defmethod(self, fun, name): - yield "(defmethod %s" % (repr_fun_name(name)) + yield "(defmethod %s" % (clrepr(name, symbol=True)) arglist = fun.getargs() selfvar = clrepr(arglist[0]) - clsname = repr_class_name(arglist[0].concretetype._name) + clsname = clrepr(arglist[0].concretetype._name, symbol=True) args = " ".join(map(clrepr, arglist[1:])) if args: yield "((%s %s) %s)" % (selfvar, clsname, args) From nik at codespeak.net Fri Apr 28 03:46:28 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 03:46:28 +0200 (CEST) Subject: [pypy-svn] r26473 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060428014628.3B6D1100AD@code0.codespeak.net> Author: nik Date: Fri Apr 28 03:46:23 2006 New Revision: 26473 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Log: make it possible to initialize the value type of a Dict lazily. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 28 03:46:23 2006 @@ -352,15 +352,19 @@ KEYTYPE_T = object() VALUETYPE_T = object() - def __init__(self, KEYTYPE, VALUETYPE): + def __init__(self, KEYTYPE, VALUETYPE=None): self._KEYTYPE = KEYTYPE self._VALUETYPE = VALUETYPE self._null = _null_dict(self) + if VALUETYPE is not None: + self._init_methods() + + def _init_methods(self): generic_types = { self.SELFTYPE_T: self, - self.KEYTYPE_T: KEYTYPE, - self.VALUETYPE_T: VALUETYPE + self.KEYTYPE_T: self._KEYTYPE, + self.VALUETYPE_T: self._VALUETYPE } self._GENERIC_METHODS = frozendict({ @@ -386,6 +390,18 @@ return '%s%s' % (self.__class__.__name__, saferecursive(str, "(...)")((self._KEYTYPE, self._VALUETYPE))) + def __eq__(self, other): + if not isinstance(other, Dict): + return False + if self._VALUETYPE is None or other._VALUETYPE is None: + raise TypeError("Can't compare uninitialized Dict type.") + return BuiltinADTType.__eq__(self, other) + + def __hash__(self): + if self._VALUETYPE is None: + raise TypeError("Can't hash uninitialized Dict type.") + return BuiltinADTType.__hash__(self) + def _get_interp_class(self): return _dict @@ -394,6 +410,10 @@ VALUETYPE = self._specialize_type(self._VALUETYPE, generic_types) return self.__class__(KEYTYPE, VALUETYPE) + def _set_valuetype(self, VALUETYPE): + self._VALUETYPE = VALUETYPE + self._init_methods() + class DictItemsIterator(BuiltinADTType): SELFTYPE_T = object() KEYTYPE_T = object() @@ -966,5 +986,11 @@ def hasItemType(LIST): return LIST._ITEMTYPE is not None +def setValueType(DICT, VALUETYPE): + return DICT._set_valuetype(VALUETYPE) + +def hasValueType(DICT): + return DICT._VALUETYPE is not None + ROOT = Instance('Root', None, _is_root=True) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Fri Apr 28 03:46:23 2006 @@ -1,6 +1,6 @@ import py from pypy.rpython.test.test_llinterp import interpret -from pypy.rpython.ootypesystem.ootype import Signed, Float, Dict, new, typeOf +from pypy.rpython.ootypesystem.ootype import Signed, Float, Dict, new, typeOf, setValueType def test_new(): DT = Dict(Signed, Float) @@ -30,3 +30,16 @@ items.append((it.ll_current_key(), it.ll_current_value())) items.sort() assert items == [(42, 43.0), (52, 53.0)] + +def test_optional_valuetype(): + DT = Dict(Signed) + DT2 = Dict(Signed, Float) + assert DT != Signed + py.test.raises(TypeError, "DT == DT2") + py.test.raises(TypeError, "DT2 == DT") + py.test.raises(TypeError, hash, DT) + setValueType(DT, Float) + assert DT == DT2 + assert DT2 == DT + assert hash(DT) == hash(DT2) + From nik at codespeak.net Fri Apr 28 03:50:51 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 03:50:51 +0200 (CEST) Subject: [pypy-svn] r26474 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060428015051.C6600100AD@code0.codespeak.net> Author: nik Date: Fri Apr 28 03:50:47 2006 New Revision: 26474 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Log: make str and hash of recursive Dict types work. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 28 03:50:47 2006 @@ -387,8 +387,8 @@ # equal. def __str__(self): - return '%s%s' % (self.__class__.__name__, - saferecursive(str, "(...)")((self._KEYTYPE, self._VALUETYPE))) + return '%s(%s, %s)' % (self.__class__.__name__, + self._KEYTYPE, saferecursive(str, "...")(self._VALUETYPE)) def __eq__(self, other): if not isinstance(other, Dict): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Fri Apr 28 03:50:47 2006 @@ -43,3 +43,9 @@ assert DT2 == DT assert hash(DT) == hash(DT2) +def test_recursive_str_hash(): + DT = Dict(Signed) + setValueType(DT, DT) + assert isinstance(str(DT), str) + assert isinstance(hash(DT), int) + From dialtone at codespeak.net Fri Apr 28 03:56:06 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Fri, 28 Apr 2006 03:56:06 +0200 (CEST) Subject: [pypy-svn] r26475 - pypy/dist/pypy/translator/cl Message-ID: <20060428015606.BD04E100B2@code0.codespeak.net> Author: dialtone Date: Fri Apr 28 03:56:03 2006 New Revision: 26475 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: everything has to be repred in the right way Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 03:56:03 2006 @@ -37,7 +37,7 @@ yield line def nop(self, result, arg): - yield "(setf %s %s)" % (result, arg) + yield "(setf %s %s)" % (clrepr(result, True), clrepr(arg, True)) op_same_as = nop op_ooupcast = nop @@ -45,7 +45,9 @@ def make_unary_op(cl_op): def _(self, result, arg): - yield "(setf %s (%s %s))" % (result, cl_op, arg) + yield "(setf %s (%s %s))" % (clrepr(result, True), + cl_op, + clrepr(arg, True)) return _ op_bool_not = make_unary_op("not") @@ -56,7 +58,10 @@ def make_binary_op(cl_op): def _(self, result, arg1, arg2): - yield "(setf %s (%s %s %s))" % (result, cl_op, arg1, arg2) + yield "(setf %s (%s %s %s))" % (clrepr(result, True), + cl_op, + clrepr(arg1, True), + clrepr(arg2, True)) return _ op_int_add = make_binary_op("+") @@ -76,34 +81,42 @@ op_char_ne = make_binary_op("char/=") def op_int_is_true(self, result, arg): - yield "(setf %s (not (zerop %s)))" % (result, arg) + yield "(setf %s (not (zerop %s)))" % (clrepr(result, True), + clrepr(arg, True)) def op_direct_call(self, result, fun, *args): funobj = self.args[0].value self.gen.pendinggraphs.append(funobj) args = " ".join(args) - yield "(setf %s (%s %s))" % (result, fun, args) + yield "(setf %s (%s %s))" % (clrepr(result, True), + clrepr(fun, True), + clrepr(args, True)) def op_new(self, result, clsname): cls = self.args[0].value if isinstance(cls, List): - yield "(setf %s (make-array 0 :adjustable t))" % (result,) + yield "(setf %s (make-array 0 :adjustable t))" % (clrepr(result, True),) elif isinstance(cls, Record): clsname = self.gen.declare_struct(cls) - yield "(setf %s (make-%s))" % (result, clsname) + yield "(setf %s (make-%s))" % (clrepr(result, True), + clrepr(clsname, True)) elif isinstance(cls, Instance): self.gen.declare_class(cls) clsname = clrepr(cls) - yield "(setf %s (make-instance %s))" % (result, clsname) + yield "(setf %s (make-instance %s))" % (clrepr(result, True), + clrepr(clsname, True)) else: raise NotImplementedError() def op_runtimenew(self, result, arg): - yield "(setf %s (make-instance %s))" % (result, arg) + yield "(setf %s (make-instance %s))" % (clrepr(result, True), + clrepr(arg, True)) def op_instanceof(self, result, arg, clsname): clsname = clrepr(self.args[1].value) - yield "(setf %s (typep %s %s))" % (result, arg, clsname) + yield "(setf %s (typep %s %s))" % (clrepr(result, True), + clrepr(arg, True), + clrepr(clsname, True)) def op_oosend(self, result, *ignore): method = self.args[0].value @@ -113,7 +126,7 @@ if isinstance(cls, List): impl = ListImpl(receiver) code = getattr(impl, method)(*args) - yield "(setf %s %s)" % (result, code) + yield "(setf %s %s)" % (clrepr(result, True), clrepr(code, True)) elif isinstance(cls, Instance): methodobj = cls._methods[method] methodobj._method_name = method # XXX @@ -123,23 +136,34 @@ args = map(self.gen.check_declaration, args) args = " ".join(args) if args: - yield "(setf %s (%s %s %s))" % (result, name, selfvar, args) + yield "(setf %s (%s %s %s))" % (clrepr(result, True), + clrepr(name, True), + clrepr(selfvar, True), + clrepr(args, True)) else: - yield "(setf %s (%s %s))" % (result, name, selfvar) + yield "(setf %s (%s %s))" % (clrepr(result, True), + clrepr(name, True), + clrepr(selfvar, True)) def op_oogetfield(self, result, obj, _): fieldname = self.args[1].value - yield "(setf %s (slot-value %s '%s))" % (result, obj, fieldname) + yield "(setf %s (slot-value %s '%s))" % (clrepr(result, True), + clrepr(obj, True), + clrepr(fieldname, True)) def op_oosetfield(self, result, obj, _, value): fieldname = self.args[1].value - yield "(setf (slot-value %s '%s) %s)" % (obj, fieldname, value) + yield "(setf (slot-value %s '%s) %s)" % (clrepr(obj, True), + clrepr(fieldname, True), + clrepr(value, True)) def op_ooidentityhash(self, result, arg): - yield "(setf %s (sxhash %s))" % (result, arg) + yield "(setf %s (sxhash %s))" % (clrepr(result, True), + clrepr(arg, True)) def op_oononnull(self, result, arg): - yield "(setf %s (not (null %s)))" % (result, arg) + yield "(setf %s (not (null %s)))" % (clrepr(result, True), + clrepr(arg, True)) class ListImpl: @@ -193,7 +217,7 @@ def declare_class(self, cls): # cls is Instance name = clrepr(cls._name, symbol=True) - field_declaration = ['('+field+')' for field in cls._fields] + field_declaration = ['('+clrepr(field, True)+')' for field in cls._fields] field_declaration = " ".join(field_declaration) if cls._superclass is ROOT: class_declaration = "(defclass %s () (%s))" % (name, field_declaration) @@ -211,14 +235,18 @@ inst = oodowncast(INST, const.value) cls = clrepr(INST) const_declaration = [] - const_declaration.append("(setf %s (make-instance %s))" % (name, cls)) + const_declaration.append("(setf %s (make-instance %s))" % (clrepr(name, True), + clrepr(cls, True))) fields = INST._allfields() for fieldname in fields: fieldvalue = getattr(inst, fieldname) if isinstance(fieldvalue, _class): self.declare_class(fieldvalue._INSTANCE) fieldvaluerepr = clrepr(getattr(inst, fieldname)) - const_declaration.append("(setf (slot-value %s '%s) %s)" % (name, fieldname, fieldvaluerepr)) + ### XXX + const_declaration.append("(setf (slot-value %s '%s) %s)" % (clrepr(name, True), + clrepr(fieldname, True), + clrepr(fieldvaluerepr, True))) const_declaration = "\n".join(const_declaration) self.declarations[const] = const_declaration self.constcount += 1 @@ -260,7 +288,7 @@ def emit_defun(self, fun): yield "(defun " + clrepr(fun.name, symbol=True) arglist = fun.getargs() - args = " ".join(map(clrepr, arglist)) + args = " ".join(map(lambda item: clrepr(item, True), arglist)) yield "(%s)" % (args,) for line in self.emit_body(fun, arglist): yield line @@ -268,13 +296,16 @@ def emit_defmethod(self, fun, name): yield "(defmethod %s" % (clrepr(name, symbol=True)) arglist = fun.getargs() - selfvar = clrepr(arglist[0]) + selfvar = clrepr(arglist[0], True) clsname = clrepr(arglist[0].concretetype._name, symbol=True) - args = " ".join(map(clrepr, arglist[1:])) + args = " ".join(map(lambda item: clrepr(item, True), arglist[1:])) if args: - yield "((%s %s) %s)" % (selfvar, clsname, args) + yield "((%s %s) %s)" % (clrepr(selfvar, True), + clrepr(clsname, True), + clrepr(args, True)) else: - yield "((%s %s))" % (selfvar, clsname) + yield "((%s %s))" % (clrepr(selfvar, True), + clrepr(clsname, True)) for line in self.emit_body(fun, arglist): yield line @@ -293,9 +324,10 @@ for var in vardict: varname = clrepr(var) if var in arglist: - varnames.append("(%s %s)" % (varname, varname)) + varnames.append("(%s %s)" % (clrepr(varname, True), + clrepr(varname, True))) else: - varnames.append(varname) + varnames.append(clrepr(varname, True)) varnames = " ".join(varnames) yield "(%s)" % (varnames,) for block in blocklist: @@ -306,7 +338,7 @@ def emit_block(self, block): self.cur_block = block tag = self.blockref[block] - yield "tag" + str(tag) + yield "tag" + clrepr(str(tag), True) for op in block.operations: emit_op = Op(self, op) for line in emit_op: @@ -320,7 +352,7 @@ if (len(exits) == 2 and exits[0].exitcase == False and exits[1].exitcase == True): - yield "(if " + clrepr(block.exitswitch) + yield "(if " + clrepr(block.exitswitch, True) yield "(progn" for line in self.emit_link(exits[1]): yield line @@ -334,32 +366,33 @@ # shouldn't be needed but in Python 2.2 we can't tell apart # 0 vs nil and 1 vs t :-( for exit in exits[:-1]: - yield "(if (equalp " + clrepr(block.exitswitch) - yield clrepr(exit.exitcase) + ')' + yield "(if (equalp " + clrepr(block.exitswitch, True) + yield clrepr(exit.exitcase, True) + ')' yield "(progn" for line in self.emit_link(exit): yield line yield ")" - yield "(progn ; else should be %s" % clrepr(exits[-1].exitcase) + yield "(progn ; else should be %s" % clrepr(exits[-1].exitcase, True) for line in self.emit_link(exits[-1]): yield line yield ")" * len(exits) elif len(block.inputargs) == 2: # exc_cls, exc_value - exc_cls = clrepr(block.inputargs[0]) - exc_value = clrepr(block.inputargs[1]) - yield "(something-like-throw-exception %s %s)" % (exc_cls, exc_value) + exc_cls = clrepr(block.inputargs[0], True) + exc_value = clrepr(block.inputargs[1], True) + yield "(something-like-throw-exception %s %s)" % (clrepr(exc_cls, True), + clrepr(exc_value, True)) else: retval = clrepr(block.inputargs[0]) - yield "(return %s)" % retval + yield "(return %s)" % clrepr(retval, True) def format_jump(self, block): tag = self.blockref[block] - return "(go tag" + str(tag) + ")" + return "(go tag" + clrepr(str(tag), True) + ")" def emit_link(self, link): source = map(self.check_declaration, link.args) target = map(clrepr, link.target.inputargs) - couples = [ "%s %s" % (t, s) for (s, t) in zip(source, target)] + couples = [ "%s %s" % (clrepr(t, True), clrepr(s, True)) for (s, t) in zip(source, target)] couples = " ".join(couples) yield "(setf %s)" % (couples,) yield self.format_jump(link.target) From sanxiyn at codespeak.net Fri Apr 28 04:02:33 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 04:02:33 +0200 (CEST) Subject: [pypy-svn] r26476 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428020233.5581C1009F@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 04:02:30 2006 New Revision: 26476 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py Log: (dialtone, sanxiyn) Make a test showing duplicate method generation to fail by counting defmethod in the code Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 04:02:30 2006 @@ -26,7 +26,7 @@ cl_inc = make_cl_func(inc, [int]) assert cl_inc(5) == 6 -def test_twice(): +def test_method_twice(): class IntHolder: def __init__(self, number): self.number = number @@ -39,8 +39,10 @@ obj.inc() obj.inc() return obj.get() - cl_inc_two = make_cl_func(inc_two, [int]) - assert cl_inc_two(5) == 7 + code = generate_cl_func(inc_two, [int]) + print code + methodcount = code.count("defmethod") + assert methodcount == 3 def test_inherit(): class Foo: From nik at codespeak.net Fri Apr 28 04:04:11 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 04:04:11 +0200 (CEST) Subject: [pypy-svn] r26477 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060428020411.BEEE01009F@code0.codespeak.net> Author: nik Date: Fri Apr 28 04:04:08 2006 New Revision: 26477 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Log: i discovered we don't only want lazily initialized value types for Dicts, but also key types. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 28 04:04:08 2006 @@ -352,14 +352,17 @@ KEYTYPE_T = object() VALUETYPE_T = object() - def __init__(self, KEYTYPE, VALUETYPE=None): + def __init__(self, KEYTYPE=None, VALUETYPE=None): self._KEYTYPE = KEYTYPE self._VALUETYPE = VALUETYPE self._null = _null_dict(self) - if VALUETYPE is not None: + if self._is_initialized(): self._init_methods() + def _is_initialized(self): + return self._KEYTYPE is not None and self._VALUETYPE is not None + def _init_methods(self): generic_types = { self.SELFTYPE_T: self, @@ -393,12 +396,12 @@ def __eq__(self, other): if not isinstance(other, Dict): return False - if self._VALUETYPE is None or other._VALUETYPE is None: + if not self._is_initialized() or not other._is_initialized(): raise TypeError("Can't compare uninitialized Dict type.") return BuiltinADTType.__eq__(self, other) def __hash__(self): - if self._VALUETYPE is None: + if not self._is_initialized(): raise TypeError("Can't hash uninitialized Dict type.") return BuiltinADTType.__hash__(self) @@ -410,7 +413,8 @@ VALUETYPE = self._specialize_type(self._VALUETYPE, generic_types) return self.__class__(KEYTYPE, VALUETYPE) - def _set_valuetype(self, VALUETYPE): + def _set_types(self, KEYTYPE, VALUETYPE): + self._KEYTYPE = KEYTYPE self._VALUETYPE = VALUETYPE self._init_methods() @@ -986,11 +990,11 @@ def hasItemType(LIST): return LIST._ITEMTYPE is not None -def setValueType(DICT, VALUETYPE): - return DICT._set_valuetype(VALUETYPE) +def setDictTypes(DICT, KEYTYPE, VALUETYPE): + return DICT._set_types(KEYTYPE, VALUETYPE) -def hasValueType(DICT): - return DICT._VALUETYPE is not None +def hasDictTypes(DICT): + return DICT._is_initialized() ROOT = Instance('Root', None, _is_root=True) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oodict.py Fri Apr 28 04:04:08 2006 @@ -1,6 +1,6 @@ import py from pypy.rpython.test.test_llinterp import interpret -from pypy.rpython.ootypesystem.ootype import Signed, Float, Dict, new, typeOf, setValueType +from pypy.rpython.ootypesystem.ootype import Signed, Float, Dict, new, typeOf, setDictTypes def test_new(): DT = Dict(Signed, Float) @@ -31,21 +31,21 @@ items.sort() assert items == [(42, 43.0), (52, 53.0)] -def test_optional_valuetype(): - DT = Dict(Signed) +def test_optional_types(): + DT = Dict() DT2 = Dict(Signed, Float) assert DT != Signed py.test.raises(TypeError, "DT == DT2") py.test.raises(TypeError, "DT2 == DT") py.test.raises(TypeError, hash, DT) - setValueType(DT, Float) + setDictTypes(DT, Signed, Float) assert DT == DT2 assert DT2 == DT assert hash(DT) == hash(DT2) def test_recursive_str_hash(): - DT = Dict(Signed) - setValueType(DT, DT) + DT = Dict() + setDictTypes(DT, Signed, DT) assert isinstance(str(DT), str) assert isinstance(hash(DT), int) From ericvrp at codespeak.net Fri Apr 28 04:15:33 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 28 Apr 2006 04:15:33 +0200 (CEST) Subject: [pypy-svn] r26478 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060428021533.0F61D100BD@code0.codespeak.net> Author: ericvrp Date: Fri Apr 28 04:15:29 2006 New Revision: 26478 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: todays status/planning meeting Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Fri Apr 28 04:15:29 2006 @@ -174,3 +174,44 @@ - progress further on rpystone gencl translation (Valentino) - general wizardry (Samuele) + +Friday, 28th April 2006 +************************ + +Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Nik, Anders + +Status after day 4: + + - gencl (Sanghyeon, Valentino) + * constant instances now supported which enables the metaclass + model and class variables + * refactoring is ongoing + + - Pickle support for cell objects (Eric) + * cell, code, function partly there but needs moduledict support + + - OOTypeSystem (Nik, Samuele) + * introducted recordtype for interators and tuples + * random smaller improvements + + - ctypes _socket (and select ;)) (Arre, Anders) + * passing all tests + * uses only ctypes now but doesn't translate yet + +Pairings today: + + - Pickling cells (Eric) + * add support for additional objects like code objects. + + - ctypes _socket (Arre, Anders) + * continue translation work. + + - OOTypeSystem (Nik, Sanghyeon) + * Dictionaries need to be lazy too. + * String type + + - gencl (Valentino, Sanghyeon) + * exception handling + * rpystone translation + + - general wizardry (Samuele) From nik at codespeak.net Fri Apr 28 04:18:35 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 04:18:35 +0200 (CEST) Subject: [pypy-svn] r26479 - in pypy/dist/pypy/rpython: ootypesystem test Message-ID: <20060428021835.39F1E100A4@code0.codespeak.net> Author: nik Date: Fri Apr 28 04:18:30 2006 New Revision: 26479 Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py pypy/dist/pypy/rpython/test/test_rdict.py Log: get rid of the ForwardReference scheme in ootypesystem DictRepr, use lazy Dict type initialization instead. Modified: pypy/dist/pypy/rpython/ootypesystem/rdict.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rdict.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rdict.py Fri Apr 28 04:18:30 2006 @@ -35,7 +35,7 @@ if already_computed: self.DICT = ootype.Dict(key_repr.lowleveltype, value_repr.lowleveltype) else: - self.DICT = ootype.ForwardReference() + self.DICT = ootype.Dict() self.lowleveltype = self.DICT self.dictkey = dictkey @@ -51,9 +51,9 @@ if 'value_repr' not in self.__dict__: self.external_value_repr, self.value_repr = self.pickrepr(self._value_repr_computer()) - if isinstance(self.DICT, ootype.ForwardReference): - self.lowleveltype.become(ootype.Dict(self.key_repr.lowleveltype, - self.value_repr.lowleveltype)) + if not ootype.hasDictTypes(self.DICT): + ootype.setDictTypes(self.DICT, self.key_repr.lowleveltype, + self.value_repr.lowleveltype) def send_message(self, hop, message, can_raise=False, v_args=None): if v_args is None: Modified: pypy/dist/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rdict.py (original) +++ pypy/dist/pypy/rpython/test/test_rdict.py Fri Apr 28 04:18:30 2006 @@ -894,7 +894,6 @@ assert self.interpret(func, ()) == 0 def test_recursive(self): - py.test.skip("work-in-progress") def func(i): dic = {i: {}} dic[i] = dic From nik at codespeak.net Fri Apr 28 04:22:16 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 04:22:16 +0200 (CEST) Subject: [pypy-svn] r26480 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060428022216.2AF981009D@code0.codespeak.net> Author: nik Date: Fri Apr 28 04:22:12 2006 New Revision: 26480 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Log: get rid of the last traces of ForwardReference in ootypesystem. we should now have full support for recursive types in ootypesystem. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 28 04:22:12 2006 @@ -465,18 +465,6 @@ VALUETYPE = self._specialize_type(self._VALUETYPE, generic_types) return self.__class__(KEYTYPE, VALUETYPE) - -class ForwardReference(OOType): - def become(self, real_instance): - if not isinstance(real_instance, (Instance, BuiltinADTType)): - raise TypeError("ForwardReference can only be to an instance, " - "not %r" % (real_instance,)) - self.__class__ = real_instance.__class__ - self.__dict__ = real_instance.__dict__ - - def __hash__(self): - raise TypeError("%r object is not hashable" % self.__class__.__name__) - # ____________________________________________________________ class _class(object): Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oolist.py Fri Apr 28 04:22:12 2006 @@ -73,16 +73,14 @@ assert hash(LT) == hash(LT2) def test_recursive(): - FORWARD = ForwardReference() - LT = List(FORWARD) - FORWARD.become(LT) + LT = List() + setItemType(LT, LT) assert LT == LT assert hash(LT) == hash(LT) str(LT) # make sure this doesn't recurse infinitely - FORWARD2 = ForwardReference() - LT2 = List(FORWARD2) - FORWARD2.become(LT2) + LT2 = List() + setItemType(LT2, LT2) assert LT == LT2 assert hash(LT) == hash(LT2) From sanxiyn at codespeak.net Fri Apr 28 04:40:29 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 04:40:29 +0200 (CEST) Subject: [pypy-svn] r26481 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428024029.5BB1C100B3@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 04:40:25 2006 New Revision: 26481 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py Log: (dialtone, sanxiyn) Add a test for r26475 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 04:40:25 2006 @@ -94,6 +94,26 @@ assert cl_dynamic_class(True) == 1 assert cl_dynamic_class(False) == 2 +def test_runtimenew(): + class Foo: + pass + class Bar(Foo): + pass + class Baz(Foo): + pass + def pick_class(flag): + if flag: + return Bar + else: + return Baz + def runtime_new(flag): + cls = pick_class(flag) + cls() + return 0 + cl_runtime_new = make_cl_func(runtime_new, [bool]) + assert cl_runtime_new(True) == 0 + assert cl_runtime_new(False) == 0 + def test_instance(): py.test.skip("TODO") class Foo: From sanxiyn at codespeak.net Fri Apr 28 04:44:31 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 04:44:31 +0200 (CEST) Subject: [pypy-svn] r26482 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060428024431.7BA05100B3@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 04:44:28 2006 New Revision: 26482 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Fix typos Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Fri Apr 28 04:44:28 2006 @@ -191,7 +191,7 @@ * cell, code, function partly there but needs moduledict support - OOTypeSystem (Nik, Samuele) - * introducted recordtype for interators and tuples + * introduced record type for iterators and tuples * random smaller improvements - ctypes _socket (and select ;)) (Arre, Anders) From pedronis at codespeak.net Fri Apr 28 05:01:07 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 28 Apr 2006 05:01:07 +0200 (CEST) Subject: [pypy-svn] r26483 - pypy/dist/pypy/rpython/test Message-ID: <20060428030107.C3124100A4@code0.codespeak.net> Author: pedronis Date: Fri Apr 28 05:01:04 2006 New Revision: 26483 Modified: pypy/dist/pypy/rpython/test/test_rclass.py Log: add variation on this test. (remove a left-over print) Modified: pypy/dist/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rclass.py (original) +++ pypy/dist/pypy/rpython/test/test_rclass.py Fri Apr 28 05:01:04 2006 @@ -58,7 +58,6 @@ assert res == 12 def test_classattr_both(self): - print self.ts class A: a = 1 class B(A): @@ -78,6 +77,28 @@ res = interpret(dummyfn, [1], type_system=self.ts) assert res == 4 + def test_classattr_both2(self): + class Base(object): + a = 0 + class A(Base): + a = 1 + class B(Base): + a = 2 + def pick(i): + if i == 0: + return A + else: + return B + + def dummyfn(i): + C = pick(i) + i = C() + return C.a + i.a + res = interpret(dummyfn, [0], type_system=self.ts) + assert res == 2 + res = interpret(dummyfn, [1], type_system=self.ts) + assert res == 4 + def test_classattr_as_defaults(self): def dummyfn(): x = Random() From sanxiyn at codespeak.net Fri Apr 28 05:13:03 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 05:13:03 +0200 (CEST) Subject: [pypy-svn] r26484 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060428031303.7DCE610092@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 05:12:59 2006 New Revision: 26484 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_oo.py Log: (dialtone, sanxiyn) Generate all methods of classes up front, not only those which are "apparently" called. Pass a test. Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 05:12:59 2006 @@ -128,9 +128,6 @@ code = getattr(impl, method)(*args) yield "(setf %s %s)" % (clrepr(result, True), clrepr(code, True)) elif isinstance(cls, Instance): - methodobj = cls._methods[method] - methodobj._method_name = method # XXX - self.gen.pendinggraphs.append(methodobj) name = clrepr(method, symbol=True) selfvar = clrepr(receiver) args = map(self.gen.check_declaration, args) @@ -226,6 +223,10 @@ supername = clrepr(cls._superclass._name, symbol=True) class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) self.declarations[name] = class_declaration + for method in cls._methods: + methodobj = cls._methods[method] + methodobj._method_name = method + self.pendinggraphs.append(methodobj) def declare_constant_instance(self, const): # const.concretetype is Instance Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 05:12:59 2006 @@ -115,7 +115,6 @@ assert cl_runtime_new(False) == 0 def test_instance(): - py.test.skip("TODO") class Foo: value = 0 class Bar(Foo): From sanxiyn at codespeak.net Fri Apr 28 05:20:20 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 05:20:20 +0200 (CEST) Subject: [pypy-svn] r26485 - pypy/dist/pypy/translator/cl Message-ID: <20060428032020.4AABD1009F@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 05:20:17 2006 New Revision: 26485 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (dialtone, sanxiyn) Don't generate a class twice. Apparently pass a method duplicate test too. Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 05:20:17 2006 @@ -214,6 +214,8 @@ def declare_class(self, cls): # cls is Instance name = clrepr(cls._name, symbol=True) + if name in self.declarations: + return field_declaration = ['('+clrepr(field, True)+')' for field in cls._fields] field_declaration = " ".join(field_declaration) if cls._superclass is ROOT: From pedronis at codespeak.net Fri Apr 28 05:31:50 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 28 Apr 2006 05:31:50 +0200 (CEST) Subject: [pypy-svn] r26486 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060428033150.F2B39100A4@code0.codespeak.net> Author: pedronis Date: Fri Apr 28 05:31:47 2006 New Revision: 26486 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: fix ReST Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Fri Apr 28 05:31:47 2006 @@ -133,13 +133,14 @@ - Conceptual design decisions about ootypesystem (tuples list...) (Samuele, Nik) Thursday, 27th April 2006 -************************ +************************* Present: Samuele, Arre, Valentino, Eric, Sanghyeon, Nik, Anders Status after day 3: - gencl (Sanghyeon, Valentino) + * refactored another bit * superclasses and inheritance and isinstance and range * Record type @@ -148,25 +149,30 @@ * works but breaks translation - OOTypeSystem (Nik, Samuele) + * Improved forward references and made the List type lazily inittable. * Introduced the Record type * Introduced Range support in the OOTypeSystem. - ctypes _socket (and select ;)) (Arre, Anders) + * Passing 14 tests for the translatable module * It should be translatable now (although not tested) Pairings today: - Pickling cells (Eric) + * fix translation * add support for additional objects like code objects. - ctypes _socket (Arre, Anders) + * Pass all previous tests. * start proper translation work. - OOTypeSystem (Nik, Sanghyeon) + * Dictionaries need to be lazy too. * Record type for iterator and tuples and make it generic * String type @@ -183,34 +189,42 @@ Status after day 4: - gencl (Sanghyeon, Valentino) - * constant instances now supported which enables the metaclass + + * constant instances now supported which enables the metaclass model and class variables * refactoring is ongoing - Pickle support for cell objects (Eric) + * cell, code, function partly there but needs moduledict support - OOTypeSystem (Nik, Samuele) + * introduced record type for iterators and tuples * random smaller improvements - ctypes _socket (and select ;)) (Arre, Anders) + * passing all tests * uses only ctypes now but doesn't translate yet Pairings today: - Pickling cells (Eric) + * add support for additional objects like code objects. - ctypes _socket (Arre, Anders) + * continue translation work. - OOTypeSystem (Nik, Sanghyeon) + * Dictionaries need to be lazy too. * String type - gencl (Valentino, Sanghyeon) + * exception handling * rpystone translation From sanxiyn at codespeak.net Fri Apr 28 05:35:45 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 05:35:45 +0200 (CEST) Subject: [pypy-svn] r26487 - pypy/dist/pypy/translator/cl Message-ID: <20060428033545.9B831100A4@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 05:35:42 2006 New Revision: 26487 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (dialtone, sanxiyn) Keep the name as well as the code in the declarations dictionary Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 05:35:42 2006 @@ -203,19 +203,21 @@ def declare_struct(self, cls): # cls is Record + if cls in self.declarations: + return self.declarations[cls][0] name = "struct" + str(self.structcount) field_declaration = cls._fields.keys() field_declaration = " ".join(field_declaration) struct_declaration = "(defstruct %s %s)" % (name, field_declaration) - self.declarations[name] = struct_declaration + self.declarations[cls] = (name, struct_declaration) self.structcount += 1 return name def declare_class(self, cls): # cls is Instance - name = clrepr(cls._name, symbol=True) - if name in self.declarations: + if cls in self.declarations: return + name = clrepr(cls._name, symbol=True) field_declaration = ['('+clrepr(field, True)+')' for field in cls._fields] field_declaration = " ".join(field_declaration) if cls._superclass is ROOT: @@ -224,7 +226,7 @@ self.declare_class(cls._superclass) supername = clrepr(cls._superclass._name, symbol=True) class_declaration = "(defclass %s (%s) (%s))" % (name, supername, field_declaration) - self.declarations[name] = class_declaration + self.declarations[cls] = (name, class_declaration) for method in cls._methods: methodobj = cls._methods[method] methodobj._method_name = method @@ -232,6 +234,10 @@ def declare_constant_instance(self, const): # const.concretetype is Instance + if const in self.declarations: + return self.declarations[const][0] + if const.value is const.concretetype._null: + return "nil" name = "const" + str(self.constcount) INST = dynamicType(const.value) self.declare_class(INST) @@ -251,7 +257,7 @@ clrepr(fieldname, True), clrepr(fieldvaluerepr, True))) const_declaration = "\n".join(const_declaration) - self.declarations[const] = const_declaration + self.declarations[const] = (name, const_declaration) self.constcount += 1 return name @@ -264,7 +270,7 @@ def emitcode(self): lines = list(self.emit()) - declarations = "\n".join(self.declarations.values()) + declarations = "\n".join([d[1] for d in self.declarations.values()]) code = "\n".join(lines) if declarations: return declarations + "\n" + code + "\n" From nik at codespeak.net Fri Apr 28 05:52:59 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 05:52:59 +0200 (CEST) Subject: [pypy-svn] r26488 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060428035259.6EC841009F@code0.codespeak.net> Author: nik Date: Fri Apr 28 05:52:55 2006 New Revision: 26488 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/test/test_ootype.py Log: define a hash method on _views. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Fri Apr 28 05:52:55 2006 @@ -599,6 +599,9 @@ assert isinstance(other, _view) return self._inst == other._inst + def __hash__(self): + return hash(self._inst) + def __nonzero__(self): return bool(self._inst) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_ootype.py Fri Apr 28 05:52:55 2006 @@ -381,3 +381,13 @@ assert a.zz() == 1 assert b.zz() == 3 +def test_view_instance_hash(): + I = Instance("Foo", ROOT) + + inst = new(I) + inst_up = ooupcast(ROOT, inst) + inst_up2 = ooupcast(ROOT, inst) + + assert inst_up == inst_up2 + assert hash(inst_up) == hash(inst_up2) + From dialtone at codespeak.net Fri Apr 28 05:56:10 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Fri, 28 Apr 2006 05:56:10 +0200 (CEST) Subject: [pypy-svn] r26489 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428035610.67D43100B6@code0.codespeak.net> Author: dialtone Date: Fri Apr 28 05:56:07 2006 New Revision: 26489 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py Log: (sanxiyn, dialtone) thanks nik Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 05:56:07 2006 @@ -135,7 +135,7 @@ assert cl_dynamic_instance(False) == 2 def test_nullable(): - py.test.skip("fails with null inst") + #py.test.skip("fails with null inst") class Foo: def __init__(self, value): self.value = value From pedronis at codespeak.net Fri Apr 28 06:43:57 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 28 Apr 2006 06:43:57 +0200 (CEST) Subject: [pypy-svn] r26490 - pypy/dist/pypy/module/__builtin__ Message-ID: <20060428044357.BDC241009B@code0.codespeak.net> Author: pedronis Date: Fri Apr 28 06:43:53 2006 New Revision: 26490 Modified: pypy/dist/pypy/module/__builtin__/__init__.py pypy/dist/pypy/module/__builtin__/special.py Log: _pdb() builtin, useful to get a rough idea how stack traces look like. Modified: pypy/dist/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/__init__.py (original) +++ pypy/dist/pypy/module/__builtin__/__init__.py Fri Apr 28 06:43:53 2006 @@ -79,6 +79,7 @@ # default __metaclass__ '__metaclass__' : '(space.w_type)', '_isfake' : 'special._isfake', + '_pdb' : 'special._pdb', # interp-level function definitions 'abs' : 'operation.abs', Modified: pypy/dist/pypy/module/__builtin__/special.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/special.py (original) +++ pypy/dist/pypy/module/__builtin__/special.py Fri Apr 28 06:43:53 2006 @@ -22,3 +22,12 @@ return space.wrap(rarithmetic.formatd(fmt, x)) _formatd.unwrap_spec = [gateway.ObjSpace, int, int, str, float] + + +def __pdb(): + import pdb + pdb.set_trace() +__pdb._annspecialcase_ = 'override:ignore' + +def _pdb(space): + __pdb() From sanxiyn at codespeak.net Fri Apr 28 07:12:59 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 07:12:59 +0200 (CEST) Subject: [pypy-svn] r26491 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428051259.5BAA1100A7@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 07:12:56 2006 New Revision: 26491 Added: pypy/dist/pypy/translator/cl/test/test_exception.py Log: Add a failing test for exception handling Added: pypy/dist/pypy/translator/cl/test/test_exception.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/test/test_exception.py Fri Apr 28 07:12:56 2006 @@ -0,0 +1,21 @@ +import py +from pypy.translator.cl.buildcl import make_cl_func + +def test_handle_exception(): + py.test.skip("TODO") + class MyException(Exception): + pass + def raise_exception(): + # This is in a separate function to fool RTyper + raise MyException() + def handle_exception(flag): + try: + if flag: + raise_exception() + else: + return 2 + except MyException: + return 1 + cl_handle_exception = make_cl_func(handle_exception, [bool]) + assert cl_handle_exception(True) == 1 + assert cl_handle_exception(False) == 2 From nik at codespeak.net Fri Apr 28 08:42:56 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 08:42:56 +0200 (CEST) Subject: [pypy-svn] r26493 - in pypy/dist/pypy: jit/llabstractinterp/test jit/timeshifter module/_socket/rpython rpython rpython/lltypesystem rpython/module rpython/ootypesystem rpython/rctypes rpython/rctypes/test translator/c translator/llvm Message-ID: <20060428064256.E8DD7100A7@code0.codespeak.net> Author: nik Date: Fri Apr 28 08:42:26 2006 New Revision: 26493 Added: pypy/dist/pypy/rpython/lltypesystem/rstr.py (contents, props changed) pypy/dist/pypy/rpython/ootypesystem/rstr.py (contents, props changed) Modified: pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tl.py pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tlr.py pypy/dist/pypy/jit/llabstractinterp/test/test_llabstractinterp.py pypy/dist/pypy/jit/llabstractinterp/test/test_vlist.py pypy/dist/pypy/jit/timeshifter/rtyper.py pypy/dist/pypy/module/_socket/rpython/ll__socket.py pypy/dist/pypy/rpython/lltypesystem/rclass.py pypy/dist/pypy/rpython/lltypesystem/rlist.py pypy/dist/pypy/rpython/module/ll_os.py pypy/dist/pypy/rpython/module/ll_os_path.py pypy/dist/pypy/rpython/module/support.py pypy/dist/pypy/rpython/rctypes/achar_p.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rfloat.py pypy/dist/pypy/rpython/rint.py pypy/dist/pypy/rpython/rstr.py pypy/dist/pypy/rpython/typesystem.py pypy/dist/pypy/translator/c/extfunc.py pypy/dist/pypy/translator/llvm/extfunchelper.py pypy/dist/pypy/translator/llvm/gc.py Log: split rstr into an lltype and minimal ootype version. fix a lot of imports of rstr, but not all of them. some stuff relies too much on the fact that there is only an lltype rstr and will have to be fixed somehwere along the line. Modified: pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tl.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tl.py (original) +++ pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tl.py Fri Apr 28 08:42:26 2006 @@ -4,7 +4,7 @@ from pypy.translator.translator import TranslationContext from pypy.jit.tl import tl from pypy.jit.llabstractinterp.llabstractinterp import LLAbstractInterp -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.llinterp import LLInterpreter from pypy.jit.llabstractinterp.test.test_llabstractinterp import summary #from pypy.translator.backendopt import inline Modified: pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tlr.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tlr.py (original) +++ pypy/dist/pypy/jit/llabstractinterp/test/test_jit_tlr.py Fri Apr 28 08:42:26 2006 @@ -4,7 +4,7 @@ from pypy.jit.llabstractinterp.llabstractinterp import LLAbstractInterp from pypy.jit.llabstractinterp.test.test_llabstractinterp import summary from pypy.rpython.llinterp import LLInterpreter -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.jit.tl import tlr Modified: pypy/dist/pypy/jit/llabstractinterp/test/test_llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp/test/test_llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp/test/test_llabstractinterp.py Fri Apr 28 08:42:26 2006 @@ -3,7 +3,7 @@ from pypy.rpython.annlowlevel import annotate_lowlevel_helper from pypy.rpython.lltypesystem import lltype from pypy.rpython.llinterp import LLInterpreter -from pypy.rpython import rstr +from pypy.rpython.lltypesystem import rstr from pypy.annotation import model as annmodel from pypy.jit.llabstractinterp.llabstractinterp import LLAbstractInterp, Policy from pypy.objspace.flow import model as flowmodel Modified: pypy/dist/pypy/jit/llabstractinterp/test/test_vlist.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp/test/test_vlist.py (original) +++ pypy/dist/pypy/jit/llabstractinterp/test/test_vlist.py Fri Apr 28 08:42:26 2006 @@ -3,7 +3,7 @@ from pypy.jit.llabstractinterp.llabstractinterp import LLAbstractInterp, Policy from pypy.jit.llabstractinterp.test.test_llabstractinterp import summary from pypy.rpython.llinterp import LLInterpreter -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.objectmodel import hint policy = Policy(inlining=True, const_propagate=True, concrete_args=False, Modified: pypy/dist/pypy/jit/timeshifter/rtyper.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/rtyper.py (original) +++ pypy/dist/pypy/jit/timeshifter/rtyper.py Fri Apr 28 08:42:26 2006 @@ -4,7 +4,7 @@ from pypy.rpython import annlowlevel from pypy.rpython.rtyper import RPythonTyper, LowLevelOpList, TyperError from pypy.rpython.rmodel import Repr, inputconst -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.typesystem import TypeSystem from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython import rgenop Modified: pypy/dist/pypy/module/_socket/rpython/ll__socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/rpython/ll__socket.py (original) +++ pypy/dist/pypy/module/_socket/rpython/ll__socket.py Fri Apr 28 08:42:26 2006 @@ -1,6 +1,6 @@ import _socket -from pypy.rpython.rstr import STR +from pypy.rpython.lltypesystem.rstr import STR from pypy.rpython.lltypesystem.lltype import GcStruct, Signed, Array, Char, Ptr, malloc from pypy.rpython.module.support import to_rstr, from_rstr from pypy.rpython.module.support import to_opaque_object, from_opaque_object Modified: pypy/dist/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rclass.py Fri Apr 28 08:42:26 2006 @@ -538,7 +538,7 @@ def ll_str(self, i): # doesn't work for non-gc classes! instance = cast_pointer(OBJECTPTR, i) - from pypy.rpython import rstr + from pypy.rpython.lltypesystem import rstr nameLen = len(instance.typeptr.name) nameString = malloc(rstr.STR, nameLen-1) i = 0 Modified: pypy/dist/pypy/rpython/lltypesystem/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rlist.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rlist.py Fri Apr 28 08:42:26 2006 @@ -14,7 +14,7 @@ GcForwardReference, Ptr, GcArray, GcStruct, \ Void, Signed, malloc, typeOf, Primitive, \ Bool, nullptr, typeMethod -from pypy.rpython import rstr +from pypy.rpython.lltypesystem import rstr from pypy.rpython import robject # ____________________________________________________________ Added: pypy/dist/pypy/rpython/lltypesystem/rstr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/rstr.py Fri Apr 28 08:42:26 2006 @@ -0,0 +1,37 @@ +from pypy.rpython.rstr import string_repr, StringRepr, STR, AbstractStringIteratorRepr, \ + ll_strconcat, instance_str_prefix, instance_str_suffix, list_str_open_bracket, ll_join, list_str_sep, list_str_close_bracket +from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, Signed + +class __extend__(StringRepr): + + lowleveltype = Ptr(STR) + + def make_iterator_repr(self): + return string_iterator_repr + + +class StringIteratorRepr(AbstractStringIteratorRepr): + + lowleveltype = Ptr(GcStruct('stringiter', + ('string', string_repr.lowleveltype), + ('index', Signed))) + + def __init__(self): + self.ll_striter = ll_striter + self.ll_strnext = ll_strnext + +def ll_striter(string): + iter = malloc(string_iterator_repr.lowleveltype.TO) + iter.string = string + iter.index = 0 + return iter + +def ll_strnext(iter): + chars = iter.string.chars + index = iter.index + if index >= len(chars): + raise StopIteration + iter.index = index + 1 + return chars[index] + +string_iterator_repr = StringIteratorRepr() Modified: pypy/dist/pypy/rpython/module/ll_os.py ============================================================================== --- pypy/dist/pypy/rpython/module/ll_os.py (original) +++ pypy/dist/pypy/rpython/module/ll_os.py Fri Apr 28 08:42:26 2006 @@ -14,7 +14,7 @@ # and buffer preparation stuff is not useful. import os, errno -from pypy.rpython.rstr import STR +from pypy.rpython.lltypesystem.rstr import STR from pypy.rpython.lltypesystem.lltype import \ GcStruct, Signed, Array, Char, Ptr, malloc from pypy.rpython.module.support import to_rstr, from_rstr, ll_strcpy, _ll_strfill Modified: pypy/dist/pypy/rpython/module/ll_os_path.py ============================================================================== --- pypy/dist/pypy/rpython/module/ll_os_path.py (original) +++ pypy/dist/pypy/rpython/module/ll_os_path.py Fri Apr 28 08:42:26 2006 @@ -6,7 +6,7 @@ import os import stat -from pypy.rpython.rstr import STR +from pypy.rpython.lltypesystem.rstr import STR from pypy.rpython.module.support import to_rstr, from_rstr, ll_strcpy from pypy.rpython.module.ll_os import ll_os_stat Modified: pypy/dist/pypy/rpython/module/support.py ============================================================================== --- pypy/dist/pypy/rpython/module/support.py (original) +++ pypy/dist/pypy/rpython/module/support.py Fri Apr 28 08:42:26 2006 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython import extfunctable -from pypy.rpython.rstr import STR +from pypy.rpython.lltypesystem.rstr import STR from pypy.rpython.lltypesystem.lltype import \ GcStruct, Signed, Array, Char, Ptr, malloc Added: pypy/dist/pypy/rpython/ootypesystem/rstr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/rstr.py Fri Apr 28 08:42:26 2006 @@ -0,0 +1,17 @@ +from pypy.rpython.rstr import string_repr, StringRepr, STR, AbstractStringIteratorRepr +from pypy.rpython.lltypesystem.lltype import Ptr +from pypy.rpython.ootypesystem.ootype import Signed, Record + +class __extend__(StringRepr): + + lowleveltype = Ptr(STR) + + def make_iterator_repr(self): + return string_iterator_repr + + +class StringIteratorRepr(AbstractStringIteratorRepr): + + lowleveltype = Record({'string': string_repr.lowleveltype, 'index': Signed}) + +string_iterator_repr = StringIteratorRepr Modified: pypy/dist/pypy/rpython/rctypes/achar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/achar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/achar_p.py Fri Apr 28 08:42:26 2006 @@ -9,7 +9,7 @@ _about_ = c_char_p def specialize_call(self, hop): - from pypy.rpython.rstr import string_repr + string_repr = hop.rtyper.type_system.rstr.string_repr r_char_p = hop.r_result v_result = r_char_p.allocate_instance(hop.llops) if len(hop.args_s): Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Fri Apr 28 08:42:26 2006 @@ -1,5 +1,5 @@ from ctypes import ARRAY, c_int -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.lltypesystem import lltype from pypy.annotation.pairtype import pairtype Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Fri Apr 28 08:42:26 2006 @@ -1,6 +1,7 @@ from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype -from pypy.rpython.rstr import StringRepr, string_repr +from pypy.rpython.rstr import StringRepr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO from pypy.rpython.rctypes.rarray import ArrayRepr from pypy.rpython.rctypes.rstringbuf import StringBufRepr Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Fri Apr 28 08:42:26 2006 @@ -10,7 +10,7 @@ from pypy.rpython.test.test_llinterp import interpret from pypy.translator.c.test.test_genc import compile from pypy import conftest -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.lltypesystem import lltype from ctypes import cdll, pythonapi, PyDLL, _FUNCFLAG_PYTHONAPI Modified: pypy/dist/pypy/rpython/rfloat.py ============================================================================== --- pypy/dist/pypy/rpython/rfloat.py (original) +++ pypy/dist/pypy/rpython/rfloat.py Fri Apr 28 08:42:26 2006 @@ -7,7 +7,6 @@ from pypy.rpython.rmodel import IntegerRepr, BoolRepr, StringRepr from pypy.rpython.robject import PyObjRepr, pyobj_repr from pypy.rpython.rstr import string_repr -from pypy.rpython import rstr from pypy.rpython.rmodel import log import math @@ -87,6 +86,7 @@ class __extend__(pairtype(StringRepr, FloatRepr)): def rtype_mod(_, hop): + rstr = hop.rtyper.type_system.rstr return rstr.do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) #Helpers FloatRepr,FloatRepr Modified: pypy/dist/pypy/rpython/rint.py ============================================================================== --- pypy/dist/pypy/rpython/rint.py (original) +++ pypy/dist/pypy/rpython/rint.py Fri Apr 28 08:42:26 2006 @@ -307,7 +307,7 @@ return vlist[0] def ll_str(self, i): - from pypy.rpython.rstr import STR + from pypy.rpython.lltypesystem.rstr import STR temp = malloc(CHAR_ARRAY, 20) len = 0 sign = 0 @@ -358,7 +358,7 @@ hex_chars[i] = "%x"%i def ll_int2hex(i, addPrefix): - from pypy.rpython.rstr import STR + from pypy.rpython.lltypesystem.rstr import STR temp = malloc(CHAR_ARRAY, 20) len = 0 sign = 0 @@ -391,7 +391,7 @@ return result def ll_int2oct(i, addPrefix): - from pypy.rpython.rstr import STR + from pypy.rpython.lltypesystem.rstr import STR if i == 0: result = malloc(STR, 1) result.chars[0] = '0' Modified: pypy/dist/pypy/rpython/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/rstr.py (original) +++ pypy/dist/pypy/rpython/rstr.py Fri Apr 28 08:42:26 2006 @@ -33,7 +33,7 @@ class __extend__(annmodel.SomeString): def rtyper_makerepr(self, rtyper): - return string_repr + return rtyper.type_system.rstr.string_repr def rtyper_makekey(self): return self.__class__, @@ -56,7 +56,6 @@ class __extend__(StringRepr): - lowleveltype = Ptr(STR) def convert_const(self, value): if value is None: @@ -224,9 +223,6 @@ def ll_str(self, s): return s - def make_iterator_repr(self): - return string_iterator_repr - class __extend__(pairtype(StringRepr, IntegerRepr)): def rtype_getitem(_, hop): v_str, v_index = hop.inputargs(string_repr, Signed) @@ -1109,35 +1105,18 @@ # # Iteration. -class StringIteratorRepr(IteratorRepr): - lowleveltype = Ptr(GcStruct('stringiter', - ('string', string_repr.lowleveltype), - ('index', Signed))) +class AbstractStringIteratorRepr(IteratorRepr): + def newiter(self, hop): v_str, = hop.inputargs(string_repr) - return hop.gendirectcall(ll_striter, v_str) + return hop.gendirectcall(self.ll_striter, v_str) def rtype_next(self, hop): v_iter, = hop.inputargs(self) hop.has_implicit_exception(StopIteration) # record that we know about it hop.exception_is_here() - return hop.gendirectcall(ll_strnext, v_iter) - -string_iterator_repr = StringIteratorRepr() + return hop.gendirectcall(self.ll_strnext, v_iter) -def ll_striter(string): - iter = malloc(string_iterator_repr.lowleveltype.TO) - iter.string = string - iter.index = 0 - return iter - -def ll_strnext(iter): - chars = iter.string.chars - index = iter.index - if index >= len(chars): - raise StopIteration - iter.index = index + 1 - return chars[index] # these should be in rclass, but circular imports prevent (also it's # not that insane that a string constant is built in this file). Modified: pypy/dist/pypy/rpython/typesystem.py ============================================================================== --- pypy/dist/pypy/rpython/typesystem.py (original) +++ pypy/dist/pypy/rpython/typesystem.py Fri Apr 28 08:42:26 2006 @@ -21,7 +21,7 @@ except ImportError: return None if name in ('rclass', 'rpbc', 'rbuiltin', 'rtuple', 'rlist', - 'rslice', 'rdict', 'rrange', 'exceptiondata'): + 'rslice', 'rdict', 'rrange', 'rstr', 'exceptiondata'): mod = load(name) if mod is not None: setattr(self, name, mod) Modified: pypy/dist/pypy/translator/c/extfunc.py ============================================================================== --- pypy/dist/pypy/translator/c/extfunc.py (original) +++ pypy/dist/pypy/translator/c/extfunc.py Fri Apr 28 08:42:26 2006 @@ -2,8 +2,8 @@ from pypy.objspace.flow.model import FunctionGraph from pypy.rpython.lltypesystem import lltype from pypy.translator.c.support import cdecl -from pypy.rpython.rstr import STR -from pypy.rpython import rstr +from pypy.rpython.lltypesystem.rstr import STR +from pypy.rpython.lltypesystem import rstr from pypy.rpython.lltypesystem import rlist from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack Modified: pypy/dist/pypy/translator/llvm/extfunchelper.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfunchelper.py (original) +++ pypy/dist/pypy/translator/llvm/extfunchelper.py Fri Apr 28 08:42:26 2006 @@ -2,8 +2,8 @@ from pypy.objspace.flow.model import FunctionGraph from pypy.rpython.lltypesystem import lltype from pypy.translator.c.support import cdecl -from pypy.rpython.rstr import STR -from pypy.rpython import rstr +from pypy.rpython.lltypesystem.rstr import STR +from pypy.rpython.lltypesystem import rstr from pypy.rpython.lltypesystem import rlist from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Fri Apr 28 08:42:26 2006 @@ -1,5 +1,5 @@ import sys -from pypy.rpython.rstr import STR +from pypy.rpython.lltypesystem.rstr import STR from pypy.translator.llvm.log import log log = log.gc From nik at codespeak.net Fri Apr 28 09:28:30 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 09:28:30 +0200 (CEST) Subject: [pypy-svn] r26496 - in pypy/dist/pypy/rpython/ootypesystem: . test Message-ID: <20060428072830.5840C1009D@code0.codespeak.net> Author: nik Date: Fri Apr 28 09:28:26 2006 New Revision: 26496 Modified: pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Log: (sanxiyn, dialtone, samuele, nik) added a helper to exceptiondata to check if an Instance is derived from Exception. the interesting part of this was writing a test for it. ;) Modified: pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/exceptiondata.py Fri Apr 28 09:28:26 2006 @@ -8,6 +8,18 @@ class ExceptionData(AbstractExceptionData): """Public information for the code generators to help with exceptions.""" + def __init__(self, rtyper): + AbstractExceptionData.__init__(self, rtyper) + self._compute_exception_instance(rtyper) + + def _compute_exception_instance(self, rtyper): + excdef = rtyper.annotator.bookkeeper.getuniqueclassdef(Exception) + excrepr = rclass.getinstancerepr(rtyper, excdef) + self._EXCEPTION_INST = excrepr.lowleveltype + + def is_exception_instance(self, INSTANCE): + return ootype.isSubclass(INSTANCE, self._EXCEPTION_INST) + def make_helpers(self, rtyper): self.fn_exception_match = self.make_exception_matcher(rtyper) self.fn_pyexcclass2exc = self.make_pyexcclass2exc(rtyper) Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oortype.py Fri Apr 28 09:28:26 2006 @@ -172,4 +172,18 @@ res = interpret(oof, [], type_system='ootype') +def test_is_exception_instance(): + def f(): + return NameError() + + t = TranslationContext() + t.buildannotator().build_types(f, []) + if conftest.option.view: + t.view() + rtyper = t.buildrtyper(type_system="ootype") + rtyper.specialize() + graph = graphof(t, f) + + INST = graph.getreturnvar().concretetype + assert rtyper.exceptiondata.is_exception_instance(INST) From ac at codespeak.net Fri Apr 28 09:40:12 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 28 Apr 2006 09:40:12 +0200 (CEST) Subject: [pypy-svn] r26497 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060428074012.521801009D@code0.codespeak.net> Author: ac Date: Fri Apr 28 09:40:06 2006 New Revision: 26497 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/rpython/rctypes/avoid_p.py pypy/dist/pypy/rpython/rctypes/rstringbuf.py pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Log: (aleale, arre) Support slicing and casting to pointer on stringbuffer. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Fri Apr 28 09:40:06 2006 @@ -785,10 +785,13 @@ pass def getitem((s_cto, s_slice)): - try: - listdef = ListDef(None, s_cto.knowntype._type_.annotator_type) - except AttributeError: - listdef = ListDef(None, SomeCTypesObject(s_cto.knowntype._type_)) + result_ctype = s_cto.knowntype._type_ + s_result = SomeCTypesObject(result_ctype, + memorystate=SomeCTypesObject.MEMORYALIAS) + list_item = s_result.return_annotation() + if isinstance(list_item, SomeChar): + return SomeString() + listdef = ListDef(None, list_item) return SomeList(listdef) class __extend__(pairtype(SomeCTypesObject, SomeCTypesObject)): Modified: pypy/dist/pypy/rpython/rctypes/avoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/avoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/avoid_p.py Fri Apr 28 09:40:06 2006 @@ -2,7 +2,8 @@ from pypy.rpython.rctypes.implementation import CTypesCallEntry, CTypesObjEntry from pypy.annotation.model import SomeCTypesObject -from ctypes import c_void_p, c_int, POINTER, cast +from ctypes import c_void_p, c_int, POINTER, cast, c_char, c_char_p +from pypy.rpython.rctypes.astringbuf import StringBufferType PointerType = type(POINTER(c_int)) @@ -39,18 +40,27 @@ assert s_type.is_constant(), ( "cast(p, %r): argument 2 must be constant" % (s_type,)) type = s_type.const - self.checkptr(s_arg.knowntype) self.checkptr(type) + if s_arg.knowntype == StringBufferType: + pass + else: + self.checkptr(s_arg.knowntype) return SomeCTypesObject(type, SomeCTypesObject.OWNSMEMORY) def specialize_call(self, hop): from pypy.rpython.rctypes.rpointer import PointerRepr from pypy.rpython.rctypes.rvoid_p import CVoidPRepr + from pypy.rpython.rctypes.rstringbuf import StringBufRepr from pypy.rpython.lltypesystem import lltype, llmemory - assert isinstance(hop.args_r[0], (PointerRepr, CVoidPRepr)) + assert isinstance(hop.args_r[0], (PointerRepr, CVoidPRepr, + StringBufRepr)) targetctype = hop.args_s[1].const v_box, c_targetctype = hop.inputargs(hop.args_r[0], lltype.Void) - v_adr = hop.args_r[0].getvalue(hop.llops, v_box) + if isinstance(hop.args_r[0], StringBufRepr): + v_index = hop.inputconst(lltype.Signed, 0) + v_adr = hop.args_r[0].get_c_data_of_item(hop.llops, v_box, v_index) + else: + v_adr = hop.args_r[0].getvalue(hop.llops, v_box) if v_adr.concretetype != llmemory.Address: v_adr = hop.genop('cast_ptr_to_adr', [v_adr], resulttype = llmemory.Address) Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Fri Apr 28 09:40:06 2006 @@ -3,7 +3,8 @@ from pypy.rpython.rmodel import IntegerRepr, inputconst from pypy.rpython.rctypes.rmodel import CTypesRefRepr from pypy.objspace.flow.model import Constant - +from pypy.rpython.rslice import AbstractSliceRepr +from pypy.rpython.rstr import string_repr class StringBufRepr(CTypesRefRepr): @@ -32,6 +33,7 @@ resulttype = ONE_CHAR_PTR) return v_char_p + ONE_CHAR_PTR = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) @@ -49,5 +51,44 @@ v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) hop.genop('setarrayitem', [v_array, v_index, v_item]) +class __extend__(pairtype(StringBufRepr, AbstractSliceRepr)): + def rtype_getitem((r_stringbuf, r_slice), hop): + rs = r_stringbuf.rtyper.type_system.rslice + if r_slice == rs.startonly_slice_repr: + v_stringbuf, v_start = hop.inputargs(r_stringbuf, rs.startonly_slice_repr) + v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) + return hop.gendirectcall(ll_slice_startonly, v_array, v_start) + if r_slice == rs.startstop_slice_repr: + v_stringbuf, v_slice = hop.inputargs(r_stringbuf, rs.startstop_slice_repr) + v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) + return hop.gendirectcall(ll_slice, v_array, v_slice) + raise TyperError('getitem does not support slices with %r' % (r_slice,)) + +def ll_slice_startonly(sbuf, start): + return ll_slice_start_stop(sbuf, start, len(sbuf)) + +def ll_slice(sbuf, slice): + return ll_slice_start_stop(sbuf, slice.start, slice.stop) + +def ll_slice_start_stop(sbuf, start, stop): + length = len(sbuf) + if start < 0: + start = length + start + if start < 0: + start = 0 + if stop < 0: + stop = length + stop + if stop < 0: + stop = 0 + if stop > length: + stop = length + if start > stop: + start = stop + newlength = stop - start + newstr = lltype.malloc(string_repr.lowleveltype.TO, newlength) + for i in range(newlength): + newstr.chars[i] = sbuf[start + i] + return newstr + STRBUFTYPE = lltype.Array(lltype.Char) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Fri Apr 28 09:40:06 2006 @@ -12,7 +12,8 @@ import sys from pypy.rpython.test.test_llinterp import interpret -from ctypes import create_string_buffer, sizeof, c_int +from ctypes import create_string_buffer, cast, POINTER, c_void_p, c_char +from ctypes import c_char_p, c_long, pointer, sizeof, c_int from pypy.rpython.rctypes.astringbuf import StringBufferType @@ -54,6 +55,33 @@ if conftest.option.view: a.translator.view() + def test_annotate_slice(self): + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + return buf[:2] + + a = RPythonAnnotator() + s = a.build_types(func, [int]) + assert s == annmodel.SomeString() + + def test_annotate_cast_to_ptr(self): + charp = POINTER(c_char) + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + buf[2] = 'z' + cp = cast(buf, charp) + vp = cast(buf, c_void_p) + return cp[0] + cast(vp, charp)[2] + + a = RPythonAnnotator() + s = a.build_types(func, [int]) + assert s == annmodel.SomeString() + + class Test_specialization: def test_specialize_create(self): def func(n): @@ -96,6 +124,31 @@ res = interpret(func, [12]) assert ''.join(res.chars) == "xy" + def test_specialize_slice(self): + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + buf[2] = 'z' + return buf[:2] + '_' + buf[1:3] + '_' + buf[9:] + + res = interpret(func, [12]) + assert ''.join(res.chars) == "xy_yz_\0\0\0" + + def test_specialize_cast_to_ptr(self): + charp = POINTER(c_char) + def func(n): + buf = create_string_buffer(n) + buf[0] = 'x' + buf[1] = 'y' + buf[2] = 'z' + cp = cast(buf, charp) + vp = cast(buf, c_void_p) + return cp[0] + '_' + cast(vp, charp)[2] + + res = interpret(func, [12]) + assert ''.join(res.chars) == 'x_z' + def test_specialize_sizeof(self): def func(n): buf = create_string_buffer(n) @@ -117,3 +170,17 @@ assert res[1] == sizeof(c_int) * 42 assert res[2] == sizeof(c_int) assert res[3] == sizeof(c_int) * 42 + + def test_compile_cast_to_ptr(self): + charp = POINTER(c_char) + def func(n): + c_n = c_long(n) + c_n_ptr = cast(pointer(c_n), POINTER(c_char)) + buf = create_string_buffer(sizeof(c_long)) + for i in range(sizeof(c_long)): + buf[i] = c_n_ptr[i] + c_long_ptr = cast(buf, POINTER(c_long)) + return c_long_ptr.contents.value + fn = compile(func, [int]) + res = fn(0x12345678) + assert res == 0x12345678 From arigo at codespeak.net Fri Apr 28 10:34:49 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 10:34:49 +0200 (CEST) Subject: [pypy-svn] r26499 - pypy/extradoc/soc-2006 Message-ID: <20060428083449.AD97A100A4@code0.codespeak.net> Author: arigo Date: Fri Apr 28 10:34:43 2006 New Revision: 26499 Modified: pypy/extradoc/soc-2006/planning.txt Log: Registered as a mentor. Modified: pypy/extradoc/soc-2006/planning.txt ============================================================================== --- pypy/extradoc/soc-2006/planning.txt (original) +++ pypy/extradoc/soc-2006/planning.txt Fri Apr 28 10:34:43 2006 @@ -4,7 +4,6 @@ mentors ------------------------ -Armin Rigo Michael Hudson Christian Tismer Anders Chrigstroem @@ -16,6 +15,7 @@ Carl Friedrich Bolz Aurelien Campeas (pending) Samuele Pedroni +Armin Rigo On the 27th of April or so, mwh will send a list of all not-yet-registered mentors to Neal Norwitz. From sanxiyn at codespeak.net Fri Apr 28 10:50:36 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 10:50:36 +0200 (CEST) Subject: [pypy-svn] r26501 - pypy/dist/pypy/rpython/test Message-ID: <20060428085036.55E4D100AD@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 10:50:33 2006 New Revision: 26501 Modified: pypy/dist/pypy/rpython/test/test_rclass.py Log: Add a failing test_runtime_exception. Please help! Modified: pypy/dist/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rclass.py (original) +++ pypy/dist/pypy/rpython/test/test_rclass.py Fri Apr 28 10:50:33 2006 @@ -1,7 +1,8 @@ +import py from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.ootypesystem import ootype -from pypy.rpython.test.test_llinterp import interpret +from pypy.rpython.test.test_llinterp import interpret, interpret_raises from pypy.rpython.rarithmetic import intmask @@ -99,6 +100,19 @@ res = interpret(dummyfn, [1], type_system=self.ts) assert res == 4 + def test_runtime_exception(self): + py.test.skip("God help us") + def pick(flag): + if flag: + return TypeError + else: + return ValueError + def f(flag): + ex = pick(flag) + raise ex() + interpret_raises(TypeError, f, [True], type_system=self.ts) + interpret_raises(ValueError, f, [False], type_system=self.ts) + def test_classattr_as_defaults(self): def dummyfn(): x = Random() From auc at codespeak.net Fri Apr 28 10:56:42 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 28 Apr 2006 10:56:42 +0200 (CEST) Subject: [pypy-svn] r26502 - pypy/dist/pypy/doc/discussion Message-ID: <20060428085642.CF5A6100AD@code0.codespeak.net> Author: auc Date: Fri Apr 28 10:56:41 2006 New Revision: 26502 Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt Log: abstract spec Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/logic-plan.txt (original) +++ pypy/dist/pypy/doc/discussion/logic-plan.txt Fri Apr 28 10:56:41 2006 @@ -191,3 +191,51 @@ It is expected that PyPy coroutines/greenlets/microthreads be at some point picklable. One could implement clone() using the thread pickling machinery. + +Such is the abstract interface of computation spaces (the +inherit-from-threads part is a bit speculative) : + + class MicroThread ... + "provided by PyPy, stackless team" + + class CompSpace(Microthread): + + """solver API""" + + def ask(self): + """ + blocks until the space is stable, which means that + the embedded computation has reached a fixpoint + returns int in [0,1,n] + 0 : space is failed (further calls to the + space raise an exception) + 1 : space is entailed (solution found) + n : space is distributable (n choice points) + """ + + def commit(self, choice): + """tells the space which choice point to pick""" + + def clone(self): + """returns a cloned computation space""" + + def merge(self): + """ + extract solution values from an entailed space + furher calls to space methods will raise + an exception + """ + + """distributor API""" + + def choose(self): + """ + blocks until commit is called + returns the choice value that + was given to the commit call + """ + + +Programs runing in a computation spaces should not be allowed to alter +the top-level space (i.e interaction with the global program state and +outside world). From arigo at codespeak.net Fri Apr 28 11:11:38 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 11:11:38 +0200 (CEST) Subject: [pypy-svn] r26503 - pypy/dist/pypy/doc/discussion Message-ID: <20060428091138.37AD9100B0@code0.codespeak.net> Author: arigo Date: Fri Apr 28 11:11:36 2006 New Revision: 26503 Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt Log: Restification. Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/logic-plan.txt (original) +++ pypy/dist/pypy/doc/discussion/logic-plan.txt Fri Apr 28 11:11:36 2006 @@ -193,7 +193,7 @@ machinery. Such is the abstract interface of computation spaces (the -inherit-from-threads part is a bit speculative) : +inherit-from-threads part is a bit speculative):: class MicroThread ... "provided by PyPy, stackless team" From auc at codespeak.net Fri Apr 28 11:15:13 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 28 Apr 2006 11:15:13 +0200 (CEST) Subject: [pypy-svn] r26504 - pypy/dist/pypy/translator/cl/test Message-ID: <20060428091513.D0842100B5@code0.codespeak.net> Author: auc Date: Fri Apr 28 11:15:12 2006 New Revision: 26504 Added: pypy/dist/pypy/translator/cl/test/test_dict.py Log: failing dict test Added: pypy/dist/pypy/translator/cl/test/test_dict.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/cl/test/test_dict.py Fri Apr 28 11:15:12 2006 @@ -0,0 +1,15 @@ +from pypy.translator.cl.buildcl import make_cl_func + +def notest_dict_length(): + def dict_length_one(key, val): + dic = {key:val} + return len(dic) + cl_dict_length_one = make_cl_func(dict_length_one, [int,int]) + assert cl_dict_length_one(42,42) == 1 + +def notest_dict_key_access(): + def dict_key(number): + dic = {42:43} + return dic[number] + cl_dict_key = make_cl_func(dict_key, [int]) + assert cl_dict_key(42) == 43 From pedronis at codespeak.net Fri Apr 28 11:29:16 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 28 Apr 2006 11:29:16 +0200 (CEST) Subject: [pypy-svn] r26505 - pypy/dist/pypy/interpreter Message-ID: <20060428092916.3E64F100AD@code0.codespeak.net> Author: pedronis Date: Fri Apr 28 11:29:13 2006 New Revision: 26505 Modified: pypy/dist/pypy/interpreter/mixedmodule.py Log: heavy-handed code to assure that builtin modules functions have the name matching their exposed name. Modified: pypy/dist/pypy/interpreter/mixedmodule.py ============================================================================== --- pypy/dist/pypy/interpreter/mixedmodule.py (original) +++ pypy/dist/pypy/interpreter/mixedmodule.py Fri Apr 28 11:29:13 2006 @@ -52,6 +52,7 @@ bltin = BuiltinFunction(func) bltin.w_module = self.w_name func._builtinversion_ = bltin + bltin.name = name w_value = space.wrap(bltin) space.setitem(self.w_dict, w_name, w_value) return w_value From sanxiyn at codespeak.net Fri Apr 28 11:31:35 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 11:31:35 +0200 (CEST) Subject: [pypy-svn] r26506 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060428093135.D9A28100AD@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 11:31:31 2006 New Revision: 26506 Modified: pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_exception.py pypy/dist/pypy/translator/cl/test/test_oo.py Log: (dialtone, sanxiyn) Work in progress on exception handling Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 11:31:31 2006 @@ -1,7 +1,7 @@ import types from pypy.tool.udir import udir -from pypy.objspace.flow.model import Constant +from pypy.objspace.flow.model import Constant, c_last_exception from pypy.translator.translator import graphof from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _class, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT @@ -101,10 +101,13 @@ yield "(setf %s (make-%s))" % (clrepr(result, True), clrepr(clsname, True)) elif isinstance(cls, Instance): - self.gen.declare_class(cls) clsname = clrepr(cls) - yield "(setf %s (make-instance %s))" % (clrepr(result, True), - clrepr(clsname, True)) + if self.gen.is_exception_instance(cls): + self.gen.declare_exception(cls) + yield "(setf %s (make-condition %s))" % (result, clsname) + else: + self.gen.declare_class(cls) + yield "(setf %s (make-instance %s))" % (result, clsname) else: raise NotImplementedError() @@ -195,6 +198,10 @@ self.constcount = 0 self.structcount = 0 + def is_exception_instance(self, INST): + exceptiondata = self.context.rtyper.exceptiondata + return exceptiondata.is_exception_instance(INST) + def check_declaration(self, arg): if isinstance(arg, Constant): if isinstance(arg.concretetype, Instance): @@ -232,6 +239,21 @@ methodobj._method_name = method self.pendinggraphs.append(methodobj) + def declare_exception(self, cls): + # cls is Instance + assert self.is_exception_instance(cls) + if cls in self.declarations: + return self.declarations[cls][0] + name = clrepr(cls._name, symbol=True) + if cls._superclass is OBJECT: + exception_declaration = "(define-condition %s () ())" % (name,) + else: + self.declare_exception(cls._superclass) + supername = clrepr(cls._superclass._name, symbol=True) + exception_declaration = "(define-condition %s (%s) ())" % (name, supername) + self.declarations[cls] = (name, exception_declaration) + return name + def declare_constant_instance(self, const): # const.concretetype is Instance if const in self.declarations: @@ -348,6 +370,8 @@ self.cur_block = block tag = self.blockref[block] yield "tag" + clrepr(str(tag), True) + if block.exitswitch is c_last_exception: + yield "(handler-case" for op in block.operations: emit_op = Op(self, op) for line in emit_op: @@ -370,6 +394,21 @@ for line in self.emit_link(exits[0]): yield line yield "))" + elif block.exitswitch is c_last_exception: + body = None + exceptions = {} + for exit in exits: + if exit.exitcase is None: + body = exit + else: + cls = exit.llexitcase.class_._INSTANCE + exception = self.declare_exception(cls) + exceptions[exception] = exit + self.emit_link(body) + for exception in exceptions: + yield "(%s ()" % (exception,) + self.emit_link(exceptions[exception]) + yield ")" else: # this is for the more general case. The previous special case # shouldn't be needed but in Python 2.2 we can't tell apart @@ -385,14 +424,14 @@ for line in self.emit_link(exits[-1]): yield line yield ")" * len(exits) - elif len(block.inputargs) == 2: # exc_cls, exc_value - exc_cls = clrepr(block.inputargs[0], True) + elif len(block.inputargs) == 2: exc_value = clrepr(block.inputargs[1], True) - yield "(something-like-throw-exception %s %s)" % (clrepr(exc_cls, True), - clrepr(exc_value, True)) + yield "(error %s)" % (exc_value,) else: retval = clrepr(block.inputargs[0]) yield "(return %s)" % clrepr(retval, True) + if block.exitswitch is c_last_exception: + yield ")" def format_jump(self, block): tag = self.blockref[block] Modified: pypy/dist/pypy/translator/cl/test/test_exception.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_exception.py (original) +++ pypy/dist/pypy/translator/cl/test/test_exception.py Fri Apr 28 11:31:31 2006 @@ -2,7 +2,6 @@ from pypy.translator.cl.buildcl import make_cl_func def test_handle_exception(): - py.test.skip("TODO") class MyException(Exception): pass def raise_exception(): Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Fri Apr 28 11:31:31 2006 @@ -135,7 +135,6 @@ assert cl_dynamic_instance(False) == 2 def test_nullable(): - #py.test.skip("fails with null inst") class Foo: def __init__(self, value): self.value = value From pedronis at codespeak.net Fri Apr 28 11:33:40 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 28 Apr 2006 11:33:40 +0200 (CEST) Subject: [pypy-svn] r26507 - in pypy/dist/pypy/rpython: . test Message-ID: <20060428093340.D47AC100AD@code0.codespeak.net> Author: pedronis Date: Fri Apr 28 11:33:36 2006 New Revision: 26507 Modified: pypy/dist/pypy/rpython/exceptiondata.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/rpython/test/test_rclass.py Log: try to make sure that reprs are created only after normalisation. split make_standard_exceptions in two. Add a finish to ExceptionData. Modified: pypy/dist/pypy/rpython/exceptiondata.py ============================================================================== --- pypy/dist/pypy/rpython/exceptiondata.py (original) +++ pypy/dist/pypy/rpython/exceptiondata.py Fri Apr 28 11:33:36 2006 @@ -21,5 +21,10 @@ bk = rtyper.annotator.bookkeeper for cls in self.standardexceptions: classdef = bk.getuniqueclassdef(cls) + + def finish(self, rtyper): + bk = rtyper.annotator.bookkeeper + for cls in self.standardexceptions: + classdef = bk.getuniqueclassdef(cls) rclass.getclassrepr(rtyper, classdef).setup() Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Fri Apr 28 11:33:36 2006 @@ -155,6 +155,7 @@ # first make sure that all functions called in a group have exactly # the same signature, by hacking their flow graphs if needed self.type_system.perform_normalizations(self) + self.exceptiondata.finish(self) # new blocks can be created as a result of specialize_block(), so # we need to be careful about the loop here. self.already_seen = {} Modified: pypy/dist/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rclass.py (original) +++ pypy/dist/pypy/rpython/test/test_rclass.py Fri Apr 28 11:33:36 2006 @@ -101,7 +101,6 @@ assert res == 4 def test_runtime_exception(self): - py.test.skip("God help us") def pick(flag): if flag: return TypeError From sanxiyn at codespeak.net Fri Apr 28 11:48:30 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Fri, 28 Apr 2006 11:48:30 +0200 (CEST) Subject: [pypy-svn] r26511 - pypy/dist/pypy/translator/cl Message-ID: <20060428094830.9FD8F100AD@code0.codespeak.net> Author: sanxiyn Date: Fri Apr 28 11:48:27 2006 New Revision: 26511 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: (dialtone, sanxiyn) Exceptions work in Lisp backend! (well, in the most primitive form at least) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Fri Apr 28 11:48:27 2006 @@ -223,7 +223,7 @@ def declare_class(self, cls): # cls is Instance if cls in self.declarations: - return + return self.declarations[cls][0] name = clrepr(cls._name, symbol=True) field_declaration = ['('+clrepr(field, True)+')' for field in cls._fields] field_declaration = " ".join(field_declaration) @@ -238,6 +238,7 @@ methodobj = cls._methods[method] methodobj._method_name = method self.pendinggraphs.append(methodobj) + return name def declare_exception(self, cls): # cls is Instance @@ -246,10 +247,10 @@ return self.declarations[cls][0] name = clrepr(cls._name, symbol=True) if cls._superclass is OBJECT: - exception_declaration = "(define-condition %s () ())" % (name,) + supername = self.declare_class(OBJECT) + exception_declaration = "(define-condition %s (condition %s) ())" % (name, supername) else: - self.declare_exception(cls._superclass) - supername = clrepr(cls._superclass._name, symbol=True) + supername = self.declare_exception(cls._superclass) exception_declaration = "(define-condition %s (%s) ())" % (name, supername) self.declarations[cls] = (name, exception_declaration) return name @@ -407,7 +408,8 @@ self.emit_link(body) for exception in exceptions: yield "(%s ()" % (exception,) - self.emit_link(exceptions[exception]) + for line in self.emit_link(exceptions[exception]): + yield line yield ")" else: # this is for the more general case. The previous special case From auc at codespeak.net Fri Apr 28 11:49:36 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 28 Apr 2006 11:49:36 +0200 (CEST) Subject: [pypy-svn] r26512 - pypy/dist/pypy/objspace Message-ID: <20060428094936.9D39D10079@code0.codespeak.net> Author: auc Date: Fri Apr 28 11:49:35 2006 New Revision: 26512 Modified: pypy/dist/pypy/objspace/logic.py Log: cut crap Modified: pypy/dist/pypy/objspace/logic.py ============================================================================== --- pypy/dist/pypy/objspace/logic.py (original) +++ pypy/dist/pypy/objspace/logic.py Fri Apr 28 11:49:35 2006 @@ -768,6 +768,7 @@ # ---- constraints ---------------- from pypy.objspace.constraint import constraint +all_mms.update(constraint.all_mms) #----- distributors --------------- from pypy.objspace.constraint import distributor @@ -836,8 +837,6 @@ #-- contraint ---- space.setitem(space.builtin.w_dict, space.wrap('make_expression'), space.wrap(constraint.app_make_expression)) - space.setitem(space.builtin.w_dict, space.wrap('make_filter'), - space.wrap(constraint.app_make_filter)) space.setitem(space.builtin.w_dict, space.wrap('AllDistinct'), space.wrap(constraint.app_make_alldistinct)) #-- distributor -- From arigo at codespeak.net Fri Apr 28 12:09:07 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 12:09:07 +0200 (CEST) Subject: [pypy-svn] r26515 - in pypy/dist/pypy/rpython/rctypes: test tool Message-ID: <20060428100907.4303C1008F@code0.codespeak.net> Author: arigo Date: Fri Apr 28 12:09:06 2006 New Revision: 26515 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/tool/util.py Log: A ctypes helper to raise an exception containing the current C-level errno. The next step would be to support it in rctypes. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Fri Apr 28 12:09:06 2006 @@ -3,7 +3,7 @@ """ import py -import sys +import sys, errno import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext, graphof @@ -106,6 +106,21 @@ s2 = ctime(byref(c_long(N))) assert s1.strip() == s2.strip() +def test_open(): + if sys.platform == 'win32': + py.test.skip("Unix only") + open = mylib.open + fd = open("/_rctypes_test_rfunc/this/directory/does/not/exist/at/all!", + 0, 0) + try: + util.setfromerrno() + except OSError, e: + pass + else: + raise AssertionError("util.setfromerrno() should have raised") + assert fd == -1 + assert e.errno == errno.ENOENT + ##def test_callback(): ## assert callback(100) == 103 ## assert pycallback(100) == 103 Modified: pypy/dist/pypy/rpython/rctypes/tool/util.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/util.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/util.py Fri Apr 28 12:09:06 2006 @@ -1,9 +1,16 @@ """ -This is the module 'ctypes.util', copied from ctypes 0.9.9.6. +This is the module 'ctypes.util', copied from ctypes 0.9.9.6, +with an extra setfromerrno() helper. """ import sys, os import ctypes +def setfromerrno(exc=OSError): + """Raise an exception of the given class with the last failed C library + function's errno.""" + ctypes.pythonapi.PyErr_SetFromErrno(ctypes.py_object(exc)) + + # find_library(name) returns the pathname of a library, or None. if os.name == "nt": def find_library(name): From auc at codespeak.net Fri Apr 28 12:16:48 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 28 Apr 2006 12:16:48 +0200 (CEST) Subject: [pypy-svn] r26516 - pypy/dist/pypy/doc/discussion Message-ID: <20060428101648.6F3ED100A4@code0.codespeak.net> Author: auc Date: Fri Apr 28 12:16:47 2006 New Revision: 26516 Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt Log: more api Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/logic-plan.txt (original) +++ pypy/dist/pypy/doc/discussion/logic-plan.txt Fri Apr 28 12:16:47 2006 @@ -218,7 +218,7 @@ def clone(self): """returns a cloned computation space""" - + def merge(self): """ extract solution values from an entailed space @@ -235,6 +235,25 @@ was given to the commit call """ + def fail(self): + """ + mark the space as failed + """ + + """ + constraint stuff : a space acts as/contains a constraint + store which holds : logic variables, constraint variables + (logic vars augmented with finite domains), constraints + """ + + def var(self, domain, name=None): + """ + creates a possibly named contraint variable inside + the space, returns the variable object + """ + + def tell(self, constraint): + """register a constraint""" Programs runing in a computation spaces should not be allowed to alter the top-level space (i.e interaction with the global program state and From nik at codespeak.net Fri Apr 28 12:38:55 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 12:38:55 +0200 (CEST) Subject: [pypy-svn] r26517 - pypy/dist/pypy/rpython/test Message-ID: <20060428103855.A54FD100B6@code0.codespeak.net> Author: nik Date: Fri Apr 28 12:38:52 2006 New Revision: 26517 Modified: pypy/dist/pypy/rpython/test/test_rlist.py Log: enable some skipped tests that pass by now. Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Fri Apr 28 12:38:52 2006 @@ -356,7 +356,6 @@ assert self.ll_to_list(res) == [5, 6, 7, 8, 9] def test_slice(self): - self._skip_oo('tuples of lists') def dummyfn(): l = [5, 6, 7, 8, 9] return l[:2], l[1:4], l[3:] @@ -406,7 +405,6 @@ assert res == 0 def test_setslice(self): - self._skip_oo('tuples of lists') def dummyfn(): l = [10, 9, 8, 7] l[:2] = [6, 5] @@ -1051,7 +1049,6 @@ assert res == 2 def test_type_erase_fixed_size(self): - self._skip_oo('tuples of lists') class A(object): pass class B(object): @@ -1076,7 +1073,6 @@ assert r_A_list.lowleveltype == r_B_list.lowleveltype def test_type_erase_var_size(self): - self._skip_oo('tuples of lists') class A(object): pass class B(object): From nik at codespeak.net Fri Apr 28 12:50:47 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Fri, 28 Apr 2006 12:50:47 +0200 (CEST) Subject: [pypy-svn] r26519 - in pypy/dist/pypy: jit/timeshifter/test rpython rpython/lltypesystem rpython/ootypesystem rpython/rctypes translator/goal Message-ID: <20060428105047.8977D1009F@code0.codespeak.net> Author: nik Date: Fri Apr 28 12:50:36 2006 New Revision: 26519 Modified: pypy/dist/pypy/jit/timeshifter/test/test_timeshift.py pypy/dist/pypy/rpython/lltypesystem/rstr.py pypy/dist/pypy/rpython/ootypesystem/rstr.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rfloat.py pypy/dist/pypy/rpython/rmodel.py pypy/dist/pypy/rpython/rstr.py pypy/dist/pypy/translator/goal/targetpypymain.py Log: actually have separate StringRepr classes for lltypes and ootypes. this fixes the strange test failures due to the double extension of StringRepr in both type systems. Modified: pypy/dist/pypy/jit/timeshifter/test/test_timeshift.py ============================================================================== --- pypy/dist/pypy/jit/timeshifter/test/test_timeshift.py (original) +++ pypy/dist/pypy/jit/timeshifter/test/test_timeshift.py Fri Apr 28 12:50:36 2006 @@ -9,7 +9,8 @@ from pypy.jit.llabstractinterp.test.test_llabstractinterp import summary from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.objectmodel import hint, keepalive_until_here -from pypy.rpython import rgenop, rstr +from pypy.rpython import rgenop +from pypy.rpython.lltypesystem import rstr from pypy.annotation import model as annmodel from pypy.rpython.llinterp import LLInterpreter from pypy.objspace.flow.model import checkgraph Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rstr.py Fri Apr 28 12:50:36 2006 @@ -1,14 +1,124 @@ -from pypy.rpython.rstr import string_repr, StringRepr, STR, AbstractStringIteratorRepr, \ - ll_strconcat, instance_str_prefix, instance_str_suffix, list_str_open_bracket, ll_join, list_str_sep, list_str_close_bracket +from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr, \ + ll_strconcat from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, Signed -class __extend__(StringRepr): +class StringRepr(AbstractStringRepr): lowleveltype = Ptr(STR) + def __init__(self, *args): + AbstractStringRepr.__init__(self, *args) + self.ll_strip = ll_strip + self.ll_upper = ll_upper + self.ll_lower = ll_lower + self.ll_join = ll_join + def make_iterator_repr(self): return string_iterator_repr + def can_ll_be_null(self, s_value): + if self is string_repr: + return s_value.can_be_none() + else: + return True # for CharRepr/UniCharRepr subclasses, + # where NULL is always valid: it is chr(0) + +def ll_strip(s, ch, left, right): + s_len = len(s.chars) + if s_len == 0: + return emptystr + lpos = 0 + rpos = s_len - 1 + if left: + while lpos < rpos and s.chars[lpos] == ch: + lpos += 1 + if right: + while lpos < rpos and s.chars[rpos] == ch: + rpos -= 1 + r_len = rpos - lpos + 1 + result = malloc(STR, r_len) + i = 0 + j = lpos + while i < r_len: + result.chars[i] = s.chars[j] + i += 1 + j += 1 + return result + +def ll_upper(s): + s_chars = s.chars + s_len = len(s_chars) + if s_len == 0: + return emptystr + i = 0 + result = malloc(STR, s_len) + while i < s_len: + ch = s_chars[i] + if 'a' <= ch <= 'z': + ch = chr(ord(ch) - 32) + result.chars[i] = ch + i += 1 + return result + +def ll_lower(s): + s_chars = s.chars + s_len = len(s_chars) + if s_len == 0: + return emptystr + i = 0 + result = malloc(STR, s_len) + while i < s_len: + ch = s_chars[i] + if 'A' <= ch <= 'Z': + ch = chr(ord(ch) + 32) + result.chars[i] = ch + i += 1 + return result + +def ll_join(s, length, items): + s_chars = s.chars + s_len = len(s_chars) + num_items = length + if num_items == 0: + return emptystr + itemslen = 0 + i = 0 + while i < num_items: + itemslen += len(items[i].chars) + i += 1 + result = malloc(STR, itemslen + s_len * (num_items - 1)) + res_chars = result.chars + res_index = 0 + i = 0 + item_chars = items[i].chars + item_len = len(item_chars) + j = 0 + while j < item_len: + res_chars[res_index] = item_chars[j] + j += 1 + res_index += 1 + i += 1 + while i < num_items: + j = 0 + while j < s_len: + res_chars[res_index] = s_chars[j] + j += 1 + res_index += 1 + + item_chars = items[i].chars + item_len = len(item_chars) + j = 0 + while j < item_len: + res_chars[res_index] = item_chars[j] + j += 1 + res_index += 1 + i += 1 + return result + +string_repr = StringRepr() + +emptystr = string_repr.convert_const("") + class StringIteratorRepr(AbstractStringIteratorRepr): @@ -35,3 +145,14 @@ return chars[index] string_iterator_repr = StringIteratorRepr() + + +# these should be in rclass, but circular imports prevent (also it's +# not that insane that a string constant is built in this file). + +instance_str_prefix = string_repr.convert_const("<") +instance_str_suffix = string_repr.convert_const(" object>") + +list_str_open_bracket = string_repr.convert_const("[") +list_str_close_bracket = string_repr.convert_const("]") +list_str_sep = string_repr.convert_const(", ") Modified: pypy/dist/pypy/rpython/ootypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rstr.py Fri Apr 28 12:50:36 2006 @@ -1,14 +1,15 @@ -from pypy.rpython.rstr import string_repr, StringRepr, STR, AbstractStringIteratorRepr +from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr from pypy.rpython.lltypesystem.lltype import Ptr from pypy.rpython.ootypesystem.ootype import Signed, Record -class __extend__(StringRepr): +class StringRepr(AbstractStringRepr): lowleveltype = Ptr(STR) def make_iterator_repr(self): return string_iterator_repr +string_repr = StringRepr() class StringIteratorRepr(AbstractStringIteratorRepr): Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Fri Apr 28 12:50:36 2006 @@ -1,6 +1,6 @@ from pypy.rpython.rmodel import inputconst from pypy.rpython.lltypesystem import lltype -from pypy.rpython.rstr import StringRepr +from pypy.rpython.rstr import AbstractStringRepr from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO from pypy.rpython.rctypes.rarray import ArrayRepr @@ -56,7 +56,7 @@ self.setstring(hop.llops, v_char_p, v_value) -class __extend__(pairtype(StringRepr, CCharPRepr)): +class __extend__(pairtype(AbstractStringRepr, CCharPRepr)): def convert_from_to((r_from, r_to), v, llops): # r_from could be char_repr: first convert it to string_repr v = llops.convertvar(v, r_from, string_repr) Modified: pypy/dist/pypy/rpython/rfloat.py ============================================================================== --- pypy/dist/pypy/rpython/rfloat.py (original) +++ pypy/dist/pypy/rpython/rfloat.py Fri Apr 28 12:50:36 2006 @@ -4,9 +4,8 @@ Signed, Unsigned, Bool, Float, Void, pyobjectptr from pypy.rpython.error import TyperError from pypy.rpython.rmodel import FloatRepr -from pypy.rpython.rmodel import IntegerRepr, BoolRepr, StringRepr +from pypy.rpython.rmodel import IntegerRepr, BoolRepr, AbstractStringRepr from pypy.rpython.robject import PyObjRepr, pyobj_repr -from pypy.rpython.rstr import string_repr from pypy.rpython.rmodel import log import math @@ -84,7 +83,7 @@ def rtype_ge(_, hop): return _rtype_compare_template(hop, 'ge') -class __extend__(pairtype(StringRepr, FloatRepr)): +class __extend__(pairtype(AbstractStringRepr, FloatRepr)): def rtype_mod(_, hop): rstr = hop.rtyper.type_system.rstr return rstr.do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) @@ -140,6 +139,8 @@ from pypy.rpython.module.ll_strtod import ll_strtod_formatd return ll_strtod_formatd(percent_f, f) +# XXX this will need to be type system independent +from pypy.rpython.lltypesystem.rstr import string_repr percent_f = string_repr.convert_const("%f") TAKE_NEXT = float(2**31) Modified: pypy/dist/pypy/rpython/rmodel.py ============================================================================== --- pypy/dist/pypy/rpython/rmodel.py (original) +++ pypy/dist/pypy/rpython/rmodel.py Fri Apr 28 12:50:36 2006 @@ -327,10 +327,10 @@ from pypy.rpython.rint import signed_repr self.as_int = signed_repr -class StringRepr(Repr): +class AbstractStringRepr(Repr): pass -class CharRepr(StringRepr): +class CharRepr(AbstractStringRepr): lowleveltype = Char class UniCharRepr(Repr): Modified: pypy/dist/pypy/rpython/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/rstr.py (original) +++ pypy/dist/pypy/rpython/rstr.py Fri Apr 28 12:50:36 2006 @@ -3,7 +3,7 @@ from pypy.annotation import model as annmodel from pypy.rpython.error import TyperError from pypy.rpython.rmodel import IntegerRepr, IteratorRepr -from pypy.rpython.rmodel import StringRepr, CharRepr, inputconst, UniCharRepr +from pypy.rpython.rmodel import AbstractStringRepr, CharRepr, inputconst, UniCharRepr from pypy.rpython.rarithmetic import _hash_string from pypy.rpython.robject import PyObjRepr, pyobj_repr from pypy.rpython.lltypesystem.rtuple import TupleRepr # XXX type system! @@ -50,12 +50,12 @@ return self.__class__, CONST_STR_CACHE = WeakValueDictionary() -string_repr = StringRepr() +#string_repr = StringRepr() char_repr = CharRepr() unichar_repr = UniCharRepr() -class __extend__(StringRepr): +class __extend__(AbstractStringRepr): def convert_const(self, value): if value is None: @@ -82,43 +82,42 @@ def get_ll_fasthash_function(self): return ll_strfasthash - def can_ll_be_null(self, s_value): - if self is string_repr: - return s_value.can_be_none() - else: - return True # for CharRepr/UniCharRepr subclasses, - # where NULL is always valid: it is chr(0) - def rtype_len(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, = hop.inputargs(string_repr) return hop.gendirectcall(ll_strlen, v_str) def rtype_is_true(self, hop): s_str = hop.args_s[0] if s_str.can_be_None: + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, = hop.inputargs(string_repr) return hop.gendirectcall(ll_str_is_true, v_str) else: # defaults to checking the length - return super(StringRepr, self).rtype_is_true(hop) + return super(AbstractStringRepr, self).rtype_is_true(hop) def rtype_ord(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, = hop.inputargs(string_repr) c_zero = inputconst(Signed, 0) v_chr = hop.gendirectcall(ll_stritem_nonneg, v_str, c_zero) return hop.genop('cast_char_to_int', [v_chr], resulttype=Signed) def rtype_method_startswith(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, v_value = hop.inputargs(string_repr, string_repr) hop.exception_cannot_occur() return hop.gendirectcall(ll_startswith, v_str, v_value) def rtype_method_endswith(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, v_value = hop.inputargs(string_repr, string_repr) hop.exception_cannot_occur() return hop.gendirectcall(ll_endswith, v_str, v_value) def rtype_method_find(_, hop, reverse=False): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str = hop.inputarg(string_repr, arg=0) if hop.args_r[1] == char_repr: v_value = hop.inputarg(char_repr, arg=1) @@ -144,12 +143,13 @@ def rtype_method_rfind(self, hop): return self.rtype_method_find(hop, reverse=True) - def rtype_method_strip(_, hop, left=True, right=True): + def rtype_method_strip(self, hop, left=True, right=True): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str = hop.inputarg(string_repr, arg=0) v_char = hop.inputarg(char_repr, arg=1) v_left = hop.inputconst(Bool, left) v_right = hop.inputconst(Bool, right) - return hop.gendirectcall(ll_strip, v_str, v_char, v_left, v_right) + return hop.gendirectcall(self.ll_strip, v_str, v_char, v_left, v_right) def rtype_method_lstrip(self, hop): return self.rtype_method_strip(hop, left=True, right=False) @@ -157,18 +157,21 @@ def rtype_method_rstrip(self, hop): return self.rtype_method_strip(hop, left=False, right=True) - def rtype_method_upper(_, hop): + def rtype_method_upper(self, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, = hop.inputargs(string_repr) hop.exception_cannot_occur() - return hop.gendirectcall(ll_upper, v_str) + return hop.gendirectcall(self.ll_upper, v_str) - def rtype_method_lower(_, hop): + def rtype_method_lower(self, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, = hop.inputargs(string_repr) hop.exception_cannot_occur() - return hop.gendirectcall(ll_lower, v_str) + return hop.gendirectcall(self.ll_lower, v_str) - def rtype_method_join(_, hop): + def rtype_method_join(self, hop): hop.exception_cannot_occur() + string_repr = hop.rtyper.type_system.rstr.string_repr if hop.s_result.is_constant(): return inputconst(string_repr, hop.s_result.const) r_lst = hop.args_r[1] @@ -189,12 +192,13 @@ return hop.gendirectcall(llfn, v_length, v_items) else: if r_lst.item_repr == string_repr: - llfn = ll_join + llfn = self.ll_join else: raise TyperError("sep.join() of non-string list: %r" % r_lst) return hop.gendirectcall(llfn, v_str, v_length, v_items) def rtype_method_split(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, v_chr = hop.inputargs(string_repr, char_repr) cLIST = hop.inputconst(Void, hop.r_result.lowleveltype.TO) hop.exception_cannot_occur() @@ -203,12 +207,14 @@ def rtype_method_replace(_, hop): if not (hop.args_r[1] == char_repr and hop.args_r[2] == char_repr): raise TyperError, 'replace only works for char args' + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, v_c1, v_c2 = hop.inputargs(string_repr, char_repr, char_repr) hop.exception_cannot_occur() return hop.gendirectcall(ll_replace_chr_chr, v_str, v_c1, v_c2) def rtype_int(_, hop): hop.has_implicit_exception(ValueError) # record that we know about it + string_repr = hop.rtyper.type_system.rstr.string_repr if hop.nb_args == 1: v_str, = hop.inputargs(string_repr) c_base = inputconst(Signed, 10) @@ -223,8 +229,9 @@ def ll_str(self, s): return s -class __extend__(pairtype(StringRepr, IntegerRepr)): +class __extend__(pairtype(AbstractStringRepr, IntegerRepr)): def rtype_getitem(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, v_index = hop.inputargs(string_repr, Signed) if hop.has_implicit_exception(IndexError): if hop.args_s[1].nonneg: @@ -243,9 +250,10 @@ return do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) -class __extend__(pairtype(StringRepr, SliceRepr)): +class __extend__(pairtype(AbstractStringRepr, SliceRepr)): def rtype_getitem((_, r_slic), hop): + string_repr = hop.rtyper.type_system.rstr.string_repr if r_slic == startonly_slice_repr: v_str, v_start = hop.inputargs(string_repr, startonly_slice_repr) return hop.gendirectcall(ll_stringslice_startonly, v_str, v_start) @@ -258,40 +266,47 @@ raise TyperError(r_slic) -class __extend__(pairtype(StringRepr, StringRepr)): +class __extend__(pairtype(AbstractStringRepr, AbstractStringRepr)): def rtype_add(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) return hop.gendirectcall(ll_strconcat, v_str1, v_str2) rtype_inplace_add = rtype_add def rtype_eq(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) return hop.gendirectcall(ll_streq, v_str1, v_str2) def rtype_ne(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) vres = hop.gendirectcall(ll_streq, v_str1, v_str2) return hop.genop('bool_not', [vres], resulttype=Bool) def rtype_lt(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) vres = hop.gendirectcall(ll_strcmp, v_str1, v_str2) return hop.genop('int_lt', [vres, hop.inputconst(Signed, 0)], resulttype=Bool) def rtype_le(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) vres = hop.gendirectcall(ll_strcmp, v_str1, v_str2) return hop.genop('int_le', [vres, hop.inputconst(Signed, 0)], resulttype=Bool) def rtype_ge(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) vres = hop.gendirectcall(ll_strcmp, v_str1, v_str2) return hop.genop('int_ge', [vres, hop.inputconst(Signed, 0)], resulttype=Bool) def rtype_gt(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str1, v_str2 = hop.inputargs(string_repr, string_repr) vres = hop.gendirectcall(ll_strcmp, v_str1, v_str2) return hop.genop('int_gt', [vres, hop.inputconst(Signed, 0)], @@ -300,8 +315,9 @@ def rtype_mod(_, hop): return do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) -class __extend__(pairtype(StringRepr, CharRepr)): +class __extend__(pairtype(AbstractStringRepr, CharRepr)): def rtype_contains(_, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, v_chr = hop.inputargs(string_repr, char_repr) return hop.gendirectcall(ll_contains, v_str, v_chr) @@ -373,6 +389,7 @@ else: raise TyperError, "%%%s is not RPython" % (code, ) else: + from pypy.rpython.lltypesystem.rstr import string_repr vchunk = inputconst(string_repr, thing) i = inputconst(Signed, i) hop.genop('setarrayitem', [vtemp, i, vchunk]) @@ -381,7 +398,7 @@ return hop.gendirectcall(ll_join_strs, size, vtemp) -class __extend__(pairtype(StringRepr, TupleRepr)): +class __extend__(pairtype(AbstractStringRepr, TupleRepr)): def rtype_mod(_, hop): r_tuple = hop.args_r[1] v_tuple = hop.args_v[1] @@ -515,20 +532,22 @@ # # _________________________ Conversions _________________________ -class __extend__(pairtype(CharRepr, StringRepr)): +class __extend__(pairtype(CharRepr, AbstractStringRepr)): def convert_from_to((r_from, r_to), v, llops): + string_repr = llops.rtyper.type_system.rstr.string_repr if r_from == char_repr and r_to == string_repr: return llops.gendirectcall(ll_chr2str, v) return NotImplemented -class __extend__(pairtype(StringRepr, CharRepr)): +class __extend__(pairtype(AbstractStringRepr, CharRepr)): def convert_from_to((r_from, r_to), v, llops): + string_repr = llops.rtyper.type_system.rstr.string_repr if r_from == string_repr and r_to == char_repr: c_zero = inputconst(Signed, 0) return llops.gendirectcall(ll_stritem_nonneg, v, c_zero) return NotImplemented -class __extend__(pairtype(PyObjRepr, StringRepr)): +class __extend__(pairtype(PyObjRepr, AbstractStringRepr)): def convert_from_to((r_from, r_to), v, llops): v_len = llops.gencapicall('PyString_Size', [v], resulttype=Signed) cstr = inputconst(Void, STR) @@ -538,11 +557,13 @@ v_chars = llops.genop('getsubstruct', [v_result, cchars], resulttype=Ptr(STR.chars)) llops.gencapicall('PyString_ToLLCharArray', [v, v_chars]) + string_repr = llops.rtyper.type_system.rstr.string_repr v_result = llops.convertvar(v_result, string_repr, r_to) return v_result -class __extend__(pairtype(StringRepr, PyObjRepr)): +class __extend__(pairtype(AbstractStringRepr, PyObjRepr)): def convert_from_to((r_from, r_to), v, llops): + string_repr = llops.rtyper.type_system.rstr.string_repr v = llops.convertvar(v, r_from, string_repr) cchars = inputconst(Void, "chars") v_chars = llops.genop('getsubstruct', [v, cchars], @@ -842,100 +863,6 @@ i = e return -1 -emptystr = string_repr.convert_const("") - -def ll_strip(s, ch, left, right): - s_len = len(s.chars) - if s_len == 0: - return emptystr - lpos = 0 - rpos = s_len - 1 - if left: - while lpos < rpos and s.chars[lpos] == ch: - lpos += 1 - if right: - while lpos < rpos and s.chars[rpos] == ch: - rpos -= 1 - r_len = rpos - lpos + 1 - result = malloc(STR, r_len) - i = 0 - j = lpos - while i < r_len: - result.chars[i] = s.chars[j] - i += 1 - j += 1 - return result - -def ll_upper(s): - s_chars = s.chars - s_len = len(s_chars) - if s_len == 0: - return emptystr - i = 0 - result = malloc(STR, s_len) - while i < s_len: - ch = s_chars[i] - if 'a' <= ch <= 'z': - ch = chr(ord(ch) - 32) - result.chars[i] = ch - i += 1 - return result - -def ll_lower(s): - s_chars = s.chars - s_len = len(s_chars) - if s_len == 0: - return emptystr - i = 0 - result = malloc(STR, s_len) - while i < s_len: - ch = s_chars[i] - if 'A' <= ch <= 'Z': - ch = chr(ord(ch) + 32) - result.chars[i] = ch - i += 1 - return result - -def ll_join(s, length, items): - s_chars = s.chars - s_len = len(s_chars) - num_items = length - if num_items == 0: - return emptystr - itemslen = 0 - i = 0 - while i < num_items: - itemslen += len(items[i].chars) - i += 1 - result = malloc(STR, itemslen + s_len * (num_items - 1)) - res_chars = result.chars - res_index = 0 - i = 0 - item_chars = items[i].chars - item_len = len(item_chars) - j = 0 - while j < item_len: - res_chars[res_index] = item_chars[j] - j += 1 - res_index += 1 - i += 1 - while i < num_items: - j = 0 - while j < s_len: - res_chars[res_index] = s_chars[j] - j += 1 - res_index += 1 - - item_chars = items[i].chars - item_len = len(item_chars) - j = 0 - while j < item_len: - res_chars[res_index] = item_chars[j] - j += 1 - res_index += 1 - i += 1 - return result - def ll_join_strs(length, items): num_items = length itemslen = 0 @@ -1108,6 +1035,7 @@ class AbstractStringIteratorRepr(IteratorRepr): def newiter(self, hop): + string_repr = hop.rtyper.type_system.rstr.string_repr v_str, = hop.inputargs(string_repr) return hop.gendirectcall(self.ll_striter, v_str) @@ -1117,13 +1045,3 @@ hop.exception_is_here() return hop.gendirectcall(self.ll_strnext, v_iter) - -# these should be in rclass, but circular imports prevent (also it's -# not that insane that a string constant is built in this file). - -instance_str_prefix = string_repr.convert_const("<") -instance_str_suffix = string_repr.convert_const(" object>") - -list_str_open_bracket = string_repr.convert_const("[") -list_str_close_bracket = string_repr.convert_const("]") -list_str_sep = string_repr.convert_const(", ") Modified: pypy/dist/pypy/translator/goal/targetpypymain.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpypymain.py (original) +++ pypy/dist/pypy/translator/goal/targetpypymain.py Fri Apr 28 12:50:36 2006 @@ -77,7 +77,7 @@ return entry_point, [SomeString()], PyPyAnnotatorPolicy() def get_llinterp_args(): - from pypy.rpython import rstr + from pypy.rpython.lltypesystem import rstr ll_str = rstr.string_repr.convert_const("app_example.py") return [ll_str] From ac at codespeak.net Fri Apr 28 12:51:42 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 28 Apr 2006 12:51:42 +0200 (CEST) Subject: [pypy-svn] r26520 - in pypy/dist/pypy: module/_socket rpython/rctypes/socketmodule Message-ID: <20060428105142.662AE1009F@code0.codespeak.net> Author: ac Date: Fri Apr 28 12:51:41 2006 New Revision: 26520 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: (aleale, arre) Now _socket annotates (but does not specialize!) Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Fri Apr 28 12:51:41 2006 @@ -182,16 +182,21 @@ if not hostent: raise w_get_socketherror(space, None, _c.h_errno.value) aliases = [] - for alias in hostent.contents.h_aliases: - if alias is None: - break + i = 0 + h_aliases = hostent.contents.h_aliases + alias = h_aliases[0] + while alias is not None: aliases.append(space.wrap(alias)) + i += 1 + alias = h_aliases[i] address_list = [] h_addr_list = ctypes.cast(hostent.contents.h_addr_list, ctypes.POINTER(ctypes.POINTER(_c.in_addr))) - for addr in h_addr_list: - if not addr: - break + i = 0 + addr = h_addr_list[0] + while addr is not None: address_list.append(space.wrap(_c.inet_ntoa(addr.contents))) + i += 1 + addr = h_addr_list[i] return space.newtuple([space.wrap(hostent.contents.h_name), space.newlist(aliases), space.newlist(address_list)]) @@ -290,7 +295,7 @@ sock_type = space.int_w(w_sock_type) if w_proto is not None: proto = space.int_w(w_proto) - result = ctypes.ARRAY(ctypes.c_int, 2) + result = _c.socketpair_t() error = _c.socketpair(family, sock_type, proto, result) if error < 0: raise w_get_socketerror(space, None, _c.errno.value) @@ -673,7 +678,7 @@ if res: _c.freeaddrinfo(res) if error: - raise w_get_socketgaierror(None, error) + raise w_get_socketgaierror(space, None, error) return space.newtuple([space.wrap(hostbuf.value), space.wrap(portbuf.value)]) getnameinfo.unwrap_spec = [ObjSpace, W_Root, int] @@ -775,11 +780,11 @@ if retval != 0: raise w_get_socketgaierror(space, None, retval) addrinfo = res.contents - addrlen = addrinfo.ai_addrlen - caddr_ptr = ctypes.create_string_buffer(addrlen) - _c.memcpy(caddr_ptr, addrinfo.ai_addr, addrlen) + addrlen = addrinfo.ai_addrlen + caddr_buf = ctypes.create_string_buffer(int(addrlen)) # XXX forcing a long to an int + _c.memcpy(caddr_buf, addrinfo.ai_addr, addrlen) - sockaddr_ptr = ctypes.cast(caddr_ptr, _c.sockaddr_ptr) + sockaddr_ptr = ctypes.cast(caddr_buf, _c.sockaddr_ptr) return sockaddr_ptr, addrlen else: @@ -794,10 +799,10 @@ """ peeraddr = _c.pointer(_c.sockaddr()) peeraddrlen = _c.socklen_t(_c.sockaddr_size) - newfd = _c.socketaccept(self._fd, peeraddr, + newfd = _c.socketaccept(self.fd, peeraddr, _c.pointer(peeraddrlen)) if newfd < 0: - raise w_get_socketerror(_c.errno.value) + raise w_get_socketerror(space, None, _c.errno.value) newsocket = Socket(space, newfd, self.family, self.type, self.proto) return space.newtuple([newsocket, w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto)]) accept.unwrap_spec = ['self', ObjSpace] @@ -810,7 +815,7 @@ sockets the address is a tuple (ifname, proto [,pkttype [,hatype]]) """ caddr_ptr, caddr_len = self._getsockaddr(space, w_addr) - res = _c.socketbind(self._fd, caddr_ptr, caddr_len) + res = _c.socketbind(self.fd, caddr_ptr, caddr_len) if res < 0: raise w_get_socketerror(space, None, _c.errno.value) bind.unwrap_spec = ['self', ObjSpace, W_Root] @@ -834,12 +839,12 @@ Connect the socket to a remote address. For IP sockets, the address is a pair (host, port). """ - errno = self.connect_ex(space, w_addr) + errno = self._connect_ex(space, w_addr) if errno: raise w_get_socketerror(space, None, errno) connect.unwrap_spec = ['self', ObjSpace, W_Root] - def connect_ex(self, space, w_addr): + def _connect_ex(self, space, w_addr): """connect_ex(address) -> errno This is like connect(address), but returns an error code (the errno value) @@ -854,6 +859,9 @@ pass return errno return 0 + + def connect_ex(self, space, w_addr): + return space.wrap(self._connect_ex(space, w_addr)) connect_ex.unwrap_spec = ['self', ObjSpace, W_Root] def dup(self, space): @@ -921,7 +929,7 @@ c_buffersize = _c.c_int(space.int_w(w_buffersize)) buffer = ctypes.create_string_buffer(c_buffersize.value) err = _c.socketgetsockopt(self.fd, level, option, buffer, - ctypes.POINTER(c_buffersize)) + ctypes.pointer(c_buffersize)) if err: raise w_get_socketerror(space, None, _c.errno.value) return space.wrap(buffer[:c_buffersize.value]) @@ -1038,10 +1046,7 @@ setblocking(True) is equivalent to settimeout(None); setblocking(False) is equivalent to settimeout(0.0). """ - if flag: - self.settimeout(space, None) - else: - self.settimeout(space, 0.0) + setblocking(self.fd, bool(flag)) setblocking.unwrap_spec = ['self', ObjSpace, int] def setsockopt(self, space, level, option, w_value): @@ -1089,7 +1094,7 @@ raise OperationError(space.w_ValueError, space.wrap("Timeout value out of range")) self.timeout = timeout - self.setblocking(space, timeout < 0.0) + setblocking(self.fd, timeout < 0.0) settimeout.unwrap_spec = ['self', ObjSpace, W_Root] def shutdown(self, space, how): Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Fri Apr 28 12:51:41 2006 @@ -350,6 +350,7 @@ memcpy.argtypes = [c_void_p, c_void_p, size_t] memcpy.restype = c_void_p +socketpair_t = ARRAY(c_int, 2) socketpair = socketdll.socketpair socketpair.argtypes = [c_int, c_int, c_int, POINTER(c_int)] socketpair.restype = c_int From ericvrp at codespeak.net Fri Apr 28 12:59:17 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 28 Apr 2006 12:59:17 +0200 (CEST) Subject: [pypy-svn] r26522 - pypy/branch/picklesupport Message-ID: <20060428105917.3E15C1009D@code0.codespeak.net> Author: ericvrp Date: Fri Apr 28 12:59:13 2006 New Revision: 26522 Added: pypy/branch/picklesupport/ - copied from r26521, pypy/dist/pypy/ Log: working on pickling support (similar to stackless pickling support) From ericvrp at codespeak.net Fri Apr 28 13:01:41 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 28 Apr 2006 13:01:41 +0200 (CEST) Subject: [pypy-svn] r26523 - in pypy/branch/picklesupport: interpreter interpreter/test module/_pickle_support Message-ID: <20060428110141.2CC2210095@code0.codespeak.net> Author: ericvrp Date: Fri Apr 28 13:01:28 2006 New Revision: 26523 Modified: pypy/branch/picklesupport/interpreter/function.py pypy/branch/picklesupport/interpreter/module.py pypy/branch/picklesupport/interpreter/test/test_pickle.py pypy/branch/picklesupport/interpreter/typedef.py pypy/branch/picklesupport/module/_pickle_support/__init__.py pypy/branch/picklesupport/module/_pickle_support/maker.py Log: moving to snake for translating tests Modified: pypy/branch/picklesupport/interpreter/function.py ============================================================================== --- pypy/branch/picklesupport/interpreter/function.py (original) +++ pypy/branch/picklesupport/interpreter/function.py Fri Apr 28 13:01:28 2006 @@ -169,7 +169,7 @@ def descr_function_repr(self): return self.getrepr(self.space, 'function %s' % (self.name,)) - def descr__reduce__(self, space): + def descr_function__reduce__(self, space): from pypy.interpreter.mixedmodule import MixedModule w_mod = space.getbuiltinmodule('_pickle_support') mod = space.interp_w(MixedModule, w_mod) @@ -181,7 +181,6 @@ w_closure = space.newtuple([w(cell) for cell in self.closure]) tup = [ w(self.code), - #space.newdict([]), #XXX because pickle.py has no _pickle_moduledict yet... self.w_func_globals, w(self.name), space.newtuple(self.defs_w), @@ -390,6 +389,19 @@ return space.w_False return space.eq(self.w_function, other.w_function) + def descr_method__reduce__(self, space): + from pypy.interpreter.mixedmodule import MixedModule + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + new_inst = mod.get('method_new') + w = space.wrap + w_instance = self.w_instance or space.w_None + if space.is_w( self.w_class, space.w_None ): + tup = [self.w_function, w_instance] + else: + tup = [self.w_function, w_instance, self.w_class] + return space.newtuple([new_inst, space.newtuple(tup)]) + class StaticMethod(Wrappable): """A static method. Note that there is one class staticmethod at app-level too currently; this is only used for __new__ methods.""" Modified: pypy/branch/picklesupport/interpreter/module.py ============================================================================== --- pypy/branch/picklesupport/interpreter/module.py (original) +++ pypy/branch/picklesupport/interpreter/module.py Fri Apr 28 13:01:28 2006 @@ -35,3 +35,24 @@ w_doc = space.w_None space.setitem(self.w_dict, space.new_interned_str('__name__'), w_name) space.setitem(self.w_dict, space.new_interned_str('__doc__'), w_doc) + + def descr__reduce__(self, space): + w_name = space.finditem(self.w_dict, space.wrap('__name__')) + if (w_name is None or + not space.is_true(space.isinstance(w_name, space.w_str))): + # maybe raise exception here (XXX this path is untested) + return space.w_None + w_modules = space.sys.get('modules') + if space.finditem(w_modules, w_name) is None: + #not imported case + from pypy.interpreter.mixedmodule import MixedModule + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + new_inst = mod.get('module_new') + return space.newtuple([new_inst, space.newtuple([w_name, + self.getdict()]), + ]) + #already imported case + w_import = space.builtin.get('__import__') + return space.newtuple([w_import, space.newtuple([w_name])]) + Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py ============================================================================== --- pypy/branch/picklesupport/interpreter/test/test_pickle.py (original) +++ pypy/branch/picklesupport/interpreter/test/test_pickle.py Fri Apr 28 13:01:28 2006 @@ -24,21 +24,40 @@ assert func is result del sys.modules['mod'] + def test_pickle_not_imported_module(self): + import new + mod = new.module('mod') + mod.__dict__['a'] = 1 + import pickle + pckl = pickle.dumps(mod) + result = pickle.loads(pckl) + assert mod.__name__ == result.__name__ + assert mod.__dict__ == result.__dict__ + #print mod.__dict__ + def test_pickle_builtin_func(self): import pickle pckl = pickle.dumps(map) result = pickle.loads(pckl) assert map is result - def test_pickle_nested_func(self): - skip("work in progress") + def test_pickle_non_top_reachable_func(self): def func(): return 42 + global a + a = 42 + del globals()['test_pickle_non_top_reachable_func'] import pickle pckl = pickle.dumps(func) result = pickle.loads(pckl) - assert func == result - + assert func.func_name == result.func_name + assert func.func_closure == result.func_closure + assert func.func_code == result.func_code + assert func.func_defaults == result.func_defaults + assert func.func_dict == result.func_dict + assert func.func_doc == result.func_doc + assert func.func_globals == result.func_globals + def test_pickle_cell(self): def g(): x = [42] @@ -83,25 +102,26 @@ result = pickle.loads(pckl) assert tb == result - def test_pickle_module(self): #XXX this passes for the wrong reason! - skip("work in progress") - def f(): - pass + def test_pickle_module(self): import pickle - mod = f.__module__ #XXX returns a string? + mod = pickle pckl = pickle.dumps(mod) result = pickle.loads(pckl) - assert mod == result + assert mod is result - def test_pickle_moduledict(self): #XXX this test is not correct! - skip("work in progress") - def f(): - pass + def test_pickle_moduledict(self): import pickle - modedict = f.__module__.__dict__ + moddict = pickle.__dict__ pckl = pickle.dumps(moddict) result = pickle.loads(pckl) - assert mod == result + assert moddict is result + + def test_pickle_bltins_module(self): + import pickle + mod = __builtins__ + pckl = pickle.dumps(mod) + result = pickle.loads(pckl) + assert mod is result def test_pickle_iter(self): skip("work in progress") @@ -127,9 +147,14 @@ def test_pickle_enum(self): skip("work in progress") - - def test_pickle_enumfactory(self): - skip("work in progress") + import pickle + e = enumerate([]) + pckl = pickle.dumps(e) + result = pickle.loads(pckl) + assert e == result + + #def test_pickle_enumfactory(self): + # skip("work in progress") def test_pickle_sequenceiter(self): ''' @@ -144,7 +169,7 @@ result = pickle.loads(pckl) assert liter == result - def test_pickle_rangeiter(self): + def test_pickle_xrangeiter(self): skip("work in progress") import pickle riter = iter(xrange(5)) Modified: pypy/branch/picklesupport/interpreter/typedef.py ============================================================================== --- pypy/branch/picklesupport/interpreter/typedef.py (original) +++ pypy/branch/picklesupport/interpreter/typedef.py Fri Apr 28 13:01:28 2006 @@ -450,6 +450,8 @@ __new__ = interp2app(Module.descr_module__new__.im_func, unwrap_spec=[ObjSpace, W_Root, Arguments]), __init__ = interp2app(Module.descr_module__init__), + __reduce__ = interp2app(Module.descr__reduce__, + unwrap_spec=['self', ObjSpace]), __dict__ = GetSetProperty(descr_get_dict, cls=Module), # module dictionaries are readonly attributes __doc__ = 'module(name[, doc])\n\nCreate a module object.\nThe name must be a string; the optional doc argument can have any type.' ) @@ -483,7 +485,7 @@ unwrap_spec=['self', Arguments]), __get__ = interp2app(descr_function_get), __repr__ = interp2app(Function.descr_function_repr), - __reduce__ = interp2app(Function.descr__reduce__, + __reduce__ = interp2app(Function.descr_function__reduce__, unwrap_spec=['self', ObjSpace]), func_code = getset_func_code, func_doc = getset_func_doc, @@ -510,7 +512,9 @@ __getattribute__ = interp2app(Method.descr_method_getattribute), __eq__ = interp2app(Method.descr_method_eq), __ne__ = descr_generic_ne, - __repr__ = interp2app(Method.descr_method_repr), + __repr__ = interp2app(Method.descr_method_repr), + __reduce__ = interp2app(Method.descr_method__reduce__, + unwrap_spec=['self', ObjSpace]), # XXX getattribute/setattribute etc.pp ) Modified: pypy/branch/picklesupport/module/_pickle_support/__init__.py ============================================================================== --- pypy/branch/picklesupport/module/_pickle_support/__init__.py (original) +++ pypy/branch/picklesupport/module/_pickle_support/__init__.py Fri Apr 28 13:01:28 2006 @@ -7,7 +7,9 @@ } interpleveldefs = { - 'cell_new': 'maker.cell_new', - 'code_new': 'maker.code_new', - 'func_new': 'maker.func_new', + 'cell_new' : 'maker.cell_new', + 'code_new' : 'maker.code_new', + 'func_new' : 'maker.func_new', + 'module_new' : 'maker.module_new', + 'method_new' : 'maker.method_new', } Modified: pypy/branch/picklesupport/module/_pickle_support/maker.py ============================================================================== --- pypy/branch/picklesupport/module/_pickle_support/maker.py (original) +++ pypy/branch/picklesupport/module/_pickle_support/maker.py Fri Apr 28 13:01:28 2006 @@ -1,9 +1,10 @@ from pypy.interpreter.nestedscope import Cell from pypy.interpreter.pycode import PyCode -from pypy.interpreter.function import Function +from pypy.interpreter.function import Function, Method +from pypy.interpreter.module import Module from pypy.rpython.objectmodel import instantiate from pypy.interpreter.argument import Arguments -from pypy.interpreter.baseobjspace import ObjSpace +from pypy.interpreter.baseobjspace import ObjSpace, W_Root #note: for now we don't use the actual value when creating the Cell. @@ -22,3 +23,12 @@ w_type = space.gettypeobject(Function.typedef) return space.call_args(w_type, __args__) func_new.unwrap_spec = [ObjSpace, Arguments] + +def module_new(space, w_name, w_dict): #XXX untested + new_mod = Module(space, w_name, w_dict) + return space.wrap(new_mod) + +def method_new(space, __args__): #XXX untested + w_type = space.gettypeobject(Method.typedef) + return space.call_args(w_type, __args__) +method_new.unwrap_spec = [ObjSpace, Arguments] From ac at codespeak.net Fri Apr 28 17:10:45 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 28 Apr 2006 17:10:45 +0200 (CEST) Subject: [pypy-svn] r26524 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060428151045.47BF01009B@code0.codespeak.net> Author: ac Date: Fri Apr 28 17:10:44 2006 New Revision: 26524 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: (aleale, arre) Structures with pointer to them selves do *not* work Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Fri Apr 28 17:10:44 2006 @@ -13,7 +13,7 @@ from pypy.rpython.test.test_llinterp import interpret from ctypes import c_int, c_short, Structure, POINTER, pointer, c_char_p -from ctypes import c_char +from ctypes import c_char, SetPointerType class tagpoint(Structure): _fields_ = [("x", c_int), @@ -151,6 +151,23 @@ res = interpret(func, []) assert res == 121 + def test_struct_with_pointer_to_self(self): + py.test.skip('In-preogress') + PS = POINTER('S') + class S(Structure): + _fields_ = [('l', PS), ('r', PS)] + SetPointerType(PS, S) + + def func(): + s0 = S() + s0.r = s0 + s0.l = pointer(S()) + s0.l.r = s0 + + return bool(s0.r.contents.l.contents.r) + res = interpret(func, []) + assert res is False + def test_specialize_keepalive(self): class S(Structure): _fields_ = [('x', c_int)] From mwh at codespeak.net Fri Apr 28 18:11:50 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 28 Apr 2006 18:11:50 +0200 (CEST) Subject: [pypy-svn] r26525 - in pypy/dist/pypy/translator: c/test stackless stackless/test Message-ID: <20060428161150.7BA811009F@code0.codespeak.net> Author: mwh Date: Fri Apr 28 18:11:48 2006 New Revision: 26525 Added: pypy/dist/pypy/translator/stackless/test/test_yield_current_frame_to_caller.py (contents, props changed) Modified: pypy/dist/pypy/translator/c/test/test_stackless.py pypy/dist/pypy/translator/stackless/code.py Log: a failed attempt to implement yield_current_frame_to_caller & some tests. off on holiday for a week now, yippee! Modified: pypy/dist/pypy/translator/c/test/test_stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_stackless.py (original) +++ pypy/dist/pypy/translator/c/test/test_stackless.py Fri Apr 28 18:11:48 2006 @@ -178,6 +178,18 @@ data = self.wrap_stackless_function(f) assert int(data.strip()) == 1234567 + def test_foo(self): + def f(): + c = g() + c.switch() + return 1 + def g(): + d = yield_current_frame_to_caller() + return d + data = self.wrap_stackless_function(f) + assert data.strip() == '1' + + def test_yield_noswitch_frame(self): # this time we make sure that function 'g' does not # need to switch and even does not need to be stackless Modified: pypy/dist/pypy/translator/stackless/code.py ============================================================================== --- pypy/dist/pypy/translator/stackless/code.py (original) +++ pypy/dist/pypy/translator/stackless/code.py Fri Apr 28 18:11:48 2006 @@ -14,30 +14,86 @@ return (currentframe.function, currentframe.retval_type, currentframe.restartstate) -decode_state.stackless_explict = True +decode_state.stackless_explicit = True SWITCH_STATE = lltype.GcStruct('state_switch', ('header', STATE_HEADER), ('c', llmemory.Address)) -def switch(c): - # this is untested so far! - if not global_state.restartstate: +class Frame(object): + def __init__(self, state): + self.state = state + __init__.stackless_explicit = True + def switch(self): + if global_state.restart_substate == 0: + u = UnwindException() + s = lltype.malloc(SWITCH_STATE) + s.header.restartstate = 1 + # the next three lines are pure rtyper-pleasing hacks + f = Frame.switch + if global_state.restart_substate: + f = None + s.c = llmemory.cast_ptr_to_adr(self.state) + s.header.function = llmemory.cast_ptr_to_adr(f) + add_frame_state(u, s.header) + raise u + elif global_state.restart_substate == 1: + top = global_state.top + state = lltype.cast_pointer(lltype.Ptr(SWITCH_STATE), top) + u = UnwindException() + s.header.restartstate = 2 + c = lltype.cast_adr_to_ptr(lltype.Ptr(STATE_HEADER), top) + s.header.function = c.function + s.reader.retval_type = RETVAL_VOID_P + # the next three lines are pure rtyper-pleasing hacks + f = Frame.switch + if global_state.restart_substate: + f = None + add_frame_state(u, s.header) + raise u + else: + top = global_state.top + global_state.restart_substate = 0 + r = top.f_back + state = lltype.cast_pointer(lltype.Ptr(SWITCH_STATE), top) + global_state.top = lltype.cast_adr_to_ptr(lltype.Ptr(STATE_HEADER), state.c) + #global_state.restart_substate = state.header.restartstate + return r + switch.stackless_explicit = True + +def yield_current_frame_to_caller(): + if global_state.restart_substate == 0: + u = UnwindException() + s = lltype.malloc(STATE_HEADER) + s.restartstate = 1 + # the next three lines are pure rtyper-pleasing hacks + f = yield_current_frame_to_caller + if global_state.restart_substate: + f = None + s.function = llmemory.cast_ptr_to_adr(f) + s.retval_type = RETVAL_VOID_P + add_frame_state(u, s) + raise u + elif global_state.restart_substate == 1: + ycftc_state = global_state.top + our_caller_state = ycftc_state.f_back + caller_state = our_caller_state.f_back + cur = caller_state + while cur.f_back: + cur = cur.f_back + bot = cur u = UnwindException() - s = lltype.malloc(SWITCH_STATE) - s.c = llmemory.cast_ptr_to_adr(c) - s.header.restartstate = 1 - s.header.function = llmemory.cast_ptr_to_adr(switch) - s.header.retval_type = RETVAL_VOID_P - add_frame_state(u, s.header) + u.frame_top = caller_state + u.frame_bottom = bot + global_state.retval_void_p = llmemory.cast_ptr_to_adr(Frame(ycftc_state)) + global_state.restart_substate = 2 raise u else: - top = global_state.top - s = lltype.cast_pointer(lltype.Ptr(SWITCH_STATE), top) - global_state.top = s.c - return top.f_back -switch.stackless_explicit = True - + pass + + +yield_current_frame_to_caller.stackless_explicit = True + def stack_frames_depth(): if not global_state.restart_substate: Added: pypy/dist/pypy/translator/stackless/test/test_yield_current_frame_to_caller.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/stackless/test/test_yield_current_frame_to_caller.py Fri Apr 28 18:11:48 2006 @@ -0,0 +1,47 @@ +from pypy.translator.stackless.test.test_transform import \ + llinterp_stackless_function, run_stackless_function +from pypy.translator.stackless import code +import py +import os + +py.test.skip('in progress') + +def test_simple(): + def f(ignored): + c = g() + #c.switch() + return 1 + def g(): + d = code.yield_current_frame_to_caller() + return d + + data = llinterp_stackless_function(f) + assert data == 1234567 + + +def test_yield_frame(): + + def g(lst): + lst.append(2) + frametop_before_5 = code.yield_current_frame_to_caller() + lst.append(4) + frametop_before_7 = frametop_before_5.switch() + lst.append(6) + return frametop_before_7 + + def f(ignored): + lst = [1] + frametop_before_4 = g(lst) + lst.append(3) + frametop_before_6 = frametop_before_4.switch() + lst.append(5) + frametop_after_return = frametop_before_6.switch() + lst.append(7) + assert frametop_after_return is None + n = 0 + for i in lst: + n = n*10 + i + return n + + data = llinterp_stackless_function(f) + assert data == 1234567 From arigo at codespeak.net Fri Apr 28 18:20:42 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 18:20:42 +0200 (CEST) Subject: [pypy-svn] r26526 - in pypy/dist/pypy: objspace/flow rpython Message-ID: <20060428162042.3C55D100A2@code0.codespeak.net> Author: arigo Date: Fri Apr 28 18:20:41 2006 New Revision: 26526 Modified: pypy/dist/pypy/objspace/flow/model.py pypy/dist/pypy/rpython/llinterp.py Log: Tracing for the llinterpreter: this dumps html files in /tmp/usession-*/. Viewed with a Javascript-enabled browser, we can show and hide subcalls by clicking on them. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Fri Apr 28 18:20:41 2006 @@ -67,12 +67,14 @@ return getsource(self.func) source = roproperty(getsource) - def __repr__(self): + def __str__(self): if hasattr(self, 'func'): - fnrepr = nice_repr_for_func(self.func, self.name) + return nice_repr_for_func(self.func, self.name) else: - fnrepr = self.name - return '' % (fnrepr, uid(self)) + return self.name + + def __repr__(self): + return '' % (self, uid(self)) def iterblocks(self): block = self.startblock Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Apr 28 18:20:41 2006 @@ -36,6 +36,8 @@ class LLInterpreter(object): """ low level interpreter working with concrete values. """ + TRACING = True + def __init__(self, typer, heap=lltype): self.bindings = {} self.typer = typer @@ -47,19 +49,41 @@ if hasattr(heap, "prepare_graphs_and_create_gc"): flowgraphs = typer.annotator.translator.graphs self.gc = heap.prepare_graphs_and_create_gc(self, flowgraphs) + if self.TRACING: + self.tracer = Tracer() + else: + self.tracer = None def eval_graph(self, graph, args=()): llframe = LLFrame(graph, args, self) + if self.tracer: + self.tracer.start() + retval = None try: - return llframe.eval() - except LLException, e: - log.error("LLEXCEPTION: %s" % (e, )) - self.print_traceback() - raise - except Exception, e: - log.error("AN ERROR OCCURED: %s" % (e, )) - self.print_traceback() - raise + try: + retval = llframe.eval() + except LLException, e: + log.error("LLEXCEPTION: %s" % (e, )) + self.print_traceback() + if self.tracer: + self.tracer.dump('LLException: %s\n' % (e,)) + raise + except Exception, e: + log.error("AN ERROR OCCURED: %s" % (e, )) + self.print_traceback() + if self.tracer: + line = str(e) + if line: + line = ': ' + line + line = '* %s' % (e.__class__.__name__,) + line + self.tracer.dump(line + '\n') + raise + finally: + if self.tracer: + if retval is not None: + self.tracer.dump(' ---> %r\n' % (retval,)) + self.tracer.stop() + return retval def print_traceback(self): frame = self.active_frame @@ -68,11 +92,12 @@ frames.append(frame) frame = frame.f_back frames.reverse() + lines = [] for frame in frames: logline = frame.graph.name if frame.curr_block is None: logline += " " - log.traceback(logline) + lines.append(logline) continue try: logline += " " + self.typer.annotator.annotated[frame.curr_block].__module__ @@ -80,13 +105,19 @@ # if the graph is from the GC it was not produced by the same # translator :-( logline += " " - log.traceback(logline) + lines.append(logline) for i, operation in enumerate(frame.curr_block.operations): if i == frame.curr_operation_index: logline = "E %s" else: logline = " %s" - log.traceback(logline % (operation, )) + lines.append(logline % (operation, )) + if self.tracer: + self.tracer.dump('Traceback\n', bold=True) + for line in lines: + self.tracer.dump(line + '\n') + for line in lines: + log.traceback(line) def find_roots(self): #log.findroots("starting") @@ -204,19 +235,25 @@ def eval(self): self.llinterpreter.active_frame = self graph = self.graph - #log.frame("evaluating", graph.name) - nextblock = graph.startblock - args = self.args - while 1: - self.clear() - self.fillvars(nextblock, args) - nextblock, args = self.eval_block(nextblock) - if nextblock is None: - self.llinterpreter.active_frame = self.f_back - for obj in self.alloca_objects: - #XXX slighly unclean - obj._setobj(None) - return args + tracer = self.llinterpreter.tracer + if tracer: + tracer.enter(graph) + try: + nextblock = graph.startblock + args = self.args + while 1: + self.clear() + self.fillvars(nextblock, args) + nextblock, args = self.eval_block(nextblock) + if nextblock is None: + self.llinterpreter.active_frame = self.f_back + for obj in self.alloca_objects: + #XXX slighly unclean + obj._setobj(None) + return args + finally: + if tracer: + tracer.leave() def eval_block(self, block): """ return (nextblock, values) tuple. If nextblock @@ -237,16 +274,20 @@ # determine nextblock and/or return value if len(block.exits) == 0: # return block + tracer = self.llinterpreter.tracer if len(block.inputargs) == 2: # exception + if tracer: + tracer.dump('raise') etypevar, evaluevar = block.getvariables() etype = self.getval(etypevar) evalue = self.getval(evaluevar) # watch out, these are _ptr's raise LLException(etype, evalue) + if tracer: + tracer.dump('return') resultvar, = block.getvariables() result = self.getval(resultvar) - #log.operation("returning", repr(result)) return None, result elif block.exitswitch is None: # single-exit block @@ -283,7 +324,9 @@ return link.target, [self.getval(x) for x in link.args] def eval_operation(self, operation): - #log.operation("considering", operation) + tracer = self.llinterpreter.tracer + if tracer: + tracer.dump(str(operation)) ophandler = self.getoperationhandler(operation.opname) # XXX slighly unnice but an important safety check if operation.opname == 'direct_call': @@ -313,6 +356,11 @@ else: self.handle_cleanup(operation) self.setvar(operation.result, retval) + if tracer: + if retval is None: + tracer.dump('\n') + else: + tracer.dump(' ---> %r\n' % (retval,)) def handle_cleanup(self, operation, exception=False): cleanup = getattr(operation, 'cleanup', None) @@ -953,6 +1001,86 @@ def op_ooidentityhash(self, inst): return ootype.ooidentityhash(inst) + +class Tracer(object): + Counter = 0 + file = None + + HEADER = """ + + + + 
+    """
+
+    FOOTER = """
""" + + ENTER = ('''\n\t%s''' + '''\n
\t''') + LEAVE = '''\n
\t''' + + def htmlquote(self, s, text_to_html={}): + # HTML quoting, lazily initialized + if not text_to_html: + import htmlentitydefs + for key, value in htmlentitydefs.entitydefs.items(): + text_to_html[value] = '&' + key + ';' + return ''.join([text_to_html.get(c, c) for c in s]) + + def start(self): + # start of a dump file + from pypy.tool.udir import udir + n = Tracer.Counter + Tracer.Counter += 1 + self.file = udir.join('llinterp_trace_%d.html' % n).open('w') + print >> self.file, self.HEADER + self.count = 0 + self.indentation = '' + + def stop(self): + # end of a dump file + if self.file: + print >> self.file, self.FOOTER + self.file.close() + self.file = None + + def enter(self, graph): + # enter evaluation of a graph + if self.file: + s = self.htmlquote(str(graph)) + i = s.rfind(')') + s = s[:i+1] + '' + s[i+1:] + '' + if self.count == 0: + display = 'block' + else: + display = 'none' + text = self.ENTER % (self.count, s, self.count, display) + self.indentation += ' ' + self.file.write(text.replace('\t', self.indentation)) + self.count += 1 + + def leave(self): + # leave evaluation of a graph + if self.file: + self.indentation = self.indentation[:-4] + self.file.write(self.LEAVE.replace('\t', self.indentation)) + + def dump(self, text, bold=False): + if self.file: + text = self.htmlquote(text) + if bold: + text = '%s' % (text,) + self.file.write(text.replace('\n', '\n'+self.indentation)) + + # by default we route all logging messages to nothingness # e.g. tests can then switch on logging to get more help # for failing tests From ale at codespeak.net Fri Apr 28 18:21:15 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 28 Apr 2006 18:21:15 +0200 (CEST) Subject: [pypy-svn] r26527 - pypy/dist/pypy/tool/pytest Message-ID: <20060428162115.40287100B6@code0.codespeak.net> Author: ale Date: Fri Apr 28 18:21:12 2006 New Revision: 26527 Modified: pypy/dist/pypy/tool/pytest/genreportdata.py Log: Genreportdata only works with python2.3 (mimelib changes ?) ?) Modified: pypy/dist/pypy/tool/pytest/genreportdata.py ============================================================================== --- pypy/dist/pypy/tool/pytest/genreportdata.py (original) +++ pypy/dist/pypy/tool/pytest/genreportdata.py Fri Apr 28 18:21:12 2006 @@ -2,6 +2,9 @@ import autopath import py import sys +if sys.version_info[:2] != (2,3): + raise RuntimeError("Genreportdata.py needs Python 2.3") + mydir = py.magic.autopath().dirpath().realpath() from pypy.tool.pytest import htmlreport from pypy.tool.pytest import confpath From arigo at codespeak.net Fri Apr 28 18:25:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 18:25:47 +0200 (CEST) Subject: [pypy-svn] r26528 - pypy/dist/pypy/rpython Message-ID: <20060428162547.A16C7100A6@code0.codespeak.net> Author: arigo Date: Fri Apr 28 18:25:46 2006 New Revision: 26528 Modified: pypy/dist/pypy/rpython/llinterp.py Log: Javascript trick (the previous version made the browser jump to the start of the page). Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Apr 28 18:25:46 2006 @@ -1023,7 +1023,7 @@ FOOTER = """""" - ENTER = ('''\n\t%s''' + ENTER = ('''\n\t%s''' '''\n
\t''') LEAVE = '''\n
\t''' From arigo at codespeak.net Fri Apr 28 18:29:30 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 18:29:30 +0200 (CEST) Subject: [pypy-svn] r26529 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060428162930.C1DFF1009D@code0.codespeak.net> Author: arigo Date: Fri Apr 28 18:29:29 2006 New Revision: 26529 Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py Log: Forgotten fix to the import. Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Fri Apr 28 18:29:29 2006 @@ -4,7 +4,7 @@ from pypy.rpython.rctypes.rmodel import CTypesRefRepr from pypy.objspace.flow.model import Constant from pypy.rpython.rslice import AbstractSliceRepr -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr class StringBufRepr(CTypesRefRepr): From arigo at codespeak.net Fri Apr 28 18:58:21 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 28 Apr 2006 18:58:21 +0200 (CEST) Subject: [pypy-svn] r26530 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060428165821.06AD8100B7@code0.codespeak.net> Author: arigo Date: Fri Apr 28 18:58:21 2006 New Revision: 26530 Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Log: Fixed the recursive ctypes structure problem. The rpointers are now lazily following the type of what they point to. Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Fri Apr 28 18:58:21 2006 @@ -9,20 +9,27 @@ class PointerRepr(CTypesValueRepr): def __init__(self, rtyper, s_pointer): - ptr_ctype = s_pointer.knowntype - ref_ctype = ptr_ctype._type_ + # For recursive types, getting the r_contents is delayed until + # _setup_repr(). + ll_contents = lltype.Ptr(lltype.ForwardReference()) + self.keepalive_box_type = lltype.GcForwardReference() + super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) + def _setup_repr(self): # Find the repr and low-level type of the contents from its ctype + rtyper = self.rtyper + ref_ctype = self.ctype._type_ self.r_contents = rtyper.getrepr(SomeCTypesObject(ref_ctype, SomeCTypesObject.MEMORYALIAS)) - - ll_contents = lltype.Ptr(self.r_contents.c_data_type) - - super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) + if isinstance(self.ll_type.TO, lltype.ForwardReference): + self.ll_type.TO.become(self.r_contents.c_data_type) + if isinstance(self.keepalive_box_type, lltype.GcForwardReference): + self.keepalive_box_type.become( + self.r_contents.r_memoryowner.lowleveltype.TO) def get_content_keepalive_type(self): "Keepalive for the box that holds the data that 'self' points to." - return self.r_contents.r_memoryowner.lowleveltype + return lltype.Ptr(self.keepalive_box_type) def setkeepalive(self, llops, v_box, v_owner): inputargs = [v_box, inputconst(lltype.Void, 'keepalive'), Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Fri Apr 28 18:58:21 2006 @@ -152,7 +152,6 @@ assert res == 121 def test_struct_with_pointer_to_self(self): - py.test.skip('In-preogress') PS = POINTER('S') class S(Structure): _fields_ = [('l', PS), ('r', PS)] @@ -160,11 +159,12 @@ def func(): s0 = S() - s0.r = s0 - s0.l = pointer(S()) - s0.l.r = s0 + s0.r.contents = s0 + s0.l.contents = S() + s0.l.contents.r.contents = s0 - return bool(s0.r.contents.l.contents.r) + return bool(s0.r.contents.l.contents.l) + assert not func() res = interpret(func, []) assert res is False From bea at codespeak.net Fri Apr 28 19:41:32 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Fri, 28 Apr 2006 19:41:32 +0200 (CEST) Subject: [pypy-svn] r26531 - pypy/extradoc/talk/agile2006 Message-ID: <20060428174132.CEAF5100A7@code0.codespeak.net> Author: bea Date: Fri Apr 28 19:41:30 2006 New Revision: 26531 Added: pypy/extradoc/talk/agile2006/ pypy/extradoc/talk/agile2006/draftpaper_agile2006.doc (contents, props changed) pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt Log: dir for agile talk Added: pypy/extradoc/talk/agile2006/draftpaper_agile2006.doc ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt Fri Apr 28 19:41:30 2006 @@ -0,0 +1,163 @@ +Title: "Trouble in Paradise: Open Source projects, EU-funding and Agile practices" +------------------------------------------------------------------------------------- + +Abstract: +(200 words - write last) + +1.Introduction: +This paper will present the story and the experiences gathered so far in the PyPy project concerning +how to integrate such diverse perspectives as agile practices being used in a distributed and dispersed development +style in an Open Source project that is partly funded by the European Union. + +The PyPy project started as a grass-root effort among core developers in the Python language community, aiming at +building a Python implementation purely built in Python (todays main language implementation is based on C, called Cpython. +It was a language implementation project that was purely driven from a non-profit perspective which +increased in complexity when the Open Source project applied for EU-funding. +We believe that the practice of ?sprinting?, +as being used in the PyPy project, makes Agile and Distributed/dispersed workstyles more possible to +combine. We also believe it is a very useful practice for creating value and ensuring quality in a +projects with hybrid cultures and methodologies. It is our aim to start to show-case this with +experiences from the first funded year of PyPy, combining Agile and Distributed/dispersed +Development in an Open Source Community with the Plan-driven approaches through EU-Funding. + +1.1 Project context +Objectives: +The Technical objective of the PyPy project is to build a implementation of the Open Source +Programming Language in the Python language itself. By employing novel combinations of known +techniques for building optimized compilers and interpreters the project will build an implementation +of Python that is easier to read and understand than the existing implementations, while only paying +a small performance penalty. + +The Scientific objectives of the project are to investigate a number of novel techniques based on +Aspect Oriented Programming code generation and abstract interpretation for the implementation of +practical dynamic languages The goal is to break the compromise between flexibility, simplicity and +performance trade-offs and to expand the range of directly supportable runtime platforms. + +The Method objective is to showcase a novel software engineering process: Sprint Driven Development. +This is an Agile methodology, providing a dynamic and adaptive environment, suitable for co-operative and +distributed development. + +Strategy: +The strategy of the project is to leverage the community of PyPy and Python through open and transparent +communication and working style. The challenge has been to implement this strategy not only in the +F/OSS community part of the project but also in the partially funded consortium structure of the project. +The structure of the project and the consortium needed to adhere to this strategy while also conforming +to the actual requirements of the 6th Framework Programme. + +The consortium consists of 8 partners: DFKI (Germany), Ab Strakt (Sweden), Logilab (France), merlinux GmbH +(Germany), tismerysoft GmbH (Germany), Change Maker (Sweden) , Impara GmbH (Germany) and Heinrich Heine +Universit?t D?sseldorf (Germany) and 4 physical person partners: Laura Creighton (Sweden), Richard Emslie (UK), +Eric Van Riet Paap (Netherlands) , Niklaus Haldiman (Switzerland). The project effort of work for the 2 years of +funding consists of 14 work-packages and in total 58 deliverables. + +History: +Mid 2003 the idea of trying to get EU-funding for the project was born. It became clear that the project had an +arbitrarily large scale and that receiving some funding would dramatically increase the pace and seriousness of the +project - because funded developers can dedicate more of their time to the project. The involved developers and +people stretched outside of the Open Source ecologies to try to gather as much information and contacts as +possible in order to answer the question: "Should we go for it?" to which the answer quickly became "Let's see +how far we get!". + +There had been a growing interest from the European Commission, IST division to look closer at the Open Source world +and its achievements. Several funded research projects in the 5th framework programme studied the phenomenon +(FLOSS-POLS, FLOSS) - its organization, business models and licensing. A few other funded software projects used +Open Source in their work as tools (languages and applications). There was no previous experience of an Open Source +community based project making a bid for funding. + +The areas in the 6th Framework programme (second call) fit very well with the objectives of PyPy. +The idea of strengthening the European Software development companies and businesses with supporting +an open source language implementation was new but appealing to the EU. However, being an Open Source +project was not enough. The challenges and the idea of a flexible, configurable "translator" or +"compiler" met the research targets of the FP6, as well as trying out and documenting the agile methodology +being used. + +In short, we argued that EU funding allowed the project to go from reaching a critical mass and position +to continue to evolve from there, and that it would help European organizations to make some ground. + +Acting on this strategy proved to be a more difficult task. The entire proposal and negotiation process took +over a year (Autumn 2003 until November 2004).A proper description of planned work, necessary to satisfy +formal requirements, had not previously been part of the development focus and both the EU and the parties +involved had to adapt to the situation. Yet, drafting the high-level requirements (in total 14 work-packages +and 58 deliverables) was done using the same version-control/open-communication based work style, including +evolving the proposal at sprints. Writing the proposal and specifying according objectives on a higher level +has proved to be generally useful for clarifying goals on a longer term. +It also helps others to understand the project better. + +Unfortunately the negotiations with the EU got stuck in organizational limbo and the project is still +suffering from the effects of this even today. The goal of funding contributors especially coming to +sprints was originally based on a non-profit association. This solution wasn't seen as realistic or +feasible by the EU although we think it remains a viable approach for the future. During negotiations, +we got to an alternative solution which had a few drawbacks: contributors have to become Contract +Partners within the EU-level Consortium (which is by itself not difficult) and can then at least claim travel and +accommodation costs when attending sprints. + +However, this construction does not allow them to get paid for work time and also has some formal requirements. +In practice this leads to current considerations of developers to shift private money between them in order to +circumvent the current problems withimplementing an agile model within the EU contract framing. + + +2. Influencing factors: the F/OSS Python community culture +2.1 The Python community +2.2 The PyPy community +2.3 Supporting infrastructure +2.4 Supporting practices + +3. Influencing factors: agile practices in PyPy +3.1 TDD +3.2 XP +3.3 Sprints + +4. Influencing factors: EU-project practices +The formal project organization required by the EU imposed more structure on the previous more free-floating +agile process. Roles and responsibilities where staked out,conforming with the requirements of the roles +but delegating as much as possible of the responsibilities and decision-making to the core developers. The +strategy was to keep "conceptual integrity" (Brooks) of the vision and the idea in the hands of the core +developers. A somewhat negative result was the added workload and responsibility on developers regarding +EU related work. It was also a conscious strategy to employ the same version-control/review based scheme +regarding EU documents and Consortium level issues as those being used in the technical development. + +It remains a challenge for all partners of the consortium, universities and companies alike, to connect an +ongoing medium-scale open-source project with EU regulations and requirements - not to speak of the fact that +companies need to fund 50% of the costs themselves. It is, in fact, too early to judge the overall success +of our approaches although we are confident that things work out reasonably well. + +4.1 Consortium structure +4.2 Resource tracking and reporting +4.3 Communication and documentation + +5. Troubles in Paradise:striking a balance +5.1 Developer driven versus formal project structure +5.2 Agile strategies versus formal EU-contractual requirements +5.3 F/OSS community versus hierarchies for "conceptual integrity" + +6. Conclusion: +The important question is the following (and is also part of the methodological objective of the project): +is there enough room to manage a project and Ope Source community within the plan-driven inspired methods +that are required in EU-funded projects, while still working agile and distributed? + +We believe so. The one clear dominating factor to make all this succeed is, as always, the people factor, +the CRACK performers as Boehm and Turner calls them (?Collaborative, Representative, Authorized, Committed, +Knowledgeable?) (4). + +The core developers of the PyPy project had the right mix of various skills in order to succeed in setting +up a hybrid environment - enabling them to work full time on a project they strongly believed in. +The most crucial mix of skills for making this possible was/are: + +- social (the ability to communicate, "manage" groups and handle conflicts) +- leadership abilities (both on technical levels as well on general social levels) +- ability to network (to find and invite actors in the various cultures) +- entrepreneurs (to instigate the idea of EU-funding and also to create consortium and companies) +- curious and collaborative (towards other Python implementations and other languages and research approaches) +- technical skills (programming language and implementation aspects, frameworks, mathematics, computer science etc) +- ability to balance "conceptual integrity" (Brooks) with open and transparent communication through sprints, documentation, tutorials, mentoring, sync-meetings, thus also increasing the group of the core developers during the project + +Drawing from these different skills within the community of developers in the PyPy project one possible conclusion +would be that a truly agile approach dominating the workstyle of an Open Source project will increase the ability +of the community to spread the strategy of agility to other domains such as entrepreneurship and business models +(whether this is done by separate individuals, subgroups or the whole group). By doing this hybrid projects such +as PyPy and others (Zope Europe among others) are made possible. + +6.1 The vision factor +6.2 Managing diversities +6.3 Process design +6.4 Group learning \ No newline at end of file From antocuni at codespeak.net Fri Apr 28 20:58:00 2006 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 28 Apr 2006 20:58:00 +0200 (CEST) Subject: [pypy-svn] r26533 - pypy/dist/pypy/doc/discussion Message-ID: <20060428185800.0696F10091@code0.codespeak.net> Author: antocuni Date: Fri Apr 28 20:57:52 2006 New Revision: 26533 Added: pypy/dist/pypy/doc/discussion/cli-optimizations.txt (contents, props changed) Log: Some ideas on how to optimize gencli. Added: pypy/dist/pypy/doc/discussion/cli-optimizations.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/discussion/cli-optimizations.txt Fri Apr 28 20:57:52 2006 @@ -0,0 +1,196 @@ +Possible optimizations for the CLI backend +========================================== + +Stack push/pop optimitazion +--------------------------- + +The CLI's VM is a stack based machine: this fact doesn't play nicely +with the SSI form the flowgraphs are generated in. At the moment +gencli does a literal translation of the SSI statements, allocating a +new local variable for each variable of the flowgraph. + +For example, consider the following RPython code and the corresponding +flowgraph:: + + def bar(x, y): + foo(x+y, x-y) + + + inputargs: x_0 y_0 + v0 = int_add(x_0, y_0) + v1 = int_sub(x_0, y_0) + v2 = directcall((sm foo), v0, v1) + +This is the IL code generated by the CLI backend:: + + .locals init (int32 v0, int32 v1, int32 v2) + + block0: + ldarg 'x_0' + ldarg 'y_0' + add + stloc 'v0' + ldarg 'x_0' + ldarg 'y_0' + sub + stloc 'v1' + ldloc 'v0' + ldloc 'v1' + call int32 foo(int32, int32) + stloc 'v2' + +As you can see, the results of 'add' and 'sub' are stored in v0 and +v1, respectively, then v0 and v1 are reloaded onto stack. These +store/load is redundant, since the code would work nicely even without +them:: + + .locals init (int32 v2) + + block0: + ldarg 'x_0' + ldarg 'y_0' + add + ldarg 'x_0' + ldarg 'y_0' + sub + call int32 foo(int32, int32) + stloc 'v2' + +I've checked the native code generated by the Mono Jit on x86 and I've +seen that it does not optimize it. I haven't checked the native code +generated by Microsoft CLR, yet. + +Thus, we might consider to optimize it manually; it should not be so +difficult, but it is not trivial becasue we have to make sure that the +dropped locals are used only once. + + +Mapping RPython exceptions to native CLI exceptions +--------------------------------------------------- + +Both RPython and CLI have its own set of exception classes: some of +these are pretty similar; e.g., we have OverflowError, +ZeroDivisionError and IndexError on the first side and +OverflowException, DivideByZeroException and IndexOutOfRangeException +on the other side. + +The first attempt was to map RPython classes to their corresponding +CLI ones: this worked for simple cases, but it would have triggered +subtle bugs in more complex ones, because the two exception +hierarchies don't completely overlap. + +For now I've choosen to build an RPython exception hierarchy +completely indipendent from the CLI one, but this means that we can't +rely on exceptions raised by standard operations. The currently +implemented solution is to do an exception translation on-the-fly; for +example, the 'ind_add_ovf' is translated into the following IL code:: + + .try + { + ldarg 'x_0' + ldarg 'y_0' + add.ovf + stloc 'v1' + leave __check_block_2 + } + catch [mscorlib]System.OverflowException + { + newobj instance void class exceptions.OverflowError::.ctor() + dup + ldsfld class Object_meta pypy.runtime.Constants::exceptions_OverflowError_meta + stfld class Object_meta Object::meta + throw + } + +I.e., it catches the builtin OverflowException and raises a RPython +OverflowError. + +I haven't misured timings yet, but I guess that this machinery brings +to some performance penalties even in the non-overflow case; a +possible optimization is to do the on-the-fly translation only when it +is strictly necessary, i.e. only when the except clause catches an +exception class whose subclass hierarchy is compatible with the +builtin one. As an example, consider the following RPython code:: + + try: + return mylist[0] + except IndexError: + return -1 + +Given that IndexError has no subclasses, we can map it to +IndexOutOfBoundException and directly catch this one: + + try + { + ldloc 'mylist' + ldc.i4 0 + call int32 getitem(MyListType, int32) + ... + } + catch [mscorlib]System.IndexOutOfBoundException + { + // return -1 + ... + } + +By contrast we can't do so if the except clause catches classes that +don't directly map to any builtin class, such as LookupError:: + + try: + return mylist[0] + except LookupError: + return -1 + +Has to be translated in the old way:: + + .try + { + ldloc 'mylist' + ldc.i4 0 + + .try + { + call int32 getitem(MyListType, int32) + } + catch [mscorlib]System.IndexOutOfBoundException + { + // translate IndexOutOfBoundException into IndexError + newobj instance void class exceptions.IndexError::.ctor() + dup + ldsfld class Object_meta pypy.runtime.Constants::exceptions_IndexError_meta + stfld class Object_meta Object::meta + throw + } + ... + } + .catch exceptions.LookupError + { + // return -1 + ... + } + + +Specializing methods of List +---------------------------- + +Most methods of RPython lists are implemented by ll_* helpers placed +in rpython/rlist.py. For some of those we have a direct correspondent +already implemented in .NET List<>; we could use the oopspec attribute +for doing an on-the-fly replacement of these low level helpers with +their builtin correspondent. As an example the 'append' method is +already mapped to pypylib.List.append. Thanks to Armin Rigo for the +idea of using oopspec. + + +Doing some caching on Dict +-------------------------- + +The current implementations of ll_dict_getitem and ll_dict_get in +ootypesystem.rdict do two consecutive lookups (calling ll_contains and +ll_get) on the same key. We might cache the result of +pypylib.Dict.ll_contains so that the succesive ll_get don't need a +lookup. Btw, we need some profiling before choosing the best way. Or +we could directly refactor ootypesystem.rdict for doing a single +lookup. + + From ericvrp at codespeak.net Sat Apr 29 03:32:05 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 03:32:05 +0200 (CEST) Subject: [pypy-svn] r26541 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060429013205.3E2E71008B@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 03:31:53 2006 New Revision: 26541 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: Tokyo sprint last day status and planning Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Sat Apr 29 03:31:53 2006 @@ -229,3 +229,56 @@ * rpystone translation - general wizardry (Samuele) + +Saturday, 29th April 2006 +*************************** + +Present: Samuele, Arre, Valentino, Eric, Anders, Yusei, Koichi + +Status after day 5: + + - gencl (Sanghyeon, Valentino) + + * Somewhat exception handling support using clisp. + Needs more work. + + - Pickle support for objects (Eric, Samuele) + + * We added module, moduledict, functions + + - OOTypeSystem (Nik, Samuele) + + * fixed very obscure bug (see r26507 for the dirty details) + * splitting stringsupport between ootype and lltype system + (will need more work, since this broke translation) + + - ctypes _socket (and select ;)) (Arre, Anders) + + * annotation workds, rtyping in progress. + * now supporting recursive structures in rctypes + * errno needs a bit more support in rctypes. More specifically, + errno can be many different things (a define, a function Xi + (sometimes thread specific)) but all platform specific. + * ipv6 support is not (ahum) complete + * no windows support yet + +Pairings today: + + - Pickling (Eric, Yusei) + + * add support for additional objects like code objects. + + - ctypes _socket (Arre, Anders) + + * continue translation work. + + - OOTypeSystem (Nik) + + * fix String type translation issue + + - gencl (Valentino) + + * exception handling + * rpystone translation + + - general wizardry (Samuele) From tismer at codespeak.net Sat Apr 29 03:43:25 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 29 Apr 2006 03:43:25 +0200 (CEST) Subject: [pypy-svn] r26542 - in pypy/dist/pypy: rpython rpython/lltypesystem translator/c translator/c/src translator/c/test Message-ID: <20060429014325.F2BF51008B@code0.codespeak.net> Author: tismer Date: Sat Apr 29 03:43:15 2006 New Revision: 26542 Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py pypy/dist/pypy/rpython/robject.py pypy/dist/pypy/translator/c/pyobj.py pypy/dist/pypy/translator/c/src/pyobj.h pypy/dist/pypy/translator/c/src/support.h pypy/dist/pypy/translator/c/test/test_wrapping.py pypy/dist/pypy/translator/c/wrapper.py Log: support for true methods, properties, ... checking in with a bug right now. Modified: pypy/dist/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lloperation.py Sat Apr 29 03:43:15 2006 @@ -317,6 +317,7 @@ 'same_as': LLOp(canfold=True), 'hint': LLOp(), 'check_no_more_arg': LLOp(canraise=(Exception,)), + 'check_self_nonzero': LLOp(canraise=(Exception,)), 'decode_arg': LLOp(canraise=(Exception,)), 'decode_arg_def': LLOp(canraise=(Exception,)), 'getslice': LLOp(canraise=(Exception,)), Modified: pypy/dist/pypy/rpython/robject.py ============================================================================== --- pypy/dist/pypy/rpython/robject.py (original) +++ pypy/dist/pypy/rpython/robject.py Sat Apr 29 03:43:15 2006 @@ -1,7 +1,7 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem.lltype import \ - PyObject, Ptr, Void, pyobjectptr, nullptr + PyObject, Ptr, Void, pyobjectptr, nullptr, Bool from pypy.rpython.rmodel import Repr, VoidRepr, inputconst from pypy.rpython import rclass from pypy.tool.sourcetools import func_with_new_name @@ -69,3 +69,10 @@ for opname in annmodel.BINARY_OPERATIONS: make_operation(opname, pairtype(PyObjRepr, Repr)) make_operation(opname, pairtype(Repr, PyObjRepr)) + + +class __extend__(pairtype(PyObjRepr, PyObjRepr)): + def rtype_contains((r_seq, r_item), hop): + v_seq, v_item = hop.inputargs(r_seq, r_item) + return hop.llops.gencapicall('PySequence_Contains_with_exc', + [v_seq, v_item], resulttype=Bool) Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Sat Apr 29 03:43:15 2006 @@ -81,6 +81,8 @@ self.initcode.append("%s = %s" % (name, pyexpr)) def nameof_object(self, value): + if isinstance(object, property): + return self.nameof_property(value) if type(value) is not object: raise Exception, "nameof(%r)" % (value,) name = self.uniquename('g_object') @@ -579,6 +581,15 @@ self.name_for_meth[value] = fname if self.use_true_methods: self.is_method[value] = True + elif isinstance(value, property): + fget, fset, fdel, doc = value.fget, value.fset, value.fdel, value.__doc__ + for f in fget, fset, fdel: + if f and self.use_true_methods: + self.is_method[f] = True + stuff = [self.nameof(x) for x in fget, fset, fdel, doc] + yield '%s.%s = property(%s, %s, %s, %s)' % ((name, key) + + tuple(stuff)) + continue yield '%s.%s = %s' % (name, key, self.nameof(value)) if not init_seen: log.WARNING('No __init__ found for %s - you cannot build instances' % @@ -603,3 +614,13 @@ pycfunctionobj = self.uniquename('gfunc_' + newname) self.wrappers[pycfunctionobj] = g.func.__name__, self.getvalue(fwrapper), g.func.__doc__ return pycfunctionobj + + def nameof_property(self, p): + fget, fset, fdel, doc = p.fget, p.fset, p.fdel, p.__doc__ + for f in fget, fset, fdel: + if f and self.use_true_methods: + self.is_method[f] = True + stuff = [self.nameof(x) for x in fget, fset, fdel, doc] + name = self.uniquename('gprop') + expr = 'property(%s, %s, %s, %s)' % (tuple(stuff)) + self.initcode_python(name, expr) Modified: pypy/dist/pypy/translator/c/src/pyobj.h ============================================================================== --- pypy/dist/pypy/translator/c/src/pyobj.h (original) +++ pypy/dist/pypy/translator/c/src/pyobj.h Sat Apr 29 03:43:15 2006 @@ -225,7 +225,9 @@ #define OP_DECODE_ARG_DEF(fname, pos, name, vargs, vkwds, def, r) \ if (!(r=decode_arg(fname, pos, name, vargs, vkwds, def))) CFAIL() #define OP_CHECK_NO_MORE_ARG(fname, n, vargs, r) \ - if (check_no_more_arg(fname, n, vargs) < 0) CFAIL() + if ((r=check_no_more_arg(fname, n, vargs)) < 0) CFAIL() +#define OP_CHECK_SELF_NONZERO(fname, self, r) \ + if ((r=check_self_nonzero(fname, self)) < 0) CFAIL() unsigned long long RPyLong_AsUnsignedLongLong(PyObject *v); long long RPyLong_AsLongLong(PyObject *v); Modified: pypy/dist/pypy/translator/c/src/support.h ============================================================================== --- pypy/dist/pypy/translator/c/src/support.h (original) +++ pypy/dist/pypy/translator/c/src/support.h Sat Apr 29 03:43:15 2006 @@ -46,9 +46,10 @@ PyObject* decode_arg(PyObject* fname, int position, PyObject* name, PyObject* vargs, PyObject* vkwds, PyObject* def); int check_no_more_arg(PyObject* fname, int n, PyObject* vargs); +int check_self_nonzero(PyObject* fname, PyObject* self); PyObject *PyTuple_GetItem_WithIncref(PyObject *tuple, int index); int PyTuple_SetItem_WithIncref(PyObject *tuple, int index, PyObject *o); - +int PySequence_Contains_with_exc(PyObject *seq, PyObject *ob); /* implementations */ @@ -387,6 +388,17 @@ return 0; } +int check_self_nonzero(PyObject* fname, PyObject* self) +{ + if (!self) { + PyErr_Format(PyExc_TypeError, + "%s() expects instance first arg", + PyString_AS_STRING(fname)); + return -1; + } + return 0; +} + /************************************************************/ PyObject *PyTuple_GetItem_WithIncref(PyObject *tuple, int index) @@ -402,6 +414,15 @@ return PyTuple_SetItem(tuple, index, o); } +int PySequence_Contains_with_exc(PyObject *seq, PyObject *ob) +{ + int ret = PySequence_Contains(seq, ob); + + if (ret < 0) + CFAIL(); + return ret; +} + #endif /* PYPY_STANDALONE */ #endif /* PYPY_NOT_MAIN_FILE */ Modified: pypy/dist/pypy/translator/c/test/test_wrapping.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_wrapping.py (original) +++ pypy/dist/pypy/translator/c/test/test_wrapping.py Sat Apr 29 03:43:15 2006 @@ -4,15 +4,15 @@ from pypy.rpython.extregistry import ExtRegistryEntry from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem import lltype -from pypy.rpython import robject, rclass +from pypy.rpython import robject, rclass, rint from pypy.translator.tool.cbuild import enable_fast_compilation import sys, types P = False # debug printing -def get_annotation(func): - argstypelist = [] +def get_annotation(func, pre=[]): + argstypelist = pre[:] if func.func_defaults: for spec in func.func_defaults: if isinstance(spec, tuple): @@ -46,15 +46,26 @@ for obj in exports: if isinstance(obj, type): - clsdef = bk.getuniqueclassdef(obj) + cls = obj + clsdef = bk.getuniqueclassdef(cls) rtyper.add_wrapper(clsdef) + for obj in cls.__dict__.values(): + if isinstance(obj, types.FunctionType): + if not ann.bookkeeper.getdesc(obj).querycallfamily(): + # not annotated, so enforce it + ann.build_types(obj, get_annotation(obj, [cls]), complete_now=False) + elif isinstance(obj, property): + for obj in obj.fget, obj.fset, obj.fdel: + if obj and not ann.bookkeeper.getdesc(obj).querycallfamily(): + ann.build_types(obj, get_annotation(obj, [cls]), complete_now=False) elif isinstance(obj, types.FunctionType): if not ann.bookkeeper.getdesc(obj).querycallfamily(): # not annotated, so enforce it - ann.build_types(obj, get_annotation(obj)) + ann.build_types(obj, get_annotation(obj), complete_now=False) if obj.__name__ == '__init__': pyobj_options['use_true_methods'] = True + ann.build_types(func, get_annotation(func)) if view: t.viewcg() rtyper.specialize() @@ -172,6 +183,10 @@ #return type(self)(self.a + other.a, self.b + other.b) return DemoClass(self.a + other.a, self.b + other.b) + def ab(self): + return self.a + self.b + ab = property(ab) + # see if we get things exported with subclassing class DemoSubclass(DemoClass): def __init__(self, a, b, c): @@ -252,6 +267,10 @@ def specialize_call(self, hop): v_idx, = hop.inputargs(*hop.args_r) + if hop.args_r[0] <> rint.signed_repr: + v_idx = hop.llops.convertvar(v_idx, + r_from=hop.args_r[0], + r_to=rint.signed_repr) v_res = hop.llops.gencapicall('postsetup_get_methodname', [v_idx], resulttype=robject.pyobj_repr) return v_res @@ -265,6 +284,12 @@ def specialize_call(self, hop): v_idx, v_type = hop.inputargs(*hop.args_r) + if hop.args_r[0] <> rint.signed_repr: + v_idx = hop.llops.convertvar(v_idx, + r_from=hop.args_r[0], + r_to=rint.signed_repr) + assert hop.args_r[1] == robject.pyobj_repr, ( + 'build_method works for Python types only') v_res = hop.llops.gencapicall('postsetup_build_method', [v_idx, v_type], resulttype=robject.pyobj_repr) return v_res @@ -278,6 +303,8 @@ def specialize_call(self, hop): v_type, = hop.inputargs(*hop.args_r) + assert hop.args_r[0] == robject.pyobj_repr, ( + 'get_typedict works for Python types only') v_res = hop.llops.gencapicall('postsetup_get_typedict', [v_type], resulttype=robject.pyobj_repr) return v_res @@ -307,6 +334,43 @@ dic[methname] = meth idx += 1 +def __init__(mod): + import types + import __builtin__ as bltn + hasattr = bltn.hasattr + isinstance = bltn.isinstance + + funcs = bltn.dict() # no hashing function for PyObject + idx = 0 + while 1: + name = get_methodname(idx) + if not name: + break + func = getattr(mod, name) + funcs[func] = idx + idx += 1 + + for name in mod.__all__: + obj = getattr(mod, name) + if isinstance(obj, type) and hasattr(obj, '__self__'): + cls = obj + dic = get_typedict(cls) + for name, value in dic.items(): + if isinstance(value, types.BuiltinFunctionType) and value in funcs: + idx = funcs[value] + meth = build_method(idx, cls) + dic[name] = meth + elif isinstance(value, property): + stuff = [value.fget, value.fset, value.fdel, value.__doc__] + for i, fn in enumerate(stuff): + if fn in funcs: + idx = funcs[fn] + stuff[i] = build_method(idx, cls) + if not stuff[-1]: + # use fget's doc if we don't ahve one + stuff[-1] = getattr(stuff[0], '__doc__', None) + dic[name] = property(*stuff) + # creating an object, wrapping, unwrapping, call function, check whether __del__ is called def test_wrap_call_dtor(): f = getcompiled(democlass_helper, use_boehm=not True, exports=[DemoSubclass]) @@ -317,7 +381,7 @@ # exposing and using classes from a generasted extension module def test_expose_classes(): m = get_compiled_module(democlass_helper2, use_boehm=not True, exports=[ - DemoClass, DemoSubclass, DemoNotAnnotated, __init__]) + DemoClass, DemoSubclass, __init__, DemoNotAnnotated]) obj = m.DemoClass(2, 3) res = obj.demo() assert res == DemoClass(2, 3).demo() Modified: pypy/dist/pypy/translator/c/wrapper.py ============================================================================== --- pypy/dist/pypy/translator/c/wrapper.py (original) +++ pypy/dist/pypy/translator/c/wrapper.py Sat Apr 29 03:43:15 2006 @@ -101,12 +101,14 @@ inputconst(Signed, nb_positional_args), vargs, ] - newops.genop('check_no_more_arg', vlist) + newops.genop('check_no_more_arg', vlist, resulttype=Signed) # use the rtyper to produce the conversions inputargs = f._obj.graph.getargs() if as_method: varguments.insert(0, vself) + vlist = [vfname, vself] + newops.genop('check_self_nonzero', vlist, resulttype=Signed) for i in range(len(varguments)): if FUNCTYPE.ARGS[i] != PyObjPtr: # "argument_i = type_conversion_operations(argument_i)" From nik at codespeak.net Sat Apr 29 04:18:39 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 04:18:39 +0200 (CEST) Subject: [pypy-svn] r26543 - in pypy/dist/pypy/rpython: . lltypesystem Message-ID: <20060429021839.9F2BA10089@code0.codespeak.net> Author: nik Date: Sat Apr 29 04:18:34 2006 New Revision: 26543 Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py pypy/dist/pypy/rpython/rstr.py Log: fix to the rstr reorganisation. this fixes a translation problem. sorry about that. Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rstr.py Sat Apr 29 04:18:34 2006 @@ -1,5 +1,5 @@ from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr, \ - ll_strconcat + ll_strconcat, do_stringformat from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, Signed class StringRepr(AbstractStringRepr): Modified: pypy/dist/pypy/rpython/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/rstr.py (original) +++ pypy/dist/pypy/rpython/rstr.py Sat Apr 29 04:18:34 2006 @@ -247,7 +247,8 @@ return hop.gendirectcall(llfn, v_str, v_index) def rtype_mod(_, hop): - return do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) + rstr = hop.rtyper.type_system.rstr + return rstr.do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) class __extend__(pairtype(AbstractStringRepr, SliceRepr)): @@ -313,7 +314,8 @@ resulttype=Bool) def rtype_mod(_, hop): - return do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) + rstr = hop.rtyper.type_system.rstr + return rstr.do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) class __extend__(pairtype(AbstractStringRepr, CharRepr)): def rtype_contains(_, hop): @@ -410,7 +412,8 @@ resulttype=r_arg) sourcevars.append((vitem, r_arg)) - return do_stringformat(hop, sourcevars) + rstr = hop.rtyper.type_system.rstr + return rstr.do_stringformat(hop, sourcevars) class __extend__(CharRepr): From nik at codespeak.net Sat Apr 29 04:45:02 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 04:45:02 +0200 (CEST) Subject: [pypy-svn] r26544 - in pypy/dist/pypy/translator/squeak: . test Message-ID: <20060429024502.4C87D10094@code0.codespeak.net> Author: nik Date: Sat Apr 29 04:44:56 2006 New Revision: 26544 Modified: pypy/dist/pypy/translator/squeak/gensqueak.py pypy/dist/pypy/translator/squeak/test/runtest.py pypy/dist/pypy/translator/squeak/test/test_oo.py Log: don't let the GenSqueak constructor have so many side-effects. have a separate method "gen" to actually start source generation. Modified: pypy/dist/pypy/translator/squeak/gensqueak.py ============================================================================== --- pypy/dist/pypy/translator/squeak/gensqueak.py (original) +++ pypy/dist/pypy/translator/squeak/gensqueak.py Sat Apr 29 04:44:56 2006 @@ -22,6 +22,7 @@ self.generated_nodes = set() self.constant_insts = {} + def gen(self): graph = self.translator.graphs[0] self.pending_nodes.append(FunctionNode(self, graph)) self.filename = '%s.st' % graph.name @@ -30,6 +31,7 @@ self.pending_nodes.append(SetupNode(self, self.constant_insts)) self.gen_source(file) file.close() + return self.filename def gen_source(self, file): while self.pending_nodes: Modified: pypy/dist/pypy/translator/squeak/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/squeak/test/runtest.py (original) +++ pypy/dist/pypy/translator/squeak/test/runtest.py Sat Apr 29 04:44:56 2006 @@ -67,7 +67,9 @@ self.graph = t.graphs[0] if conftest.option.view: t.viewcg() - return GenSqueak(udir, t) + gen = GenSqueak(udir, t) + gen.gen() + return gen def _write_startup(self): startup_st = udir.join("startup.st") Modified: pypy/dist/pypy/translator/squeak/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/squeak/test/test_oo.py (original) +++ pypy/dist/pypy/translator/squeak/test/test_oo.py Sat Apr 29 04:44:56 2006 @@ -13,7 +13,8 @@ t.buildrtyper(type_system="ootype").specialize() if view or conftest.option.view: t.viewcg() - GenSqueak(udir, t) + gen = GenSqueak(udir, t) + gen.gen() C = Instance("test", ROOT, {'a': (Signed, 3)}) From ac at codespeak.net Sat Apr 29 04:45:32 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sat, 29 Apr 2006 04:45:32 +0200 (CEST) Subject: [pypy-svn] r26545 - pypy/dist/pypy/rpython/rctypes/tool Message-ID: <20060429024532.33B5A1009B@code0.codespeak.net> Author: ac Date: Sat Apr 29 04:45:31 2006 New Revision: 26545 Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Log: (aleale, arre) Fieldnames may not start with '_' Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Sat Apr 29 04:45:31 2006 @@ -187,7 +187,7 @@ for i, cell in enumerate(layout): if cell is not None: continue - layout_addfield(layout, i, ctypes.c_char, '_pad%d' % (n,)) + layout_addfield(layout, i, ctypes.c_char, 'pad%d' % (n,)) n += 1 # build the ctypes Structure From ac at codespeak.net Sat Apr 29 04:46:23 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sat, 29 Apr 2006 04:46:23 +0200 (CEST) Subject: [pypy-svn] r26546 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060429024623.CC1B410089@code0.codespeak.net> Author: ac Date: Sat Apr 29 04:46:22 2006 New Revision: 26546 Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: (aleale, arre) Support calling a function taking a c_void_p with a stringbuffer. Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rvoid_p.py Sat Apr 29 04:46:22 2006 @@ -1,5 +1,30 @@ -from pypy.rpython.rctypes.rmodel import CTypesValueRepr - +from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO +from pypy.rpython.rctypes.rstringbuf import StringBufRepr +from pypy.annotation.pairtype import pairtype +from pypy.rpython.rctypes.rchar_p import CCharPRepr +from pypy.rpython.lltypesystem import llmemory class CVoidPRepr(CTypesValueRepr): pass # No operations supported on c_void_p instances so far + +class __extend__(pairtype(StringBufRepr, CVoidPRepr)): + def convert_from_to((r_from, r_to), v, llops): + # warning: no keepalives, only for short-lived conversions like + # in argument passing + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + v_c_array = r_from.get_c_data_of_item(llops, v, C_ZERO) + v_adr = llops.genop('cast_ptr_to_adr', [v_c_array], + resulttype = llmemory.Address) + r_temp.setvalue(llops, v_owned_box, v_adr) + return llops.convertvar(v_owned_box, r_temp, r_to) + # XXX some code duplication above + +class __extend__(pairtype(CCharPRepr, CVoidPRepr)): + def convert_from_to((r_from, r_to), v, llops): + v_ptr = r_from.getvalue(llops, v) + v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], + resulttype = llmemory.Address) + + return r_to.return_value(llops, v_adr) + Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Sat Apr 29 04:46:22 2006 @@ -11,10 +11,11 @@ from pypy.translator.c.test.test_genc import compile from pypy import conftest from pypy.rpython.lltypesystem.rstr import string_repr -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from ctypes import cdll, pythonapi, PyDLL, _FUNCFLAG_PYTHONAPI -from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer +from ctypes import c_int, c_long, c_char_p, c_void_p, c_char +from ctypes import create_string_buffer, cast from ctypes import POINTER, py_object, byref, Structure from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 @@ -63,6 +64,18 @@ ##PyIntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int, callconv=PyDLL) ##pycallback = PyIntIntCallback(mycallback) +def ll_memcpy(dst, src, length): + C_ARRAY = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) + c_src = llmemory.cast_adr_to_ptr(src, C_ARRAY) + c_dst = llmemory.cast_adr_to_ptr(dst, C_ARRAY) + for i in range(length): + c_dst[i] = c_src[i] + return dst + +memcpy = mylib.memcpy +memcpy.argtypes = [c_void_p, c_void_p, c_long] +memcpy.restype = c_void_p +memcpy.llinterp_friendly_version = ll_memcpy def test_labs(n=6): assert labs(n) == abs(n) @@ -159,6 +172,18 @@ a.build_types(ep, []) if conftest.option.view: a.translator.view() + def test_annotate_call_void_p_arg_with_stringbuf(self): + string = 'abc xyz' + def f(x): + buf = create_string_buffer(len(string) + 1) + res = memcpy(buf, string, len(string)) + return buf.value + a = RPythonAnnotator() + s = a.build_types(f, [int]) + if conftest.option.view: + a.translator.view() + assert s.knowntype == str + ## def test_annotate_callback(self): ## def fn(n): @@ -232,6 +257,17 @@ res = interpret(fn, [11]) assert res == 42 + def test_specialize_call_void_p_arg_with_stringbuf(self): + string = 'abc xyz' + def f(): + buf = create_string_buffer(len(string) + 1) + res = memcpy(buf, c_char_p(string), len(string)) + return buf.value + assert f() == string + res = interpret(f, []) + assert ''.join(res.chars) == string + + class Test_compile: def test_compile_labs(self): fn = compile(test_labs, [int]) @@ -300,3 +336,14 @@ s1 = time.ctime(N) s2 = fn(N) assert s1.strip() == s2.strip() + + def test_compile_call_void_p_arg_with_stringbuf(self): + string = 'abc xyz' + def f(): + buf = create_string_buffer(len(string) + 1) + res = memcpy(buf, c_char_p(string), len(string)) + return buf.value + assert f() == string + fn = compile(f, []) + assert fn() == string + From nik at codespeak.net Sat Apr 29 04:51:37 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 04:51:37 +0200 (CEST) Subject: [pypy-svn] r26547 - in pypy/dist/pypy/translator: . goal Message-ID: <20060429025137.62B5010091@code0.codespeak.net> Author: nik Date: Sat Apr 29 04:51:32 2006 New Revision: 26547 Modified: pypy/dist/pypy/translator/driver.py pypy/dist/pypy/translator/goal/translate.py Log: enable squeak translations of standalone targets. Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Sat Apr 29 04:51:32 2006 @@ -375,6 +375,24 @@ task_run_cl = taskdef(task_run_cl, ['compile_cl'], 'XXX') + def task_source_squeak(self): + from pypy.translator.squeak.gensqueak import GenSqueak + self.gen = GenCL(dir, self.translator) + filename = self.gen.gen() + self.log.info("Wrote %s" % (filename,)) + task_source_squeak = taskdef(task_source_squeak, ['ootype'], + 'Generating Squeak source') + + def task_compile_squeak(self): + pass + task_compile_squeak = taskdef(task_compile_squeak, ['source_squeak'], + 'XXX') + + def task_run_squeak(self): + pass + task_run_squeak = taskdef(task_run_squeak, ['compile_squeak'], + 'XXX') + def proceed(self, goals): if not goals: if self.default_goal: Modified: pypy/dist/pypy/translator/goal/translate.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate.py (original) +++ pypy/dist/pypy/translator/goal/translate.py Sat Apr 29 04:51:32 2006 @@ -46,7 +46,7 @@ '0_source': [OPT(('-s', '--source'), "Generate source code", GOAL), OPT(('--no-source',), "Don't generate source code", SKIP_GOAL)], - '1_backend': [OPT(('-b', '--backend'), "Backend", ['c', 'llvm', 'cl'])], + '1_backend': [OPT(('-b', '--backend'), "Backend", ['c', 'llvm', 'cl', 'squeak'])], '2_gc': [OPT(('--gc',), "Garbage collector", ['boehm', 'ref', 'framework', 'none'])], '3_stackless': [OPT(('--stackless',), "Stackless code generation", True)], From nik at codespeak.net Sat Apr 29 05:07:35 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 05:07:35 +0200 (CEST) Subject: [pypy-svn] r26548 - pypy/dist/pypy/translator Message-ID: <20060429030735.23D0010095@code0.codespeak.net> Author: nik Date: Sat Apr 29 05:07:32 2006 New Revision: 26548 Modified: pypy/dist/pypy/translator/driver.py Log: fix typo (thanks sanxiyn) Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Sat Apr 29 05:07:32 2006 @@ -377,7 +377,7 @@ def task_source_squeak(self): from pypy.translator.squeak.gensqueak import GenSqueak - self.gen = GenCL(dir, self.translator) + self.gen = GenSqueak(dir, self.translator) filename = self.gen.gen() self.log.info("Wrote %s" % (filename,)) task_source_squeak = taskdef(task_source_squeak, ['ootype'], From ac at codespeak.net Sat Apr 29 05:20:26 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sat, 29 Apr 2006 05:20:26 +0200 (CEST) Subject: [pypy-svn] r26549 - pypy/dist/pypy/rpython/test Message-ID: <20060429032026.930ED1009D@code0.codespeak.net> Author: ac Date: Sat Apr 29 05:20:26 2006 New Revision: 26549 Modified: pypy/dist/pypy/rpython/test/test_rstr.py Log: (aleale, arre) A failing test for nik. Modified: pypy/dist/pypy/rpython/test/test_rstr.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rstr.py (original) +++ pypy/dist/pypy/rpython/test/test_rstr.py Sat Apr 29 05:20:26 2006 @@ -302,6 +302,9 @@ res = interpret(lambda: ''.join(['abc', 'de', 'fghi']), []) assert ''.join(res.chars) == "abcdefghi" + + #res = interpret(lambda: '.'.join(['abc', 'def']), []) + #assert ''.join(res.chars) == 'abc.def' def fn(i, j): s1 = [ '', ',', ' and '] From nik at codespeak.net Sat Apr 29 05:31:30 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 05:31:30 +0200 (CEST) Subject: [pypy-svn] r26550 - in pypy/dist/pypy/rpython: lltypesystem test Message-ID: <20060429033130.4CEC61009D@code0.codespeak.net> Author: nik Date: Sat Apr 29 05:31:26 2006 New Revision: 26550 Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py pypy/dist/pypy/rpython/test/test_rstr.py Log: fix another rstr-related translation failure, uncovered by anders + anders. Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rstr.py Sat Apr 29 05:31:26 2006 @@ -1,4 +1,4 @@ -from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr, \ +from pypy.rpython.rstr import AbstractStringRepr, char_repr, STR, AbstractStringIteratorRepr, \ ll_strconcat, do_stringformat from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, Signed @@ -115,6 +115,11 @@ i += 1 return result +char_repr.ll_strip = ll_strip +char_repr.ll_upper = ll_upper +char_repr.ll_lower = ll_lower +char_repr.ll_join = ll_join + string_repr = StringRepr() emptystr = string_repr.convert_const("") Modified: pypy/dist/pypy/rpython/test/test_rstr.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rstr.py (original) +++ pypy/dist/pypy/rpython/test/test_rstr.py Sat Apr 29 05:31:26 2006 @@ -303,8 +303,8 @@ res = interpret(lambda: ''.join(['abc', 'de', 'fghi']), []) assert ''.join(res.chars) == "abcdefghi" - #res = interpret(lambda: '.'.join(['abc', 'def']), []) - #assert ''.join(res.chars) == 'abc.def' + res = interpret(lambda: '.'.join(['abc', 'def']), []) + assert ''.join(res.chars) == 'abc.def' def fn(i, j): s1 = [ '', ',', ' and '] From nik at codespeak.net Sat Apr 29 06:14:49 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 06:14:49 +0200 (CEST) Subject: [pypy-svn] r26551 - in pypy/dist/pypy/rpython: lltypesystem ootypesystem Message-ID: <20060429041449.CFE5910094@code0.codespeak.net> Author: nik Date: Sat Apr 29 06:14:44 2006 New Revision: 26551 Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py pypy/dist/pypy/rpython/ootypesystem/ootype.py pypy/dist/pypy/rpython/ootypesystem/rstr.py Log: pasing test for string constants in ootypesystem. Modified: pypy/dist/pypy/rpython/lltypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/rstr.py Sat Apr 29 06:14:44 2006 @@ -1,6 +1,9 @@ +from weakref import WeakValueDictionary from pypy.rpython.rstr import AbstractStringRepr, char_repr, STR, AbstractStringIteratorRepr, \ - ll_strconcat, do_stringformat -from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, Signed + ll_strconcat, do_stringformat, ll_strhash +from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, nullptr, Signed + +CONST_STR_CACHE = WeakValueDictionary() class StringRepr(AbstractStringRepr): @@ -13,6 +16,22 @@ self.ll_lower = ll_lower self.ll_join = ll_join + def convert_const(self, value): + if value is None: + return nullptr(STR) + #value = getattr(value, '__self__', value) # for bound string methods + if not isinstance(value, str): + raise TyperError("not a str: %r" % (value,)) + try: + return CONST_STR_CACHE[value] + except KeyError: + p = malloc(STR, len(value)) + for i in range(len(value)): + p.chars[i] = value[i] + ll_strhash(p) # precompute the hash + CONST_STR_CACHE[value] = p + return p + def make_iterator_repr(self): return string_iterator_repr Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sat Apr 29 06:14:44 2006 @@ -181,6 +181,13 @@ StaticMethod.__init__(self, args, result) +class String(OOType): + + def _defl(self): + return "" + +String = String() + class BuiltinType(OOType): def _example(self): @@ -642,6 +649,10 @@ else: instance_impl = _instance +def make_string(value): + assert isinstance(value, str) + return _string(value) + def make_instance(INSTANCE): inst = _instance(INSTANCE) if STATICNESS: @@ -724,6 +735,9 @@ callb, checked_args = self.meth._checkargs(args) return callb(self.inst, *checked_args) +class _string(str): + _TYPE = String + class _builtin_type(object): def __getattribute__(self, name): TYPE = object.__getattribute__(self, "_TYPE") Modified: pypy/dist/pypy/rpython/ootypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rstr.py Sat Apr 29 06:14:44 2006 @@ -1,10 +1,18 @@ from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr from pypy.rpython.lltypesystem.lltype import Ptr -from pypy.rpython.ootypesystem.ootype import Signed, Record +from pypy.rpython.ootypesystem.ootype import Signed, Record, String, make_string class StringRepr(AbstractStringRepr): - lowleveltype = Ptr(STR) + lowleveltype = String + + def convert_const(self, value): + # XXX what do we do about null strings? + #if value is None: + # return nullptr(STR) + if not isinstance(value, str): + raise TyperError("not a str: %r" % (value,)) + return make_string(value) def make_iterator_repr(self): return string_iterator_repr From nik at codespeak.net Sat Apr 29 06:18:08 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 06:18:08 +0200 (CEST) Subject: [pypy-svn] r26552 - pypy/dist/pypy/rpython/ootypesystem/test Message-ID: <20060429041808.8120210095@code0.codespeak.net> Author: nik Date: Sat Apr 29 06:18:04 2006 New Revision: 26552 Added: pypy/dist/pypy/rpython/ootypesystem/test/test_oostring.py (contents, props changed) Log: actually add the test file that was supposed to go with the last commit ... Added: pypy/dist/pypy/rpython/ootypesystem/test/test_oostring.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oostring.py Sat Apr 29 06:18:04 2006 @@ -0,0 +1,8 @@ +from pypy.rpython.test.test_llinterp import interpret + +def test_constant_string(): + def f(): + return "foo" + res = interpret(f, [], type_system="ootype") + assert res == "foo" + From ericvrp at codespeak.net Sat Apr 29 06:22:09 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 06:22:09 +0200 (CEST) Subject: [pypy-svn] r26553 - in pypy/branch/picklesupport: doc/discussion interpreter interpreter/test module/__builtin__ module/_pickle_support objspace/flow rpython rpython/lltypesystem rpython/ootypesystem rpython/rctypes rpython/rctypes/test rpython/rctypes/tool rpython/test Message-ID: <20060429042209.017051009F@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 06:21:31 2006 New Revision: 26553 Added: pypy/branch/picklesupport/doc/discussion/cli-optimizations.txt - copied unchanged from r26551, pypy/dist/pypy/doc/discussion/cli-optimizations.txt Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py pypy/branch/picklesupport/interpreter/typedef.py pypy/branch/picklesupport/module/__builtin__/__init__.py pypy/branch/picklesupport/module/__builtin__/app_functional.py pypy/branch/picklesupport/module/_pickle_support/maker.py pypy/branch/picklesupport/objspace/flow/model.py pypy/branch/picklesupport/rpython/llinterp.py pypy/branch/picklesupport/rpython/lltypesystem/lloperation.py pypy/branch/picklesupport/rpython/lltypesystem/rstr.py pypy/branch/picklesupport/rpython/ootypesystem/ootype.py pypy/branch/picklesupport/rpython/ootypesystem/rstr.py pypy/branch/picklesupport/rpython/rctypes/rpointer.py pypy/branch/picklesupport/rpython/rctypes/rstringbuf.py pypy/branch/picklesupport/rpython/rctypes/rvoid_p.py pypy/branch/picklesupport/rpython/rctypes/test/test_rfunc.py pypy/branch/picklesupport/rpython/rctypes/test/test_rstruct.py pypy/branch/picklesupport/rpython/rctypes/tool/ctypes_platform.py pypy/branch/picklesupport/rpython/robject.py pypy/branch/picklesupport/rpython/rstr.py pypy/branch/picklesupport/rpython/test/test_rstr.py Log: get changes from the trunk Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py ============================================================================== --- pypy/branch/picklesupport/interpreter/test/test_pickle.py (original) +++ pypy/branch/picklesupport/interpreter/test/test_pickle.py Sat Apr 29 06:21:31 2006 @@ -128,15 +128,39 @@ def test_pickle_method(self): skip("work in progress") - class C(object): + class myclass(object): def f(self): pass import pickle - method = C.f + method = myclass.f pckl = pickle.dumps(method) result = pickle.loads(pckl) assert method == result + def test_pickle_staticmethod(self): + skip("work in progress") + class myclass(object): + def f(self): + pass + f = staticmethod(f) + import pickle + method = myclass.f + pckl = pickle.dumps(method) + result = pickle.loads(pckl) + assert method == result + + def test_pickle_classmethod(self): + skip("work in progress") + class myclass(object): + def f(self): + pass + f = classmethod(f) + import pickle + method = myclass.f + pckl = pickle.dumps(method) + result = pickle.loads(pckl) + assert method == result + def test_pickle_dictiter(self): skip("work in progress") import pickle @@ -146,7 +170,7 @@ assert diter == result def test_pickle_enum(self): - skip("work in progress") + #skip("work in progress") import pickle e = enumerate([]) pckl = pickle.dumps(e) @@ -170,12 +194,14 @@ assert liter == result def test_pickle_xrangeiter(self): - skip("work in progress") import pickle riter = iter(xrange(5)) + riter.next() + riter.next() pckl = pickle.dumps(riter) result = pickle.loads(pckl) - assert riter == result + assert type(riter) is type(result) + assert list(result) == [2,3,4] def test_pickle_generator(self): skip("work in progress") Modified: pypy/branch/picklesupport/interpreter/typedef.py ============================================================================== --- pypy/branch/picklesupport/interpreter/typedef.py (original) +++ pypy/branch/picklesupport/interpreter/typedef.py Sat Apr 29 06:21:31 2006 @@ -23,20 +23,20 @@ # xxx used by faking self.fakedcpytype = None self.add_entries(**rawdict) - + def add_entries(self, **rawdict): # xxx fix the names of the methods to match what app-level expects for key, value in rawdict.items(): if isinstance(value, (interp2app, GetSetProperty)): value.name = key self.rawdict.update(rawdict) - + def _freeze_(self): # hint for the annotator: track individual constant instances of TypeDef return True -def get_unique_interplevel_subclass(cls, hasdict, wants_slots, needsdel=False): +def get_unique_interplevel_subclass(cls, hasdict, wants_slots, needsdel=False): key = cls, hasdict, wants_slots, needsdel try: return _subclass_cache[key] @@ -53,10 +53,10 @@ def _buildusercls(cls, hasdict, wants_slots, wants_del): "NOT_RPYTHON: initialization-time only" typedef = cls.typedef - + if hasdict and typedef.hasdict: return get_unique_interplevel_subclass(cls, False, wants_slots, wants_del) - + name = ['User'] if not hasdict: name.append('NoDict') @@ -65,9 +65,9 @@ if wants_del: name.append('WithDel') name.append(cls.__name__) - + name = ''.join(name) - + if wants_del: supercls = get_unique_interplevel_subclass(cls, hasdict, wants_slots, False) class Proto(object): @@ -79,29 +79,29 @@ e.clear(self.space) # break up reference cycles elif wants_slots: supercls = get_unique_interplevel_subclass(cls, hasdict, False, False) - + class Proto(object): def user_setup_slots(self, nslots): - self.slots_w = [None] * nslots - + self.slots_w = [None] * nslots + def setslotvalue(self, index, w_value): self.slots_w[index] = w_value - + def getslotvalue(self, index): return self.slots_w[index] elif hasdict: supercls = get_unique_interplevel_subclass(cls, False, False, False) - + class Proto(object): def getdict(self): return self.w__dict__ - + def setdict(self, space, w_dict): if not space.is_true(space.isinstance(w_dict, space.w_dict)): raise OperationError(space.w_TypeError, space.wrap("setting dictionary to a non-dict")) self.w__dict__ = w_dict - + def user_setup(self, space, w_subtype, nslots): self.space = space self.w__class__ = w_subtype @@ -111,29 +111,29 @@ supercls = cls class Proto(object): - + def getclass(self, space): return self.w__class__ - + def setclass(self, space, w_subtype): # only used by descr_set___class__ self.w__class__ = w_subtype - - + + def user_setup(self, space, w_subtype, nslots): self.space = space self.w__class__ = w_subtype self.user_setup_slots(nslots) - + def user_setup_slots(self, nslots): assert nslots == 0 - + body = dict([(key, value) for key, value in Proto.__dict__.items() if not key.startswith('_') or key == '__del__']) subcls = type(name, (supercls,), body) - + return subcls def make_descr_typecheck_wrapper(func, extraargs=(), cls=None): @@ -144,7 +144,7 @@ if hasattr(func, 'im_func'): assert func.im_class is cls func = func.im_func - + miniglobals = { func.__name__: func, 'OperationError': OperationError @@ -171,12 +171,12 @@ return %(name)s(space, obj, %(extra)s) """ miniglobals[cls_name] = cls - + name = func.__name__ extra = ', '.join(extraargs) source = py.code.Source(source % locals()) exec source.compile() in miniglobals - return miniglobals['descr_typecheck_%s' % func.__name__] + return miniglobals['descr_typecheck_%s' % func.__name__] def unknown_objclass_getter(space): raise OperationError(space.w_TypeError, @@ -213,16 +213,16 @@ def __init__(self, fget, fset=None, fdel=None, doc=None, cls=None): "NOT_RPYTHON: initialization-time only" objclass_getter, cls = make_objclass_getter(fget, cls) - fget = make_descr_typecheck_wrapper(fget, cls=cls) + fget = make_descr_typecheck_wrapper(fget, cls=cls) fset = make_descr_typecheck_wrapper(fset, ('w_value',), cls=cls) - fdel = make_descr_typecheck_wrapper(fdel, cls=cls) + fdel = make_descr_typecheck_wrapper(fdel, cls=cls) self.fget = fget self.fset = fset self.fdel = fdel self.doc = doc self.name = '' self.objclass_getter = objclass_getter - + def descr_property_get(space, property, w_obj, w_cls=None): """property.__get__(obj[, type]) -> value Read the value of the property of the given obj.""" @@ -233,7 +233,7 @@ return space.wrap(property) else: return property.fget(space, w_obj) - + def descr_property_set(space, property, w_obj, w_value): """property.__set__(obj, value) Change the value of the property of the given obj.""" @@ -242,7 +242,7 @@ raise OperationError(space.w_TypeError, space.wrap("readonly attribute")) fset(space, w_obj, w_value) - + def descr_property_del(space, property, w_obj): """property.__delete__(obj) Delete the value of the property from the given obj.""" @@ -251,7 +251,7 @@ raise OperationError(space.w_AttributeError, space.wrap("cannot delete attribute")) fdel(space, w_obj) - + def descr_get_objclass(space, property): return property.objclass_getter(space) @@ -268,8 +268,8 @@ if w_value is None: return space.w_None else: - return w_value - + return w_value + return GetSetProperty(fget, cls=cls) GetSetProperty.typedef = TypeDef( @@ -294,13 +294,13 @@ self.index = index self.name = name self.w_cls = w_cls - + def typecheck(self, space, w_obj): if not space.is_true(space.isinstance(w_obj, self.w_cls)): raise OperationError(space.w_TypeError, space.wrap("descriptor '%s' for '%s' objects doesn't apply to '%s' object" % (self.name, self.w_cls.name, space.type(w_obj).name))) - + def descr_member_get(space, member, w_obj, w_w_cls=None): """member.__get__(obj[, type]) -> value Read the slot 'member' of the given 'obj'.""" @@ -314,14 +314,14 @@ raise OperationError(space.w_AttributeError, space.wrap(self.name)) # XXX better message return w_result - + def descr_member_set(space, member, w_obj, w_value): """member.__set__(obj, value) Write into the slot 'member' of the given 'obj'.""" self = member self.typecheck(space, w_obj) w_obj.setslotvalue(self.index, w_value) - + def descr_member_del(space, member, w_obj): """member.__delete__(obj) Delete the value of the slot 'member' from the given 'obj'.""" @@ -355,7 +355,7 @@ from pypy.interpreter.function import Function, Method, StaticMethod from pypy.interpreter.function import BuiltinFunction, descr_function_get from pypy.interpreter.pytraceback import PyTraceback -from pypy.interpreter.generator import GeneratorIterator +from pypy.interpreter.generator import GeneratorIterator from pypy.interpreter.nestedscope import Cell from pypy.interpreter.special import NotImplemented, Ellipsis @@ -415,7 +415,7 @@ __new__ = interp2app(PyCode.descr_code__new__.im_func), __eq__ = interp2app(PyCode.descr_code__eq__), __ne__ = descr_generic_ne, - __reduce__ = interp2app(PyCode.descr__reduce__, + __reduce__ = interp2app(PyCode.descr__reduce__, unwrap_spec=['self', ObjSpace]), co_argcount = interp_attrproperty('co_argcount', cls=PyCode), co_nlocals = interp_attrproperty('co_nlocals', cls=PyCode), @@ -438,7 +438,7 @@ f_lineno = GetSetProperty(PyFrame.fget_f_lineno, PyFrame.fset_f_lineno), f_back = GetSetProperty(PyFrame.fget_f_back), f_lasti = GetSetProperty(PyFrame.fget_f_lasti), - f_trace = GetSetProperty(PyFrame.fget_f_trace, PyFrame.fset_f_trace, + f_trace = GetSetProperty(PyFrame.fget_f_trace, PyFrame.fset_f_trace, PyFrame.fdel_f_trace), f_exc_type = GetSetProperty(PyFrame.fget_f_exc_type), f_exc_value = GetSetProperty(PyFrame.fget_f_exc_value), @@ -450,7 +450,7 @@ __new__ = interp2app(Module.descr_module__new__.im_func, unwrap_spec=[ObjSpace, W_Root, Arguments]), __init__ = interp2app(Module.descr_module__init__), - __reduce__ = interp2app(Module.descr__reduce__, + __reduce__ = interp2app(Module.descr__reduce__, unwrap_spec=['self', ObjSpace]), __dict__ = GetSetProperty(descr_get_dict, cls=Module), # module dictionaries are readonly attributes __doc__ = 'module(name[, doc])\n\nCreate a module object.\nThe name must be a string; the optional doc argument can have any type.' @@ -480,14 +480,14 @@ getset_func_dict = GetSetProperty(descr_get_dict, descr_set_dict, cls=Function) Function.typedef = TypeDef("function", - __new__ = interp2app(Function.descr_method__new__.im_func), + __new__ = interp2app(Function.descr_method__new__.im_func), __call__ = interp2app(Function.descr_function_call, unwrap_spec=['self', Arguments]), __get__ = interp2app(descr_function_get), __repr__ = interp2app(Function.descr_function_repr), - __reduce__ = interp2app(Function.descr_function__reduce__, + __reduce__ = interp2app(Function.descr_function__reduce__, unwrap_spec=['self', ObjSpace]), - func_code = getset_func_code, + func_code = getset_func_code, func_doc = getset_func_doc, func_name = getset_func_name, func_dict = getset_func_dict, @@ -506,16 +506,16 @@ __call__ = interp2app(Method.descr_method_call, unwrap_spec=['self', Arguments]), __get__ = interp2app(Method.descr_method_get), - im_func = interp_attrproperty_w('w_function', cls=Method), - im_self = interp_attrproperty_w('w_instance', cls=Method), + im_func = interp_attrproperty_w('w_function', cls=Method), + im_self = interp_attrproperty_w('w_instance', cls=Method), im_class = interp_attrproperty_w('w_class', cls=Method), __getattribute__ = interp2app(Method.descr_method_getattribute), __eq__ = interp2app(Method.descr_method_eq), __ne__ = descr_generic_ne, - __repr__ = interp2app(Method.descr_method_repr), - __reduce__ = interp2app(Method.descr_method__reduce__, - unwrap_spec=['self', ObjSpace]), - # XXX getattribute/setattribute etc.pp + __repr__ = interp2app(Method.descr_method_repr), + __reduce__ = interp2app(Method.descr_method__reduce__, + unwrap_spec=['self', ObjSpace]), + # XXX getattribute/setattribute etc.pp ) StaticMethod.typedef = TypeDef("staticmethod", @@ -543,31 +543,29 @@ GeneratorIterator.typedef = TypeDef("generator", next = interp2app(GeneratorIterator.descr_next), __iter__ = interp2app(GeneratorIterator.descr__iter__), - gi_running = interp_attrproperty('running', cls=GeneratorIterator), - gi_frame = interp_attrproperty('frame', cls=GeneratorIterator), + gi_running = interp_attrproperty('running', cls=GeneratorIterator), + gi_frame = interp_attrproperty('frame', cls=GeneratorIterator), ) Cell.typedef = TypeDef("cell", __eq__ = interp2app(Cell.descr__eq__, unwrap_spec=['self', ObjSpace, W_Root]), __ne__ = descr_generic_ne, - __reduce__ = interp2app(Cell.descr__reduce__, + __reduce__ = interp2app(Cell.descr__reduce__, unwrap_spec=['self', ObjSpace]), __setstate__ = interp2app(Cell.descr__setstate__, - unwrap_spec=['self', ObjSpace, W_Root]), + unwrap_spec=['self', ObjSpace, W_Root]), ) -Ellipsis.typedef = TypeDef("Ellipsis", +Ellipsis.typedef = TypeDef("Ellipsis", __repr__ = interp2app(Ellipsis.descr__repr__), ) -NotImplemented.typedef = TypeDef("NotImplemented", - __repr__ = interp2app(NotImplemented.descr__repr__), +NotImplemented.typedef = TypeDef("NotImplemented", + __repr__ = interp2app(NotImplemented.descr__repr__), ) ControlFlowException.typedef = TypeDef("ControlFlowException") interptypes = [ val.typedef for name,val in globals().items() if hasattr(val,'__bases__') and hasattr(val,'typedef') ] - - Modified: pypy/branch/picklesupport/module/__builtin__/__init__.py ============================================================================== --- pypy/branch/picklesupport/module/__builtin__/__init__.py (original) +++ pypy/branch/picklesupport/module/__builtin__/__init__.py Sat Apr 29 06:21:31 2006 @@ -31,6 +31,8 @@ 'any' : 'app_functional.any', 'enumerate' : 'app_functional.enumerate', 'xrange' : 'app_functional.xrange', + '_install_pickle_support_for_xrange_iterator': + 'app_functional._install_pickle_support_for_xrange_iterator', 'sorted' : 'app_functional.sorted', 'reversed' : 'app_functional.reversed', @@ -137,3 +139,15 @@ builtin = module.Module(space, None) space.setitem(builtin.w_dict, space.wrap('None'), space.w_None) return builtin + + def setup_after_space_initialization(self): + """NOT_RPYTHON""" + space = self.space + # call installations for pickle support + for name in self.loaders.keys(): + if name.startswith('_install_pickle_support_for_'): + w_install = self.get(name) + space.call_function(w_install) + # xxx hide the installer + space.delitem(self.w_dict, space.wrap(name)) + del self.loaders[name] \ No newline at end of file Modified: pypy/branch/picklesupport/module/__builtin__/app_functional.py ============================================================================== --- pypy/branch/picklesupport/module/__builtin__/app_functional.py (original) +++ pypy/branch/picklesupport/module/__builtin__/app_functional.py Sat Apr 29 06:21:31 2006 @@ -362,6 +362,18 @@ def __len__(self): return self._remaining + def __reduce__(self): + tup = (self._current, self._remaining, self._step) + return (make_xrange_iterator, tup) + +def make_xrange_iterator(*args): + return xrange_iterator(*args) + +def _install_pickle_support_for_xrange_iterator(): + import _pickle_support + make_xrange_iterator.__module__ = '_pickle_support' + _pickle_support.make_xrange_iterator = make_xrange_iterator + # ____________________________________________________________ def sorted(lst, cmp=None, key=None, reverse=None): Modified: pypy/branch/picklesupport/module/_pickle_support/maker.py ============================================================================== --- pypy/branch/picklesupport/module/_pickle_support/maker.py (original) +++ pypy/branch/picklesupport/module/_pickle_support/maker.py Sat Apr 29 06:21:31 2006 @@ -24,11 +24,11 @@ return space.call_args(w_type, __args__) func_new.unwrap_spec = [ObjSpace, Arguments] -def module_new(space, w_name, w_dict): #XXX untested +def module_new(space, w_name, w_dict): new_mod = Module(space, w_name, w_dict) return space.wrap(new_mod) -def method_new(space, __args__): #XXX untested +def method_new(space, __args__): w_type = space.gettypeobject(Method.typedef) return space.call_args(w_type, __args__) method_new.unwrap_spec = [ObjSpace, Arguments] Modified: pypy/branch/picklesupport/objspace/flow/model.py ============================================================================== --- pypy/branch/picklesupport/objspace/flow/model.py (original) +++ pypy/branch/picklesupport/objspace/flow/model.py Sat Apr 29 06:21:31 2006 @@ -67,12 +67,14 @@ return getsource(self.func) source = roproperty(getsource) - def __repr__(self): + def __str__(self): if hasattr(self, 'func'): - fnrepr = nice_repr_for_func(self.func, self.name) + return nice_repr_for_func(self.func, self.name) else: - fnrepr = self.name - return '' % (fnrepr, uid(self)) + return self.name + + def __repr__(self): + return '' % (self, uid(self)) def iterblocks(self): block = self.startblock Modified: pypy/branch/picklesupport/rpython/llinterp.py ============================================================================== --- pypy/branch/picklesupport/rpython/llinterp.py (original) +++ pypy/branch/picklesupport/rpython/llinterp.py Sat Apr 29 06:21:31 2006 @@ -36,6 +36,8 @@ class LLInterpreter(object): """ low level interpreter working with concrete values. """ + TRACING = True + def __init__(self, typer, heap=lltype): self.bindings = {} self.typer = typer @@ -47,19 +49,41 @@ if hasattr(heap, "prepare_graphs_and_create_gc"): flowgraphs = typer.annotator.translator.graphs self.gc = heap.prepare_graphs_and_create_gc(self, flowgraphs) + if self.TRACING: + self.tracer = Tracer() + else: + self.tracer = None def eval_graph(self, graph, args=()): llframe = LLFrame(graph, args, self) + if self.tracer: + self.tracer.start() + retval = None try: - return llframe.eval() - except LLException, e: - log.error("LLEXCEPTION: %s" % (e, )) - self.print_traceback() - raise - except Exception, e: - log.error("AN ERROR OCCURED: %s" % (e, )) - self.print_traceback() - raise + try: + retval = llframe.eval() + except LLException, e: + log.error("LLEXCEPTION: %s" % (e, )) + self.print_traceback() + if self.tracer: + self.tracer.dump('LLException: %s\n' % (e,)) + raise + except Exception, e: + log.error("AN ERROR OCCURED: %s" % (e, )) + self.print_traceback() + if self.tracer: + line = str(e) + if line: + line = ': ' + line + line = '* %s' % (e.__class__.__name__,) + line + self.tracer.dump(line + '\n') + raise + finally: + if self.tracer: + if retval is not None: + self.tracer.dump(' ---> %r\n' % (retval,)) + self.tracer.stop() + return retval def print_traceback(self): frame = self.active_frame @@ -68,11 +92,12 @@ frames.append(frame) frame = frame.f_back frames.reverse() + lines = [] for frame in frames: logline = frame.graph.name if frame.curr_block is None: logline += " " - log.traceback(logline) + lines.append(logline) continue try: logline += " " + self.typer.annotator.annotated[frame.curr_block].__module__ @@ -80,13 +105,19 @@ # if the graph is from the GC it was not produced by the same # translator :-( logline += " " - log.traceback(logline) + lines.append(logline) for i, operation in enumerate(frame.curr_block.operations): if i == frame.curr_operation_index: logline = "E %s" else: logline = " %s" - log.traceback(logline % (operation, )) + lines.append(logline % (operation, )) + if self.tracer: + self.tracer.dump('Traceback\n', bold=True) + for line in lines: + self.tracer.dump(line + '\n') + for line in lines: + log.traceback(line) def find_roots(self): #log.findroots("starting") @@ -204,19 +235,25 @@ def eval(self): self.llinterpreter.active_frame = self graph = self.graph - #log.frame("evaluating", graph.name) - nextblock = graph.startblock - args = self.args - while 1: - self.clear() - self.fillvars(nextblock, args) - nextblock, args = self.eval_block(nextblock) - if nextblock is None: - self.llinterpreter.active_frame = self.f_back - for obj in self.alloca_objects: - #XXX slighly unclean - obj._setobj(None) - return args + tracer = self.llinterpreter.tracer + if tracer: + tracer.enter(graph) + try: + nextblock = graph.startblock + args = self.args + while 1: + self.clear() + self.fillvars(nextblock, args) + nextblock, args = self.eval_block(nextblock) + if nextblock is None: + self.llinterpreter.active_frame = self.f_back + for obj in self.alloca_objects: + #XXX slighly unclean + obj._setobj(None) + return args + finally: + if tracer: + tracer.leave() def eval_block(self, block): """ return (nextblock, values) tuple. If nextblock @@ -237,16 +274,20 @@ # determine nextblock and/or return value if len(block.exits) == 0: # return block + tracer = self.llinterpreter.tracer if len(block.inputargs) == 2: # exception + if tracer: + tracer.dump('raise') etypevar, evaluevar = block.getvariables() etype = self.getval(etypevar) evalue = self.getval(evaluevar) # watch out, these are _ptr's raise LLException(etype, evalue) + if tracer: + tracer.dump('return') resultvar, = block.getvariables() result = self.getval(resultvar) - #log.operation("returning", repr(result)) return None, result elif block.exitswitch is None: # single-exit block @@ -283,7 +324,9 @@ return link.target, [self.getval(x) for x in link.args] def eval_operation(self, operation): - #log.operation("considering", operation) + tracer = self.llinterpreter.tracer + if tracer: + tracer.dump(str(operation)) ophandler = self.getoperationhandler(operation.opname) # XXX slighly unnice but an important safety check if operation.opname == 'direct_call': @@ -313,6 +356,11 @@ else: self.handle_cleanup(operation) self.setvar(operation.result, retval) + if tracer: + if retval is None: + tracer.dump('\n') + else: + tracer.dump(' ---> %r\n' % (retval,)) def handle_cleanup(self, operation, exception=False): cleanup = getattr(operation, 'cleanup', None) @@ -953,6 +1001,86 @@ def op_ooidentityhash(self, inst): return ootype.ooidentityhash(inst) + +class Tracer(object): + Counter = 0 + file = None + + HEADER = """ + + + + 
+    """
+
+    FOOTER = """
""" + + ENTER = ('''\n\t%s''' + '''\n
\t''') + LEAVE = '''\n
\t''' + + def htmlquote(self, s, text_to_html={}): + # HTML quoting, lazily initialized + if not text_to_html: + import htmlentitydefs + for key, value in htmlentitydefs.entitydefs.items(): + text_to_html[value] = '&' + key + ';' + return ''.join([text_to_html.get(c, c) for c in s]) + + def start(self): + # start of a dump file + from pypy.tool.udir import udir + n = Tracer.Counter + Tracer.Counter += 1 + self.file = udir.join('llinterp_trace_%d.html' % n).open('w') + print >> self.file, self.HEADER + self.count = 0 + self.indentation = '' + + def stop(self): + # end of a dump file + if self.file: + print >> self.file, self.FOOTER + self.file.close() + self.file = None + + def enter(self, graph): + # enter evaluation of a graph + if self.file: + s = self.htmlquote(str(graph)) + i = s.rfind(')') + s = s[:i+1] + '' + s[i+1:] + '' + if self.count == 0: + display = 'block' + else: + display = 'none' + text = self.ENTER % (self.count, s, self.count, display) + self.indentation += ' ' + self.file.write(text.replace('\t', self.indentation)) + self.count += 1 + + def leave(self): + # leave evaluation of a graph + if self.file: + self.indentation = self.indentation[:-4] + self.file.write(self.LEAVE.replace('\t', self.indentation)) + + def dump(self, text, bold=False): + if self.file: + text = self.htmlquote(text) + if bold: + text = '%s' % (text,) + self.file.write(text.replace('\n', '\n'+self.indentation)) + + # by default we route all logging messages to nothingness # e.g. tests can then switch on logging to get more help # for failing tests Modified: pypy/branch/picklesupport/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/picklesupport/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/picklesupport/rpython/lltypesystem/lloperation.py Sat Apr 29 06:21:31 2006 @@ -317,6 +317,7 @@ 'same_as': LLOp(canfold=True), 'hint': LLOp(), 'check_no_more_arg': LLOp(canraise=(Exception,)), + 'check_self_nonzero': LLOp(canraise=(Exception,)), 'decode_arg': LLOp(canraise=(Exception,)), 'decode_arg_def': LLOp(canraise=(Exception,)), 'getslice': LLOp(canraise=(Exception,)), Modified: pypy/branch/picklesupport/rpython/lltypesystem/rstr.py ============================================================================== --- pypy/branch/picklesupport/rpython/lltypesystem/rstr.py (original) +++ pypy/branch/picklesupport/rpython/lltypesystem/rstr.py Sat Apr 29 06:21:31 2006 @@ -1,6 +1,9 @@ -from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr, \ - ll_strconcat -from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, Signed +from weakref import WeakValueDictionary +from pypy.rpython.rstr import AbstractStringRepr, char_repr, STR, AbstractStringIteratorRepr, \ + ll_strconcat, do_stringformat, ll_strhash +from pypy.rpython.lltypesystem.lltype import malloc, GcStruct, Ptr, nullptr, Signed + +CONST_STR_CACHE = WeakValueDictionary() class StringRepr(AbstractStringRepr): @@ -13,6 +16,22 @@ self.ll_lower = ll_lower self.ll_join = ll_join + def convert_const(self, value): + if value is None: + return nullptr(STR) + #value = getattr(value, '__self__', value) # for bound string methods + if not isinstance(value, str): + raise TyperError("not a str: %r" % (value,)) + try: + return CONST_STR_CACHE[value] + except KeyError: + p = malloc(STR, len(value)) + for i in range(len(value)): + p.chars[i] = value[i] + ll_strhash(p) # precompute the hash + CONST_STR_CACHE[value] = p + return p + def make_iterator_repr(self): return string_iterator_repr @@ -115,6 +134,11 @@ i += 1 return result +char_repr.ll_strip = ll_strip +char_repr.ll_upper = ll_upper +char_repr.ll_lower = ll_lower +char_repr.ll_join = ll_join + string_repr = StringRepr() emptystr = string_repr.convert_const("") Modified: pypy/branch/picklesupport/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/branch/picklesupport/rpython/ootypesystem/ootype.py (original) +++ pypy/branch/picklesupport/rpython/ootypesystem/ootype.py Sat Apr 29 06:21:31 2006 @@ -181,6 +181,13 @@ StaticMethod.__init__(self, args, result) +class String(OOType): + + def _defl(self): + return "" + +String = String() + class BuiltinType(OOType): def _example(self): @@ -642,6 +649,10 @@ else: instance_impl = _instance +def make_string(value): + assert isinstance(value, str) + return _string(value) + def make_instance(INSTANCE): inst = _instance(INSTANCE) if STATICNESS: @@ -724,6 +735,9 @@ callb, checked_args = self.meth._checkargs(args) return callb(self.inst, *checked_args) +class _string(str): + _TYPE = String + class _builtin_type(object): def __getattribute__(self, name): TYPE = object.__getattribute__(self, "_TYPE") Modified: pypy/branch/picklesupport/rpython/ootypesystem/rstr.py ============================================================================== --- pypy/branch/picklesupport/rpython/ootypesystem/rstr.py (original) +++ pypy/branch/picklesupport/rpython/ootypesystem/rstr.py Sat Apr 29 06:21:31 2006 @@ -1,10 +1,18 @@ from pypy.rpython.rstr import AbstractStringRepr, STR, AbstractStringIteratorRepr from pypy.rpython.lltypesystem.lltype import Ptr -from pypy.rpython.ootypesystem.ootype import Signed, Record +from pypy.rpython.ootypesystem.ootype import Signed, Record, String, make_string class StringRepr(AbstractStringRepr): - lowleveltype = Ptr(STR) + lowleveltype = String + + def convert_const(self, value): + # XXX what do we do about null strings? + #if value is None: + # return nullptr(STR) + if not isinstance(value, str): + raise TyperError("not a str: %r" % (value,)) + return make_string(value) def make_iterator_repr(self): return string_iterator_repr Modified: pypy/branch/picklesupport/rpython/rctypes/rpointer.py ============================================================================== --- pypy/branch/picklesupport/rpython/rctypes/rpointer.py (original) +++ pypy/branch/picklesupport/rpython/rctypes/rpointer.py Sat Apr 29 06:21:31 2006 @@ -9,20 +9,27 @@ class PointerRepr(CTypesValueRepr): def __init__(self, rtyper, s_pointer): - ptr_ctype = s_pointer.knowntype - ref_ctype = ptr_ctype._type_ + # For recursive types, getting the r_contents is delayed until + # _setup_repr(). + ll_contents = lltype.Ptr(lltype.ForwardReference()) + self.keepalive_box_type = lltype.GcForwardReference() + super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) + def _setup_repr(self): # Find the repr and low-level type of the contents from its ctype + rtyper = self.rtyper + ref_ctype = self.ctype._type_ self.r_contents = rtyper.getrepr(SomeCTypesObject(ref_ctype, SomeCTypesObject.MEMORYALIAS)) - - ll_contents = lltype.Ptr(self.r_contents.c_data_type) - - super(PointerRepr, self).__init__(rtyper, s_pointer, ll_contents) + if isinstance(self.ll_type.TO, lltype.ForwardReference): + self.ll_type.TO.become(self.r_contents.c_data_type) + if isinstance(self.keepalive_box_type, lltype.GcForwardReference): + self.keepalive_box_type.become( + self.r_contents.r_memoryowner.lowleveltype.TO) def get_content_keepalive_type(self): "Keepalive for the box that holds the data that 'self' points to." - return self.r_contents.r_memoryowner.lowleveltype + return lltype.Ptr(self.keepalive_box_type) def setkeepalive(self, llops, v_box, v_owner): inputargs = [v_box, inputconst(lltype.Void, 'keepalive'), Modified: pypy/branch/picklesupport/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/branch/picklesupport/rpython/rctypes/rstringbuf.py (original) +++ pypy/branch/picklesupport/rpython/rctypes/rstringbuf.py Sat Apr 29 06:21:31 2006 @@ -4,7 +4,7 @@ from pypy.rpython.rctypes.rmodel import CTypesRefRepr from pypy.objspace.flow.model import Constant from pypy.rpython.rslice import AbstractSliceRepr -from pypy.rpython.rstr import string_repr +from pypy.rpython.lltypesystem.rstr import string_repr class StringBufRepr(CTypesRefRepr): Modified: pypy/branch/picklesupport/rpython/rctypes/rvoid_p.py ============================================================================== --- pypy/branch/picklesupport/rpython/rctypes/rvoid_p.py (original) +++ pypy/branch/picklesupport/rpython/rctypes/rvoid_p.py Sat Apr 29 06:21:31 2006 @@ -1,5 +1,30 @@ -from pypy.rpython.rctypes.rmodel import CTypesValueRepr - +from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO +from pypy.rpython.rctypes.rstringbuf import StringBufRepr +from pypy.annotation.pairtype import pairtype +from pypy.rpython.rctypes.rchar_p import CCharPRepr +from pypy.rpython.lltypesystem import llmemory class CVoidPRepr(CTypesValueRepr): pass # No operations supported on c_void_p instances so far + +class __extend__(pairtype(StringBufRepr, CVoidPRepr)): + def convert_from_to((r_from, r_to), v, llops): + # warning: no keepalives, only for short-lived conversions like + # in argument passing + r_temp = r_to.r_memoryowner + v_owned_box = r_temp.allocate_instance(llops) + v_c_array = r_from.get_c_data_of_item(llops, v, C_ZERO) + v_adr = llops.genop('cast_ptr_to_adr', [v_c_array], + resulttype = llmemory.Address) + r_temp.setvalue(llops, v_owned_box, v_adr) + return llops.convertvar(v_owned_box, r_temp, r_to) + # XXX some code duplication above + +class __extend__(pairtype(CCharPRepr, CVoidPRepr)): + def convert_from_to((r_from, r_to), v, llops): + v_ptr = r_from.getvalue(llops, v) + v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], + resulttype = llmemory.Address) + + return r_to.return_value(llops, v_adr) + Modified: pypy/branch/picklesupport/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/branch/picklesupport/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/branch/picklesupport/rpython/rctypes/test/test_rfunc.py Sat Apr 29 06:21:31 2006 @@ -11,10 +11,11 @@ from pypy.translator.c.test.test_genc import compile from pypy import conftest from pypy.rpython.lltypesystem.rstr import string_repr -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from ctypes import cdll, pythonapi, PyDLL, _FUNCFLAG_PYTHONAPI -from ctypes import c_int, c_long, c_char_p, c_char, create_string_buffer +from ctypes import c_int, c_long, c_char_p, c_void_p, c_char +from ctypes import create_string_buffer, cast from ctypes import POINTER, py_object, byref, Structure from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 @@ -63,6 +64,18 @@ ##PyIntIntCallback = CALLBACK_FUNCTYPE(c_int, c_int, callconv=PyDLL) ##pycallback = PyIntIntCallback(mycallback) +def ll_memcpy(dst, src, length): + C_ARRAY = lltype.Ptr(lltype.FixedSizeArray(lltype.Char, 1)) + c_src = llmemory.cast_adr_to_ptr(src, C_ARRAY) + c_dst = llmemory.cast_adr_to_ptr(dst, C_ARRAY) + for i in range(length): + c_dst[i] = c_src[i] + return dst + +memcpy = mylib.memcpy +memcpy.argtypes = [c_void_p, c_void_p, c_long] +memcpy.restype = c_void_p +memcpy.llinterp_friendly_version = ll_memcpy def test_labs(n=6): assert labs(n) == abs(n) @@ -159,6 +172,18 @@ a.build_types(ep, []) if conftest.option.view: a.translator.view() + def test_annotate_call_void_p_arg_with_stringbuf(self): + string = 'abc xyz' + def f(x): + buf = create_string_buffer(len(string) + 1) + res = memcpy(buf, string, len(string)) + return buf.value + a = RPythonAnnotator() + s = a.build_types(f, [int]) + if conftest.option.view: + a.translator.view() + assert s.knowntype == str + ## def test_annotate_callback(self): ## def fn(n): @@ -232,6 +257,17 @@ res = interpret(fn, [11]) assert res == 42 + def test_specialize_call_void_p_arg_with_stringbuf(self): + string = 'abc xyz' + def f(): + buf = create_string_buffer(len(string) + 1) + res = memcpy(buf, c_char_p(string), len(string)) + return buf.value + assert f() == string + res = interpret(f, []) + assert ''.join(res.chars) == string + + class Test_compile: def test_compile_labs(self): fn = compile(test_labs, [int]) @@ -300,3 +336,14 @@ s1 = time.ctime(N) s2 = fn(N) assert s1.strip() == s2.strip() + + def test_compile_call_void_p_arg_with_stringbuf(self): + string = 'abc xyz' + def f(): + buf = create_string_buffer(len(string) + 1) + res = memcpy(buf, c_char_p(string), len(string)) + return buf.value + assert f() == string + fn = compile(f, []) + assert fn() == string + Modified: pypy/branch/picklesupport/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/branch/picklesupport/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/branch/picklesupport/rpython/rctypes/test/test_rstruct.py Sat Apr 29 06:21:31 2006 @@ -13,7 +13,7 @@ from pypy.rpython.test.test_llinterp import interpret from ctypes import c_int, c_short, Structure, POINTER, pointer, c_char_p -from ctypes import c_char +from ctypes import c_char, SetPointerType class tagpoint(Structure): _fields_ = [("x", c_int), @@ -151,6 +151,23 @@ res = interpret(func, []) assert res == 121 + def test_struct_with_pointer_to_self(self): + PS = POINTER('S') + class S(Structure): + _fields_ = [('l', PS), ('r', PS)] + SetPointerType(PS, S) + + def func(): + s0 = S() + s0.r.contents = s0 + s0.l.contents = S() + s0.l.contents.r.contents = s0 + + return bool(s0.r.contents.l.contents.l) + assert not func() + res = interpret(func, []) + assert res is False + def test_specialize_keepalive(self): class S(Structure): _fields_ = [('x', c_int)] Modified: pypy/branch/picklesupport/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/branch/picklesupport/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/branch/picklesupport/rpython/rctypes/tool/ctypes_platform.py Sat Apr 29 06:21:31 2006 @@ -187,7 +187,7 @@ for i, cell in enumerate(layout): if cell is not None: continue - layout_addfield(layout, i, ctypes.c_char, '_pad%d' % (n,)) + layout_addfield(layout, i, ctypes.c_char, 'pad%d' % (n,)) n += 1 # build the ctypes Structure Modified: pypy/branch/picklesupport/rpython/robject.py ============================================================================== --- pypy/branch/picklesupport/rpython/robject.py (original) +++ pypy/branch/picklesupport/rpython/robject.py Sat Apr 29 06:21:31 2006 @@ -1,7 +1,7 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.rpython.lltypesystem.lltype import \ - PyObject, Ptr, Void, pyobjectptr, nullptr + PyObject, Ptr, Void, pyobjectptr, nullptr, Bool from pypy.rpython.rmodel import Repr, VoidRepr, inputconst from pypy.rpython import rclass from pypy.tool.sourcetools import func_with_new_name @@ -69,3 +69,10 @@ for opname in annmodel.BINARY_OPERATIONS: make_operation(opname, pairtype(PyObjRepr, Repr)) make_operation(opname, pairtype(Repr, PyObjRepr)) + + +class __extend__(pairtype(PyObjRepr, PyObjRepr)): + def rtype_contains((r_seq, r_item), hop): + v_seq, v_item = hop.inputargs(r_seq, r_item) + return hop.llops.gencapicall('PySequence_Contains_with_exc', + [v_seq, v_item], resulttype=Bool) Modified: pypy/branch/picklesupport/rpython/rstr.py ============================================================================== --- pypy/branch/picklesupport/rpython/rstr.py (original) +++ pypy/branch/picklesupport/rpython/rstr.py Sat Apr 29 06:21:31 2006 @@ -247,7 +247,8 @@ return hop.gendirectcall(llfn, v_str, v_index) def rtype_mod(_, hop): - return do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) + rstr = hop.rtyper.type_system.rstr + return rstr.do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) class __extend__(pairtype(AbstractStringRepr, SliceRepr)): @@ -313,7 +314,8 @@ resulttype=Bool) def rtype_mod(_, hop): - return do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) + rstr = hop.rtyper.type_system.rstr + return rstr.do_stringformat(hop, [(hop.args_v[1], hop.args_r[1])]) class __extend__(pairtype(AbstractStringRepr, CharRepr)): def rtype_contains(_, hop): @@ -410,7 +412,8 @@ resulttype=r_arg) sourcevars.append((vitem, r_arg)) - return do_stringformat(hop, sourcevars) + rstr = hop.rtyper.type_system.rstr + return rstr.do_stringformat(hop, sourcevars) class __extend__(CharRepr): Modified: pypy/branch/picklesupport/rpython/test/test_rstr.py ============================================================================== --- pypy/branch/picklesupport/rpython/test/test_rstr.py (original) +++ pypy/branch/picklesupport/rpython/test/test_rstr.py Sat Apr 29 06:21:31 2006 @@ -302,6 +302,9 @@ res = interpret(lambda: ''.join(['abc', 'de', 'fghi']), []) assert ''.join(res.chars) == "abcdefghi" + + res = interpret(lambda: '.'.join(['abc', 'def']), []) + assert ''.join(res.chars) == 'abc.def' def fn(i, j): s1 = [ '', ',', ' and '] From ericvrp at codespeak.net Sat Apr 29 06:26:28 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 06:26:28 +0200 (CEST) Subject: [pypy-svn] r26554 - pypy/dist/lib-python/modified-2.4.1 Message-ID: <20060429042628.589681009F@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 06:26:25 2006 New Revision: 26554 Modified: pypy/dist/lib-python/modified-2.4.1/pickle.py Log: moduledict pickling support Modified: pypy/dist/lib-python/modified-2.4.1/pickle.py ============================================================================== --- pypy/dist/lib-python/modified-2.4.1/pickle.py (original) +++ pypy/dist/lib-python/modified-2.4.1/pickle.py Sat Apr 29 06:26:25 2006 @@ -212,14 +212,25 @@ self.proto = int(protocol) self.bin = protocol >= 1 self.fast = 0 - ## Stackless addition BEGIN - #XXX support not included yet in pypy + + def _pickle_moduledict(self, obj): try: - from stackless import _pickle_moduledict - except ImportError: - _pickle_moduledict = lambda self, obj:None - self._pickle_moduledict = _pickle_moduledict - ## Stackless addition END + modict = self.module_dict_ids + except AttributeError: + modict = {} + from sys import modules + for mod in modules.values(): + if isinstance(mod, ModuleType): + modict[id(mod.__dict__)] = mod + self.module_dict_ids = modict + + thisid = id(obj) + try: + themodule = modict[thisid] + except KeyError: + return None + from __builtin__ import getattr + return getattr, (themodule, '__dict__') def clear_memo(self): """Clears the pickler's "memo". @@ -661,10 +672,11 @@ def save_dict(self, obj): ## Stackless addition BEGIN - modict_saver = self._pickle_moduledict(self, obj) + modict_saver = self._pickle_moduledict(obj) if modict_saver is not None: return self.save_reduce(*modict_saver) ## Stackless addition END + write = self.write if self.bin: From ericvrp at codespeak.net Sat Apr 29 06:27:41 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 06:27:41 +0200 (CEST) Subject: [pypy-svn] r26555 - pypy/branch/picklesupport/interpreter/test Message-ID: <20060429042741.D7315100A3@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 06:27:39 2006 New Revision: 26555 Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py Log: oops, this test actually is not passing yet Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py ============================================================================== --- pypy/branch/picklesupport/interpreter/test/test_pickle.py (original) +++ pypy/branch/picklesupport/interpreter/test/test_pickle.py Sat Apr 29 06:27:39 2006 @@ -170,7 +170,7 @@ assert diter == result def test_pickle_enum(self): - #skip("work in progress") + skip("work in progress") import pickle e = enumerate([]) pckl = pickle.dumps(e) From dialtone at codespeak.net Sat Apr 29 06:47:26 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Sat, 29 Apr 2006 06:47:26 +0200 (CEST) Subject: [pypy-svn] r26556 - pypy/dist/pypy/translator/cl Message-ID: <20060429044726.1F74910094@code0.codespeak.net> Author: dialtone Date: Sat Apr 29 06:47:22 2006 New Revision: 26556 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: use accessors to avoid violating the clhs for conditions, introduce accessors for all objects (classes included) except for Records that still use the old way Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Sat Apr 29 06:47:22 2006 @@ -147,15 +147,25 @@ def op_oogetfield(self, result, obj, _): fieldname = self.args[1].value - yield "(setf %s (slot-value %s '%s))" % (clrepr(result, True), - clrepr(obj, True), - clrepr(fieldname, True)) + if isinstance(self.args[0].concretetype, Record): + yield "(setf %s (slot-value %s '%s))" % (clrepr(result, True), + clrepr(obj, True), + clrepr(fieldname, True)) + else: + yield "(setf %s (%s %s))" % (clrepr(result, True), + clrepr(fieldname, True), + clrepr(obj, True)) def op_oosetfield(self, result, obj, _, value): fieldname = self.args[1].value - yield "(setf (slot-value %s '%s) %s)" % (clrepr(obj, True), - clrepr(fieldname, True), - clrepr(value, True)) + if isinstance(self.args[0].concretetype, Record): + yield "(setf (slot-value %s '%s) %s)" % (clrepr(obj, True), + clrepr(fieldname, True), + clrepr(value, True)) + else: + yield "(setf (%s %s) %s)" % (clrepr(fieldname, True), + clrepr(obj, True), + clrepr(value, True)) def op_ooidentityhash(self, result, arg): yield "(setf %s (sxhash %s))" % (clrepr(result, True), @@ -225,7 +235,10 @@ if cls in self.declarations: return self.declarations[cls][0] name = clrepr(cls._name, symbol=True) - field_declaration = ['('+clrepr(field, True)+')' for field in cls._fields] + field_declaration = [] + for field in cls._fields: + field = clrepr(field, True) + field_declaration.append('('+field+' :accessor '+field+')') field_declaration = " ".join(field_declaration) if cls._superclass is ROOT: class_declaration = "(defclass %s () (%s))" % (name, field_declaration) @@ -247,8 +260,7 @@ return self.declarations[cls][0] name = clrepr(cls._name, symbol=True) if cls._superclass is OBJECT: - supername = self.declare_class(OBJECT) - exception_declaration = "(define-condition %s (condition %s) ())" % (name, supername) + exception_declaration = "(define-condition %s () ((meta :accessor meta)))" % (name) else: supername = self.declare_exception(cls._superclass) exception_declaration = "(define-condition %s (%s) ())" % (name, supername) From ale at codespeak.net Sat Apr 29 06:48:21 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sat, 29 Apr 2006 06:48:21 +0200 (CEST) Subject: [pypy-svn] r26557 - pypy/dist/lib-python/modified-2.4.1 Message-ID: <20060429044821.3E32410094@code0.codespeak.net> Author: ale Date: Sat Apr 29 06:48:17 2006 New Revision: 26557 Added: pypy/dist/lib-python/modified-2.4.1/tarfile.py - copied, changed from r26540, pypy/dist/lib-python/2.4.1/tarfile.py Log: issue 13 : testing The test_tarfile passes on CPython, because it relies on base conversion of strings to ints ignoring null characters ie. Python 2.4.2 (#1, Apr 24 2006, 10:25:58) [GCC 4.0.0 20041026 (Apple Computer, Inc. build 4061)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> int('123\x00',10) 123 This behaviour is only if use base conversion ie. >>> int('123\x00') Traceback (most recent call last): File "", line 1, in ? ValueError: null byte in argument for int() Rather than changing PyPy's behaviour I have patched tarfile.py, to replace int and long with functions truncating the inputstrings on null characters From ericvrp at codespeak.net Sat Apr 29 08:10:04 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 08:10:04 +0200 (CEST) Subject: [pypy-svn] r26558 - in pypy/branch/picklesupport: interpreter/test rpython/ootypesystem/test Message-ID: <20060429061004.6F9EE10095@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 08:09:58 2006 New Revision: 26558 Added: pypy/branch/picklesupport/rpython/ootypesystem/test/test_oostring.py - copied unchanged from r26557, pypy/dist/pypy/rpython/ootypesystem/test/test_oostring.py Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py Log: get changes from the trunk (again) Modified: pypy/branch/picklesupport/interpreter/test/test_pickle.py ============================================================================== --- pypy/branch/picklesupport/interpreter/test/test_pickle.py (original) +++ pypy/branch/picklesupport/interpreter/test/test_pickle.py Sat Apr 29 08:09:58 2006 @@ -170,7 +170,7 @@ assert diter == result def test_pickle_enum(self): - skip("work in progress") + #skip("work in progress") import pickle e = enumerate([]) pckl = pickle.dumps(e) From nik at codespeak.net Sat Apr 29 08:19:59 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 08:19:59 +0200 (CEST) Subject: [pypy-svn] r26559 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060429061959.75C7C10097@code0.codespeak.net> Author: nik Date: Sat Apr 29 08:19:54 2006 New Revision: 26559 Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py Log: return a runtime string from _defl and _example of the String type. Modified: pypy/dist/pypy/rpython/ootypesystem/ootype.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/ootype.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/ootype.py Sat Apr 29 08:19:54 2006 @@ -184,7 +184,10 @@ class String(OOType): def _defl(self): - return "" + return make_string("") + + def _example(self): + return self._defl() String = String() From ac at codespeak.net Sat Apr 29 08:24:32 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sat, 29 Apr 2006 08:24:32 +0200 (CEST) Subject: [pypy-svn] r26560 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060429062432.C3C881008B@code0.codespeak.net> Author: ac Date: Sat Apr 29 08:24:32 2006 New Revision: 26560 Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Log: (aleale, arre) Support conversion POINTER(c_type) -> c_void_p. Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rvoid_p.py Sat Apr 29 08:24:32 2006 @@ -3,6 +3,7 @@ from pypy.annotation.pairtype import pairtype from pypy.rpython.rctypes.rchar_p import CCharPRepr from pypy.rpython.lltypesystem import llmemory +from pypy.rpython.rctypes.rpointer import PointerRepr class CVoidPRepr(CTypesValueRepr): pass # No operations supported on c_void_p instances so far @@ -28,3 +29,11 @@ return r_to.return_value(llops, v_adr) +class __extend__(pairtype(PointerRepr, CVoidPRepr)): + def convert_from_to((r_from, r_to), v, llops): + v_ptr = r_from.getvalue(llops, v) + v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], + resulttype = llmemory.Address) + + return r_to.return_value(llops, v_adr) + Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Sat Apr 29 08:24:32 2006 @@ -176,13 +176,27 @@ string = 'abc xyz' def f(x): buf = create_string_buffer(len(string) + 1) - res = memcpy(buf, string, len(string)) + res = memcpy(buf, c_char_p(string), len(string)) return buf.value a = RPythonAnnotator() s = a.build_types(f, [int]) if conftest.option.view: a.translator.view() assert s.knowntype == str + + def test_annotate_call_void_p_arg_with_ptr(self): + string = 'abc xyz' + c_char_ptr = POINTER(c_char) + def f(): + buf = create_string_buffer(len(string) + 1) + cp = cast(buf, c_char_ptr) + res = memcpy(cp, c_char_p(string), len(string)) + return buf.value + a = RPythonAnnotator() + s = a.build_types(f, []) + if conftest.option.view: + a.translator.view() + assert s.knowntype == str ## def test_annotate_callback(self): @@ -268,6 +282,17 @@ assert ''.join(res.chars) == string + def test_specialize_call_void_p_arg_with_ptr(self): + string = 'abc xyz' + c_char_ptr = POINTER(c_char) + def f(): + buf = create_string_buffer(len(string) + 1) + cp = cast(buf, c_char_ptr) + res = memcpy(cp, c_char_p(string), len(string)) + return buf.value + res = interpret(f, []) + assert ''.join(res.chars) == string + class Test_compile: def test_compile_labs(self): fn = compile(test_labs, [int]) @@ -347,3 +372,14 @@ fn = compile(f, []) assert fn() == string + def test_compile_call_void_p_arg_with_ptr(self): + string = 'abc xyz' + c_char_ptr = POINTER(c_char) + def f(): + buf = create_string_buffer(len(string) + 1) + cp = cast(buf, c_char_ptr) + res = memcpy(cp, c_char_p(string), len(string)) + return buf.value + fn = compile(f, []) + assert fn() == string + From ericvrp at codespeak.net Sat Apr 29 09:27:46 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 09:27:46 +0200 (CEST) Subject: [pypy-svn] r26563 - in pypy/dist/pypy: interpreter interpreter/test module/__builtin__ module/_pickle_support Message-ID: <20060429072746.756F31009B@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 09:27:37 2006 New Revision: 26563 Modified: pypy/dist/pypy/interpreter/function.py pypy/dist/pypy/interpreter/module.py pypy/dist/pypy/interpreter/test/test_pickle.py pypy/dist/pypy/interpreter/typedef.py pypy/dist/pypy/module/__builtin__/__init__.py pypy/dist/pypy/module/__builtin__/app_functional.py pypy/dist/pypy/module/_pickle_support/__init__.py pypy/dist/pypy/module/_pickle_support/maker.py Log: Merge picklesupport brancj back into the trunk. Modified: pypy/dist/pypy/interpreter/function.py ============================================================================== --- pypy/dist/pypy/interpreter/function.py (original) +++ pypy/dist/pypy/interpreter/function.py Sat Apr 29 09:27:37 2006 @@ -169,7 +169,7 @@ def descr_function_repr(self): return self.getrepr(self.space, 'function %s' % (self.name,)) - def descr__reduce__(self, space): + def descr_function__reduce__(self, space): from pypy.interpreter.mixedmodule import MixedModule w_mod = space.getbuiltinmodule('_pickle_support') mod = space.interp_w(MixedModule, w_mod) @@ -181,7 +181,6 @@ w_closure = space.newtuple([w(cell) for cell in self.closure]) tup = [ w(self.code), - #space.newdict([]), #XXX because pickle.py has no _pickle_moduledict yet... self.w_func_globals, w(self.name), space.newtuple(self.defs_w), @@ -390,6 +389,19 @@ return space.w_False return space.eq(self.w_function, other.w_function) + def descr_method__reduce__(self, space): + from pypy.interpreter.mixedmodule import MixedModule + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + new_inst = mod.get('method_new') + w = space.wrap + w_instance = self.w_instance or space.w_None + if space.is_w( self.w_class, space.w_None ): + tup = [self.w_function, w_instance] + else: + tup = [self.w_function, w_instance, self.w_class] + return space.newtuple([new_inst, space.newtuple(tup)]) + class StaticMethod(Wrappable): """A static method. Note that there is one class staticmethod at app-level too currently; this is only used for __new__ methods.""" Modified: pypy/dist/pypy/interpreter/module.py ============================================================================== --- pypy/dist/pypy/interpreter/module.py (original) +++ pypy/dist/pypy/interpreter/module.py Sat Apr 29 09:27:37 2006 @@ -35,3 +35,24 @@ w_doc = space.w_None space.setitem(self.w_dict, space.new_interned_str('__name__'), w_name) space.setitem(self.w_dict, space.new_interned_str('__doc__'), w_doc) + + def descr__reduce__(self, space): + w_name = space.finditem(self.w_dict, space.wrap('__name__')) + if (w_name is None or + not space.is_true(space.isinstance(w_name, space.w_str))): + # maybe raise exception here (XXX this path is untested) + return space.w_None + w_modules = space.sys.get('modules') + if space.finditem(w_modules, w_name) is None: + #not imported case + from pypy.interpreter.mixedmodule import MixedModule + w_mod = space.getbuiltinmodule('_pickle_support') + mod = space.interp_w(MixedModule, w_mod) + new_inst = mod.get('module_new') + return space.newtuple([new_inst, space.newtuple([w_name, + self.getdict()]), + ]) + #already imported case + w_import = space.builtin.get('__import__') + return space.newtuple([w_import, space.newtuple([w_name])]) + Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Sat Apr 29 09:27:37 2006 @@ -24,21 +24,40 @@ assert func is result del sys.modules['mod'] + def test_pickle_not_imported_module(self): + import new + mod = new.module('mod') + mod.__dict__['a'] = 1 + import pickle + pckl = pickle.dumps(mod) + result = pickle.loads(pckl) + assert mod.__name__ == result.__name__ + assert mod.__dict__ == result.__dict__ + #print mod.__dict__ + def test_pickle_builtin_func(self): import pickle pckl = pickle.dumps(map) result = pickle.loads(pckl) assert map is result - def test_pickle_nested_func(self): - skip("work in progress") + def test_pickle_non_top_reachable_func(self): def func(): return 42 + global a + a = 42 + del globals()['test_pickle_non_top_reachable_func'] import pickle pckl = pickle.dumps(func) result = pickle.loads(pckl) - assert func == result - + assert func.func_name == result.func_name + assert func.func_closure == result.func_closure + assert func.func_code == result.func_code + assert func.func_defaults == result.func_defaults + assert func.func_dict == result.func_dict + assert func.func_doc == result.func_doc + assert func.func_globals == result.func_globals + def test_pickle_cell(self): def g(): x = [42] @@ -83,40 +102,65 @@ result = pickle.loads(pckl) assert tb == result - def test_pickle_module(self): #XXX this passes for the wrong reason! - skip("work in progress") - def f(): - pass + def test_pickle_module(self): import pickle - mod = f.__module__ #XXX returns a string? + mod = pickle pckl = pickle.dumps(mod) result = pickle.loads(pckl) - assert mod == result + assert mod is result - def test_pickle_moduledict(self): #XXX this test is not correct! - skip("work in progress") - def f(): - pass + def test_pickle_moduledict(self): import pickle - modedict = f.__module__.__dict__ + moddict = pickle.__dict__ pckl = pickle.dumps(moddict) result = pickle.loads(pckl) - assert mod == result + assert moddict is result + + def test_pickle_bltins_module(self): + import pickle + mod = __builtins__ + pckl = pickle.dumps(mod) + result = pickle.loads(pckl) + assert mod is result def test_pickle_iter(self): skip("work in progress") def test_pickle_method(self): skip("work in progress") - class C(object): + class myclass(object): def f(self): pass import pickle - method = C.f + method = myclass.f pckl = pickle.dumps(method) result = pickle.loads(pckl) assert method == result + def test_pickle_staticmethod(self): + skip("work in progress") + class myclass(object): + def f(self): + pass + f = staticmethod(f) + import pickle + method = myclass.f + pckl = pickle.dumps(method) + result = pickle.loads(pckl) + assert method == result + + def test_pickle_classmethod(self): + skip("work in progress") + class myclass(object): + def f(self): + pass + f = classmethod(f) + import pickle + method = myclass.f + pckl = pickle.dumps(method) + result = pickle.loads(pckl) + assert method == result + def test_pickle_dictiter(self): skip("work in progress") import pickle @@ -127,9 +171,14 @@ def test_pickle_enum(self): skip("work in progress") - - def test_pickle_enumfactory(self): - skip("work in progress") + import pickle + e = enumerate([]) + pckl = pickle.dumps(e) + result = pickle.loads(pckl) + assert e == result + + #def test_pickle_enumfactory(self): + # skip("work in progress") def test_pickle_sequenceiter(self): ''' @@ -144,13 +193,15 @@ result = pickle.loads(pckl) assert liter == result - def test_pickle_rangeiter(self): - skip("work in progress") + def test_pickle_xrangeiter(self): import pickle riter = iter(xrange(5)) + riter.next() + riter.next() pckl = pickle.dumps(riter) result = pickle.loads(pckl) - assert riter == result + assert type(riter) is type(result) + assert list(result) == [2,3,4] def test_pickle_generator(self): skip("work in progress") Modified: pypy/dist/pypy/interpreter/typedef.py ============================================================================== --- pypy/dist/pypy/interpreter/typedef.py (original) +++ pypy/dist/pypy/interpreter/typedef.py Sat Apr 29 09:27:37 2006 @@ -23,20 +23,20 @@ # xxx used by faking self.fakedcpytype = None self.add_entries(**rawdict) - + def add_entries(self, **rawdict): # xxx fix the names of the methods to match what app-level expects for key, value in rawdict.items(): if isinstance(value, (interp2app, GetSetProperty)): value.name = key self.rawdict.update(rawdict) - + def _freeze_(self): # hint for the annotator: track individual constant instances of TypeDef return True -def get_unique_interplevel_subclass(cls, hasdict, wants_slots, needsdel=False): +def get_unique_interplevel_subclass(cls, hasdict, wants_slots, needsdel=False): key = cls, hasdict, wants_slots, needsdel try: return _subclass_cache[key] @@ -53,10 +53,10 @@ def _buildusercls(cls, hasdict, wants_slots, wants_del): "NOT_RPYTHON: initialization-time only" typedef = cls.typedef - + if hasdict and typedef.hasdict: return get_unique_interplevel_subclass(cls, False, wants_slots, wants_del) - + name = ['User'] if not hasdict: name.append('NoDict') @@ -65,9 +65,9 @@ if wants_del: name.append('WithDel') name.append(cls.__name__) - + name = ''.join(name) - + if wants_del: supercls = get_unique_interplevel_subclass(cls, hasdict, wants_slots, False) class Proto(object): @@ -79,29 +79,29 @@ e.clear(self.space) # break up reference cycles elif wants_slots: supercls = get_unique_interplevel_subclass(cls, hasdict, False, False) - + class Proto(object): def user_setup_slots(self, nslots): - self.slots_w = [None] * nslots - + self.slots_w = [None] * nslots + def setslotvalue(self, index, w_value): self.slots_w[index] = w_value - + def getslotvalue(self, index): return self.slots_w[index] elif hasdict: supercls = get_unique_interplevel_subclass(cls, False, False, False) - + class Proto(object): def getdict(self): return self.w__dict__ - + def setdict(self, space, w_dict): if not space.is_true(space.isinstance(w_dict, space.w_dict)): raise OperationError(space.w_TypeError, space.wrap("setting dictionary to a non-dict")) self.w__dict__ = w_dict - + def user_setup(self, space, w_subtype, nslots): self.space = space self.w__class__ = w_subtype @@ -111,29 +111,29 @@ supercls = cls class Proto(object): - + def getclass(self, space): return self.w__class__ - + def setclass(self, space, w_subtype): # only used by descr_set___class__ self.w__class__ = w_subtype - - + + def user_setup(self, space, w_subtype, nslots): self.space = space self.w__class__ = w_subtype self.user_setup_slots(nslots) - + def user_setup_slots(self, nslots): assert nslots == 0 - + body = dict([(key, value) for key, value in Proto.__dict__.items() if not key.startswith('_') or key == '__del__']) subcls = type(name, (supercls,), body) - + return subcls def make_descr_typecheck_wrapper(func, extraargs=(), cls=None): @@ -144,7 +144,7 @@ if hasattr(func, 'im_func'): assert func.im_class is cls func = func.im_func - + miniglobals = { func.__name__: func, 'OperationError': OperationError @@ -171,12 +171,12 @@ return %(name)s(space, obj, %(extra)s) """ miniglobals[cls_name] = cls - + name = func.__name__ extra = ', '.join(extraargs) source = py.code.Source(source % locals()) exec source.compile() in miniglobals - return miniglobals['descr_typecheck_%s' % func.__name__] + return miniglobals['descr_typecheck_%s' % func.__name__] def unknown_objclass_getter(space): raise OperationError(space.w_TypeError, @@ -213,16 +213,16 @@ def __init__(self, fget, fset=None, fdel=None, doc=None, cls=None): "NOT_RPYTHON: initialization-time only" objclass_getter, cls = make_objclass_getter(fget, cls) - fget = make_descr_typecheck_wrapper(fget, cls=cls) + fget = make_descr_typecheck_wrapper(fget, cls=cls) fset = make_descr_typecheck_wrapper(fset, ('w_value',), cls=cls) - fdel = make_descr_typecheck_wrapper(fdel, cls=cls) + fdel = make_descr_typecheck_wrapper(fdel, cls=cls) self.fget = fget self.fset = fset self.fdel = fdel self.doc = doc self.name = '' self.objclass_getter = objclass_getter - + def descr_property_get(space, property, w_obj, w_cls=None): """property.__get__(obj[, type]) -> value Read the value of the property of the given obj.""" @@ -233,7 +233,7 @@ return space.wrap(property) else: return property.fget(space, w_obj) - + def descr_property_set(space, property, w_obj, w_value): """property.__set__(obj, value) Change the value of the property of the given obj.""" @@ -242,7 +242,7 @@ raise OperationError(space.w_TypeError, space.wrap("readonly attribute")) fset(space, w_obj, w_value) - + def descr_property_del(space, property, w_obj): """property.__delete__(obj) Delete the value of the property from the given obj.""" @@ -251,7 +251,7 @@ raise OperationError(space.w_AttributeError, space.wrap("cannot delete attribute")) fdel(space, w_obj) - + def descr_get_objclass(space, property): return property.objclass_getter(space) @@ -268,8 +268,8 @@ if w_value is None: return space.w_None else: - return w_value - + return w_value + return GetSetProperty(fget, cls=cls) GetSetProperty.typedef = TypeDef( @@ -294,13 +294,13 @@ self.index = index self.name = name self.w_cls = w_cls - + def typecheck(self, space, w_obj): if not space.is_true(space.isinstance(w_obj, self.w_cls)): raise OperationError(space.w_TypeError, space.wrap("descriptor '%s' for '%s' objects doesn't apply to '%s' object" % (self.name, self.w_cls.name, space.type(w_obj).name))) - + def descr_member_get(space, member, w_obj, w_w_cls=None): """member.__get__(obj[, type]) -> value Read the slot 'member' of the given 'obj'.""" @@ -314,14 +314,14 @@ raise OperationError(space.w_AttributeError, space.wrap(self.name)) # XXX better message return w_result - + def descr_member_set(space, member, w_obj, w_value): """member.__set__(obj, value) Write into the slot 'member' of the given 'obj'.""" self = member self.typecheck(space, w_obj) w_obj.setslotvalue(self.index, w_value) - + def descr_member_del(space, member, w_obj): """member.__delete__(obj) Delete the value of the slot 'member' from the given 'obj'.""" @@ -355,7 +355,7 @@ from pypy.interpreter.function import Function, Method, StaticMethod from pypy.interpreter.function import BuiltinFunction, descr_function_get from pypy.interpreter.pytraceback import PyTraceback -from pypy.interpreter.generator import GeneratorIterator +from pypy.interpreter.generator import GeneratorIterator from pypy.interpreter.nestedscope import Cell from pypy.interpreter.special import NotImplemented, Ellipsis @@ -415,7 +415,7 @@ __new__ = interp2app(PyCode.descr_code__new__.im_func), __eq__ = interp2app(PyCode.descr_code__eq__), __ne__ = descr_generic_ne, - __reduce__ = interp2app(PyCode.descr__reduce__, + __reduce__ = interp2app(PyCode.descr__reduce__, unwrap_spec=['self', ObjSpace]), co_argcount = interp_attrproperty('co_argcount', cls=PyCode), co_nlocals = interp_attrproperty('co_nlocals', cls=PyCode), @@ -438,7 +438,7 @@ f_lineno = GetSetProperty(PyFrame.fget_f_lineno, PyFrame.fset_f_lineno), f_back = GetSetProperty(PyFrame.fget_f_back), f_lasti = GetSetProperty(PyFrame.fget_f_lasti), - f_trace = GetSetProperty(PyFrame.fget_f_trace, PyFrame.fset_f_trace, + f_trace = GetSetProperty(PyFrame.fget_f_trace, PyFrame.fset_f_trace, PyFrame.fdel_f_trace), f_exc_type = GetSetProperty(PyFrame.fget_f_exc_type), f_exc_value = GetSetProperty(PyFrame.fget_f_exc_value), @@ -450,6 +450,8 @@ __new__ = interp2app(Module.descr_module__new__.im_func, unwrap_spec=[ObjSpace, W_Root, Arguments]), __init__ = interp2app(Module.descr_module__init__), + __reduce__ = interp2app(Module.descr__reduce__, + unwrap_spec=['self', ObjSpace]), __dict__ = GetSetProperty(descr_get_dict, cls=Module), # module dictionaries are readonly attributes __doc__ = 'module(name[, doc])\n\nCreate a module object.\nThe name must be a string; the optional doc argument can have any type.' ) @@ -478,14 +480,14 @@ getset_func_dict = GetSetProperty(descr_get_dict, descr_set_dict, cls=Function) Function.typedef = TypeDef("function", - __new__ = interp2app(Function.descr_method__new__.im_func), + __new__ = interp2app(Function.descr_method__new__.im_func), __call__ = interp2app(Function.descr_function_call, unwrap_spec=['self', Arguments]), __get__ = interp2app(descr_function_get), __repr__ = interp2app(Function.descr_function_repr), - __reduce__ = interp2app(Function.descr__reduce__, + __reduce__ = interp2app(Function.descr_function__reduce__, unwrap_spec=['self', ObjSpace]), - func_code = getset_func_code, + func_code = getset_func_code, func_doc = getset_func_doc, func_name = getset_func_name, func_dict = getset_func_dict, @@ -504,14 +506,16 @@ __call__ = interp2app(Method.descr_method_call, unwrap_spec=['self', Arguments]), __get__ = interp2app(Method.descr_method_get), - im_func = interp_attrproperty_w('w_function', cls=Method), - im_self = interp_attrproperty_w('w_instance', cls=Method), + im_func = interp_attrproperty_w('w_function', cls=Method), + im_self = interp_attrproperty_w('w_instance', cls=Method), im_class = interp_attrproperty_w('w_class', cls=Method), __getattribute__ = interp2app(Method.descr_method_getattribute), __eq__ = interp2app(Method.descr_method_eq), __ne__ = descr_generic_ne, - __repr__ = interp2app(Method.descr_method_repr), - # XXX getattribute/setattribute etc.pp + __repr__ = interp2app(Method.descr_method_repr), + __reduce__ = interp2app(Method.descr_method__reduce__, + unwrap_spec=['self', ObjSpace]), + # XXX getattribute/setattribute etc.pp ) StaticMethod.typedef = TypeDef("staticmethod", @@ -539,31 +543,29 @@ GeneratorIterator.typedef = TypeDef("generator", next = interp2app(GeneratorIterator.descr_next), __iter__ = interp2app(GeneratorIterator.descr__iter__), - gi_running = interp_attrproperty('running', cls=GeneratorIterator), - gi_frame = interp_attrproperty('frame', cls=GeneratorIterator), + gi_running = interp_attrproperty('running', cls=GeneratorIterator), + gi_frame = interp_attrproperty('frame', cls=GeneratorIterator), ) Cell.typedef = TypeDef("cell", __eq__ = interp2app(Cell.descr__eq__, unwrap_spec=['self', ObjSpace, W_Root]), __ne__ = descr_generic_ne, - __reduce__ = interp2app(Cell.descr__reduce__, + __reduce__ = interp2app(Cell.descr__reduce__, unwrap_spec=['self', ObjSpace]), __setstate__ = interp2app(Cell.descr__setstate__, - unwrap_spec=['self', ObjSpace, W_Root]), + unwrap_spec=['self', ObjSpace, W_Root]), ) -Ellipsis.typedef = TypeDef("Ellipsis", +Ellipsis.typedef = TypeDef("Ellipsis", __repr__ = interp2app(Ellipsis.descr__repr__), ) -NotImplemented.typedef = TypeDef("NotImplemented", - __repr__ = interp2app(NotImplemented.descr__repr__), +NotImplemented.typedef = TypeDef("NotImplemented", + __repr__ = interp2app(NotImplemented.descr__repr__), ) ControlFlowException.typedef = TypeDef("ControlFlowException") interptypes = [ val.typedef for name,val in globals().items() if hasattr(val,'__bases__') and hasattr(val,'typedef') ] - - Modified: pypy/dist/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/__init__.py (original) +++ pypy/dist/pypy/module/__builtin__/__init__.py Sat Apr 29 09:27:37 2006 @@ -31,6 +31,8 @@ 'any' : 'app_functional.any', 'enumerate' : 'app_functional.enumerate', 'xrange' : 'app_functional.xrange', + '_install_pickle_support_for_xrange_iterator': + 'app_functional._install_pickle_support_for_xrange_iterator', 'sorted' : 'app_functional.sorted', 'reversed' : 'app_functional.reversed', @@ -137,3 +139,15 @@ builtin = module.Module(space, None) space.setitem(builtin.w_dict, space.wrap('None'), space.w_None) return builtin + + def setup_after_space_initialization(self): + """NOT_RPYTHON""" + space = self.space + # call installations for pickle support + for name in self.loaders.keys(): + if name.startswith('_install_pickle_support_for_'): + w_install = self.get(name) + space.call_function(w_install) + # xxx hide the installer + space.delitem(self.w_dict, space.wrap(name)) + del self.loaders[name] \ No newline at end of file Modified: pypy/dist/pypy/module/__builtin__/app_functional.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/app_functional.py (original) +++ pypy/dist/pypy/module/__builtin__/app_functional.py Sat Apr 29 09:27:37 2006 @@ -362,6 +362,18 @@ def __len__(self): return self._remaining + def __reduce__(self): + tup = (self._current, self._remaining, self._step) + return (make_xrange_iterator, tup) + +def make_xrange_iterator(*args): + return xrange_iterator(*args) + +def _install_pickle_support_for_xrange_iterator(): + import _pickle_support + make_xrange_iterator.__module__ = '_pickle_support' + _pickle_support.make_xrange_iterator = make_xrange_iterator + # ____________________________________________________________ def sorted(lst, cmp=None, key=None, reverse=None): Modified: pypy/dist/pypy/module/_pickle_support/__init__.py ============================================================================== --- pypy/dist/pypy/module/_pickle_support/__init__.py (original) +++ pypy/dist/pypy/module/_pickle_support/__init__.py Sat Apr 29 09:27:37 2006 @@ -7,7 +7,9 @@ } interpleveldefs = { - 'cell_new': 'maker.cell_new', - 'code_new': 'maker.code_new', - 'func_new': 'maker.func_new', + 'cell_new' : 'maker.cell_new', + 'code_new' : 'maker.code_new', + 'func_new' : 'maker.func_new', + 'module_new' : 'maker.module_new', + 'method_new' : 'maker.method_new', } Modified: pypy/dist/pypy/module/_pickle_support/maker.py ============================================================================== --- pypy/dist/pypy/module/_pickle_support/maker.py (original) +++ pypy/dist/pypy/module/_pickle_support/maker.py Sat Apr 29 09:27:37 2006 @@ -1,9 +1,10 @@ from pypy.interpreter.nestedscope import Cell from pypy.interpreter.pycode import PyCode -from pypy.interpreter.function import Function +from pypy.interpreter.function import Function, Method +from pypy.interpreter.module import Module from pypy.rpython.objectmodel import instantiate from pypy.interpreter.argument import Arguments -from pypy.interpreter.baseobjspace import ObjSpace +from pypy.interpreter.baseobjspace import ObjSpace, W_Root #note: for now we don't use the actual value when creating the Cell. @@ -22,3 +23,12 @@ w_type = space.gettypeobject(Function.typedef) return space.call_args(w_type, __args__) func_new.unwrap_spec = [ObjSpace, Arguments] + +def module_new(space, w_name, w_dict): + new_mod = Module(space, w_name, w_dict) + return space.wrap(new_mod) + +def method_new(space, __args__): + w_type = space.gettypeobject(Method.typedef) + return space.call_args(w_type, __args__) +method_new.unwrap_spec = [ObjSpace, Arguments] From ericvrp at codespeak.net Sat Apr 29 10:28:51 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 10:28:51 +0200 (CEST) Subject: [pypy-svn] r26564 - in pypy/dist/pypy: interpreter/test module/__builtin__ Message-ID: <20060429082851.1E05610094@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 10:28:44 2006 New Revision: 26564 Modified: pypy/dist/pypy/interpreter/test/test_pickle.py pypy/dist/pypy/module/__builtin__/__init__.py pypy/dist/pypy/module/__builtin__/app_functional.py Log: pickling support for reversed_iterator Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Sat Apr 29 10:28:44 2006 @@ -123,9 +123,6 @@ result = pickle.loads(pckl) assert mod is result - def test_pickle_iter(self): - skip("work in progress") - def test_pickle_method(self): skip("work in progress") class myclass(object): @@ -179,7 +176,7 @@ #def test_pickle_enumfactory(self): # skip("work in progress") - + def test_pickle_sequenceiter(self): ''' In PyPy there is no distinction here between listiterator and @@ -203,6 +200,16 @@ assert type(riter) is type(result) assert list(result) == [2,3,4] + def test_pickle_reversed_iterator(self): + import pickle + i = reversed(xrange(5)) + i.next() + i.next() + pckl = pickle.dumps(i) + result = pickle.loads(pckl) + assert type(i) is type(result) + assert list(result) == [2,1,0] + def test_pickle_generator(self): skip("work in progress") import pickle Modified: pypy/dist/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/__init__.py (original) +++ pypy/dist/pypy/module/__builtin__/__init__.py Sat Apr 29 10:28:44 2006 @@ -35,6 +35,8 @@ 'app_functional._install_pickle_support_for_xrange_iterator', 'sorted' : 'app_functional.sorted', 'reversed' : 'app_functional.reversed', + '_install_pickle_support_for_reversed_iterator': + 'app_functional._install_pickle_support_for_reversed_iterator', 'issubclass' : 'app_inspect.issubclass', 'isinstance' : 'app_inspect.isinstance', Modified: pypy/dist/pypy/module/__builtin__/app_functional.py ============================================================================== --- pypy/dist/pypy/module/__builtin__/app_functional.py (original) +++ pypy/dist/pypy/module/__builtin__/app_functional.py Sat Apr 29 10:28:44 2006 @@ -418,3 +418,21 @@ if self.remaining > len(self.seq): self.remaining = 0 return self.remaining + + def __reduce__(self): + tup = (self.seq, self.remaining) + return (make_reversed_iterator, tup) + +def make_reversed_iterator(seq, remaining): + ri = reversed_iterator.__new__(reversed_iterator) + ri.seq = seq + #or "ri = reversed_iterator(seq)" but that executes len(seq) + ri.remaining = remaining + return ri + +def _install_pickle_support_for_reversed_iterator(): + import _pickle_support + make_reversed_iterator.__module__ = '_pickle_support' + _pickle_support.make_reversed_iterator = make_reversed_iterator + + From dialtone at codespeak.net Sat Apr 29 10:34:59 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Sat, 29 Apr 2006 10:34:59 +0200 (CEST) Subject: [pypy-svn] r26565 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060429083459.9A4DE10097@code0.codespeak.net> Author: dialtone Date: Sat Apr 29 10:34:53 2006 New Revision: 26565 Modified: pypy/dist/pypy/translator/cl/clrepr.py pypy/dist/pypy/translator/cl/gencl.py pypy/dist/pypy/translator/cl/test/test_call.py pypy/dist/pypy/translator/cl/test/test_cltrans.py pypy/dist/pypy/translator/cl/test/test_exception.py Log: (nik, dialtone) made exception tests pass (and moved some of them in the right place) by fixing code generation that was not outputting the link. This revealed another problem with clrepr-ing constant instances that have constant instances as fields. Modified: pypy/dist/pypy/translator/cl/clrepr.py ============================================================================== --- pypy/dist/pypy/translator/cl/clrepr.py (original) +++ pypy/dist/pypy/translator/cl/clrepr.py Sat Apr 29 10:34:53 2006 @@ -1,5 +1,4 @@ from pypy.objspace.flow.model import Constant, Variable, Atom -from pypy.rpython.rmodel import HalfConcreteWrapper from pypy.rpython.ootypesystem.ootype import List, Record, Instance from pypy.rpython.ootypesystem.ootype import Signed, Unsigned, Float, Char from pypy.rpython.ootypesystem.ootype import Bool, Void, UniChar, Class @@ -21,9 +20,6 @@ return repr_unknown(item) def repr_const(item): - if isinstance(item.value, HalfConcreteWrapper): - item = item.concretize() - fun = dispatch.get(type(item.concretetype), None) if fun is not None: return fun(item) Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Sat Apr 29 10:34:53 2006 @@ -384,7 +384,7 @@ tag = self.blockref[block] yield "tag" + clrepr(str(tag), True) if block.exitswitch is c_last_exception: - yield "(handler-case" + yield "(handler-case (progn" for op in block.operations: emit_op = Op(self, op) for line in emit_op: @@ -417,7 +417,9 @@ cls = exit.llexitcase.class_._INSTANCE exception = self.declare_exception(cls) exceptions[exception] = exit - self.emit_link(body) + for line in self.emit_link(body): + yield line + yield ")" # closes the progn for the handler-case for exception in exceptions: yield "(%s ()" % (exception,) for line in self.emit_link(exceptions[exception]): Modified: pypy/dist/pypy/translator/cl/test/test_call.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_call.py (original) +++ pypy/dist/pypy/translator/cl/test/test_call.py Sat Apr 29 10:34:53 2006 @@ -9,23 +9,3 @@ return n + 1 cl_add_one = make_cl_func(add_one, [int]) assert cl_add_one(1) == 2 - -def test_indirect_call(): - py.test.skip("needs exception handling") - def id(n): - return n - def square(n): - return n * n - def map_sum(func, n): - sum = 0 - for i in range(1, n+1): - sum += func(i) - return sum - def sum(n): - return map_sum(id, n) - def square_sum(n): - return map_sum(square, n) - cl_sum = make_cl_func(sum, [int]) - assert cl_sum(5) == 15 - cl_square_sum = make_cl_func(square_sum, [int]) - assert cl_square_sum(5) == 55 Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Sat Apr 29 10:34:53 2006 @@ -44,16 +44,6 @@ cl_get_three = make_cl_func(get_three) assert cl_get_three() == 3 -def test_iteration(): - py.test.skip("needs exception handling") - def get_last(num): - last = 0 - for i in range(num): - last = i - return last - cl_get_last = make_cl_func(get_last, [int]) - assert cl_get_last(5) == 4 - def test_if(): cl_if = make_cl_func(t.if_then_else, [bool, int, int]) assert cl_if(True, 50, 100) == 50 @@ -99,7 +89,6 @@ assert cl_sieve() == 1028 def test_easy(): - #py.test.skip("temporarily disabled") # These are the Pyrex tests which were easy to adopt. f1 = make_cl_func(t.simple_func, [int]) assert f1(1) == 2 Modified: pypy/dist/pypy/translator/cl/test/test_exception.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_exception.py (original) +++ pypy/dist/pypy/translator/cl/test/test_exception.py Sat Apr 29 10:34:53 2006 @@ -2,6 +2,7 @@ from pypy.translator.cl.buildcl import make_cl_func def test_handle_exception(): + py.test.skip("We support exceptions, but not really it seems") class MyException(Exception): pass def raise_exception(): @@ -18,3 +19,31 @@ cl_handle_exception = make_cl_func(handle_exception, [bool]) assert cl_handle_exception(True) == 1 assert cl_handle_exception(False) == 2 + +def test_indirect_call(): + def id(n): + return n + def square(n): + return n * n + def map_sum(func, n): + sum = 0 + for i in range(1, n+1): + sum += func(i) + return sum + def sum(n): + return map_sum(id, n) + def square_sum(n): + return map_sum(square, n) + cl_sum = make_cl_func(sum, [int]) + assert cl_sum(5) == 15 + cl_square_sum = make_cl_func(square_sum, [int]) + assert cl_square_sum(5) == 55 + +def test_iteration(): + def get_last(num): + last = 0 + for i in range(num): + last = i + return last + cl_get_last = make_cl_func(get_last, [int]) + assert cl_get_last(5) == 4 From arigo at codespeak.net Sat Apr 29 10:50:28 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 10:50:28 +0200 (CEST) Subject: [pypy-svn] r26566 - in pypy/dist/pypy: rpython/rctypes rpython/rctypes/test rpython/rctypes/tool translator/c/src Message-ID: <20060429085028.9208E10097@code0.codespeak.net> Author: arigo Date: Sat Apr 29 10:50:26 2006 New Revision: 26566 Added: pypy/dist/pypy/rpython/rctypes/aerrno.py (contents, props changed) pypy/dist/pypy/rpython/rctypes/test/test_rerrno.py (contents, props changed) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py pypy/dist/pypy/rpython/rctypes/tool/util.py pypy/dist/pypy/translator/c/src/ll_os.h Log: Support geterrno() from rctypes to genc. Added: pypy/dist/pypy/rpython/rctypes/aerrno.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/aerrno.py Sat Apr 29 10:50:26 2006 @@ -0,0 +1,36 @@ +""" +Helpers to access the C-level 'errno' variable. +""" +from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.annotation import model as annmodel +from ctypes import pythonapi, py_object + + +##def setfromerrno(exc=OSError): +## """Raise an exception of the given class with the last failed C library +## function's errno.""" +## pythonapi.PyErr_SetFromErrno(py_object(exc)) + +def geterrno(): + """Return the current 'errno' value.""" + try: + pythonapi.PyErr_SetFromErrno(py_object(OSError)) + except OSError, e: + return e.errno + else: + raise RuntimeError("setfromerrno() should have raised") + + +class GetErrnoFnEntry(ExtRegistryEntry): + "Annotation and rtyping of calls to geterrno()" + _about_ = geterrno + + def compute_result_annotation(self): + return annmodel.SomeInteger() + + def specialize_call(self, hop): + from pypy.rpython.lltypesystem import lltype + return hop.llops.gencapicall('geterrno', [], + resulttype = lltype.Signed, + includes = (), + _callable = geterrno) Added: pypy/dist/pypy/rpython/rctypes/test/test_rerrno.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rerrno.py Sat Apr 29 10:50:26 2006 @@ -0,0 +1,91 @@ +""" +Test errno. +""" + +import py +import sys, errno +import pypy.rpython.rctypes.implementation +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy.rpython.test.test_llinterp import interpret +from pypy.translator.c.test.test_genc import compile +from pypy import conftest +from pypy.rpython.rctypes import aerrno +from pypy.rpython.rctypes.test.test_rfunc import mylib + +from ctypes import c_char_p, c_int + + +if sys.platform == 'win32': + py.test.skip("Unix only for now") + +open = mylib.open +open.argtypes = [c_char_p, c_int, c_int] +open.restype = c_int +def ll_open(p, flags, mode): + s = '' + i = 0 + while p[i] != '\x00': + s += p[i] + i += 1 + return open(s, flags, mode) +open.llinterp_friendly_version = ll_open +open.includes = ('fcntl.h',) + + +def test_open(): + if sys.platform == 'win32': + py.test.skip("Unix only") + open = mylib.open + fd = open("/_rctypes_test_rfunc/this/directory/does/not/exist/at/all!", + 0, 0) + result = aerrno.geterrno() + assert fd == -1 + assert result == errno.ENOENT + +class Test_annotation: + def test_annotate_geterrno(self): + def f(): + return aerrno.geterrno() + a = RPythonAnnotator() + s = a.build_types(f, []) + if conftest.option.view: + a.translator.view() + assert s.knowntype == int + +class Test_specialization: + def test_specialize_geterrno(self): + if sys.platform == 'win32': + py.test.skip("Unix only") + open = mylib.open + open.argtypes = [c_char_p, c_int, c_int] + open.restype = c_int + def func(): + fd = open( + "/_rctypes_test_rfunc/this/directory/does/not/exist/at/all!", + 0, 0) + return fd, aerrno.geterrno() + + res = interpret(func, []) + assert res.item0 == -1 + # the following doesn't work because the llinterp calls many C library + # functions between open() and geterrno() + ##assert res.item1 == errno.ENOENT + +class Test_compile: + def test_compile_geterrno(self): + if sys.platform == 'win32': + py.test.skip("Unix only") + open = mylib.open + open.argtypes = [c_char_p, c_int, c_int] + open.restype = c_int + def func(): + fd = open( + "/_rctypes_test_rfunc/this/directory/does/not/exist/at/all!", + 0, 0) + return fd, aerrno.geterrno() + + fn = compile(func, []) + fd, result = fn() + assert fd == -1 + assert result == errno.ENOENT Modified: pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rfunc.py Sat Apr 29 10:50:26 2006 @@ -3,7 +3,7 @@ """ import py -import sys, errno +import sys import pypy.rpython.rctypes.implementation from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext, graphof @@ -119,21 +119,6 @@ s2 = ctime(byref(c_long(N))) assert s1.strip() == s2.strip() -def test_open(): - if sys.platform == 'win32': - py.test.skip("Unix only") - open = mylib.open - fd = open("/_rctypes_test_rfunc/this/directory/does/not/exist/at/all!", - 0, 0) - try: - util.setfromerrno() - except OSError, e: - pass - else: - raise AssertionError("util.setfromerrno() should have raised") - assert fd == -1 - assert e.errno == errno.ENOENT - ##def test_callback(): ## assert callback(100) == 103 ## assert pycallback(100) == 103 @@ -197,7 +182,6 @@ if conftest.option.view: a.translator.view() assert s.knowntype == str - ## def test_annotate_callback(self): ## def fn(n): Modified: pypy/dist/pypy/rpython/rctypes/tool/util.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/util.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/util.py Sat Apr 29 10:50:26 2006 @@ -1,16 +1,9 @@ """ -This is the module 'ctypes.util', copied from ctypes 0.9.9.6, -with an extra setfromerrno() helper. +This is the module 'ctypes.util', copied from ctypes 0.9.9.6. """ import sys, os import ctypes -def setfromerrno(exc=OSError): - """Raise an exception of the given class with the last failed C library - function's errno.""" - ctypes.pythonapi.PyErr_SetFromErrno(ctypes.py_object(exc)) - - # find_library(name) returns the pathname of a library, or None. if os.name == "nt": def find_library(name): Modified: pypy/dist/pypy/translator/c/src/ll_os.h ============================================================================== --- pypy/dist/pypy/translator/c/src/ll_os.h (original) +++ pypy/dist/pypy/translator/c/src/ll_os.h Sat Apr 29 10:50:26 2006 @@ -62,6 +62,8 @@ RPyString *LL_os_readdir(struct RPyOpaque_DIR *dir); void LL_os_closedir(struct RPyOpaque_DIR *dir); +#define geterrno() errno + /* implementations */ #ifndef PYPY_NOT_MAIN_FILE From arigo at codespeak.net Sat Apr 29 11:30:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 11:30:59 +0200 (CEST) Subject: [pypy-svn] r26567 - pypy/dist/pypy/rpython/rctypes/tool Message-ID: <20060429093059.9D6C510086@code0.codespeak.net> Author: arigo Date: Sat Apr 29 11:30:58 2006 New Revision: 26567 Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Log: Revert this change: '_pad' is not the only name that could start with an underscore. There might be real C field names that start with an underscore, so it's a problem that must be solved elsewhere... Modified: pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py (original) +++ pypy/dist/pypy/rpython/rctypes/tool/ctypes_platform.py Sat Apr 29 11:30:58 2006 @@ -187,7 +187,7 @@ for i, cell in enumerate(layout): if cell is not None: continue - layout_addfield(layout, i, ctypes.c_char, 'pad%d' % (n,)) + layout_addfield(layout, i, ctypes.c_char, '_pad%d' % (n,)) n += 1 # build the ctypes Structure From dialtone at codespeak.net Sat Apr 29 11:35:25 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Sat, 29 Apr 2006 11:35:25 +0200 (CEST) Subject: [pypy-svn] r26568 - pypy/dist/pypy/doc/discussion Message-ID: <20060429093525.9B8B61008B@code0.codespeak.net> Author: dialtone Date: Sat Apr 29 11:35:22 2006 New Revision: 26568 Added: pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt Log: add a longish discussion about pypy metaclasses in common lisp that eventually evolved into a discussion on how rclass could be improved Added: pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt Sat Apr 29 11:35:22 2006 @@ -0,0 +1,134 @@ +[09:41] arigo: is it possible to ask the backendoptimizer to completely remove all the oogetfield('meta', obj)? +[09:42] and at the same time to change all the oogetfield('somefield', meta) into oogetfield('somefield', obj) +[09:42] because then we wouldn't need the metaclass hierarchy anymore +[09:42] (at least in common lisp) +[09:42] as far as I know the idea was indeed to be able to do this kind of things +[09:43] but not necessarily in the existing backendopt +[09:44] uhmmm +[09:44] I have no idea how to do this stuff +[09:44] if I understand it correctly, as a first step you can just tweak gencl to recognize oogetfield('meta', obj) +[09:44] I'll think about it on the plane maybe +[09:44] and produce a same_as equivalent instead +[09:44] (do I make any sense at all?) +[09:44] yes +[09:45] same_as(meta, obj) +[09:45] so that the next oogetfield() will still work on meta which in reality is the obj +[09:45] yes +[09:45] thus you obtained the same thing without removing anything +[09:45] cool +[09:46] dialtone: can you explain me better what are you trying to do? +[09:46] it looks kinda simple +[09:46] am I a fool? +[09:46] antocuni: I want to get rid of the metaclass stuff in common lisp +[09:47] since common lisp supports class variables +[09:47] (DEFCLASS foo () ((bar :allocate :class))) +[09:47] cool +[09:47] but to do that I also have to get rid of the opcodes that work on the object model +[09:48] at first I thought about removing the metaclass related operations (or change them) but armin got a great idea about using same_as +[09:48] idnar (i=mithrand at unaffiliated/idnar) left irc: Remote closed the connection +[09:48] there might be a few problems, though +[09:48] and here comes the part I feared +[09:48] I'm not sure if the meta object is used for more than oogetfields +[09:49] and also, let's see if there are name clashes in the fields +[09:49] I can't understand a thing: are you trying to lookup some fields in the obj directly, instead of in the metclass, right? +[09:49] antocuni: yes +[09:50] why an object should have fields that belongs to its metaclass? +[09:50] arigo: uhmmm you can have both a class variable and an instance variable named in the same way? +[09:50] metaclass is not a real metaclass +[09:50] I don't know +[09:50] arigo - r26566 - Support geterrno() from rctypes to genc. +[09:50] dialtone: ah, now I understand +[09:50] I would expect it not to be the case, as the names come from RPython names +[09:51] arigo: indeed +[09:51] but I guess I can set different accessors maybe for class level things and for instance level things +[09:51] let's try +[09:51] no... +[09:52] so a name clash would break stuff +[09:52] but... how do you recognize an access to a class variable and one to an instance variable from RPython? +[09:53] dialtone: I think we don't have name clashes, because there is some mangling anyway +[09:53] cool +[09:53] if I see it correctly, class variable names start with 'pbc' and instance ones with 'o' +[09:53] that's what we've done in gencl yes +[09:54] ? that's what the ootyping is doing +[09:54] yes yes +[09:54] :-) +[09:54] I mean that I see the distinction in gencl :) +[09:54] sooooooo +[09:55] if I have a getfield where the first argument is meta and I simply emit the same code that I emit for the same_as I should be safe removing all the meta stuff... maybe +[09:55] seems like a tiny change in gencl +[09:55] dialtone: in RPython, the annotator says that attributes are instance fields as soon as they are written to instances, otherwise they are class attributes +[09:56] yes, it should work +[09:56] Palats (n=Pierre at izumi.palats.com) left irc: Read error: 104 (Connection reset by peer) +[09:56] unless of course metaclasses are used for something else than class variables +[09:56] ideally, you should not look for the name 'meta' but for some other hint +[09:57] I'm not completely at ease with the various levels of ootype +[09:57] neither am I\ +[09:57] all field names other than those defined by ootype (like "meta") will be mangled, so i guess checking for "meta" is good enough +[09:57] and I also have to ignore the setfield opcode that deals with metaclasses +[09:58] or make it a same_as as well +[09:59] apparently, the meta instances are used as the ootype of RPython classes +[10:00] so they can be manipulated by RPython code that passes classes around +[10:01] I guess you can also pass classes around in CL, read attributes from them, and instantiate them +[10:01] yes +[10:01] so a saner approach might be to try to have gencl use CL classes instead of these meta instances +[10:03] uhmmmmm +[10:03] which means: recognize if an ootype.Instance is actually representing an RPython class (by using a hint) +[10:03] I also have to deal with the Class_ +[10:03] but that can probably be set to standard-class +[10:03] yes, I think it's saner to make, basically, oogetfield('class_') be a same_as +[10:04] cool +[10:04] I think I'll save this irc log to put it in the svn tree for sanxiyn +[10:04] to recognize RPython class represenations: if the ootype.Instance has the superclass ootypesystem.rclass.CLASSTYPE, then it's a "metaclass" +[10:04] he is thinking about this in the plane (at least this is what he told) +[10:05] :-) +[10:05] nikh: yes +[10:05] ootype is indeed rather complicated, level-wise, to support limited languages like Java +[10:05] unfortunately, yes +[10:05] well, in a way it's very convenient for the backends +[10:05] but if you want to use more native constructs, it gets hairy quickly +[10:05] I dunno +[10:05] depends on the backend +[10:06] hum, there is still an information missing that gencl would need here +[10:06] I think if the language of the backend is powerful enough it could use an higher abstraction +[10:07] dialtone: yes, there is also the (hairly to implement) idea of producing slightly different things for different back-ends too +[10:07] using backendopts? +[10:08] would it make sense to have a kind of backend_supports=['metaclasses', 'classvariables', 'first_class_functions'...] +[10:08] maybe, but I was thinking about doing different things in ootypesystem/rclass already +[10:08] yes, such a backend_supports would be great +[10:09] dialtone: there is still an hour left to sprint, so go go go ;) +[10:09] you can do it, if you want it ;) +[10:09] what is missing is the link from the concrete Instance types, and which Instance corresponds to its meta-instance +[10:10] idnar (i=mithrand at unaffiliated/idnar) joined #pypy. +[10:10] dialtone: it's not as simple as making an oogetfield be a same_as +[10:10] KnowledgeUnboundError, Missing documentation in slot brain +[10:10] right now for CL the goal would be to generate for a normal Instance, a DEFCLASS whose :allocate :class attributes are the attributes of the meta-Instance +[10:11] we could optionally have class fields in Instances, and then operations like ooget/setclassfield +[10:11] the reason why I ask is that if we manage to do this then we could also use default Condition as Exception +[10:11] and we could map the Conditions in common lisp to exceptions in python transparently +[10:12] since the object systems will then match (and they are vaguely similar anyway) +[10:12] nice +[10:12] at least I think +[10:18] I'm still rather confused by ootypesystem/rclass +[10:18] although I think that blame would show my name on quite some bits :-) +[10:19] there are no class attributes read through instances +[10:19] they are turned into method calls +[10:19] accessor methods +[10:20] it's a bit organically grown +[10:20] accessor methods were introduced at one point, and the meta-Instance later +[10:21] uhmmm +[10:22] what was the reason for having accessor methods? +[10:22] they seem to be only generated for class vars that are overriden in subclasses. +[10:22] yes +[10:22] before we had the meta-Instance trick, it was the only way to avoid storing the value in all instances +[10:22] aha +[10:23] we could possibly get rid of these accessors +[10:23] now, yes, by storing the values in the meta-Instance +[10:23] they are alway anyway stored in the meta-Instance, I think +[10:23] no, I think that other values are stored in the meta-Instance right now +[10:24] it's the values that are only ever accessed with a syntax 'ClassName.attr', i.e. not through an instance +[10:24] ...more precisely, with 'x = ClassName or OtherClassName; x.attr' +[10:25] hm, i'm still trying to read this out of the code ... +[10:28] it's in ClassRepr._setup_repr() +[10:28] there is no clsfields here, just pbcfields +[10:28] # attributes showing up in getattrs done on the class as a PBC +[10:28] i see \ No newline at end of file From ericvrp at codespeak.net Sat Apr 29 11:39:03 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 29 Apr 2006 11:39:03 +0200 (CEST) Subject: [pypy-svn] r26569 - pypy/dist/pypy/interpreter/test Message-ID: <20060429093903.BCF0E10094@code0.codespeak.net> Author: ericvrp Date: Sat Apr 29 11:39:00 2006 New Revision: 26569 Modified: pypy/dist/pypy/interpreter/test/test_pickle.py Log: (Yusei, ericvrp) more tests that pass Modified: pypy/dist/pypy/interpreter/test/test_pickle.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_pickle.py (original) +++ pypy/dist/pypy/interpreter/test/test_pickle.py Sat Apr 29 11:39:00 2006 @@ -1,5 +1,5 @@ class AppTestInterpObjectPickling: - + def test_pickle_code(self): def f(): return 42 @@ -8,7 +8,7 @@ pckl = pickle.dumps(code) result = pickle.loads(pckl) assert code == result - + def test_pickle_global_func(self): import new mod = new.module('mod') @@ -23,7 +23,7 @@ result = pickle.loads(pckl) assert func is result del sys.modules['mod'] - + def test_pickle_not_imported_module(self): import new mod = new.module('mod') @@ -34,13 +34,13 @@ assert mod.__name__ == result.__name__ assert mod.__dict__ == result.__dict__ #print mod.__dict__ - + def test_pickle_builtin_func(self): import pickle pckl = pickle.dumps(map) result = pickle.loads(pckl) assert map is result - + def test_pickle_non_top_reachable_func(self): def func(): return 42 @@ -57,7 +57,7 @@ assert func.func_dict == result.func_dict assert func.func_doc == result.func_doc assert func.func_globals == result.func_globals - + def test_pickle_cell(self): def g(): x = [42] @@ -65,13 +65,13 @@ x[0] += 1 return x return f.func_closure[0] - import pickle + import pickle cell = g() pckl = pickle.dumps(cell) result = pickle.loads(pckl) assert cell == result assert not (cell != result) - + def test_pickle_frame(self): skip("work in progress") from sys import exc_info @@ -86,7 +86,7 @@ pckl = pickle.dumps(frame) result = pickle.loads(pckl) assert frame == result - + def test_pickle_traceback(self): skip("work in progress") def f(): @@ -101,39 +101,57 @@ pckl = pickle.dumps(tb) result = pickle.loads(pckl) assert tb == result - + def test_pickle_module(self): import pickle mod = pickle pckl = pickle.dumps(mod) result = pickle.loads(pckl) assert mod is result - + def test_pickle_moduledict(self): import pickle moddict = pickle.__dict__ pckl = pickle.dumps(moddict) result = pickle.loads(pckl) assert moddict is result - + def test_pickle_bltins_module(self): import pickle mod = __builtins__ pckl = pickle.dumps(mod) result = pickle.loads(pckl) assert mod is result - + + def test_pickle_buffer(self): + import pickle + a = buffer('ABCDEF') + pckl = pickle.dumps(a) + result = pickle.loads(pckl) + assert a == result + + def test_pickle_complex(self): + import pickle + a = complex(1.23,4.567) + pckl = pickle.dumps(a) + result = pickle.loads(pckl) + assert a == result + def test_pickle_method(self): skip("work in progress") class myclass(object): def f(self): pass + def __reduce__(self): + return (myclass,()) import pickle - method = myclass.f + myclass_inst = myclass() + method = myclass_inst.f pckl = pickle.dumps(method) result = pickle.loads(pckl) assert method == result + def test_pickle_staticmethod(self): skip("work in progress") class myclass(object): @@ -145,7 +163,7 @@ pckl = pickle.dumps(method) result = pickle.loads(pckl) assert method == result - + def test_pickle_classmethod(self): skip("work in progress") class myclass(object): @@ -157,26 +175,7 @@ pckl = pickle.dumps(method) result = pickle.loads(pckl) assert method == result - - def test_pickle_dictiter(self): - skip("work in progress") - import pickle - diter = iter({}) - pckl = pickle.dumps(diter) - result = pickle.loads(pckl) - assert diter == result - - def test_pickle_enum(self): - skip("work in progress") - import pickle - e = enumerate([]) - pckl = pickle.dumps(e) - result = pickle.loads(pckl) - assert e == result - - #def test_pickle_enumfactory(self): - # skip("work in progress") - + def test_pickle_sequenceiter(self): ''' In PyPy there is no distinction here between listiterator and @@ -189,7 +188,27 @@ pckl = pickle.dumps(liter) result = pickle.loads(pckl) assert liter == result - + + def test_pickle_dictiter(self): + skip("work in progress") + import pickle + diter = iter({}) + pckl = pickle.dumps(diter) + result = pickle.loads(pckl) + assert diter == result + + def test_pickle_enum(self): + import pickle + e = enumerate(range(10)) + e.next() + e.next() + e._iter= 42 #XXX HACK, because we have no sequence/dict iterator yet + pckl = pickle.dumps(e) + result = pickle.loads(pckl) + assert type(e) is type(result) + assert e._iter == result._iter #change when _iter is an iter(..) again + assert e._index == result._index + def test_pickle_xrangeiter(self): import pickle riter = iter(xrange(5)) @@ -199,7 +218,7 @@ result = pickle.loads(pckl) assert type(riter) is type(result) assert list(result) == [2,3,4] - + def test_pickle_reversed_iterator(self): import pickle i = reversed(xrange(5)) @@ -209,10 +228,10 @@ result = pickle.loads(pckl) assert type(i) is type(result) assert list(result) == [2,1,0] - + def test_pickle_generator(self): skip("work in progress") - import pickle + import pickle def giveme(n): x = 0 while x < n: From nik at codespeak.net Sat Apr 29 11:45:21 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 11:45:21 +0200 (CEST) Subject: [pypy-svn] r26570 - pypy/dist/pypy/rpython/ootypesystem Message-ID: <20060429094521.8011010094@code0.codespeak.net> Author: nik Date: Sat Apr 29 11:45:18 2006 New Revision: 26570 Modified: pypy/dist/pypy/rpython/ootypesystem/rstr.py Log: some comments about the state of ootype strings. unfortunately i didn't really progress as much as i would have liked. Modified: pypy/dist/pypy/rpython/ootypesystem/rstr.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/rstr.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/rstr.py Sat Apr 29 11:45:18 2006 @@ -3,6 +3,22 @@ from pypy.rpython.ootypesystem.ootype import Signed, Record, String, make_string class StringRepr(AbstractStringRepr): + """ + Some comments about the state of ootype strings at the end of Tokyo sprint + + What was accomplished: + - The rstr module was split in an lltype and ootype version. + - There is the beginnings of a String type in ootype. + - The runtime representation of Strings is a subclass of the builtin str. + The idea is that this saves us from boilerplate code implementing the + builtin str methods. + + Nothing more was done because of lack of time and paralysis in the face + of too many problems. Among other things, to write any meaningful tests + we first need conversion from Chars to Strings (because + test_llinterp.interpret won't accept strings as arguments). We will need a + new low-level operation (convert_char_to_oostring or some such) for this. + """ lowleveltype = String From nik at codespeak.net Sat Apr 29 11:59:21 2006 From: nik at codespeak.net (nik at codespeak.net) Date: Sat, 29 Apr 2006 11:59:21 +0200 (CEST) Subject: [pypy-svn] r26571 - pypy/dist/pypy/rpython/test Message-ID: <20060429095921.48E3510086@code0.codespeak.net> Author: nik Date: Sat Apr 29 11:59:18 2006 New Revision: 26571 Modified: pypy/dist/pypy/rpython/test/test_rpbc.py Log: fix failures in test_rpbc by skipping them for now. they rely on complete ootype String support and only passed by accident before. Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Sat Apr 29 11:59:18 2006 @@ -498,6 +498,8 @@ assert self.read_attr(res, "extra") == 42 def test_conv_from_None(self): + if self.ts == "ootype": + py.test.skip("disabled while ootype string support is incomplete") class A(object): pass def none(): return None @@ -969,7 +971,7 @@ def test_pbc_imprecise_attrfamily(self): if self.ts == "ootype": - py.test.skip("fix me if ootype fully supports lists") + py.test.skip("disabled while ootype string support is incomplete") fr1 = Freezing(); fr1.x = 5; fr1.y = [8] fr2 = Freezing(); fr2.x = 6; fr2.y = ["string"] def head(fr): @@ -984,6 +986,8 @@ assert res == 8 + 6 def test_multiple_specialized_functions(self): + if self.ts == "ootype": + py.test.skip("disabled while ootype string support is incomplete") def myadder(x, y): # int,int->int or str,str->str return x+y def myfirst(x, y): # int,int->int or str,str->str @@ -1008,6 +1012,8 @@ assert res == f(i) def test_specialized_method_of_frozen(self): + if self.ts == "ootype": + py.test.skip("disabled while ootype string support is incomplete") class space: def _freeze_(self): return True @@ -1035,6 +1041,8 @@ assert ''.join(res.chars) == 'tag2:hellotag2:< 42 >' def test_specialized_method(self): + if self.ts == "ootype": + py.test.skip("disabled while ootype string support is incomplete") class A: def __init__(self, tag): self.tag = tag From arigo at codespeak.net Sat Apr 29 12:07:22 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 12:07:22 +0200 (CEST) Subject: [pypy-svn] r26572 - in pypy/dist/pypy: rpython/rctypes rpython/rctypes/test translator/c Message-ID: <20060429100722.D40F710089@code0.codespeak.net> Author: arigo Date: Sat Apr 29 12:07:21 2006 New Revision: 26572 Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py pypy/dist/pypy/translator/c/node.py Log: Support for C struct field names that start with '_'. Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Sat Apr 29 12:07:21 2006 @@ -17,7 +17,7 @@ r_field = rtyper.getrepr(SomeCTypesObject(field_ctype, SomeCTypesObject.MEMORYALIAS)) self.r_fields[name] = r_field - llfields.append((name, r_field.ll_type)) + llfields.append((cmangle(name), r_field.ll_type)) # Here, self.c_data_type == self.ll_type external = getattr(struct_ctype, '_external_', False) @@ -30,11 +30,11 @@ def get_content_keepalive_type(self): "An extra struct of keepalives, one per field." keepalives = [] - for name in self.c_data_type._names: + for name, field_ctype in self.ctype._fields_: r_field = self.r_fields[name] field_keepalive_type = r_field.get_content_keepalive_type() if field_keepalive_type: - keepalives.append((name, field_keepalive_type)) + keepalives.append((cmangle(name), field_keepalive_type)) if not keepalives: return None else: @@ -45,15 +45,15 @@ llitem = r_field.convert_const(getattr(value, name)) if isinstance(r_field, CTypesRefRepr): # ByRef case - reccopy(llitem.c_data, getattr(p.c_data, name)) + reccopy(llitem.c_data, getattr(p.c_data, cmangle(name))) else: # ByValue case - setattr(p.c_data, name, llitem.c_data[0]) + setattr(p.c_data, cmangle(name), llitem.c_data[0]) def get_c_data_of_field(self, llops, v_struct, fieldname): v_c_struct = self.get_c_data(llops, v_struct) r_field = self.r_fields[fieldname] - c_fieldname = inputconst(lltype.Void, fieldname) + c_fieldname = inputconst(lltype.Void, cmangle(fieldname)) if isinstance(r_field, CTypesRefRepr): # ByRef case return llops.genop('getsubstruct', [v_c_struct, c_fieldname], @@ -69,7 +69,7 @@ r_field = self.r_fields[fieldname] assert isinstance(r_field, CTypesValueRepr) v_c_struct = self.get_c_data(llops, v_struct) - c_fieldname = inputconst(lltype.Void, fieldname) + c_fieldname = inputconst(lltype.Void, cmangle(fieldname)) return llops.genop('getfield', [v_c_struct, c_fieldname], resulttype = r_field.ll_type) @@ -112,10 +112,15 @@ v_newvalue = r_field.get_c_data_or_value(llops, v_item) # copy the new value (which might be a whole substructure) v_c_struct = self.get_c_data(llops, v_struct) - genreccopy_structfield(llops, v_newvalue, v_c_struct, name) + genreccopy_structfield(llops, v_newvalue, v_c_struct, cmangle(name)) # copy the keepalive information too v_newkeepalive = r_field.getkeepalive(llops, v_item) if v_newkeepalive is not None: v_keepalive_struct = self.getkeepalive(llops, v_struct) genreccopy_structfield(llops, v_newkeepalive, - v_keepalive_struct, name) + v_keepalive_struct, cmangle(name)) + +def cmangle(name): + # obscure: names starting with '_' are not allowed in + # lltype.Struct, so we prefix all names with 'c_' + return 'c_' + name Modified: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c (original) +++ pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Sat Apr 29 12:07:21 2006 @@ -17,6 +17,7 @@ typedef struct tagpoint { int x; int y; + int _z; } point; EXPORT(int) _testfunc_byval(point in, point *pout) @@ -34,6 +35,7 @@ int tmp = p->x; p->x = p->y; p->y = tmp; + p->_z++; } EXPORT(int) _testfunc_struct(point in) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Sat Apr 29 12:07:21 2006 @@ -41,7 +41,8 @@ # struct tagpoint class tagpoint(Structure): _fields_ = [("x", c_int), - ("y", c_int)] + ("y", c_int), + ("_z", c_int)] _external_ = True # hack to avoid redeclaration of the struct in C # _test_struct @@ -50,7 +51,7 @@ testfunc_struct.argtypes = [tagpoint] def ll_testfunc_struct(in_): - return in_.x + in_.y + return in_.c_x + in_.c_y testfunc_struct.llinterp_friendly_version = ll_testfunc_struct testfunc_struct.includes = includes @@ -61,9 +62,9 @@ def ll_testfunc_byval(in_, pout): if pout: - pout.x = in_.x - pout.y = in_.y - return in_.x + in_.y + pout.c_x = in_.c_x + pout.c_y = in_.c_y + return in_.c_x + in_.c_y testfunc_byval.llinterp_friendly_version = ll_testfunc_byval testfunc_byval.includes = includes @@ -91,7 +92,8 @@ testfunc_swap.argtypes = [tagpointptr] def ll_testfunc_swap(p): - p.x, p.y = p.y, p.x + p.c_x, p.c_y = p.c_y, p.c_x + p.c__z += 1 testfunc_swap.llinterp_friendly_version = ll_testfunc_swap testfunc_swap.includes = includes @@ -133,9 +135,11 @@ pt = tagpoint() pt.x = 5 pt.y = 9 + pt._z = 99 testfunc_swap(pointer(pt)) assert pt.x == 9 assert pt.y == 5 + assert pt._z == 100 return pt.x - pt.y # this test function is reused below class Test_annotation: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstruct.py Sat Apr 29 12:07:21 2006 @@ -100,8 +100,8 @@ res = interpret(create_struct, []) c_data = res.c_data - assert c_data.x == 0 - assert c_data.y == 0 + assert c_data.c_x == 0 + assert c_data.c_y == 0 def test_specialize_struct_access(self): def access_struct(n): @@ -198,14 +198,14 @@ return (s0, s1, s2, s3) res = interpret(func, [4, '?']) - assert res.item0.c_data.x == 4 - assert res.item0.c_data.y == '\x00' - assert res.item1.c_data.x == 4 - assert res.item1.c_data.y == '?' - assert res.item2.c_data.x == 0 - assert res.item2.c_data.y == '?' - assert res.item3.c_data.x == 4 - assert res.item3.c_data.y == '?' + assert res.item0.c_data.c_x == 4 + assert res.item0.c_data.c_y == '\x00' + assert res.item1.c_data.c_x == 4 + assert res.item1.c_data.c_y == '?' + assert res.item2.c_data.c_x == 0 + assert res.item2.c_data.c_y == '?' + assert res.item3.c_data.c_x == 4 + assert res.item3.c_data.c_y == '?' def test_specialize_bad_constructor_args(self): class S(Structure): Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Sat Apr 29 12:07:21 2006 @@ -46,7 +46,7 @@ with_number = False if STRUCT._hints.get('c_name'): self.barename = self.name = STRUCT._hints['c_name'] - self.prefix = '' + self.c_struct_field_name = self.verbatim_field_name else: (self.barename, self.name) = db.namespace.uniquename(basename, @@ -93,8 +93,14 @@ return 'struct %s @' % self.name def c_struct_field_name(self, name): + # occasionally overridden in __init__(): + # self.c_struct_field_name = self.verbatim_field_name return self.prefix + name + def verbatim_field_name(self, name): + assert name.startswith('c_') # produced in this way by rctypes + return name[2:] + def c_struct_field_type(self, name): return self.STRUCT._flds[name] From dialtone at codespeak.net Sat Apr 29 12:09:52 2006 From: dialtone at codespeak.net (dialtone at codespeak.net) Date: Sat, 29 Apr 2006 12:09:52 +0200 (CEST) Subject: [pypy-svn] r26573 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060429100952.C1DF710089@code0.codespeak.net> Author: dialtone Date: Sat Apr 29 12:09:48 2006 New Revision: 26573 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: closing session status Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Sat Apr 29 12:09:48 2006 @@ -282,3 +282,23 @@ * rpystone translation - general wizardry (Samuele) + +Saturday, 29th April - Closing session +*************************************** + + - gencl (Valentino) + * exceptions are now correctly supported + * gencl has been resurrected and can translate most of the basic operations + * rpystone has not been translated yet because of Strings (and maybe other things) + + - OOTypeSystem (Nik) + * We got rid of ForwardReferences and now have full support for recursive types + * New Record type + * preliminary string support + + - Pickling (Eric, Yusei) + * Everything is picklable except: SequenceIterators, DictIterators, + Frames, Traceback and generators + + - ctypes and _socket: Please one of the anders should comment on this. + From arigo at codespeak.net Sat Apr 29 13:41:23 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 13:41:23 +0200 (CEST) Subject: [pypy-svn] r26574 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060429114123.50BA410082@code0.codespeak.net> Author: arigo Date: Sat Apr 29 13:41:19 2006 New Revision: 26574 Modified: pypy/dist/pypy/rpython/rctypes/aarray.py pypy/dist/pypy/rpython/rctypes/achar_p.py pypy/dist/pypy/rpython/rctypes/aerrno.py pypy/dist/pypy/rpython/rctypes/afunc.py pypy/dist/pypy/rpython/rctypes/apointer.py pypy/dist/pypy/rpython/rctypes/aprimitive.py pypy/dist/pypy/rpython/rctypes/apyobject.py pypy/dist/pypy/rpython/rctypes/astringbuf.py pypy/dist/pypy/rpython/rctypes/astruct.py pypy/dist/pypy/rpython/rctypes/avoid_p.py pypy/dist/pypy/rpython/rctypes/rarray.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rpointer.py pypy/dist/pypy/rpython/rctypes/rprimitive.py pypy/dist/pypy/rpython/rctypes/rpyobject.py pypy/dist/pypy/rpython/rctypes/rstringbuf.py pypy/dist/pypy/rpython/rctypes/rstruct.py Log: Sprinkle hop.exception_cannot_occur() in rctypes. Modified: pypy/dist/pypy/rpython/rctypes/aarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/aarray.py Sat Apr 29 13:41:19 2006 @@ -14,6 +14,7 @@ from pypy.rpython.error import TyperError from pypy.rpython.rmodel import inputconst r_array = hop.r_result + hop.exception_cannot_occur() v_result = r_array.allocate_instance(hop.llops) if hop.nb_args > r_array.length: raise TyperError("too many arguments for an array of length %d" % ( Modified: pypy/dist/pypy/rpython/rctypes/achar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/achar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/achar_p.py Sat Apr 29 13:41:19 2006 @@ -11,6 +11,7 @@ def specialize_call(self, hop): string_repr = hop.rtyper.type_system.rstr.string_repr r_char_p = hop.r_result + hop.exception_cannot_occur() v_result = r_char_p.allocate_instance(hop.llops) if len(hop.args_s): v_value, = hop.inputargs(string_repr) Modified: pypy/dist/pypy/rpython/rctypes/aerrno.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aerrno.py (original) +++ pypy/dist/pypy/rpython/rctypes/aerrno.py Sat Apr 29 13:41:19 2006 @@ -30,6 +30,7 @@ def specialize_call(self, hop): from pypy.rpython.lltypesystem import lltype + hop.exception_cannot_occur() return hop.llops.gencapicall('geterrno', [], resulttype = lltype.Signed, includes = (), Modified: pypy/dist/pypy/rpython/rctypes/afunc.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/afunc.py (original) +++ pypy/dist/pypy/rpython/rctypes/afunc.py Sat Apr 29 13:41:19 2006 @@ -119,6 +119,7 @@ #else: # no 'includes': hack to trigger in GenC a PyErr_Occurred() check + hop.exception_cannot_occur() v_result = hop.llops.gencapicall(fnname, unwrapped_args_v, resulttype = RESTYPE, **kwds) Modified: pypy/dist/pypy/rpython/rctypes/apointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/apointer.py Sat Apr 29 13:41:19 2006 @@ -45,6 +45,7 @@ def specialize_call(hop): r_ptr = hop.r_result + hop.exception_cannot_occur() v_result = r_ptr.allocate_instance(hop.llops) if len(hop.args_s): v_contentsbox, = hop.inputargs(r_ptr.r_contents) @@ -70,4 +71,5 @@ def specialize_call(self, hop): assert hop.s_result.is_constant() + hop.exception_cannot_occur() return hop.inputconst(lltype.Void, hop.s_result.const) Modified: pypy/dist/pypy/rpython/rctypes/aprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/aprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/aprimitive.py Sat Apr 29 13:41:19 2006 @@ -28,6 +28,7 @@ def specialize_call(self, hop): r_primitive = hop.r_result + hop.exception_cannot_occur() v_result = r_primitive.allocate_instance(hop.llops) if len(hop.args_s): v_value, = hop.inputargs(r_primitive.ll_type) Modified: pypy/dist/pypy/rpython/rctypes/apyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/apyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/apyobject.py Sat Apr 29 13:41:19 2006 @@ -12,6 +12,7 @@ def specialize_call(self, hop): from pypy.rpython.robject import pyobj_repr r_pyobject = hop.r_result + hop.exception_cannot_occur() v_result = r_pyobject.allocate_instance(hop.llops) if len(hop.args_s): [v_input] = hop.inputargs(pyobj_repr) Modified: pypy/dist/pypy/rpython/rctypes/astringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/astringbuf.py Sat Apr 29 13:41:19 2006 @@ -27,6 +27,7 @@ from pypy.rpython.lltypesystem import lltype [v_length] = hop.inputargs(lltype.Signed) r_stringbuf = hop.r_result + hop.exception_cannot_occur() return hop.genop("malloc_varsize", [ hop.inputconst(lltype.Void, r_stringbuf.lowleveltype.TO), v_length, @@ -60,6 +61,7 @@ from pypy.rpython.error import TyperError [s_arg] = hop.args_s [r_arg] = hop.args_r + hop.exception_cannot_occur() if isinstance(s_arg, SomeCTypesObject): if s_arg.knowntype is StringBufferType: # sizeof(string_buffer) == len(string_buffer) Modified: pypy/dist/pypy/rpython/rctypes/astruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/astruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/astruct.py Sat Apr 29 13:41:19 2006 @@ -13,6 +13,7 @@ def specialize_call(self, hop, **kwds_i): from pypy.rpython.error import TyperError r_struct = hop.r_result + hop.exception_cannot_occur() v_result = r_struct.allocate_instance(hop.llops) index_by_name = {} name_by_index = {} Modified: pypy/dist/pypy/rpython/rctypes/avoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/avoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/avoid_p.py Sat Apr 29 13:41:19 2006 @@ -14,6 +14,7 @@ def specialize_call(self, hop): r_void_p = hop.r_result + hop.exception_cannot_occur() v_result = r_void_p.allocate_instance(hop.llops) return v_result @@ -72,4 +73,5 @@ # cast to pointer v_result = hop.genop('cast_adr_to_ptr', [v_adr], resulttype = hop.r_result.ll_type) + hop.exception_cannot_occur() return hop.r_result.return_value(hop.llops, v_result) Modified: pypy/dist/pypy/rpython/rctypes/rarray.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rarray.py (original) +++ pypy/dist/pypy/rpython/rctypes/rarray.py Sat Apr 29 13:41:19 2006 @@ -54,6 +54,7 @@ assert s_attr.const == 'value' assert self.r_item.ll_type == lltype.Char # .value: char arrays only v_box = hop.inputarg(self, 0) + hop.exception_cannot_occur() return hop.gendirectcall(ll_chararrayvalue, v_box) def get_c_data_of_item(self, llops, v_array, v_index): @@ -103,6 +104,7 @@ class __extend__(pairtype(ArrayRepr, IntegerRepr)): def rtype_getitem((r_array, r_int), hop): v_array, v_index = hop.inputargs(r_array, lltype.Signed) + hop.exception_cannot_occur() if isinstance(r_array.r_item, PrimitiveRepr): # primitive case (optimization; the below also works in this case) # NB. this optimization is invalid for PointerReprs! See for @@ -118,6 +120,7 @@ def rtype_setitem((r_array, r_int), hop): v_array, v_index, v_item = hop.inputargs(r_array, lltype.Signed, r_array.r_item) + hop.exception_cannot_occur() r_array.setitem(hop.llops, v_array, v_index, v_item) Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Sat Apr 29 13:41:19 2006 @@ -45,6 +45,7 @@ assert s_attr.is_constant() assert s_attr.const == 'value' v_char_p = hop.inputarg(self, 0) + hop.exception_cannot_occur() return self.getstring(hop.llops, v_char_p) def rtype_setattr(self, hop): Modified: pypy/dist/pypy/rpython/rctypes/rpointer.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpointer.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpointer.py Sat Apr 29 13:41:19 2006 @@ -57,6 +57,7 @@ assert s_attr.const == 'contents' v_ptr = hop.inputarg(self, 0) v_c_ptr = self.getvalue(hop.llops, v_ptr) + hop.exception_cannot_occur() return self.r_contents.allocate_instance_ref(hop.llops, v_c_ptr) def rtype_setattr(self, hop): @@ -74,6 +75,7 @@ self = r_ptr v_ptr, v_index = hop.inputargs(self, lltype.Signed) v_c_ptr = self.getvalue(hop.llops, v_ptr) + hop.exception_cannot_occur() if isinstance(v_index, Constant) and v_index.value == 0: pass # skip direct_ptradd else: @@ -95,4 +97,5 @@ # not supported by ctypes either raise TyperError("assignment to pointer[x] with x != 0") # copy the whole structure's content over + hop.exception_cannot_occur() genreccopy(hop.llops, v_new_c_data, v_target) Modified: pypy/dist/pypy/rpython/rctypes/rprimitive.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rprimitive.py (original) +++ pypy/dist/pypy/rpython/rctypes/rprimitive.py Sat Apr 29 13:41:19 2006 @@ -24,6 +24,7 @@ assert s_attr.is_constant() assert s_attr.const == 'value' v_primitive = hop.inputarg(self, 0) + hop.exception_cannot_occur() return self.getvalue(hop.llops, v_primitive) def rtype_setattr(self, hop): Modified: pypy/dist/pypy/rpython/rctypes/rpyobject.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rpyobject.py (original) +++ pypy/dist/pypy/rpython/rctypes/rpyobject.py Sat Apr 29 13:41:19 2006 @@ -18,6 +18,7 @@ assert s_attr.is_constant() assert s_attr.const == 'value' v_pyobj = hop.inputarg(self, 0) + hop.exception_cannot_occur() return self.getvalue(hop.llops, v_pyobj) def rtype_setattr(self, hop): Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Sat Apr 29 13:41:19 2006 @@ -20,6 +20,7 @@ assert s_attr.is_constant() assert s_attr.const == 'value' v_box = hop.inputarg(self, 0) + hop.exception_cannot_occur() return hop.gendirectcall(ll_chararrayvalue, v_box) def get_c_data_of_item(self, llops, v_stringbuf, v_index): @@ -41,6 +42,7 @@ def rtype_getitem((r_stringbuf, r_int), hop): v_stringbuf, v_index = hop.inputargs(r_stringbuf, lltype.Signed) v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) + hop.exception_cannot_occur() return hop.genop('getarrayitem', [v_array, v_index], resulttype = lltype.Char) @@ -49,6 +51,7 @@ lltype.Signed, lltype.Char) v_array = r_stringbuf.get_c_data(hop.llops, v_stringbuf) + hop.exception_cannot_occur() hop.genop('setarrayitem', [v_array, v_index, v_item]) class __extend__(pairtype(StringBufRepr, AbstractSliceRepr)): Modified: pypy/dist/pypy/rpython/rctypes/rstruct.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstruct.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstruct.py Sat Apr 29 13:41:19 2006 @@ -87,6 +87,7 @@ name = s_attr.const r_field = self.r_fields[name] v_struct, v_attr = hop.inputargs(self, lltype.Void) + hop.exception_cannot_occur() if isinstance(r_field, PrimitiveRepr): # primitive case (optimization; the below also works in this case) # NB. this optimization is invalid for PointerReprs! See for From arigo at codespeak.net Sat Apr 29 14:07:48 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 14:07:48 +0200 (CEST) Subject: [pypy-svn] r26575 - in pypy/dist/pypy: module/_socket rpython/rctypes/socketmodule Message-ID: <20060429120748.EE2A910083@code0.codespeak.net> Author: arigo Date: Sat Apr 29 14:07:47 2006 New Revision: 26575 Modified: pypy/dist/pypy/module/_socket/interp_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Log: Some small progress to make _socket compile. Modified: pypy/dist/pypy/module/_socket/interp_socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/interp_socket.py (original) +++ pypy/dist/pypy/module/_socket/interp_socket.py Sat Apr 29 14:07:47 2006 @@ -102,12 +102,12 @@ def w_get_socketherror(space, message, errno=-1): w_module = space.getbuiltinmodule('_socket') w_errortype = space.getattr(w_module, space.wrap('herror')) - if errno > -1: - if message is None: - message = _c.hstrerror(errno) - return OperationError(w_errortype, space.newtuple([space.wrap(errno), space.wrap(message)])) - else: - return OperationError(w_errortype, space.wrap(message)) + #if errno > -1: + # if message is None: + # message = _c.hstrerror(errno) + # return OperationError(w_errortype, space.newtuple([space.wrap(errno), space.wrap(message)])) + #else: + return OperationError(w_errortype, space.wrap(message)) def wrap_timeouterror(space): @@ -151,7 +151,7 @@ namebuff = ctypes.create_string_buffer(BUFFLEN) res = _c.gethostname(namebuff, BUFFLEN - 1) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.wrap(namebuff.value) gethostname.unwrap_spec = [ObjSpace] @@ -162,7 +162,7 @@ """ hostent = _c.gethostbyname(name) if not hostent: - raise w_get_socketherror(space, None, _c.h_errno.value) + raise w_get_socketherror(space, None) #, _c.h_errno.value) return space.wrap(hostent.contents.h_addr) gethostbyname.unwrap_spec = [ObjSpace, str] @@ -180,7 +180,7 @@ def common_gethost(space, hostent): if not hostent: - raise w_get_socketherror(space, None, _c.h_errno.value) + raise w_get_socketherror(space, None) #, _c.h_errno.value) aliases = [] i = 0 h_aliases = hostent.contents.h_aliases @@ -209,7 +209,7 @@ """ hostent = _c.gethostbyname(name) if not hostent: - raise w_get_socketherror(space, None, _c.h_errno.value) + raise w_get_socketherror(space, None) #, _c.h_errno.value) p_addr = hostent.contents.h_addr_list[0] hostent = _c.gethostbyaddr(p_addr, _c.in_addr_size, _c.AF_INET) @@ -269,7 +269,7 @@ newfd = _c.dup(fd) if newfd < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) if w_proto is None: return space.wrap(Socket(space, newfd, family, type)) else: @@ -298,7 +298,7 @@ result = _c.socketpair_t() error = _c.socketpair(family, sock_type, proto, result) if error < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) s0 = Socket(space, result[0], family, sock_type, proto) s1 = Socket(space, result[1], family, sock_type, proto) return space.newtuple([s0, s1]) @@ -728,7 +728,7 @@ type=_c.SOCK_STREAM, proto=0): fd = _c.socket(family, type, proto) if fd < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) # XXX If we want to support subclassing the socket type we will need # something along these lines. But allocate_instance is only defined # on the standard object space, so this is not really correct. @@ -802,7 +802,7 @@ newfd = _c.socketaccept(self.fd, peeraddr, _c.pointer(peeraddrlen)) if newfd < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) newsocket = Socket(space, newfd, self.family, self.type, self.proto) return space.newtuple([newsocket, w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto)]) accept.unwrap_spec = ['self', ObjSpace] @@ -817,7 +817,7 @@ caddr_ptr, caddr_len = self._getsockaddr(space, w_addr) res = _c.socketbind(self.fd, caddr_ptr, caddr_len) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) bind.unwrap_spec = ['self', ObjSpace, W_Root] def close(self, space): @@ -828,7 +828,7 @@ if not self.closed: res = _c.close(self.fd) if res < 0: - errno = _c.errno.value + errno = _c.geterrno() raise w_get_socketerror(space, None, errno) self.closed = True close.unwrap_spec = ['self', ObjSpace] @@ -853,7 +853,7 @@ sockaddr_ptr, sockaddr_len = self._getsockaddr(space, w_addr) err = _c.socketconnect(self.fd, sockaddr_ptr, sockaddr_len) if err: - errno = _c.errno.value + errno = _c.geterrno() if self.timeout > 0.0: # XXX timeout doesn't really work at the moment pass @@ -871,7 +871,7 @@ """ newfd = _c.dup(self.fd) if newfd < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return Socket(space, newfd, self.family, self.type, self.proto) dup.unwrap_spec = ['self', ObjSpace] @@ -899,7 +899,7 @@ res = _c.socketgetpeername(self.fd, peeraddr, ctypes.pointer(peeraddrlen)) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto) getpeername.unwrap_spec = ['self', ObjSpace] @@ -914,7 +914,7 @@ res = _c.socketgetsockname(self.fd, peeraddr, ctypes.pointer(peeraddrlen)) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return w_makesockaddr(space, peeraddr, peeraddrlen.value, self.proto) getsockname.unwrap_spec = ['self', ObjSpace] @@ -926,19 +926,21 @@ string of that length; otherwise it is an integer. """ if w_buffersize is not None: - c_buffersize = _c.c_int(space.int_w(w_buffersize)) - buffer = ctypes.create_string_buffer(c_buffersize.value) + buffersize = space.int_w(w_buffersize) + c_buffersize = _c.socklen_t(buffersize) + buffer = ctypes.create_string_buffer(buffersize) err = _c.socketgetsockopt(self.fd, level, option, buffer, ctypes.pointer(c_buffersize)) if err: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.wrap(buffer[:c_buffersize.value]) # Assume integer option optval = _c.c_int() + optlen = _c.socklen_t(_c.c_int_size) err = _c.socketgetsockopt(self.fd, level, option, _c.pointer(optval), - _c.c_int_size) + ctypes.pointer(optlen)) if err: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.wrap(optval.value) getsockopt.unwrap_spec = ['self', ObjSpace, int, int, W_Root] @@ -953,7 +955,7 @@ backlog = 1 res = _c.socketlisten(self.fd, backlog) if res == -1: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) listen.unwrap_spec = ['self', ObjSpace, int] def makefile(self, space, w_mode='r', w_buffsize=-1): @@ -971,7 +973,7 @@ buf = _c.create_string_buffer(buffersize) read_bytes = _c.socketrecv(self.fd, buf, buffersize, flags) if read_bytes < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.wrap(buf[:read_bytes]) recv.unwrap_spec = ['self', ObjSpace, int, int] @@ -987,7 +989,7 @@ _c.pointer(sockaddr), _c.sockaddr_size) w_addr = w_makesockaddr(space, _c.pointer(sockaddr), _c.sockaddr_size, self.proto) if read_bytes < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.newtuple([space.wrap(buf[:read_bytes]), w_addr]) recvfrom.unwrap_spec = ['self', ObjSpace, int, int] @@ -1000,7 +1002,7 @@ """ res = _c.send(self.fd, data, len(data), flags) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.wrap(res) send.unwrap_spec = ['self', ObjSpace, str, int] @@ -1015,7 +1017,7 @@ while data: res = _c.send(self.fd, data, len(data), flags) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) data = data[res:] sendall.unwrap_spec = ['self', ObjSpace, str, int] @@ -1035,7 +1037,7 @@ addr, addr_len = self._getsockaddr(space, w_param3) res = _c.sendto(self.fd, data, len(data), flags, addr, addr_len) if res < 0: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) return space.wrap(res) sendto.unwrap_spec = ['self', ObjSpace, str, W_Root, W_Root] @@ -1106,7 +1108,7 @@ """ err = _c.shutdown(self.fd, how) if err: - raise w_get_socketerror(space, None, _c.errno.value) + raise w_get_socketerror(space, None, _c.geterrno()) shutdown.unwrap_spec = ['self', ObjSpace, int] Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/_socket.py Sat Apr 29 14:07:47 2006 @@ -19,7 +19,7 @@ if _fd is None: self._fd = _c.socket(family, type, proto) if self._fd == -1: - raise error(_c.errno.value) + raise error(_c.geterrno()) else: self._fd = _fd @@ -38,7 +38,7 @@ paddr = cast(pointer(caddr), _c.sockaddr_ptr) res = _c.socketbind(self._fd, paddr, sizeof(caddr)) if res < 0: - raise error(_c.errno.value) + raise error(_c.geterrno()) def _getsockaddr(self, addr): if self.family == AF_INET: @@ -58,7 +58,7 @@ backlog = 1 res = _c.socketlisten(fd, backlog) if res == -1: - raise error(_c.errno.value) + raise error(_c.geterrno()) def accept(self): peeraddr = pointer(_c.sockaddr()) @@ -66,7 +66,7 @@ newfd = _c.socketaccept(self._fd, peeraddr, pointer(peeraddrlen)) if newfd < 0: - raise error(_c.errno.value) + raise error(_c.geterrno()) newsocket = socket(self.family, self.type, self.proto, newfd) return (newsocket, makesockaddr(peeraddr, peeraddrlen, self.proto)) @@ -76,7 +76,7 @@ result = _c.socketconnect(self._fd, paddr, _c.socklen_t(sizeof(caddr))) if result == -1: - return _c.errno.value + return _c.geterrno() return 0 def dup(self): @@ -91,7 +91,7 @@ res = _c.socketgetpeername(self._fd, peeraddr, pointer(peeraddrlen)) if res < 0: - raise error(_c.errno.value) + raise error(_c.geterrno()) return makesockaddr(peeraddr, peeraddrlen, self.proto) def getsockname(self): @@ -100,7 +100,7 @@ res = _c.socketgetsockname(self._fd, peeraddr, pointer(peeraddrlen)) if res < 0: - raise error(_c.errno.value) + raise error(_c.geterrno()) return makesockaddr(peeraddr, peeraddrlen, self.proto) def getsockopt(self, level, optname, buflen=-1): Modified: pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py (original) +++ pypy/dist/pypy/rpython/rctypes/socketmodule/ctypes_socket.py Sat Apr 29 14:07:47 2006 @@ -2,6 +2,7 @@ import distutils from pypy.rpython.rctypes.tool import ctypes_platform from pypy.rpython.rctypes.tool import util # ctypes.util from 0.9.9.6 +from pypy.rpython.rctypes.aerrno import geterrno from ctypes import * includes = ('sys/types.h', @@ -188,7 +189,7 @@ dup.argtypes = [c_int] dup.restype = c_int -errno = c_int.in_dll(socketdll, 'errno') +#errno = c_int.in_dll(socketdll, 'errno') strerror = socketdll.strerror strerror.argtypes = [c_int] @@ -198,11 +199,11 @@ gai_strerror.argtypes = [c_int] gai_strerror.restype = c_char_p -h_errno = c_int.in_dll(socketdll, 'h_errno') - -hstrerror = socketdll.hstrerror -hstrerror.argtypes = [c_int] -hstrerror.restype = c_char_p +#h_errno = c_int.in_dll(socketdll, 'h_errno') +# +#hstrerror = socketdll.hstrerror +#hstrerror.argtypes = [c_int] +#hstrerror.restype = c_char_p socket = socketdll.socket socket.argtypes = [c_int, c_int, c_int] From arigo at codespeak.net Sat Apr 29 15:24:04 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 15:24:04 +0200 (CEST) Subject: [pypy-svn] r26576 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060429132404.04C6F10089@code0.codespeak.net> Author: arigo Date: Sat Apr 29 15:24:04 2006 New Revision: 26576 Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Log: Assign to .value on string buffer objects in rctypes. Modified: pypy/dist/pypy/rpython/rctypes/rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/rstringbuf.py Sat Apr 29 15:24:04 2006 @@ -23,6 +23,13 @@ hop.exception_cannot_occur() return hop.gendirectcall(ll_chararrayvalue, v_box) + def rtype_setattr(self, hop): + s_attr = hop.args_s[1] + assert s_attr.is_constant() + assert s_attr.const == 'value' + v_box, v_attr, v_value = hop.inputargs(self, lltype.Void, string_repr) + hop.gendirectcall(ll_stringbuf_setvalue_from_string, v_box, v_value) + def get_c_data_of_item(self, llops, v_stringbuf, v_index): v_array = self.get_c_data(llops, v_stringbuf) v_char_p = llops.genop('direct_arrayitems', [v_array], @@ -94,4 +101,16 @@ return newstr +def ll_stringbuf_setvalue_from_string(box, s): + # Copy the string into the stringbuf. In ctypes the final \x00 is + # copied unless the string has exactly the same size as the stringbuf. + # We do the same, but unlike ctypes don't raise ValueError if the + # string is longer than the stringbuf; we just truncate instead. + # There is no support for setattr raising exceptions in RPython so far. + p = box.c_data + n = min(len(s.chars) + 1, len(p)) + for i in range(n): + p[i] = s.chars[i] + + STRBUFTYPE = lltype.Array(lltype.Char) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rstringbuf.py Sat Apr 29 15:24:04 2006 @@ -124,6 +124,18 @@ res = interpret(func, [12]) assert ''.join(res.chars) == "xy" + def test_specialize_setvalue(self): + def func(n): + buf = create_string_buffer(n) + buf.value = 'abcde' + assert buf.value == 'abcde' + buf.value = 'x' + assert buf.value == 'x' + return ord(buf[2]) + + res = interpret(func, [12]) + assert res == ord('c') # not overridden by buf.value='x' + def test_specialize_slice(self): def func(n): buf = create_string_buffer(n) From arigo at codespeak.net Sat Apr 29 16:11:28 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 16:11:28 +0200 (CEST) Subject: [pypy-svn] r26577 - in pypy/dist/pypy/rpython: lltypesystem rctypes rctypes/test Message-ID: <20060429141128.2120110082@code0.codespeak.net> Author: arigo Date: Sat Apr 29 16:11:26 2006 New Revision: 26577 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/rctypes/rchar_p.py pypy/dist/pypy/rpython/rctypes/rvoid_p.py pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Log: Some more conversions to c_char_p and c_void_p. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Sat Apr 29 16:11:26 2006 @@ -267,6 +267,13 @@ class _char_fakeaccessor(_fakeaccessor): TYPE = lltype.Char + def convert(self, value): + # hack to try to interpret the items of an array of bytes as chars + if lltype.typeOf(value) == lltype.Signed: + value = chr(value) + assert lltype.typeOf(value) == lltype.Char + return value + class _address_fakeaccessor(_fakeaccessor): TYPE = Address Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Sat Apr 29 16:11:26 2006 @@ -7,7 +7,7 @@ from pypy.rpython.rctypes.rstringbuf import StringBufRepr from pypy.annotation.pairtype import pairtype -from ctypes import c_char, c_char_p +from ctypes import c_char, c_char_p, cast class CCharPRepr(CTypesValueRepr): @@ -34,6 +34,13 @@ def setstring(self, llops, v_box, v_str): llops.gendirectcall(ll_setstring, v_box, v_str) + def convert_const(self, value): + if not isinstance(value, (str, c_char_p)): + # maybe an array of characters? cast to a c_char_p + assert type(value)._type_ == c_char + value = cast(value, c_char_p) + return super(CCharPRepr, self).convert_const(value) + def initialize_const(self, p, string): if isinstance(string, c_char_p): string = string.value Modified: pypy/dist/pypy/rpython/rctypes/rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rvoid_p.py Sat Apr 29 16:11:26 2006 @@ -1,27 +1,22 @@ from pypy.rpython.rctypes.rmodel import CTypesValueRepr, C_ZERO from pypy.rpython.rctypes.rstringbuf import StringBufRepr from pypy.annotation.pairtype import pairtype +from pypy.rpython.rstr import AbstractStringRepr +from pypy.rpython.lltypesystem.rstr import string_repr from pypy.rpython.rctypes.rchar_p import CCharPRepr -from pypy.rpython.lltypesystem import llmemory +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.rctypes.rpointer import PointerRepr +from pypy.rpython.rctypes.rarray import ArrayRepr class CVoidPRepr(CTypesValueRepr): - pass # No operations supported on c_void_p instances so far + def convert_const(self, value): + if isinstance(value, self.ctype): + return super(CVoidPRepr, self).convert_const(value) + raise NotImplementedError("XXX constant pointer passed to void* arg") -class __extend__(pairtype(StringBufRepr, CVoidPRepr)): - def convert_from_to((r_from, r_to), v, llops): - # warning: no keepalives, only for short-lived conversions like - # in argument passing - r_temp = r_to.r_memoryowner - v_owned_box = r_temp.allocate_instance(llops) - v_c_array = r_from.get_c_data_of_item(llops, v, C_ZERO) - v_adr = llops.genop('cast_ptr_to_adr', [v_c_array], - resulttype = llmemory.Address) - r_temp.setvalue(llops, v_owned_box, v_adr) - return llops.convertvar(v_owned_box, r_temp, r_to) - # XXX some code duplication above -class __extend__(pairtype(CCharPRepr, CVoidPRepr)): +class __extend__(pairtype(CCharPRepr, CVoidPRepr), + pairtype(PointerRepr, CVoidPRepr)): def convert_from_to((r_from, r_to), v, llops): v_ptr = r_from.getvalue(llops, v) v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], @@ -29,11 +24,27 @@ return r_to.return_value(llops, v_adr) -class __extend__(pairtype(PointerRepr, CVoidPRepr)): +class __extend__(pairtype(StringBufRepr, CVoidPRepr), + pairtype(ArrayRepr, CVoidPRepr)): def convert_from_to((r_from, r_to), v, llops): - v_ptr = r_from.getvalue(llops, v) + v_ptr = r_from.get_c_data_of_item(llops, v, C_ZERO) v_adr = llops.genop('cast_ptr_to_adr', [v_ptr], resulttype = llmemory.Address) return r_to.return_value(llops, v_adr) +class __extend__(pairtype(AbstractStringRepr, CVoidPRepr)): + def convert_from_to((r_from, r_to), v, llops): + # warning: no keepalives, only for short-lived conversions like + # in argument passing + # r_from could be char_repr: first convert it to string_repr + v = llops.convertvar(v, r_from, string_repr) + v_adr = llops.gendirectcall(ll_string2addr, v) + return r_to.return_value(llops, v_adr) + +def ll_string2addr(s): + if s: + ptr = lltype.direct_arrayitems(s.chars) + return llmemory.cast_ptr_to_adr(ptr) + else: + return llmemory.NULL Modified: pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_ctypes.py Sat Apr 29 16:11:26 2006 @@ -174,26 +174,59 @@ assert x.value == 42 def test_pythonapi(): - pythonapi.PyInt_AsLong.argtypes = [py_object] - pythonapi.PyInt_AsLong.restype = c_long - assert pythonapi.PyInt_AsLong(py_object(17L)) == 17 - py.test.raises(TypeError, "pythonapi.PyInt_AsLong(py_object('hello'))") + PyInt_AsLong = pythonapi.PyInt_AsLong + saved = PyInt_AsLong.__dict__.copy() + try: + PyInt_AsLong.argtypes = [py_object] + PyInt_AsLong.restype = c_long + assert PyInt_AsLong(py_object(17L)) == 17 + py.test.raises(TypeError, "PyInt_AsLong(py_object('hello'))") + finally: + PyInt_AsLong.__dict__ = saved def test_py_object_subclass(): - # automatic unwrapping of the py_object result - pythonapi.PyInt_FromLong.argtypes = [c_long] - pythonapi.PyInt_FromLong.restype = py_object - assert isinstance(pythonapi.PyInt_FromLong(17), int) - - # but not if we subclass it... - class W_Object(py_object): - pass - pythonapi.PyInt_FromLong.argtypes = [c_long] - pythonapi.PyInt_FromLong.restype = W_Object - assert isinstance(pythonapi.PyInt_FromLong(17), W_Object) + PyInt_FromLong = pythonapi.PyInt_FromLong + saved = PyInt_FromLong.__dict__.copy() + try: + # automatic unwrapping of the py_object result + PyInt_FromLong.argtypes = [c_long] + PyInt_FromLong.restype = py_object + assert isinstance(PyInt_FromLong(17), int) + + # but not if we subclass it... + class W_Object(py_object): + pass + PyInt_FromLong.argtypes = [c_long] + PyInt_FromLong.restype = W_Object + assert isinstance(PyInt_FromLong(17), W_Object) + finally: + PyInt_FromLong.__dict__ = saved def test_sizeof(): x = create_string_buffer(117) assert sizeof(x) == 117 # assumes that chars are one byte each x = (c_int * 42)() assert sizeof(x) == 42 * sizeof(c_int) + +def test_convert_pointers(): + PyString_FromString = pythonapi.PyString_FromString + saved = PyString_FromString.__dict__.copy() + try: + PyString_FromString.restype = py_object + + # automatic conversions to c_char_p + PyString_FromString.argtypes = [c_char_p] + assert PyString_FromString("hello") == "hello" + assert PyString_FromString(c_char_p("hello")) == "hello" + assert PyString_FromString((c_char * 6)(*"hello")) == "hello" + assert PyString_FromString(create_string_buffer("hello")) == "hello" + + # automatic conversions to c_void_p + PyString_FromString.argtypes = [c_void_p] + assert PyString_FromString("hello") == "hello" + assert PyString_FromString(c_char_p("hello")) == "hello" + assert PyString_FromString((c_char * 6)(*"hello")) == "hello" + assert PyString_FromString((c_byte * 6)(104,101,108,108,111)) =="hello" + assert PyString_FromString(create_string_buffer("hello")) == "hello" + finally: + PyString_FromString.__dict__ = saved Modified: pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rchar_p.py Sat Apr 29 16:11:26 2006 @@ -13,6 +13,7 @@ from pypy.rpython.test.test_llinterp import interpret from ctypes import c_char_p, pointer, Structure +from ctypes import c_int, c_char, create_string_buffer, CFUNCTYPE class Test_annotation: def test_annotate_c_char_p(self): @@ -111,6 +112,31 @@ assert not c_char_p(None) interpret(func, []) + def test_convert_pointers(self): + from pypy.rpython.rctypes.rchar_p import ll_strlen + strlen = CFUNCTYPE(c_int, c_char_p)() # not directly executable! + strlen.__name__ = 'strlen' + strlen.llinterp_friendly_version = ll_strlen + PTR = c_char_p("hello") + BUF = create_string_buffer(10) + BUF.value = "hello" + + def func(n): + # constant arguments + assert strlen("hello") == 5 + assert strlen(PTR) == 5 + assert strlen(BUF) == 5 + # variable arguments + s = chr(n) + 'bc' + assert strlen(s) == 3 + assert strlen(c_char_p(s)) == 3 + assert strlen((c_char * 6)('a', 'b')) == 2 + buf = create_string_buffer(10) + buf.value = "hello" + assert strlen(buf) == 5 + + interpret(func, [65]) + class Test_compilation: def test_compile_c_char_p(self): def func(): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rvoid_p.py Sat Apr 29 16:11:26 2006 @@ -12,6 +12,7 @@ from pypy.rpython.test.test_llinterp import interpret from ctypes import c_void_p, c_int, cast, pointer, POINTER +from ctypes import c_char, c_byte, c_char_p, create_string_buffer, CFUNCTYPE class Test_annotation: def test_annotate_c_void_p(self): @@ -48,6 +49,38 @@ assert not c_void_p() interpret(func, []) + def test_convert_pointers(self): + strlen = CFUNCTYPE(c_int, c_void_p)() + strlen.__name__ = 'strlen' + def ll_strlen_from_void_p(adr): + i = 0 + while adr.char[i] != '\x00': + i += 1 + return i + strlen.llinterp_friendly_version = ll_strlen_from_void_p + PTR = c_char_p("hello") + BUF = create_string_buffer(10) + BUF.value = "hello" + ARR = (c_byte * 10)(65, 66, 67) + + def func(n): + # constant arguments XXX in-progress + ## assert strlen("hello") == 5 + ## assert strlen(PTR) == 5 + ## assert strlen(BUF) == 5 + ## assert strlen(ARR) == 3 + # variable arguments + s = chr(n) + 'bc' + assert strlen(s) == 3 + assert strlen(c_char_p(s)) == 3 + assert strlen((c_char * 6)('a', 'b')) == 2 + assert strlen((c_byte * 6)(104,101,108,108,111)) == 5 + buf = create_string_buffer(10) + buf.value = "hello" + assert strlen(buf) == 5 + + interpret(func, [65]) + class Test_compilation: def test_compile_c_char_p(self): def func(): From cfbolz at codespeak.net Sat Apr 29 21:46:08 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 29 Apr 2006 21:46:08 +0200 (CEST) Subject: [pypy-svn] r26579 - pypy/extradoc/talk/agile2006 Message-ID: <20060429194608.3311A10091@code0.codespeak.net> Author: cfbolz Date: Sat Apr 29 21:46:05 2006 New Revision: 26579 Modified: pypy/extradoc/talk/agile2006/ (props changed) pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt (contents, props changed) Log: fixeol + restification of paper. no content was changed Modified: pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt ============================================================================== --- pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt (original) +++ pypy/extradoc/talk/agile2006/draftpaper_agile2006.txt Sat Apr 29 21:46:05 2006 @@ -1,163 +1,259 @@ -Title: "Trouble in Paradise: Open Source projects, EU-funding and Agile practices" -------------------------------------------------------------------------------------- - -Abstract: -(200 words - write last) - -1.Introduction: -This paper will present the story and the experiences gathered so far in the PyPy project concerning -how to integrate such diverse perspectives as agile practices being used in a distributed and dispersed development -style in an Open Source project that is partly funded by the European Union. - -The PyPy project started as a grass-root effort among core developers in the Python language community, aiming at -building a Python implementation purely built in Python (todays main language implementation is based on C, called Cpython. -It was a language implementation project that was purely driven from a non-profit perspective which -increased in complexity when the Open Source project applied for EU-funding. -We believe that the practice of ?sprinting?, -as being used in the PyPy project, makes Agile and Distributed/dispersed workstyles more possible to -combine. We also believe it is a very useful practice for creating value and ensuring quality in a -projects with hybrid cultures and methodologies. It is our aim to start to show-case this with -experiences from the first funded year of PyPy, combining Agile and Distributed/dispersed -Development in an Open Source Community with the Plan-driven approaches through EU-Funding. - -1.1 Project context -Objectives: -The Technical objective of the PyPy project is to build a implementation of the Open Source -Programming Language in the Python language itself. By employing novel combinations of known -techniques for building optimized compilers and interpreters the project will build an implementation -of Python that is easier to read and understand than the existing implementations, while only paying -a small performance penalty. - -The Scientific objectives of the project are to investigate a number of novel techniques based on -Aspect Oriented Programming code generation and abstract interpretation for the implementation of -practical dynamic languages The goal is to break the compromise between flexibility, simplicity and -performance trade-offs and to expand the range of directly supportable runtime platforms. - -The Method objective is to showcase a novel software engineering process: Sprint Driven Development. -This is an Agile methodology, providing a dynamic and adaptive environment, suitable for co-operative and -distributed development. - -Strategy: -The strategy of the project is to leverage the community of PyPy and Python through open and transparent -communication and working style. The challenge has been to implement this strategy not only in the -F/OSS community part of the project but also in the partially funded consortium structure of the project. -The structure of the project and the consortium needed to adhere to this strategy while also conforming -to the actual requirements of the 6th Framework Programme. - -The consortium consists of 8 partners: DFKI (Germany), Ab Strakt (Sweden), Logilab (France), merlinux GmbH -(Germany), tismerysoft GmbH (Germany), Change Maker (Sweden) , Impara GmbH (Germany) and Heinrich Heine -Universit?t D?sseldorf (Germany) and 4 physical person partners: Laura Creighton (Sweden), Richard Emslie (UK), -Eric Van Riet Paap (Netherlands) , Niklaus Haldiman (Switzerland). The project effort of work for the 2 years of -funding consists of 14 work-packages and in total 58 deliverables. - -History: -Mid 2003 the idea of trying to get EU-funding for the project was born. It became clear that the project had an -arbitrarily large scale and that receiving some funding would dramatically increase the pace and seriousness of the -project - because funded developers can dedicate more of their time to the project. The involved developers and -people stretched outside of the Open Source ecologies to try to gather as much information and contacts as -possible in order to answer the question: "Should we go for it?" to which the answer quickly became "Let's see -how far we get!". - -There had been a growing interest from the European Commission, IST division to look closer at the Open Source world -and its achievements. Several funded research projects in the 5th framework programme studied the phenomenon -(FLOSS-POLS, FLOSS) - its organization, business models and licensing. A few other funded software projects used -Open Source in their work as tools (languages and applications). There was no previous experience of an Open Source -community based project making a bid for funding. - -The areas in the 6th Framework programme (second call) fit very well with the objectives of PyPy. -The idea of strengthening the European Software development companies and businesses with supporting -an open source language implementation was new but appealing to the EU. However, being an Open Source -project was not enough. The challenges and the idea of a flexible, configurable "translator" or -"compiler" met the research targets of the FP6, as well as trying out and documenting the agile methodology -being used. - -In short, we argued that EU funding allowed the project to go from reaching a critical mass and position -to continue to evolve from there, and that it would help European organizations to make some ground. - -Acting on this strategy proved to be a more difficult task. The entire proposal and negotiation process took -over a year (Autumn 2003 until November 2004).A proper description of planned work, necessary to satisfy -formal requirements, had not previously been part of the development focus and both the EU and the parties -involved had to adapt to the situation. Yet, drafting the high-level requirements (in total 14 work-packages -and 58 deliverables) was done using the same version-control/open-communication based work style, including -evolving the proposal at sprints. Writing the proposal and specifying according objectives on a higher level -has proved to be generally useful for clarifying goals on a longer term. -It also helps others to understand the project better. - -Unfortunately the negotiations with the EU got stuck in organizational limbo and the project is still -suffering from the effects of this even today. The goal of funding contributors especially coming to -sprints was originally based on a non-profit association. This solution wasn't seen as realistic or -feasible by the EU although we think it remains a viable approach for the future. During negotiations, -we got to an alternative solution which had a few drawbacks: contributors have to become Contract -Partners within the EU-level Consortium (which is by itself not difficult) and can then at least claim travel and -accommodation costs when attending sprints. - -However, this construction does not allow them to get paid for work time and also has some formal requirements. -In practice this leads to current considerations of developers to shift private money between them in order to -circumvent the current problems withimplementing an agile model within the EU contract framing. - - -2. Influencing factors: the F/OSS Python community culture -2.1 The Python community -2.2 The PyPy community -2.3 Supporting infrastructure -2.4 Supporting practices - -3. Influencing factors: agile practices in PyPy -3.1 TDD -3.2 XP -3.3 Sprints - -4. Influencing factors: EU-project practices -The formal project organization required by the EU imposed more structure on the previous more free-floating -agile process. Roles and responsibilities where staked out,conforming with the requirements of the roles -but delegating as much as possible of the responsibilities and decision-making to the core developers. The -strategy was to keep "conceptual integrity" (Brooks) of the vision and the idea in the hands of the core -developers. A somewhat negative result was the added workload and responsibility on developers regarding -EU related work. It was also a conscious strategy to employ the same version-control/review based scheme -regarding EU documents and Consortium level issues as those being used in the technical development. - -It remains a challenge for all partners of the consortium, universities and companies alike, to connect an -ongoing medium-scale open-source project with EU regulations and requirements - not to speak of the fact that -companies need to fund 50% of the costs themselves. It is, in fact, too early to judge the overall success -of our approaches although we are confident that things work out reasonably well. - -4.1 Consortium structure -4.2 Resource tracking and reporting -4.3 Communication and documentation - -5. Troubles in Paradise:striking a balance -5.1 Developer driven versus formal project structure -5.2 Agile strategies versus formal EU-contractual requirements -5.3 F/OSS community versus hierarchies for "conceptual integrity" - -6. Conclusion: -The important question is the following (and is also part of the methodological objective of the project): -is there enough room to manage a project and Ope Source community within the plan-driven inspired methods -that are required in EU-funded projects, while still working agile and distributed? - -We believe so. The one clear dominating factor to make all this succeed is, as always, the people factor, -the CRACK performers as Boehm and Turner calls them (?Collaborative, Representative, Authorized, Committed, -Knowledgeable?) (4). - -The core developers of the PyPy project had the right mix of various skills in order to succeed in setting -up a hybrid environment - enabling them to work full time on a project they strongly believed in. -The most crucial mix of skills for making this possible was/are: - -- social (the ability to communicate, "manage" groups and handle conflicts) -- leadership abilities (both on technical levels as well on general social levels) -- ability to network (to find and invite actors in the various cultures) -- entrepreneurs (to instigate the idea of EU-funding and also to create consortium and companies) -- curious and collaborative (towards other Python implementations and other languages and research approaches) -- technical skills (programming language and implementation aspects, frameworks, mathematics, computer science etc) -- ability to balance "conceptual integrity" (Brooks) with open and transparent communication through sprints, documentation, tutorials, mentoring, sync-meetings, thus also increasing the group of the core developers during the project - -Drawing from these different skills within the community of developers in the PyPy project one possible conclusion -would be that a truly agile approach dominating the workstyle of an Open Source project will increase the ability -of the community to spread the strategy of agility to other domains such as entrepreneurship and business models -(whether this is done by separate individuals, subgroups or the whole group). By doing this hybrid projects such -as PyPy and others (Zope Europe among others) are made possible. - -6.1 The vision factor -6.2 Managing diversities -6.3 Process design -6.4 Group learning \ No newline at end of file +=================================================================================== +Title: "Trouble in Paradise: Open Source projects, EU-funding and Agile practices" +=================================================================================== + +Abstract: +(200 words - write last) + +Introduction: +============= + +This paper will present the story and the experiences gathered so far in the +PyPy project concerning how to integrate such diverse perspectives as agile +practices being used in a distributed and dispersed development style in an +Open Source project that is partly funded by the European Union. + +The PyPy project started as a grass-root effort among core developers in the +Python language community, aiming at building a Python implementation purely +built in Python (todays main language implementation is based on C, called +Cpython. It was a language implementation project that was purely driven from +a non-profit perspective which increased in complexity when the Open Source +project applied for EU-funding. We believe that the practice of "sprinting", +as being used in the PyPy project, makes Agile and Distributed/dispersed +workstyles more possible to combine. We also believe it is a very useful +practice for creating value and ensuring quality in a projects with hybrid +cultures and methodologies. It is our aim to start to show-case this with +experiences from the first funded year of PyPy, combining Agile and +Distributed/dispersed Development in an Open Source Community with the +Plan-driven approaches through EU-Funding. + +Project context ++++++++++++++++ + +Objectives: +~~~~~~~~~~~ + +The Technical objective of the PyPy project is to build a implementation of the +Open Source Programming Language in the Python language itself. By employing +novel combinations of known techniques for building optimized compilers and +interpreters the project will build an implementation of Python that is easier +to read and understand than the existing implementations, while only paying a +small performance penalty. + +The Scientific objectives of the project are to investigate a number of novel +techniques based on Aspect Oriented Programming code generation and abstract +interpretation for the implementation of practical dynamic languages The goal +is to break the compromise between flexibility, simplicity and performance +trade-offs and to expand the range of directly supportable runtime platforms. + +The Method objective is to showcase a novel software engineering process: +Sprint Driven Development. This is an Agile methodology, providing a dynamic +and adaptive environment, suitable for co-operative and distributed +development. + +Strategy: +~~~~~~~~~ + +The strategy of the project is to leverage the community of PyPy and Python +through open and transparent communication and working style. The challenge has +been to implement this strategy not only in the F/OSS community part of the +project but also in the partially funded consortium structure of the project. +The structure of the project and the consortium needed to adhere to this +strategy while also conforming to the actual requirements of the 6th Framework +Programme. + +The consortium consists of 8 partners: DFKI (Germany), Ab Strakt (Sweden), +Logilab (France), merlinux GmbH (Germany), tismerysoft GmbH (Germany), Change +Maker (Sweden) , Impara GmbH (Germany) and Heinrich Heine Universit?t +D?sseldorf (Germany) and 4 physical person partners: Laura Creighton (Sweden), +Richard Emslie (UK), Eric Van Riet Paap (Netherlands) , Niklaus Haldiman +(Switzerland). The project effort of work for the 2 years of funding consists +of 14 work-packages and in total 58 deliverables. + +History: +~~~~~~~~ + +Mid 2003 the idea of trying to get EU-funding for the project was born. It +became clear that the project had an arbitrarily large scale and that receiving +some funding would dramatically increase the pace and seriousness of the +project - because funded developers can dedicate more of their time to the +project. The involved developers and people stretched outside of the Open +Source ecologies to try to gather as much information and contacts as possible +in order to answer the question: "Should we go for it?" to which the answer +quickly became "Let's see how far we get!". + +There had been a growing interest from the European Commission, IST division to +look closer at the Open Source world and its achievements. Several funded +research projects in the 5th framework programme studied the phenomenon +(FLOSS-POLS, FLOSS) - its organization, business models and licensing. A few +other funded software projects used Open Source in their work as tools +(languages and applications). There was no previous experience of an Open +Source community based project making a bid for funding. + +The areas in the 6th Framework programme (second call) fit very well with the +objectives of PyPy. The idea of strengthening the European Software +development companies and businesses with supporting an open source language +implementation was new but appealing to the EU. However, being an Open Source +project was not enough. The challenges and the idea of a flexible, configurable +"translator" or "compiler" met the research targets of the FP6, as well as +trying out and documenting the agile methodology being used. + +In short, we argued that EU funding allowed the project to go from reaching a +critical mass and position to continue to evolve from there, and that it would +help European organizations to make some ground. + +Acting on this strategy proved to be a more difficult task. The entire proposal +and negotiation process took over a year (Autumn 2003 until November 2004).A +proper description of planned work, necessary to satisfy formal requirements, +had not previously been part of the development focus and both the EU and the +parties involved had to adapt to the situation. Yet, drafting the high-level +requirements (in total 14 work-packages and 58 deliverables) was done using the +same version-control/open-communication based work style, including evolving +the proposal at sprints. Writing the proposal and specifying according +objectives on a higher level has proved to be generally useful for clarifying +goals on a longer term. It also helps others to understand the project better. + +Unfortunately the negotiations with the EU got stuck in organizational limbo +and the project is still suffering from the effects of this even today. The +goal of funding contributors especially coming to sprints was originally based +on a non-profit association. This solution wasn't seen as realistic or +feasible by the EU although we think it remains a viable approach for the +future. During negotiations, we got to an alternative solution which had a few +drawbacks: contributors have to become Contract Partners within the EU-level +Consortium (which is by itself not difficult) and can then at least claim +travel and accommodation costs when attending sprints. + +However, this construction does not allow them to get paid for work time and +also has some formal requirements. In practice this leads to current +considerations of developers to shift private money between them in order to +circumvent the current problems withimplementing an agile model within the EU +contract framing. + + +Influencing factors: the F/OSS Python community culture +======================================================== + +The Python community +-------------------- + + +The PyPy community +------------------ + +Supporting infrastructure +-------------------------- + +Supporting practices +-------------------- + +Influencing factors: agile practices in PyPy +============================================ + +TDD +--- + +XP +-- + +Sprints +------- + +EU-project practices +==================== + +The formal project organization required by the EU imposed more structure on +the previous more free-floating agile process. Roles and responsibilities where +staked out,conforming with the requirements of the roles but delegating as much +as possible of the responsibilities and decision-making to the core developers. +The strategy was to keep "conceptual integrity" (Brooks) of the vision and the +idea in the hands of the core developers. A somewhat negative result was the +added workload and responsibility on developers regarding EU related work. It +was also a conscious strategy to employ the same version-control/review based +scheme regarding EU documents and Consortium level issues as those being used +in the technical development. + +It remains a challenge for all partners of the consortium, universities and +companies alike, to connect an ongoing medium-scale open-source project with EU +regulations and requirements - not to speak of the fact that companies need to +fund 50% of the costs themselves. It is, in fact, too early to judge the +overall success of our approaches although we are confident that things work +out reasonably well. + +Consortium structure +-------------------- + +Resource tracking and reporting +------------------------------- + +Communication and documentation +------------------------------- + +Troubles in Paradise:striking a balance +======================================= + +Developer driven versus formal project structure +------------------------------------------------ + +Agile strategies versus formal EU-contractual requirements +---------------------------------------------------------- + +F/OSS community versus hierarchies for "conceptual integrity" +------------------------------------------------------------- + +Conclusion +========== + +The important question is the following (and is also part of the methodological +objective of the project): is there enough room to manage a project and Ope +Source community within the plan-driven inspired methods that are required in +EU-funded projects, while still working agile and distributed? + +We believe so. The one clear dominating factor to make all this succeed is, as +always, the people factor, the CRACK performers as Boehm and Turner calls them +("Collaborative, Representative, Authorized, Committed, Knowledgeable") (4). + +The core developers of the PyPy project had the right mix of various skills in +order to succeed in setting up a hybrid environment - enabling them to work +full time on a project they strongly believed in. The most crucial mix of +skills for making this possible was/are: + +- social (the ability to communicate, "manage" groups and handle conflicts) + +- leadership abilities (both on technical levels as well on general social + levels) + +- ability to network (to find and invite actors in the various cultures) + +- entrepreneurs (to instigate the idea of EU-funding and also to create + consortium and companies) + +- curious and collaborative (towards other Python implementations and other + languages and research approaches) + +- technical skills (programming language and implementation aspects, + frameworks, mathematics, computer science etc) + +- ability to balance "conceptual integrity" (Brooks) with open and transparent + communication through sprints, documentation, tutorials, mentoring, + sync-meetings, thus also increasing the group of the core developers during + the project + +Drawing from these different skills within the community of developers in the +PyPy project one possible conclusion would be that a truly agile approach +dominating the workstyle of an Open Source project will increase the ability of +the community to spread the strategy of agility to other domains such as +entrepreneurship and business models (whether this is done by separate +individuals, subgroups or the whole group). By doing this hybrid projects such +as PyPy and others (Zope Europe among others) are made possible. + +The vision factor +----------------- + +Managing diversities +-------------------- + +Process design +-------------- + +Group learning +-------------- From cfbolz at codespeak.net Sat Apr 29 21:51:00 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 29 Apr 2006 21:51:00 +0200 (CEST) Subject: [pypy-svn] r26580 - in pypy/extradoc: planning planning/malaga_exhibition_feb2006 pypy.org soc-2006 sprintinfo/gothenburg-2005 sprintinfo/iceland sprintinfo/louvain-la-neuve-2006 sprintinfo/mallorca sprintinfo/paris sprintinfo/pycon06 talk talk/louvain-la-neuve-2006 talk/pycon2006 talk/pypy_euworkshop_2005-12-08 Message-ID: <20060429195100.6C3C01008B@code0.codespeak.net> Author: cfbolz Date: Sat Apr 29 21:50:50 2006 New Revision: 26580 Modified: pypy/extradoc/planning/conference-planning.txt (contents, props changed) pypy/extradoc/planning/malaga_exhibition_feb2006/ (props changed) pypy/extradoc/planning/malaga_exhibition_feb2006/information_pypy_malaga.txt (contents, props changed) pypy/extradoc/planning/sprint-planning.txt (contents, props changed) pypy/extradoc/pypy.org/links.txt (props changed) pypy/extradoc/pypy.org/revision_march2006.txt (contents, props changed) pypy/extradoc/soc-2006/ (props changed) pypy/extradoc/soc-2006/code-templating.txt (props changed) pypy/extradoc/soc-2006/constraints.txt (props changed) pypy/extradoc/soc-2006/numeric-arrays.txt (props changed) pypy/extradoc/soc-2006/planning.txt (props changed) pypy/extradoc/sprintinfo/gothenburg-2005/logistics.txt (props changed) pypy/extradoc/sprintinfo/iceland/ (props changed) pypy/extradoc/sprintinfo/iceland/people-travel-planning.txt (props changed) pypy/extradoc/sprintinfo/iceland/pypy-dev-announce.txt (props changed) pypy/extradoc/sprintinfo/louvain-la-neuve-2006/ (props changed) pypy/extradoc/sprintinfo/louvain-la-neuve-2006/people.txt (props changed) pypy/extradoc/sprintinfo/louvain-la-neuve-2006/planning.txt (props changed) pypy/extradoc/sprintinfo/louvain-la-neuve-2006/pypy-oz-agenda.txt (props changed) pypy/extradoc/sprintinfo/louvain-la-neuve-2006/report.txt (props changed) pypy/extradoc/sprintinfo/louvain-la-neuve-2006/sprint-announcement.txt (props changed) pypy/extradoc/sprintinfo/mallorca/ (props changed) pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt (props changed) pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint_report.txt (props changed) pypy/extradoc/sprintinfo/mallorca/planning.txt (props changed) pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt (contents, props changed) pypy/extradoc/sprintinfo/paris/paris-report.txt (contents, props changed) pypy/extradoc/sprintinfo/pycon06/ (props changed) pypy/extradoc/talk/gc_slides.txt (props changed) pypy/extradoc/talk/louvain-la-neuve-2006/ (props changed) pypy/extradoc/talk/pycon2006/ (props changed) pypy/extradoc/talk/pycon2006/method_talk.txt (contents, props changed) pypy/extradoc/talk/pypy_euworkshop_2005-12-08/ (props changed) pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt (contents, props changed) pypy/extradoc/talk/pypy_euworkshop_2005-12-08/pypy_alastair_part1_intro.txt (props changed) pypy/extradoc/talk/solutions-linux-paris-2006.txt (props changed) Log: fixeol (!) Modified: pypy/extradoc/planning/conference-planning.txt ============================================================================== --- pypy/extradoc/planning/conference-planning.txt (original) +++ pypy/extradoc/planning/conference-planning.txt Sat Apr 29 21:50:50 2006 @@ -60,103 +60,103 @@ Time & location: 24-26 Feburary, 2006, Addison, Texas (near Dallas) Out of the pypy group four talks were submitted. -Acceptance pending (15th November ASFAIK). +Acceptance pending (15th November ASFAIK). The 3 PyPy talks where accepted into the PyCon programme! Due: sprint planning, possibly inviting Ka Ping (regarding E-lang) - - -Calibre workshop ------------------ -CALIBRE Workshop on Distributed Development, Open Source & Industry + + +Calibre workshop +----------------- +CALIBRE Workshop on Distributed Development, Open Source & Industry - Opportunities, Suspicions, & Strategies - -Time & location: 3 of March 2006 Sk?vde Sweden - -www.calibre.ie - -No papers submitted - Bea will attend. -( Bea didnt attend - Jacob did). - -XP France: --------------- -24-25th March? Nicholas? - -Logilab attended and did a talk. - - -ACCU 2006 ----------- - -DEADLINE: 10th of February (early bird) - -Software development - C++, Agile methods and Python. - -Time & location: 19th -22nd of April, Oxford, UK - -http://www.accu.org/conference/ - -No papers submitted - attendance - Michael H, Jacob? -Michael will do a talk there about PyPy. - -XP 2006 ------------ - -eXtreme programming and agile processes in software engineering. - -Time & location: 17-22th June 2006 Oulu, Finland - -Call for papers: -Submission date: 30th of January 2006 -Experience report accepted - Bea will go and present. - -http://virtual.vtt.fi/virtual/xp2006/sivu1.html - -Agile 2006 ------------ - -DEADLINE: 23 June 2005 (end of early bird registration) - -Annual US agile conference (the main agile alliance conference) - last year were -sold out. - -Call for papers: -http://www.agile2006.org./ -- 31 January for research papers -- 28 February for experience reports - -Time and location: 23-28th of July 2006, Minneapolis, Minnesota, USA - -Experience report proposal accepted - Bea will submit the report -(first draft 14th April, final draft 30th April) and go there. - -OOPSLA 2006 ------------- -DEADLINE: registration starts 15th of June (no deadline known) - -International conference on Objecto Oriented programming, Systems -languages and applications. - -Time & location: 22nd-26th of October 2006, Portland, Oregon - -Call for papers: -Submission deadline: 18th of March - -http://www.oopsla.org/2006/ - -Samuele and Armin plan to submit paper and attend. - -No paper submitted - attendance (workshop dynamic languages)? - -MIDDLEWARE 2006 -------------------------- -DEADLINE: 3rd April 2006/scientific papers - -Time & location: November 27 - December 1, MELBOURNE, AUSTRALIA - - http://2006.middleware-conference.org/ - -ACM/IFIP/USENIX 7th International Middleware Conference - "Novel development paradigms, APIs, and languages"/PyPy? - - Suggestion by Holger. Submission/attendance? + +Time & location: 3 of March 2006 Sk?vde Sweden + +www.calibre.ie + +No papers submitted - Bea will attend. +( Bea didnt attend - Jacob did). + +XP France: +-------------- +24-25th March? Nicholas? + +Logilab attended and did a talk. + + +ACCU 2006 +---------- + +DEADLINE: 10th of February (early bird) + +Software development - C++, Agile methods and Python. + +Time & location: 19th -22nd of April, Oxford, UK + +http://www.accu.org/conference/ + +No papers submitted - attendance - Michael H, Jacob? +Michael will do a talk there about PyPy. + +XP 2006 +----------- + +eXtreme programming and agile processes in software engineering. + +Time & location: 17-22th June 2006 Oulu, Finland + +Call for papers: +Submission date: 30th of January 2006 +Experience report accepted - Bea will go and present. + +http://virtual.vtt.fi/virtual/xp2006/sivu1.html + +Agile 2006 +----------- + +DEADLINE: 23 June 2005 (end of early bird registration) + +Annual US agile conference (the main agile alliance conference) - last year were +sold out. + +Call for papers: +http://www.agile2006.org./ +- 31 January for research papers +- 28 February for experience reports + +Time and location: 23-28th of July 2006, Minneapolis, Minnesota, USA + +Experience report proposal accepted - Bea will submit the report +(first draft 14th April, final draft 30th April) and go there. + +OOPSLA 2006 +------------ +DEADLINE: registration starts 15th of June (no deadline known) + +International conference on Objecto Oriented programming, Systems +languages and applications. + +Time & location: 22nd-26th of October 2006, Portland, Oregon + +Call for papers: +Submission deadline: 18th of March + +http://www.oopsla.org/2006/ + +Samuele and Armin plan to submit paper and attend. + +No paper submitted - attendance (workshop dynamic languages)? + +MIDDLEWARE 2006 +------------------------- +DEADLINE: 3rd April 2006/scientific papers + +Time & location: November 27 - December 1, MELBOURNE, AUSTRALIA + + http://2006.middleware-conference.org/ + +ACM/IFIP/USENIX 7th International Middleware Conference + "Novel development paradigms, APIs, and languages"/PyPy? + + Suggestion by Holger. Submission/attendance? Modified: pypy/extradoc/planning/malaga_exhibition_feb2006/information_pypy_malaga.txt ============================================================================== --- pypy/extradoc/planning/malaga_exhibition_feb2006/information_pypy_malaga.txt (original) +++ pypy/extradoc/planning/malaga_exhibition_feb2006/information_pypy_malaga.txt Sat Apr 29 21:50:50 2006 @@ -1,151 +1,151 @@ -II Open Source World Conference Malaga 2006 -=============================================== - -Time: 15-17th February 2006 -Place: Malaga, Trade Fairs and Congress Centre Malaga -url: http://www.opensourceworldconference.com/malaga06/en/modules/wiwimod/index.php?page=Sobre+M%E1laga -Arranging conference: Juan Reig, ++34670644131 -Final programme: see extradoc/planning/malaga_exhibition_feb2006/ -Programme IIOSWC 2006 - FINAL.pdf - -PyPy will: - -1. Participate in the exhibition, hosted by the European Commission, DG -Information Society and Media - PyPy and tOSSad as OSS projects funded by -the Commission. - -EC will only present their DG activities (no slide show or presentations) - just -distributing advertising material. tOSSad will bring poster and brochures. -PyPy will bring mobile roll-up poster, printed PyPy Fact Sheets and a project -presentation. - -EC contacts: 3 people participating Anne-Marie Sassen,Jesus Villasante, -Valerie Stagno, (Graziana Lonero?) -tOSSad contacts: 1-3 people participating, Bulent Ozel (Kaan Erkan?) -PyPy: Jacob Hall?n, Alastair Burt - -(the names mentioned for EC and tOSSad where people involved in emails during the -preparation of the exhibition - no confirmation with the actual names was sent out. PyPy -presented our persons with names). - -2. Workshop PyPy 16th Febr 12-13 Jacob Hall?n (english) - -Alastair talked to the organizer Juan Reig, and agreed to Alastair doing the -talk in spanish. Deadline for presentations submissions was 10th Febr. -They have not changed name for speaker or language in the last version of the -programme. - --------------------------------------------- -Information from the organizers: --------------------------------------------- -Email: 2006-02-09 - -The Organization of the II Open Source World Conference would -like to provide you with updated information on the general -situation of the Conference and we enclose the final Programme. - -We would like to communicate that a change in the schedule of the -Programme of the conference has taken place, being transferred -the official opening session and the Key note speech to 12:00 -in the morning of the 15th of February, due to a series of reasons: -to take advantage of the opportunity to have like lecturer of the -opening session Michail Bletsas, from the Massachusetts Institute -of Technology (MIT), Chief Connectivity Officer in charge of -the Project 100$ Laptop and "One laptop per child", that it is -being designed on the base of an open source operating system; -we have also had to extend the number of sessions due to the -important increase of speakers and the extension of the plenary -session on patents, licences and IPR, motivated by the number of -participants anticipated in the debate. - -The Conference maintains a very high scientific and technical -level and the diversity of speakers and participants that -always we have tried and that without a doubt will enrich the -results of this event. Around 200 speakers will participate -along the next days in the sessions in Malaga. In relation to -the attendance we have confirmed over 4500 participants with -which we consider a presence throughout the three days around -6000 attendees. - -Shortly we will send you the instructions on how to get to the -Trade Fair and Conference Centre of Malaga and once there where -to pick your personalized accreditation. - -In addition, inform you that tomorrow, the 10th is the last -day to receive your presentation in case you haven?t sent it -yet. If for any circumstance you can not do it, please get in -contact with the Technical Secretariat at the Trade Centre asap. - -Please keep in mind the change in the schedule! I want to -encourage you take advantage of the opportunity of being able -to listen to the key responsible of an exciting worldwide project -for everybody. - -Best regards, -JUAN REIG -Conference Chairman -II Open Source World Conference -MALAGA 15-17 FEBRUARY 2006 -+34670644131-FAX +34670641066 -chairman at opensourceworldconference.com -www.opensourceworldconference.com - -Email: 2006-02-11 - -Dear Speaker - -For your comfort we will try to deliver your accreditation in the room of -your Hotel. If you arrive directly to the Congress Centre, you can pick-up -your accreditation at the Technical Secretariat, which will be located -between both main auditoriums next to the central patio. We remain you -that it is not necessary to go through the accreditation counters of the -entrance hall. Please verify in the reception of the hotel the schedule of -buses that leave towards the Congress Centre. - -We make available for all the attendees, free access points to high speed -Internet located within the Exhibition between the stands. In case you prefer to connect with some own portable device with wireless connectivity, you will have also the possibility since all the facility WIFI connected -for free access to the participants. In that case, you will only have to select -the option of configuration of DHCP. However, if you have any problem on -connectivity the technical personnel will be able to assist you. - -We want also to indicate that there will be a room for speakers located in -the first floor, Room n? 10, where it will be possible to update your presentation -or to use some of the computers available for you. - -Before your session we recommended you to contact with the chair of your -session directly or through the Technical Secretariat. Please indicate to -him/her how you want to de introduced and enumerate very briefly the -two or three key points of your presentation. - -We also remind you some previous information concerning your presentation -already sent to you: - - * We recommend a 15 minutes oral presentation to allow some - discussion and Q&A time at the end of the session. To protect - the rights of your other colleague speakers and the rights of the - attendees to listen to all the presentation included on the programme, - it would not be possible for any speaker to go beyond 20 minutes. The - chair has received instructions to be very strict on the timing due to the - above reasons. We ask for your kind cooperation on this issue. - - * The presentation format should be OpenOffice or PDF. Except exceptional - cases, only organization computers or laptops with preloaded presentations - will be used for the presentation at the conference rooms. In case that - you need to modify or update your presentation there will be available working - rooms at the Technical Secretariat or speaker?s room. There will be - also a technical team specialized in Open Source to support you in - case you have additional request at the Trade Fair and Conference - Centre at Malaga. If for exceptional reasons you need to use your - own laptop, please let us know at the Technical Secretariat to make - sure that everything goes smoothly. - -We wish you a pleasant trip and look forward to welcome you in -Malaga personally. - -Juan Reig - -Conference Chairman - -+34670644131 -chairman at opensourceworldconference.com +II Open Source World Conference Malaga 2006 +=============================================== + +Time: 15-17th February 2006 +Place: Malaga, Trade Fairs and Congress Centre Malaga +url: http://www.opensourceworldconference.com/malaga06/en/modules/wiwimod/index.php?page=Sobre+M%E1laga +Arranging conference: Juan Reig, ++34670644131 +Final programme: see extradoc/planning/malaga_exhibition_feb2006/ +Programme IIOSWC 2006 - FINAL.pdf + +PyPy will: + +1. Participate in the exhibition, hosted by the European Commission, DG +Information Society and Media - PyPy and tOSSad as OSS projects funded by +the Commission. + +EC will only present their DG activities (no slide show or presentations) - just +distributing advertising material. tOSSad will bring poster and brochures. +PyPy will bring mobile roll-up poster, printed PyPy Fact Sheets and a project +presentation. + +EC contacts: 3 people participating Anne-Marie Sassen,Jesus Villasante, +Valerie Stagno, (Graziana Lonero?) +tOSSad contacts: 1-3 people participating, Bulent Ozel (Kaan Erkan?) +PyPy: Jacob Hall?n, Alastair Burt + +(the names mentioned for EC and tOSSad where people involved in emails during the +preparation of the exhibition - no confirmation with the actual names was sent out. PyPy +presented our persons with names). + +2. Workshop PyPy 16th Febr 12-13 Jacob Hall?n (english) + +Alastair talked to the organizer Juan Reig, and agreed to Alastair doing the +talk in spanish. Deadline for presentations submissions was 10th Febr. +They have not changed name for speaker or language in the last version of the +programme. + +-------------------------------------------- +Information from the organizers: +-------------------------------------------- +Email: 2006-02-09 + +The Organization of the II Open Source World Conference would +like to provide you with updated information on the general +situation of the Conference and we enclose the final Programme. + +We would like to communicate that a change in the schedule of the +Programme of the conference has taken place, being transferred +the official opening session and the Key note speech to 12:00 +in the morning of the 15th of February, due to a series of reasons: +to take advantage of the opportunity to have like lecturer of the +opening session Michail Bletsas, from the Massachusetts Institute +of Technology (MIT), Chief Connectivity Officer in charge of +the Project 100$ Laptop and "One laptop per child", that it is +being designed on the base of an open source operating system; +we have also had to extend the number of sessions due to the +important increase of speakers and the extension of the plenary +session on patents, licences and IPR, motivated by the number of +participants anticipated in the debate. + +The Conference maintains a very high scientific and technical +level and the diversity of speakers and participants that +always we have tried and that without a doubt will enrich the +results of this event. Around 200 speakers will participate +along the next days in the sessions in Malaga. In relation to +the attendance we have confirmed over 4500 participants with +which we consider a presence throughout the three days around +6000 attendees. + +Shortly we will send you the instructions on how to get to the +Trade Fair and Conference Centre of Malaga and once there where +to pick your personalized accreditation. + +In addition, inform you that tomorrow, the 10th is the last +day to receive your presentation in case you haven?t sent it +yet. If for any circumstance you can not do it, please get in +contact with the Technical Secretariat at the Trade Centre asap. + +Please keep in mind the change in the schedule! I want to +encourage you take advantage of the opportunity of being able +to listen to the key responsible of an exciting worldwide project +for everybody. + +Best regards, +JUAN REIG +Conference Chairman +II Open Source World Conference +MALAGA 15-17 FEBRUARY 2006 ++34670644131-FAX +34670641066 +chairman at opensourceworldconference.com +www.opensourceworldconference.com + +Email: 2006-02-11 + +Dear Speaker + +For your comfort we will try to deliver your accreditation in the room of +your Hotel. If you arrive directly to the Congress Centre, you can pick-up +your accreditation at the Technical Secretariat, which will be located +between both main auditoriums next to the central patio. We remain you +that it is not necessary to go through the accreditation counters of the +entrance hall. Please verify in the reception of the hotel the schedule of +buses that leave towards the Congress Centre. + +We make available for all the attendees, free access points to high speed +Internet located within the Exhibition between the stands. In case you prefer to connect with some own portable device with wireless connectivity, you will have also the possibility since all the facility WIFI connected +for free access to the participants. In that case, you will only have to select +the option of configuration of DHCP. However, if you have any problem on +connectivity the technical personnel will be able to assist you. + +We want also to indicate that there will be a room for speakers located in +the first floor, Room n? 10, where it will be possible to update your presentation +or to use some of the computers available for you. + +Before your session we recommended you to contact with the chair of your +session directly or through the Technical Secretariat. Please indicate to +him/her how you want to de introduced and enumerate very briefly the +two or three key points of your presentation. + +We also remind you some previous information concerning your presentation +already sent to you: + + * We recommend a 15 minutes oral presentation to allow some + discussion and Q&A time at the end of the session. To protect + the rights of your other colleague speakers and the rights of the + attendees to listen to all the presentation included on the programme, + it would not be possible for any speaker to go beyond 20 minutes. The + chair has received instructions to be very strict on the timing due to the + above reasons. We ask for your kind cooperation on this issue. + + * The presentation format should be OpenOffice or PDF. Except exceptional + cases, only organization computers or laptops with preloaded presentations + will be used for the presentation at the conference rooms. In case that + you need to modify or update your presentation there will be available working + rooms at the Technical Secretariat or speaker?s room. There will be + also a technical team specialized in Open Source to support you in + case you have additional request at the Trade Fair and Conference + Centre at Malaga. If for exceptional reasons you need to use your + own laptop, please let us know at the Technical Secretariat to make + sure that everything goes smoothly. + +We wish you a pleasant trip and look forward to welcome you in +Malaga personally. + +Juan Reig + +Conference Chairman + ++34670644131 +chairman at opensourceworldconference.com www.opensourceworldconference.com Modified: pypy/extradoc/planning/sprint-planning.txt ============================================================================== --- pypy/extradoc/planning/sprint-planning.txt (original) +++ pypy/extradoc/planning/sprint-planning.txt Sat Apr 29 21:50:50 2006 @@ -1,88 +1,88 @@ -Author: Beatrice D?ring -20060103 - -Sprint planning PyPy ----------------------- - -January/Palma de Mallorca --------------------------- - -Time & location: 23-29th January 2006, CS Dep. of the Palma de -Mallorca University/GNU Linux lab. - -Nature: open - -Primary organizer: Bea - -February/PyCon (USA) ---------------------- - -Time & location: 27 Feb - 1st March 2006 PyCon -(24-26 Feb, 2006 Dallas, Texas, - -Nature: postsprint - -Primary organizer: Holger and Michael - -March/Europe -------------------- - -Time & location: Louvain-La-Neuve 6th-10th March - -Nature: joint sprint with OZ community and Roel Wuyts/ULB attending. -DFKI and Logilab primarily + Samuele, Carl F. - -Primary organizer: Nicolas Chauvat - -April/Leysin -------------- - -Time & location: 3rd to 9th of April, 2006, Leysin, Switzerland - -Nature: closed/core - -Primary organizer: Armin - -April/Tokyo -------------- - -Time & location: 23rd to 29th of April, 2006, host: -Yutaka Niibe, National Institute of AIST, Japan (FSIJ) -Akihabara, Tokyo, venue for 32 people. - -Nature: open, dissemination - -Primary organizer: Bea - - -July/Europython ------------------- - -Time & location: 3rd-5th of July, 2005, CERN Switzerland -(sprints at 30th of June-2nd of July and 6th of July to 9th of July) - -Nature: presprint/postsprint? - -Primary organizer: Michael H, Holger) - -August/University of Limerick -------------------------------- - -Time & location: 21th to 27th of August, 2006 at University -of Limerick, Ireland - -Nature: open, dissemination (workshop with Calibre?) - -Primary organizer: Bea - - -October/? ----------------------------- - -November/? ----------------------------- - - -Other general possibilities: -Romania, Switzerland (Leysin,Bern), -Bristol, Uganda (Nkozi), Bruxelles +Author: Beatrice D?ring +20060103 + +Sprint planning PyPy +---------------------- + +January/Palma de Mallorca +-------------------------- + +Time & location: 23-29th January 2006, CS Dep. of the Palma de +Mallorca University/GNU Linux lab. + +Nature: open + +Primary organizer: Bea + +February/PyCon (USA) +--------------------- + +Time & location: 27 Feb - 1st March 2006 PyCon +(24-26 Feb, 2006 Dallas, Texas, + +Nature: postsprint + +Primary organizer: Holger and Michael + +March/Europe +------------------- + +Time & location: Louvain-La-Neuve 6th-10th March + +Nature: joint sprint with OZ community and Roel Wuyts/ULB attending. +DFKI and Logilab primarily + Samuele, Carl F. + +Primary organizer: Nicolas Chauvat + +April/Leysin +------------- + +Time & location: 3rd to 9th of April, 2006, Leysin, Switzerland + +Nature: closed/core + +Primary organizer: Armin + +April/Tokyo +------------- + +Time & location: 23rd to 29th of April, 2006, host: +Yutaka Niibe, National Institute of AIST, Japan (FSIJ) +Akihabara, Tokyo, venue for 32 people. + +Nature: open, dissemination + +Primary organizer: Bea + + +July/Europython +------------------ + +Time & location: 3rd-5th of July, 2005, CERN Switzerland +(sprints at 30th of June-2nd of July and 6th of July to 9th of July) + +Nature: presprint/postsprint? + +Primary organizer: Michael H, Holger) + +August/University of Limerick +------------------------------- + +Time & location: 21th to 27th of August, 2006 at University +of Limerick, Ireland + +Nature: open, dissemination (workshop with Calibre?) + +Primary organizer: Bea + + +October/? +---------------------------- + +November/? +---------------------------- + + +Other general possibilities: +Romania, Switzerland (Leysin,Bern), +Bristol, Uganda (Nkozi), Bruxelles Modified: pypy/extradoc/pypy.org/revision_march2006.txt ============================================================================== --- pypy/extradoc/pypy.org/revision_march2006.txt (original) +++ pypy/extradoc/pypy.org/revision_march2006.txt Sat Apr 29 21:50:50 2006 @@ -1,20 +1,20 @@ -pypy.org;revision March 2006 --------------------------------- - -In order to have an external website servicing information in connection -to pressreleases and more external newsletter the pypy org page needs updating. - -The following changes/adds needs to be done: - -- change navigation to links about: - Home, News, Consortium, Links, Documentation, Community/Coding - - Links being a page with links to sisterprojects, to commission pages etc - - Documentation being a page to access published reports, pressreleases, talks, papers, - diagrams, tutorials, pictures, film? - -- info regarding industrial impact (why PyPy, gains? what will it provide, which industries/languages - do we target - who will have use of PyPy???? This info could be on the main page (HOME) - - -Do we want to have a link to universities, conferences and companies we are in touch with/cooperating with? -Such as UIB, UCD, Iona, x, y, z... +pypy.org;revision March 2006 +-------------------------------- + +In order to have an external website servicing information in connection +to pressreleases and more external newsletter the pypy org page needs updating. + +The following changes/adds needs to be done: + +- change navigation to links about: + Home, News, Consortium, Links, Documentation, Community/Coding + - Links being a page with links to sisterprojects, to commission pages etc + - Documentation being a page to access published reports, pressreleases, talks, papers, + diagrams, tutorials, pictures, film? + +- info regarding industrial impact (why PyPy, gains? what will it provide, which industries/languages + do we target - who will have use of PyPy???? This info could be on the main page (HOME) + + +Do we want to have a link to universities, conferences and companies we are in touch with/cooperating with? +Such as UIB, UCD, Iona, x, y, z... Modified: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt Sat Apr 29 21:50:50 2006 @@ -1,156 +1,156 @@ -Palma de Mallorca PyPy Sprint: 23rd - 29th January 2006 -============================================================ - -The next PyPy sprint is scheduled to take place January 2006 in -Palma De Mallorca, Balearic Isles, Spain. We'll give newcomer-friendly -introductions and the focus will mainly be on current JIT work, garbage -collection, alternative threading models, logic programming and on -improving the interface with external functions. To learn more about the -new Python-in-Python implementation look here: - - http://codespeak.net/pypy - -Goals and topics of the sprint ------------------------------- - -In Gothenburg we have made some first forays into the interesting topics -of Just-in-Time compilation. In Mallorca we will continue that -and have the following ideas: - - - Further work/experimentation toward Just-In-Time Compiler generation, - which was initiated with the Abstract Interpreter started in Gothenburg. - - - Integrating our garbage collection toolkit with the backends and the - code generation. - - - Heading into the direction of adding logic programming to PyPy. - - - Optimization work: our threading implementation is still incredibly - slow, we need to work on that. Furthermore there are still quite - some slow places in the interpreter that could be improved. - - - getting the socket module to a more complete state (it is - already improved but still far from complete) - - - generally improving the way we interface with external functions. - - - whatever participants want to do with PyPy (please send - suggestions to the mailing list before to allow us to plan - and give feedback) - - -Location & Accomodation ------------------------- - -The sprint will be held at the Palma University (UIB - Universitat de -les Illes Balears), in their GNU/Linux lab -(http://mnm.uib.es/phpwiki/AulaLinux). We are hosted by the Computer -Science department and Ricardo Galli is our contact person there, -helping with arranging facilities. - -The University is located 7 km away from the central Palma. Busses to -the University departs from "Plaza de Espa?a" (which is a very central -location in Palma). Take bus 19 to the UIB campus. A ticket for one -urban trip costs 1 euro. You can also buy a card that is valid for 10 -trips and costs 7.51 euros. Information about bus timetables and routes -can be found on: - - http://www.a-palma.es - -A map over the UIB campus are can be found on: - - http://www.uib.es/imagenes/planoCampus.html - -The actual address is: 3r pis de l'Anselm Turmeda which can be found on -the UIB Campus map. - -At "Plaza de Espa?a" there is a hostel (Hostal Residencia Terminus) -which has been recommended to us. It's cheap (ca 50 euros/double room -with bathroom). Some more links to accomodations (flats, student homes -and hotels): - -http://www.lodging-in-spain.com/hotel/town/Islas_Baleares,Mallorca,Palma_de_Mallorca,1/ - -http://www.uib.es/fuguib/residencia/english/index.html - -http://www.homelidays.com/EN-Holidays-Rental/110_Search/SearchList.asp?DESTINATION=Palma%20de%20Mallorca&ADR_PAYS=ES&ADR_ -LOCALISATION=ES%20ISLASBALEARES%20MALLORCA - -If you want to find a given street, you can search here: - - http://www.callejeando.com/Pueblos/pueblo7_1.htm - -To get to Palma De Mallorca almost all low fare airlines and travel -agencies have cheap tickets to get there. Information about Mallorca and -Palma (maps, tourist information, local transports, recommended air -lines, ferries and much more) can be found on: - - http://www.palmademallorca.es/portalPalma/home.jsp - -Comments on the weather: In January it is cold and wet on Mallorca - -Average temperature: 8,4 degrees Celsius -Lowest temperature: 2 degrees Celsius -Highest temperature: 14,5 degrees Celsius -Average humidity rate: 77,6 % - -So more time for coding and less time for sunbathing and beaches ;-) - -Exact times ------------ - -The public PyPy sprint is held Monday 23rd - Sunday 29th January 2006. -Hours will be from 10:00 until people have had enough. It's a good idea -to arrive a day before the sprint starts and leave a day later. In the -middle of the sprint there usually is a break day and it's usually ok to -take half-days off if you feel like it. - -For this particular break day, Thursday, we are invited to the studio of -Gin?s Qui?onero, a local artist and painter. Gin?s have also been the -person helping us getting connections to UIB and providing much -appreciated help regarding accommodation and other logistical -information. - -For those of you interested - here is his website where there also are -paintings showing his studio: - - http://www.hermetex4.com/damnans/ - -Here is a description on how to get to Gin?s studio: - -http://www.maporama.com/affiliates/popup/share/iti.asp?ItineraryID=F32E624F --BFB1-4751-A5D3-5EA4DBDC9C9B&language=en&NbStep=0&SESSIONID={CD058830-6B8B-4 -142-887F-94C01253E33C} - -For those interested in playing collectable card games, this will also -be an opportunity to get aquainted with V:TES which will be demoed by -Gin?s and Beatrice D?ring. For more information on this -cardgame - see: http://www.white-wolf.com/vtes/index.php. (The Mallorca -sprint was organized through contacts within the V:TES community). - -Network, Food, currency ------------------------- - -Currency is Euro. - -Food is available in the UIB Campus area as well as cheap restaurants in -Palma. - -You normally need a wireless network card to access the network, but we -can provide a wireless/ethernet bridge. - -230V AC plugs are used in Mallorca. - -Registration etc.pp. --------------------- - -Please subscribe to the `PyPy sprint mailing list`_, introduce yourself -and post a note that you want to come. Feel free to ask any questions -there! There also is a separate `Mallorca people`_ page tracking who is -already thought to come. If you have commit rights on codespeak then -you can modify yourself a checkout of - - http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca/people.txt - -.. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint -.. _`Mallorca people`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/people.html +Palma de Mallorca PyPy Sprint: 23rd - 29th January 2006 +============================================================ + +The next PyPy sprint is scheduled to take place January 2006 in +Palma De Mallorca, Balearic Isles, Spain. We'll give newcomer-friendly +introductions and the focus will mainly be on current JIT work, garbage +collection, alternative threading models, logic programming and on +improving the interface with external functions. To learn more about the +new Python-in-Python implementation look here: + + http://codespeak.net/pypy + +Goals and topics of the sprint +------------------------------ + +In Gothenburg we have made some first forays into the interesting topics +of Just-in-Time compilation. In Mallorca we will continue that +and have the following ideas: + + - Further work/experimentation toward Just-In-Time Compiler generation, + which was initiated with the Abstract Interpreter started in Gothenburg. + + - Integrating our garbage collection toolkit with the backends and the + code generation. + + - Heading into the direction of adding logic programming to PyPy. + + - Optimization work: our threading implementation is still incredibly + slow, we need to work on that. Furthermore there are still quite + some slow places in the interpreter that could be improved. + + - getting the socket module to a more complete state (it is + already improved but still far from complete) + + - generally improving the way we interface with external functions. + + - whatever participants want to do with PyPy (please send + suggestions to the mailing list before to allow us to plan + and give feedback) + + +Location & Accomodation +------------------------ + +The sprint will be held at the Palma University (UIB - Universitat de +les Illes Balears), in their GNU/Linux lab +(http://mnm.uib.es/phpwiki/AulaLinux). We are hosted by the Computer +Science department and Ricardo Galli is our contact person there, +helping with arranging facilities. + +The University is located 7 km away from the central Palma. Busses to +the University departs from "Plaza de Espa?a" (which is a very central +location in Palma). Take bus 19 to the UIB campus. A ticket for one +urban trip costs 1 euro. You can also buy a card that is valid for 10 +trips and costs 7.51 euros. Information about bus timetables and routes +can be found on: + + http://www.a-palma.es + +A map over the UIB campus are can be found on: + + http://www.uib.es/imagenes/planoCampus.html + +The actual address is: 3r pis de l'Anselm Turmeda which can be found on +the UIB Campus map. + +At "Plaza de Espa?a" there is a hostel (Hostal Residencia Terminus) +which has been recommended to us. It's cheap (ca 50 euros/double room +with bathroom). Some more links to accomodations (flats, student homes +and hotels): + +http://www.lodging-in-spain.com/hotel/town/Islas_Baleares,Mallorca,Palma_de_Mallorca,1/ + +http://www.uib.es/fuguib/residencia/english/index.html + +http://www.homelidays.com/EN-Holidays-Rental/110_Search/SearchList.asp?DESTINATION=Palma%20de%20Mallorca&ADR_PAYS=ES&ADR_ +LOCALISATION=ES%20ISLASBALEARES%20MALLORCA + +If you want to find a given street, you can search here: + + http://www.callejeando.com/Pueblos/pueblo7_1.htm + +To get to Palma De Mallorca almost all low fare airlines and travel +agencies have cheap tickets to get there. Information about Mallorca and +Palma (maps, tourist information, local transports, recommended air +lines, ferries and much more) can be found on: + + http://www.palmademallorca.es/portalPalma/home.jsp + +Comments on the weather: In January it is cold and wet on Mallorca + +Average temperature: 8,4 degrees Celsius +Lowest temperature: 2 degrees Celsius +Highest temperature: 14,5 degrees Celsius +Average humidity rate: 77,6 % + +So more time for coding and less time for sunbathing and beaches ;-) + +Exact times +----------- + +The public PyPy sprint is held Monday 23rd - Sunday 29th January 2006. +Hours will be from 10:00 until people have had enough. It's a good idea +to arrive a day before the sprint starts and leave a day later. In the +middle of the sprint there usually is a break day and it's usually ok to +take half-days off if you feel like it. + +For this particular break day, Thursday, we are invited to the studio of +Gin?s Qui?onero, a local artist and painter. Gin?s have also been the +person helping us getting connections to UIB and providing much +appreciated help regarding accommodation and other logistical +information. + +For those of you interested - here is his website where there also are +paintings showing his studio: + + http://www.hermetex4.com/damnans/ + +Here is a description on how to get to Gin?s studio: + +http://www.maporama.com/affiliates/popup/share/iti.asp?ItineraryID=F32E624F +-BFB1-4751-A5D3-5EA4DBDC9C9B&language=en&NbStep=0&SESSIONID={CD058830-6B8B-4 +142-887F-94C01253E33C} + +For those interested in playing collectable card games, this will also +be an opportunity to get aquainted with V:TES which will be demoed by +Gin?s and Beatrice D?ring. For more information on this +cardgame - see: http://www.white-wolf.com/vtes/index.php. (The Mallorca +sprint was organized through contacts within the V:TES community). + +Network, Food, currency +------------------------ + +Currency is Euro. + +Food is available in the UIB Campus area as well as cheap restaurants in +Palma. + +You normally need a wireless network card to access the network, but we +can provide a wireless/ethernet bridge. + +230V AC plugs are used in Mallorca. + +Registration etc.pp. +-------------------- + +Please subscribe to the `PyPy sprint mailing list`_, introduce yourself +and post a note that you want to come. Feel free to ask any questions +there! There also is a separate `Mallorca people`_ page tracking who is +already thought to come. If you have commit rights on codespeak then +you can modify yourself a checkout of + + http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca/people.txt + +.. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint +.. _`Mallorca people`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/people.html Modified: pypy/extradoc/sprintinfo/paris/paris-report.txt ============================================================================== --- pypy/extradoc/sprintinfo/paris/paris-report.txt (original) +++ pypy/extradoc/sprintinfo/paris/paris-report.txt Sat Apr 29 21:50:50 2006 @@ -1,238 +1,238 @@ -Paris sprint report 2005-10-10-2005-10-16 -Authors: Michael Hudson, Carl Friedrich Bolz, Armin Rigo - -Participants: - -Ludovic Aubry -Adrien Di Mascio -Jacob Hallen -Laura Creighton -Beatrice Duering -Armin Rigo -Samuele Pedroni -Anders Chrigstroem -Holger Krekel -Lene Wagner -Michael Hudson -Carl Friedrich Bolz -Bert Freudenberg -Anders Lehmann -Boris Feigin -Amaury Forgeot d'Arc -Andrew Thompson -Christian Tismer -Valentino Volonghi -Aurelien Campeas -Stephan Busemann -Nicholas Chauvat - ------------------------------------------------------------------------------------------------------------------------------------------ - -The largest PyPy sprint yet (in terms of attendants) where done in the offices of Logilab in Paris -2005-10-16-2005-10-16. - -Possible task on the sprint was: - -RTyper tasks: -- fixed size lists -- poor-man type erasure for rlist and rdict - -- rtyping of classes/instances/methods for target languages with direct/prevalent OO/class support: - devise a low-level model variation for this case - -Annotator related tasks: - -- support the notion of separate/external functions (and classes/PBCs) - in preparation for separate compilations - -JIT related work: -- (DONE) support addresses in the backends -- an ll interpreter written in RPython -- Saving ll graphs plus a loader written in RPython - -- Start thinking/experimenting with JIT generation at translation time - -- (DONE the starting) start a genasm back-end - -Threading/concurrency: -- release the GIL around system calls -- understand and possibly fix where the overhead, when threads are enabled, comes from - -- (DONE) generating (single-threaded) stackless C code - -Implementation/translation: -- (somewhat DONE) stack overflow detection (needed to be able to run many compliancy tests) -- try more interp. level optimisations (dicts with string keys, more agressive use of fastcall*...) - -- compute correct max stack size in compiler (?) -- cleanup our multiple compiler situation: remove - testcompiler, fix the tests to work on CPython 2.3 too, - decide what to put in lib-python/modified-2.4.1/compiler -- - stablecompiler or astcompiler? -- and submit it back to - CPython. Clean up pyparser/pythonutil.py. - -- (socket module, PEP302) - -GC related tasks: -- look into implementing weakrefs -- Boehm: fix the x=range(10**7) issue -- (improve refcounting) -- (passes toward integrating other GCs, rpython/memory) - -Refactorings/cleanups: -- cbuild/translator.Translator (use SCons?, use/generalize TranslationDriver) -- PyPy option handling unification, passing py.py options to targetpypy* -- inline (transforms in general) - -- (DONE) genc: producing multiple .h/.c files tracking Python code origin - -Larger whole projects: - -- Javascript frontend -- support for developing C extensions for CPython with RPython code -- writing a sort of flow object space for the llinterpreter - to experiment with JIT work - ----------------------------------------------------------------------------------------------------------------------------------------- - -Activities during the sprint - day-to-day: - -Monday-Wednesday: -The morning (once everyone had found their way/fought with the -metro/...) began with a tutorial for the newcomers and two discussion -groups -- one on implementing stackless-like functionality and one -titled "towards a translatable llinterpreter". - -After lunch, everyone met to hear the results of the discussion groups -and decide what to do next. The stackless group's conclusion was -"it'll be easy!" :) The llinterp group concluded "it might be doable". -More details can be found in svn at - -http://codespeak.net/svn/pypy/extradoc/sprintinfo/paris-2005-stackless-discussion.txt -http://codespeak.net/svn/pypy/extradoc/sprintinfo/paris/tllinterpreter_planning.txt - -(consistency? We're researchers!) - -Christian (he didn't get a choice), Valentino, Anders (L), Adrien, -Armin and Amaury became the stackless working group and by Tuesday -lunchtime had progressed via an unlikely sounding six-person-pair -programming methodology involving a beamer to a fully stackless C -translation, albeit with limited functionality visible even to -RPython. - -Samuele, Bert, Arre, Aurelien, and Boris became what was ultimately -known as the 'ootype group' working on a variation of the rtyper more -suited to translation to a language with a richer type system than C -(classes, lists, some vague notion of type safety, etc) such as Java, -Smalltalk, ... - -Michael and Andrew worked on a backend that emits machine code -- in -particular ppc32 machine code -- directly. By the end of Monday a toy -function doing some simple integer calculations had been translated -but on Tuesday restructuring towards re-use (and comprehensibility) -became the main goal. Oh, and not assuming an infinite supply of -registers... - -Carl and Holger started implementing Addresses in the C backend to -prepare for the coming llinterpreter work, finishing on Monday -evening. Carl then worked on data structures needed for a -translatable llinterpreter. - -Thursday: -Consortium meeting + breakday - -Friday-Sunday: -The stackless group reported good progress, having compiled a working -pypy-c with stackless support and implemented stack overflow detection -for non-stackless builds. Unfortunately for the stackless builds, -several CPOython tests expect infinite recursion to result in an error --- and do so fairly quickly, i.e. in less time than it takes to fill -the entire heap with stack frames. This groups work is basically -done for this sprint, although Anders and Christian are going to work -on compliancy testing with the new pypy-c. While waiting for builds -targeting compliancy tests, they are also going to investigate -reorganizing code generation to improve locality. - -The ootype group is also progressing well. The RTyper has mostly been -refactored to be independent of the targeted type system and work is -continuing on implementing the new OOType type system alongside the -existing LLType target. This group will be continuing, although with -somehwat different membership -- Michael joining and half of each of -Samuele and Arre leaving. - -Michael and Andrews work on the PPC backend has progressed to the -point where essentially any function that only manipulates integers -can be translated (with an exceedingly stupid register allocator). -Further work depends to a large extent on the llinterpreter work (see -below) so this work will wait until after the sprint. Andrew is -moving to work on implementing a Numeric-a-like for PyPy, together -with Ludovic. - -The LLInterpreter grouplette (Carl and 0.5 Armins and a little Holger) -did not produce much code since there are many decisions to be made -and the implications of these decisions are not understood. A -discussion group of Carl, Armin, Samuele, Holger, Christian and Arre -will try to shine lights into these shadows, and report after lunch. - -Valentino and Amaury are going to implement the socket module. This -is a step towards allowing Valentino to run Twisted on PyPy and thus -make him very happy. - -This sprint is working in quite a different way to previous sprints -- -there is lots of discussion which isnt new, but the farming off of -discussion to groups of 5-6 people who present a report to the larger -group is a novelty and seems to be working well (15+ people is too -many for a focussed technical discussion). Another difference is a -less strict emphasis on "pair" programming -- or, if you like, we are -still pair programming but we have redefined "pair" to mean a group of -two to six programmers :) - -On Friday morning another discussion group was founded and discussed - -again - the state and future of the l3interpreter (l3 = lll = low low -level), that is the translatable llinterpreter. The results were -presented after lunch, together with some ideas about the JIT. -Afterwards Carl gave a short talk on the results of his summer of code -project on writing garbage collectors in RPython. - -Boris, Michael, Bert with help from Samuele spent the whole rest of the -sprint working on the many open issues related to ootyping. Simple -programs can now be ootyped, including inheritance, methods, instance -attributes and right at the end some support for prebuilt instances. In -addition they extended the llinterpreter to understand the ootype -operations as well (we were worried that our names were starting to make -sense). - -Armin spent the last days fixing different cases that crashed pypy-c -which he found by running the CPython compliancy tests. In addition he -helped various people to find their way around the codebase. - -Adrien and Arre worked on fixing compiler and parser issues that led to -wrong line numbers and different issues that popped up. - -Ludovic and Adrien experimented with rewriting parts of the Numeric -package in RPython. - -Valentino and Amaury continued on implementing the socket module which -turned out (as expected) to be a platform dependent nightmare. They have -a kind of complete socket module now, but some functions cannot yet be -translated. - -Christian worked on an experiment to reorder functions in the created C -code to improve code locality. - -Finally, after a week of srapped attemps, much headscratching and heated -discussion there was some code written for the l3interpreter. On -Saturday afternoon Holger and Carl wrote the basic model and managed to -interpret interesting functions like x + 4. On Sunday Samuele and Carl -continued and started on a graph converter that takes ll graphs and -transforms into the form the l3interpreter expects. - -On Saturday afternoon there was a planning meeting where the actions of -the following weeks were discussed. The EU-report writing was -distributed to the different consortium members. -Furthermore we discussed the various conference and sprints planned for autumn 2005 -and spring 2006. - -All in all it was a very productive sprint but of course we all have to -recover for two weeks now. - +Paris sprint report 2005-10-10-2005-10-16 +Authors: Michael Hudson, Carl Friedrich Bolz, Armin Rigo + +Participants: + +Ludovic Aubry +Adrien Di Mascio +Jacob Hallen +Laura Creighton +Beatrice Duering +Armin Rigo +Samuele Pedroni +Anders Chrigstroem +Holger Krekel +Lene Wagner +Michael Hudson +Carl Friedrich Bolz +Bert Freudenberg +Anders Lehmann +Boris Feigin +Amaury Forgeot d'Arc +Andrew Thompson +Christian Tismer +Valentino Volonghi +Aurelien Campeas +Stephan Busemann +Nicholas Chauvat + +----------------------------------------------------------------------------------------------------------------------------------------- + +The largest PyPy sprint yet (in terms of attendants) where done in the offices of Logilab in Paris +2005-10-16-2005-10-16. + +Possible task on the sprint was: + +RTyper tasks: +- fixed size lists +- poor-man type erasure for rlist and rdict + +- rtyping of classes/instances/methods for target languages with direct/prevalent OO/class support: + devise a low-level model variation for this case + +Annotator related tasks: + +- support the notion of separate/external functions (and classes/PBCs) + in preparation for separate compilations + +JIT related work: +- (DONE) support addresses in the backends +- an ll interpreter written in RPython +- Saving ll graphs plus a loader written in RPython + +- Start thinking/experimenting with JIT generation at translation time + +- (DONE the starting) start a genasm back-end + +Threading/concurrency: +- release the GIL around system calls +- understand and possibly fix where the overhead, when threads are enabled, comes from + +- (DONE) generating (single-threaded) stackless C code + +Implementation/translation: +- (somewhat DONE) stack overflow detection (needed to be able to run many compliancy tests) +- try more interp. level optimisations (dicts with string keys, more agressive use of fastcall*...) + +- compute correct max stack size in compiler (?) +- cleanup our multiple compiler situation: remove + testcompiler, fix the tests to work on CPython 2.3 too, + decide what to put in lib-python/modified-2.4.1/compiler -- + stablecompiler or astcompiler? -- and submit it back to + CPython. Clean up pyparser/pythonutil.py. + +- (socket module, PEP302) + +GC related tasks: +- look into implementing weakrefs +- Boehm: fix the x=range(10**7) issue +- (improve refcounting) +- (passes toward integrating other GCs, rpython/memory) + +Refactorings/cleanups: +- cbuild/translator.Translator (use SCons?, use/generalize TranslationDriver) +- PyPy option handling unification, passing py.py options to targetpypy* +- inline (transforms in general) + +- (DONE) genc: producing multiple .h/.c files tracking Python code origin + +Larger whole projects: + +- Javascript frontend +- support for developing C extensions for CPython with RPython code +- writing a sort of flow object space for the llinterpreter + to experiment with JIT work + +---------------------------------------------------------------------------------------------------------------------------------------- + +Activities during the sprint - day-to-day: + +Monday-Wednesday: +The morning (once everyone had found their way/fought with the +metro/...) began with a tutorial for the newcomers and two discussion +groups -- one on implementing stackless-like functionality and one +titled "towards a translatable llinterpreter". + +After lunch, everyone met to hear the results of the discussion groups +and decide what to do next. The stackless group's conclusion was +"it'll be easy!" :) The llinterp group concluded "it might be doable". +More details can be found in svn at + +http://codespeak.net/svn/pypy/extradoc/sprintinfo/paris-2005-stackless-discussion.txt +http://codespeak.net/svn/pypy/extradoc/sprintinfo/paris/tllinterpreter_planning.txt + +(consistency? We're researchers!) + +Christian (he didn't get a choice), Valentino, Anders (L), Adrien, +Armin and Amaury became the stackless working group and by Tuesday +lunchtime had progressed via an unlikely sounding six-person-pair +programming methodology involving a beamer to a fully stackless C +translation, albeit with limited functionality visible even to +RPython. + +Samuele, Bert, Arre, Aurelien, and Boris became what was ultimately +known as the 'ootype group' working on a variation of the rtyper more +suited to translation to a language with a richer type system than C +(classes, lists, some vague notion of type safety, etc) such as Java, +Smalltalk, ... + +Michael and Andrew worked on a backend that emits machine code -- in +particular ppc32 machine code -- directly. By the end of Monday a toy +function doing some simple integer calculations had been translated +but on Tuesday restructuring towards re-use (and comprehensibility) +became the main goal. Oh, and not assuming an infinite supply of +registers... + +Carl and Holger started implementing Addresses in the C backend to +prepare for the coming llinterpreter work, finishing on Monday +evening. Carl then worked on data structures needed for a +translatable llinterpreter. + +Thursday: +Consortium meeting + breakday + +Friday-Sunday: +The stackless group reported good progress, having compiled a working +pypy-c with stackless support and implemented stack overflow detection +for non-stackless builds. Unfortunately for the stackless builds, +several CPOython tests expect infinite recursion to result in an error +-- and do so fairly quickly, i.e. in less time than it takes to fill +the entire heap with stack frames. This groups work is basically +done for this sprint, although Anders and Christian are going to work +on compliancy testing with the new pypy-c. While waiting for builds +targeting compliancy tests, they are also going to investigate +reorganizing code generation to improve locality. + +The ootype group is also progressing well. The RTyper has mostly been +refactored to be independent of the targeted type system and work is +continuing on implementing the new OOType type system alongside the +existing LLType target. This group will be continuing, although with +somehwat different membership -- Michael joining and half of each of +Samuele and Arre leaving. + +Michael and Andrews work on the PPC backend has progressed to the +point where essentially any function that only manipulates integers +can be translated (with an exceedingly stupid register allocator). +Further work depends to a large extent on the llinterpreter work (see +below) so this work will wait until after the sprint. Andrew is +moving to work on implementing a Numeric-a-like for PyPy, together +with Ludovic. + +The LLInterpreter grouplette (Carl and 0.5 Armins and a little Holger) +did not produce much code since there are many decisions to be made +and the implications of these decisions are not understood. A +discussion group of Carl, Armin, Samuele, Holger, Christian and Arre +will try to shine lights into these shadows, and report after lunch. + +Valentino and Amaury are going to implement the socket module. This +is a step towards allowing Valentino to run Twisted on PyPy and thus +make him very happy. + +This sprint is working in quite a different way to previous sprints -- +there is lots of discussion which isnt new, but the farming off of +discussion to groups of 5-6 people who present a report to the larger +group is a novelty and seems to be working well (15+ people is too +many for a focussed technical discussion). Another difference is a +less strict emphasis on "pair" programming -- or, if you like, we are +still pair programming but we have redefined "pair" to mean a group of +two to six programmers :) + +On Friday morning another discussion group was founded and discussed - +again - the state and future of the l3interpreter (l3 = lll = low low +level), that is the translatable llinterpreter. The results were +presented after lunch, together with some ideas about the JIT. +Afterwards Carl gave a short talk on the results of his summer of code +project on writing garbage collectors in RPython. + +Boris, Michael, Bert with help from Samuele spent the whole rest of the +sprint working on the many open issues related to ootyping. Simple +programs can now be ootyped, including inheritance, methods, instance +attributes and right at the end some support for prebuilt instances. In +addition they extended the llinterpreter to understand the ootype +operations as well (we were worried that our names were starting to make +sense). + +Armin spent the last days fixing different cases that crashed pypy-c +which he found by running the CPython compliancy tests. In addition he +helped various people to find their way around the codebase. + +Adrien and Arre worked on fixing compiler and parser issues that led to +wrong line numbers and different issues that popped up. + +Ludovic and Adrien experimented with rewriting parts of the Numeric +package in RPython. + +Valentino and Amaury continued on implementing the socket module which +turned out (as expected) to be a platform dependent nightmare. They have +a kind of complete socket module now, but some functions cannot yet be +translated. + +Christian worked on an experiment to reorder functions in the created C +code to improve code locality. + +Finally, after a week of srapped attemps, much headscratching and heated +discussion there was some code written for the l3interpreter. On +Saturday afternoon Holger and Carl wrote the basic model and managed to +interpret interesting functions like x + 4. On Sunday Samuele and Carl +continued and started on a graph converter that takes ll graphs and +transforms into the form the l3interpreter expects. + +On Saturday afternoon there was a planning meeting where the actions of +the following weeks were discussed. The EU-report writing was +distributed to the different consortium members. +Furthermore we discussed the various conference and sprints planned for autumn 2005 +and spring 2006. + +All in all it was a very productive sprint but of course we all have to +recover for two weeks now. + Modified: pypy/extradoc/talk/pycon2006/method_talk.txt ============================================================================== --- pypy/extradoc/talk/pycon2006/method_talk.txt (original) +++ pypy/extradoc/talk/pycon2006/method_talk.txt Sat Apr 29 21:50:50 2006 @@ -1,184 +1,184 @@ -============================================================ -Agile Open-Source Methods, Businesses and EU-funding -============================================================ - -:Authors: Bea During (Change Maker), Holger Krekel (merlinux) -:Date: 26th February 2006, PyCon, Dallas - - -What is PyPy? -================================================== - -- Next generation Python implementation -- Grass-root open source effort -- A partially funded research project -- Technical aims: flexibility and speed -- Method aims: evolve sprint-driven development - -Bootstrapping PyPy (2003) -================================================== - -- Initial sprint with Armin Rigo, Christian Tismer, - Holger Krekel, Michael Hudson and then Samuele Pedroni ... -- Driven by psyco/stackless experiences -- "Minimal Python" inital project name -- Focus on correctness of concepts, then speed - -PyPy test-driven development -================================================== - -- Identify problems/evolution by tests first -- Our own testing and development tools (py.test) -- Rule: first get the semantics and concepts right! - optimize later! -- Today around 3000 tests (plus CPython regression tests) - -PyPy is (not) a funded EU project -================================================== - -- PyPy wasn't a funded project -- PyPy isn't a funded project -- PyPy is a funded project -- Conclusion: it's more complicated - -EU project consortium -============================================================ - -- AB Strakt (Sweden) -- merlinux (Germany) -- Change Maker (Sweden) -- Heinrich Heine University (Germany) -- DFKI (Germany) -- Tismerysoft (Germany) -- Logilab (France) -- Impara (Germany) - -EU contractual work structure -=================================================== - -- "Description of Work" for two years -- 14 workpackages and 58 deliverables, 3 phases -- Sprints every 6th week (coordinating development and management work) -- EU project aspects enforced mid-term/long-term focus -- not a usual open-source approach -- **good and bad** - -Balance of interests -================================================== - -- PyPy was first and still is a network of people -- ...but EU only funds organisations -- Developers drive the technical project -- Companies co-finance 50% of all costs -- EU wants challenging research goals and tracking of goals -- **it is all about finding - good models for co-operation** - -Current developer work structure -================================================== - -- Weekly 30 minute synchronisation meetings -- Open collaborative open-source work style -- Representation through Trusted "Technical Board" - within the EU project -- Research/architecture informally guided by accepted experts - -.. Bea - - -Agile practises: the people factor -============================================================ - -- "Agile processes are designed to capitalize on each - individual and each team's unique strenghts" (Cockburn, Highsmith, 2001) -- OSS nature of teams: self-organized, intensely - collaborative - fit the agile approach -- OSS teams are an unique implementation of agile practices - why? - - -Origins of sprinting -============================================================ - -- Scrum (Agile community): 1 month long iteration of - development work, increments (also supporting activities: - planning, documentation, tracking work, evaluation) - -- Zope Foundation (Python Community): "two-day or three-day - focused development session, in which developers pair off - together in a room and focus on building a particular - subsystem". - -Sprinting the PyPy way -============================================================ - -- First-contact sprints e.g. around conferences -- Core sprints for reaching milestones, releases -- 7 days with 1 break day, every 6th week -- Typical activities: daily planning/status meetings, - closure meetings, sprint reports, pair-group programming, - tutorials, EU consortium issues -- Rotating moderation/organisation - -Sprints facilitate participation -============================================================ - -.. image:: subscribers.png - :scale: 60 - -.. - -Managing diversities -============================================================ - -- Developer driven process and formal project organization - - management team, technical board and partners - - sprint organising - - planning and focusing on technical tasks -- Constant risk of added workload of management - work on core developers - -More managing diversities -============================================================ - -- Agile strategies and Formal EU requirements - - written high level requirements - - change control structures complicated -- Constant risk of missing opportunities and not - creating/reacting to change fast enough - -Even more diversities ... -============================================================ - -- OSS community and hierarchies for "conceptual integrity" - - pypy-dev/core developers in technical board - - industrial usage vs research oriented work -- Risk for unbalancing the community - -Hitchikers guide ... -============================================================ - -.. image:: manmoon.png - -Learnings -============================================================ - -- Share a challenging vision! -- Respect and utilize strengths of different cultures and - people involved -- Design minimalistic project structures -- Learn as a group and create changes, not just react to change - -Outlook on whole project level -============================== - -- EU project to finish November 2006 -- Improve interactions with community & contribution -- Exploring Commercial opportunities ... hiring opportunities ... -- Taking care about post-EU development (2007++) -- Sprints: Lovain La Neuve, Tokyo, EuroPython, Ireland -- http://codespeak.net/pypy and http://pypy.org - - -.. |bullet| unicode:: U+02022 -.. footer:: Bea During, Holger Krekel |bullet| PyCon |bullet| 26th February 2006 - +============================================================ +Agile Open-Source Methods, Businesses and EU-funding +============================================================ + +:Authors: Bea During (Change Maker), Holger Krekel (merlinux) +:Date: 26th February 2006, PyCon, Dallas + + +What is PyPy? +================================================== + +- Next generation Python implementation +- Grass-root open source effort +- A partially funded research project +- Technical aims: flexibility and speed +- Method aims: evolve sprint-driven development + +Bootstrapping PyPy (2003) +================================================== + +- Initial sprint with Armin Rigo, Christian Tismer, + Holger Krekel, Michael Hudson and then Samuele Pedroni ... +- Driven by psyco/stackless experiences +- "Minimal Python" inital project name +- Focus on correctness of concepts, then speed + +PyPy test-driven development +================================================== + +- Identify problems/evolution by tests first +- Our own testing and development tools (py.test) +- Rule: first get the semantics and concepts right! + optimize later! +- Today around 3000 tests (plus CPython regression tests) + +PyPy is (not) a funded EU project +================================================== + +- PyPy wasn't a funded project +- PyPy isn't a funded project +- PyPy is a funded project +- Conclusion: it's more complicated + +EU project consortium +============================================================ + +- AB Strakt (Sweden) +- merlinux (Germany) +- Change Maker (Sweden) +- Heinrich Heine University (Germany) +- DFKI (Germany) +- Tismerysoft (Germany) +- Logilab (France) +- Impara (Germany) + +EU contractual work structure +=================================================== + +- "Description of Work" for two years +- 14 workpackages and 58 deliverables, 3 phases +- Sprints every 6th week (coordinating development and management work) +- EU project aspects enforced mid-term/long-term focus +- not a usual open-source approach +- **good and bad** + +Balance of interests +================================================== + +- PyPy was first and still is a network of people +- ...but EU only funds organisations +- Developers drive the technical project +- Companies co-finance 50% of all costs +- EU wants challenging research goals and tracking of goals +- **it is all about finding + good models for co-operation** + +Current developer work structure +================================================== + +- Weekly 30 minute synchronisation meetings +- Open collaborative open-source work style +- Representation through Trusted "Technical Board" + within the EU project +- Research/architecture informally guided by accepted experts + +.. Bea + + +Agile practises: the people factor +============================================================ + +- "Agile processes are designed to capitalize on each + individual and each team's unique strenghts" (Cockburn, Highsmith, 2001) +- OSS nature of teams: self-organized, intensely + collaborative - fit the agile approach +- OSS teams are an unique implementation of agile practices - why? + + +Origins of sprinting +============================================================ + +- Scrum (Agile community): 1 month long iteration of + development work, increments (also supporting activities: + planning, documentation, tracking work, evaluation) + +- Zope Foundation (Python Community): "two-day or three-day + focused development session, in which developers pair off + together in a room and focus on building a particular + subsystem". + +Sprinting the PyPy way +============================================================ + +- First-contact sprints e.g. around conferences +- Core sprints for reaching milestones, releases +- 7 days with 1 break day, every 6th week +- Typical activities: daily planning/status meetings, + closure meetings, sprint reports, pair-group programming, + tutorials, EU consortium issues +- Rotating moderation/organisation + +Sprints facilitate participation +============================================================ + +.. image:: subscribers.png + :scale: 60 + +.. + +Managing diversities +============================================================ + +- Developer driven process and formal project organization + - management team, technical board and partners + - sprint organising + - planning and focusing on technical tasks +- Constant risk of added workload of management + work on core developers + +More managing diversities +============================================================ + +- Agile strategies and Formal EU requirements + - written high level requirements + - change control structures complicated +- Constant risk of missing opportunities and not + creating/reacting to change fast enough + +Even more diversities ... +============================================================ + +- OSS community and hierarchies for "conceptual integrity" + - pypy-dev/core developers in technical board + - industrial usage vs research oriented work +- Risk for unbalancing the community + +Hitchikers guide ... +============================================================ + +.. image:: manmoon.png + +Learnings +============================================================ + +- Share a challenging vision! +- Respect and utilize strengths of different cultures and + people involved +- Design minimalistic project structures +- Learn as a group and create changes, not just react to change + +Outlook on whole project level +============================== + +- EU project to finish November 2006 +- Improve interactions with community & contribution +- Exploring Commercial opportunities ... hiring opportunities ... +- Taking care about post-EU development (2007++) +- Sprints: Lovain La Neuve, Tokyo, EuroPython, Ireland +- http://codespeak.net/pypy and http://pypy.org + + +.. |bullet| unicode:: U+02022 +.. footer:: Bea During, Holger Krekel |bullet| PyCon |bullet| 26th February 2006 + Modified: pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt ============================================================================== --- pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt (original) +++ pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt Sat Apr 29 21:50:50 2006 @@ -1,89 +1,89 @@ -SEMINAR -Best Practice in the Use and Development of Free and Open Source Software -3. Case study: - -Part 2/Holger: - -Slides: - - -1. personal background - - - worked in gaming companies, banks and car companies for - several years - - studied computer science - - left well paid job and went into open-source scenes (2001) - - various project involvements, started PyPy 2003 by - inviting people to the first "sprint" - -2. What makes Open Source communities like Python work: the people factor - - - collaborative - driven by interest - - communication - quite transparent to everyone involved - - email / IRC / version-control - - organization - rather informal - -3. technical production factors - -- automated test driven development -- specific expertise/special interest -- version control (Subversion) -- releases - -4. Typical aspects of the Python community? - - - lively community - - lots of different python implementation projects - - good contactsbetween the projects - - maybe less fragmented than other OSS communities? - -5. PyPy: the vision - -- founders came from the Python community -- "sprints" were the inital factor -- what is PyPy/Python - one of the five most used programming - languages today -- grass root approach - - -6. OSS and EU funding: PyPy as a case study - -- driven by partially EU funded and non-EU funded parties -- focus on avoiding friction and turning PyPy into a long - term project -- IBM or Sun have done similarly challenging projects - in more time and with more funding -- yet not found completely satisfying "funding" - interactions with communities. - -7. PyPy technical status - -- three public releases in 2005, well received by the - community -- core deliverables fullfilled -- contributors add different directions - -8. PyPy: It's all about communication ... -- pypy-sync meetings, 30 minutes IRC -- day-to-day IRC discussions -- "this week in PyPy" -- mailing lists: pypy-svn/eu-tracking tracks code and document changes -- around 20000 visitors per month on website -- lots of blogs and pypy-dev (developer/researcher list) -- 300-500 people across the world following the project - -9. all good and well but the main thing are: sprints -- one-week intense work-meetings with one break day -- EU and non-EU researchers/developers get together -- daily planning sessions -- pair programming -- evolving and adapting to more attendants -- organisational/management tasks happen also on sprints - -10. next -- tackling research and technical goals (challenging!) -- mid-term EU review planned for 20th january -- looking into adjusting some work planning -- increased dissemination, attending conferences (movie features?) -- start talking to and interact with commercial stakeholders - +SEMINAR +Best Practice in the Use and Development of Free and Open Source Software +3. Case study: + +Part 2/Holger: + +Slides: + + +1. personal background + + - worked in gaming companies, banks and car companies for + several years + - studied computer science + - left well paid job and went into open-source scenes (2001) + - various project involvements, started PyPy 2003 by + inviting people to the first "sprint" + +2. What makes Open Source communities like Python work: the people factor + + - collaborative - driven by interest + - communication - quite transparent to everyone involved + - email / IRC / version-control + - organization - rather informal + +3. technical production factors + +- automated test driven development +- specific expertise/special interest +- version control (Subversion) +- releases + +4. Typical aspects of the Python community? + + - lively community + - lots of different python implementation projects + - good contactsbetween the projects + - maybe less fragmented than other OSS communities? + +5. PyPy: the vision + +- founders came from the Python community +- "sprints" were the inital factor +- what is PyPy/Python - one of the five most used programming + languages today +- grass root approach + + +6. OSS and EU funding: PyPy as a case study + +- driven by partially EU funded and non-EU funded parties +- focus on avoiding friction and turning PyPy into a long + term project +- IBM or Sun have done similarly challenging projects + in more time and with more funding +- yet not found completely satisfying "funding" + interactions with communities. + +7. PyPy technical status + +- three public releases in 2005, well received by the + community +- core deliverables fullfilled +- contributors add different directions + +8. PyPy: It's all about communication ... +- pypy-sync meetings, 30 minutes IRC +- day-to-day IRC discussions +- "this week in PyPy" +- mailing lists: pypy-svn/eu-tracking tracks code and document changes +- around 20000 visitors per month on website +- lots of blogs and pypy-dev (developer/researcher list) +- 300-500 people across the world following the project + +9. all good and well but the main thing are: sprints +- one-week intense work-meetings with one break day +- EU and non-EU researchers/developers get together +- daily planning sessions +- pair programming +- evolving and adapting to more attendants +- organisational/management tasks happen also on sprints + +10. next +- tackling research and technical goals (challenging!) +- mid-term EU review planned for 20th january +- looking into adjusting some work planning +- increased dissemination, attending conferences (movie features?) +- start talking to and interact with commercial stakeholders + From arigo at codespeak.net Sat Apr 29 22:45:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 22:45:46 +0200 (CEST) Subject: [pypy-svn] r26582 - in pypy/dist/pypy: doc doc/discussion translator/cl/test Message-ID: <20060429204546.BE09710086@code0.codespeak.net> Author: arigo Date: Sat Apr 29 22:45:45 2006 New Revision: 26582 Modified: pypy/dist/pypy/doc/discussion/logic-plan.txt (props changed) pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt (props changed) pypy/dist/pypy/doc/independent-project-ideas.txt (props changed) pypy/dist/pypy/translator/cl/test/test_dict.py (props changed) pypy/dist/pypy/translator/cl/test/test_exception.py (props changed) pypy/dist/pypy/translator/cl/test/test_list.py (props changed) Log: fixeol From arigo at codespeak.net Sat Apr 29 22:46:45 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 29 Apr 2006 22:46:45 +0200 (CEST) Subject: [pypy-svn] r26583 - pypy/dist/pypy/translator/c Message-ID: <20060429204645.811FB10094@code0.codespeak.net> Author: arigo Date: Sat Apr 29 22:46:44 2006 New Revision: 26583 Modified: pypy/dist/pypy/translator/c/pyobj.py Log: Typo. Modified: pypy/dist/pypy/translator/c/pyobj.py ============================================================================== --- pypy/dist/pypy/translator/c/pyobj.py (original) +++ pypy/dist/pypy/translator/c/pyobj.py Sat Apr 29 22:46:44 2006 @@ -624,3 +624,4 @@ name = self.uniquename('gprop') expr = 'property(%s, %s, %s, %s)' % (tuple(stuff)) self.initcode_python(name, expr) + return name From cfbolz at codespeak.net Sat Apr 29 23:31:25 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 29 Apr 2006 23:31:25 +0200 (CEST) Subject: [pypy-svn] r26585 - in pypy/dist/pypy: . translator translator/c translator/goal Message-ID: <20060429213125.C5EB010091@code0.codespeak.net> Author: cfbolz Date: Sat Apr 29 23:31:23 2006 New Revision: 26585 Modified: pypy/dist/pypy/conftest.py pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/gc.py pypy/dist/pypy/translator/c/node.py pypy/dist/pypy/translator/driver.py pypy/dist/pypy/translator/goal/translate.py Log: add a new gc policy which tries to give boehm some information which fields of a structure are pointers. Annoyingly this makes boehm slower :-(. On the other hand, something like this is needed to make _weakref work on boehm. Modified: pypy/dist/pypy/conftest.py ============================================================================== --- pypy/dist/pypy/conftest.py (original) +++ pypy/dist/pypy/conftest.py Sat Apr 29 23:31:23 2006 @@ -43,7 +43,7 @@ help="view translation tests' flow graphs with Pygame"), Option('--gc', action="store", default=None, type="choice", dest="gcpolicy", - choices=['ref', 'boehm', 'none', 'framework'], + choices=['ref', 'boehm', 'none', 'framework', 'exact_boehm'], help="GcPolicy class to use for genc tests"), ) Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Sat Apr 29 23:31:23 2006 @@ -44,6 +44,8 @@ if polname is not None: if polname == 'boehm': gcpolicy = gc.BoehmGcPolicy + elif polname == 'exact_boehm': + gcpolicy = gc.MoreExactBoehmGcPolicy elif polname == 'ref': gcpolicy = gc.RefcountingGcPolicy elif polname == 'none': Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Sat Apr 29 23:31:23 2006 @@ -9,7 +9,7 @@ PyObjPtr = Ptr(PyObject) -class BasicGcPolicy: +class BasicGcPolicy(object): def __init__(self, db, thread_enabled=False): self.db = db @@ -33,6 +33,9 @@ def array_gcheader_initdata(self, defnode): return self.common_gcheader_initdata(defnode) + def struct_after_definition(self, defnode): + return [] + def gc_libraries(self): return [] @@ -155,6 +158,9 @@ class BoehmInfo: finalizer = None + # for MoreExactBoehmGcPolicy + malloc_exact = False + class BoehmGcPolicy(BasicGcPolicy): transformerclass = gctransform.BoehmGCTransformer @@ -250,6 +256,78 @@ def implementation(self): yield 'char %s /* uninitialized */;' % self.name + +class MoreExactBoehmGcPolicy(BoehmGcPolicy): + """ policy to experiment with giving some layout information to boehm. Use + new class to prevent breakage. """ + + def __init__(self, db, thread_enabled=False): + super(MoreExactBoehmGcPolicy, self).__init__(db, thread_enabled) + self.exactly_typed_structs = {} + + def get_descr_name(self, defnode): + # XXX somewhat illegal way of introducing a name + return '%s__gc_descr__' % (defnode.name, ) + + def pre_pre_gc_code(self): + for line in super(MoreExactBoehmGcPolicy, self).pre_pre_gc_code(): + yield line + yield "#include " + + def struct_setup(self, structdefnode, rtti): + self.setup_gcinfo(structdefnode) + T = structdefnode.STRUCT + if T._is_varsize(): + malloc_exact = T._flds[T._arrayfld]._is_atomic() + else: + malloc_exact = True + if malloc_exact: + if structdefnode.gcinfo is None: + structdefnode.gcinfo = BoehmInfo() + structdefnode.gcinfo.malloc_exact = True + self.exactly_typed_structs[structdefnode.STRUCT] = structdefnode + + def struct_after_definition(self, defnode): + if defnode.gcinfo and defnode.gcinfo.malloc_exact: + yield 'GC_descr %s;' % (self.get_descr_name(defnode), ) + + def gc_startup_code(self): + for line in super(MoreExactBoehmGcPolicy, self).gc_startup_code(): + yield line + for TYPE, defnode in self.exactly_typed_structs.iteritems(): + T = defnode.gettype().replace("@", "") + yield "{" + yield "GC_word T_bitmap[GC_BITMAP_SIZE(%s)] = {0};" % (T, ) + for field in TYPE._flds: + if getattr(TYPE, field) == lltype.Void: + continue + yield "GC_set_bit(T_bitmap, GC_WORD_OFFSET(%s, %s));" % ( + T, defnode.c_struct_field_name(field)) + yield "%s = GC_make_descriptor(T_bitmap, GC_WORD_LEN(%s));" % ( + self.get_descr_name(defnode), T) + yield "}" + + def zero_malloc(self, TYPE, esize, eresult): + defnode = self.db.gettypedefnode(TYPE) + gcinfo = defnode.gcinfo + if gcinfo: + if not gcinfo.malloc_exact: + assert TYPE._gcstatus() # _is_atomic() depends on this! + is_atomic = TYPE._is_atomic() + is_varsize = TYPE._is_varsize() + result = 'OP_BOEHM_ZERO_MALLOC(%s, %s, %d, %d);' % ( + esize, eresult, is_atomic, is_varsize) + else: + result = '%s = GC_MALLOC_EXPLICITLY_TYPED(%s, %s);' % ( + eresult, esize, self.get_descr_name(defnode)) + if gcinfo.finalizer: + result += ('\nGC_REGISTER_FINALIZER(%s, (GC_finalization_proc)%s, NULL, NULL, NULL);' + % (eresult, gcinfo.finalizer)) + else: + return super(MoreExactBoehmGcPolicy, self).zero_malloc( + TYPE, esize, eresult) + return result + # to get an idea how it looks like with no refcount/gc at all class NoneGcPolicy(BoehmGcPolicy): Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Sat Apr 29 23:31:23 2006 @@ -117,7 +117,6 @@ return yield 'struct %s {' % self.name is_empty = True - for name, typename in self.fields: line = '%s;' % cdecl(typename, name) if typename == PrimitiveType[Void]: @@ -128,6 +127,8 @@ if is_empty: yield '\t' + 'int _dummy; /* this struct is empty */' yield '};' + for line in self.db.gcpolicy.struct_after_definition(self): + yield line def visitor_lines(self, prefix, on_field): STRUCT = self.STRUCT Modified: pypy/dist/pypy/translator/driver.py ============================================================================== --- pypy/dist/pypy/translator/driver.py (original) +++ pypy/dist/pypy/translator/driver.py Sat Apr 29 23:31:23 2006 @@ -227,6 +227,9 @@ if opt.gc =='boehm': from pypy.translator.c import gc gcpolicy = gc.BoehmGcPolicy + if opt.gc =='exact_boehm': + from pypy.translator.c import gc + gcpolicy = gc.MoreExactBoehmGcPolicy if opt.gc == 'none': from pypy.translator.c import gc gcpolicy = gc.NoneGcPolicy Modified: pypy/dist/pypy/translator/goal/translate.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate.py (original) +++ pypy/dist/pypy/translator/goal/translate.py Sat Apr 29 23:31:23 2006 @@ -48,7 +48,7 @@ '1_backend': [OPT(('-b', '--backend'), "Backend", ['c', 'llvm', 'cl', 'squeak'])], - '2_gc': [OPT(('--gc',), "Garbage collector", ['boehm', 'ref', 'framework', 'none'])], + '2_gc': [OPT(('--gc',), "Garbage collector", ['boehm', 'ref', 'framework', 'none', 'exact_boehm'])], '3_stackless': [OPT(('--stackless',), "Stackless code generation", True)], '4_merge_if_blocks': [OPT(('--no-if-blocks-merge',), "Do not merge if ... elif ... chains and use a switch statement for them.", False)], }, From cfbolz at codespeak.net Sun Apr 30 00:39:14 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 30 Apr 2006 00:39:14 +0200 (CEST) Subject: [pypy-svn] r26586 - pypy/dist/pypy/translator/c/test Message-ID: <20060429223914.D79741008B@code0.codespeak.net> Author: cfbolz Date: Sun Apr 30 00:39:08 2006 New Revision: 26586 Modified: pypy/dist/pypy/translator/c/test/test_boehm.py Log: test all boehm tests with the exact boehm too Modified: pypy/dist/pypy/translator/c/test/test_boehm.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_boehm.py (original) +++ pypy/dist/pypy/translator/c/test/test_boehm.py Sun Apr 30 00:39:08 2006 @@ -125,3 +125,9 @@ # if res is still 0, then we haven't tested anything so fail. # it might be the test's fault though. assert res > 0 + + +class TestUsingExactBoehm(TestUsingBoehm): + from pypy.translator.c.gc import MoreExactBoehmGcPolicy as gcpolicy + + From cfbolz at codespeak.net Sun Apr 30 00:41:56 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 30 Apr 2006 00:41:56 +0200 (CEST) Subject: [pypy-svn] r26587 - pypy/dist/pypy/translator/c Message-ID: <20060429224156.97DB410091@code0.codespeak.net> Author: cfbolz Date: Sun Apr 30 00:41:54 2006 New Revision: 26587 Modified: pypy/dist/pypy/translator/c/gc.py Log: don't provide type information for types that are anyway atomic Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Sun Apr 30 00:41:54 2006 @@ -277,10 +277,13 @@ def struct_setup(self, structdefnode, rtti): self.setup_gcinfo(structdefnode) T = structdefnode.STRUCT - if T._is_varsize(): - malloc_exact = T._flds[T._arrayfld]._is_atomic() + if T._is_atomic(): + malloc_exact = False else: - malloc_exact = True + if T._is_varsize(): + malloc_exact = T._flds[T._arrayfld]._is_atomic() + else: + malloc_exact = True if malloc_exact: if structdefnode.gcinfo is None: structdefnode.gcinfo = BoehmInfo() From sanxiyn at codespeak.net Sun Apr 30 08:44:31 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 08:44:31 +0200 (CEST) Subject: [pypy-svn] r26588 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060430064431.E3C6610082@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 08:44:26 2006 New Revision: 26588 Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Log: A small correction Modified: pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/tokyo-planning.txt Sun Apr 30 08:44:26 2006 @@ -239,8 +239,8 @@ - gencl (Sanghyeon, Valentino) - * Somewhat exception handling support using clisp. - Needs more work. + * Somewhat working exception handling support, but it violates + the specification. - Pickle support for objects (Eric, Samuele) From sanxiyn at codespeak.net Sun Apr 30 08:55:47 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 08:55:47 +0200 (CEST) Subject: [pypy-svn] r26589 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060430065547.91D0610086@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 08:55:42 2006 New Revision: 26589 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/test/test_oo.py Log: Rename generate_cl_func to generate_cl_code (since the string representation of the code, not the compiled function, is returned) Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Sun Apr 30 08:55:42 2006 @@ -76,7 +76,7 @@ """ def _make_cl_func(func, cl, path, argtypes=[]): - out = generate_cl_func(func, argtypes) + code = generate_cl_code(func, argtypes) fpath = path.join("%s.lisp" % func.func_name) if clconftest.option.prettyprint: @@ -86,7 +86,7 @@ fp.close() def _(*args): - fpath.write(out) + fpath.write(code) fp = file(str(fpath), "a") print >>fp, "(write (", clrepr(func.func_name, symbol=True), for arg in args: @@ -99,7 +99,7 @@ return readlisp(output) return _ -def generate_cl_func(func, argtypes=[]): +def generate_cl_code(func, argtypes=[]): t = TranslationContext() t.buildannotator().build_types(func, argtypes) t.buildrtyper(type_system="ootype").specialize() Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Sun Apr 30 08:55:42 2006 @@ -1,5 +1,5 @@ import py -from pypy.translator.cl.buildcl import make_cl_func, generate_cl_func +from pypy.translator.cl.buildcl import make_cl_func, generate_cl_code def test_simple(): class C: @@ -39,8 +39,7 @@ obj.inc() obj.inc() return obj.get() - code = generate_cl_func(inc_two, [int]) - print code + code = generate_cl_code(inc_two, [int]) methodcount = code.count("defmethod") assert methodcount == 3 @@ -51,8 +50,7 @@ pass def check_inheritance(): Bar() - code = generate_cl_func(check_inheritance) - print code + code = generate_cl_code(check_inheritance) classcount = code.count("defclass") # Divide by two to get rid of meta hierarchy # Minus one to get rid of Object From sanxiyn at codespeak.net Sun Apr 30 09:27:10 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 09:27:10 +0200 (CEST) Subject: [pypy-svn] r26592 - pypy/dist/pypy/translator/cl Message-ID: <20060430072710.A520210081@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 09:27:05 2006 New Revision: 26592 Modified: pypy/dist/pypy/translator/cl/buildcl.py pypy/dist/pypy/translator/cl/gencl.py Log: Refactor buildcl Modified: pypy/dist/pypy/translator/cl/buildcl.py ============================================================================== --- pypy/dist/pypy/translator/cl/buildcl.py (original) +++ pypy/dist/pypy/translator/cl/buildcl.py Sun Apr 30 09:27:05 2006 @@ -76,37 +76,43 @@ """ def _make_cl_func(func, cl, path, argtypes=[]): - code = generate_cl_code(func, argtypes) - fpath = path.join("%s.lisp" % func.func_name) + gen = build_generator(func, argtypes) + fname = gen.entry_name + fpath = gen.emitfile() if clconftest.option.prettyprint: - script = path.join(".printer.lisp") - fp = file(str(script), "w") - fp.write(pretty_printer % (fpath,)) - fp.close() + printer = path.join(".printer.lisp") + code = pretty_printer % (fpath,) + printer.write(code) + py.process.cmdexec("%s %s" % (cl, printer)) - def _(*args): - fpath.write(code) - fp = file(str(fpath), "a") - print >>fp, "(write (", clrepr(func.func_name, symbol=True), - for arg in args: - print >>fp, clrepr(arg), - print >>fp, "))" + def wrapper(*args): + loader = path.join(".loader.lisp") + fp = loader.open("w") + fp.write('(load "%s")\n' % (fpath,)) + if args: + args = " ".join(map(clrepr, args)) + fp.write("(write (%s %s))\n" % (fname, args)) + else: + fp.write("(write (%s))\n" % (fname,)) fp.close() - if clconftest.option.prettyprint: - py.process.cmdexec("%s %s" % (cl, str(script))) - output = py.process.cmdexec("%s %s" % (cl, str(fpath))) + output = py.process.cmdexec("%s %s" % (cl, loader)) return readlisp(output) - return _ -def generate_cl_code(func, argtypes=[]): - t = TranslationContext() - t.buildannotator().build_types(func, argtypes) - t.buildrtyper(type_system="ootype").specialize() + return wrapper - if conftest.option.view: - t.view() - - gen = GenCL(t, func) +def generate_cl_code(func, argtypes=[]): + gen = build_generator(func, argtypes) code = gen.emitcode() return code + +def build_generator(func, argtypes=[]): + context = TranslationContext() + context.buildannotator().build_types(func, argtypes) + context.buildrtyper(type_system="ootype").specialize() + + if conftest.option.view: + context.view() + + gen = GenCL(context, func) + return gen Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Sun Apr 30 09:27:05 2006 @@ -203,6 +203,7 @@ def __init__(self, context, funobj): self.context = context self.entry_point = funobj + self.entry_name = clrepr(funobj.func_name, symbol=True) self.pendinggraphs = [funobj] self.declarations = InsertionOrderedDict() self.constcount = 0 @@ -297,7 +298,7 @@ return name def emitfile(self): - name = self.entry_point.func_name + name = self.entry_name path = udir.join("%s.lisp" % (name,)) code = self.emitcode() path.write(code) From sanxiyn at codespeak.net Sun Apr 30 09:37:29 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 09:37:29 +0200 (CEST) Subject: [pypy-svn] r26593 - pypy/dist/pypy/translator/cl/test Message-ID: <20060430073729.429E510081@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 09:37:23 2006 New Revision: 26593 Modified: pypy/dist/pypy/translator/cl/test/test_oo.py Log: Some tweaks to test_oo.py Modified: pypy/dist/pypy/translator/cl/test/test_oo.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_oo.py (original) +++ pypy/dist/pypy/translator/cl/test/test_oo.py Sun Apr 30 09:37:23 2006 @@ -43,7 +43,7 @@ methodcount = code.count("defmethod") assert methodcount == 3 -def test_inherit(): +def test_inheritance(): class Foo: pass class Bar(Foo): @@ -72,8 +72,9 @@ return isinstance(obj, Bar) cl_check_isinstance = make_cl_func(check_isinstance, [bool]) assert cl_check_isinstance(True) == True + assert cl_check_isinstance(False) == False -def test_class(): +def test_class_variable(): class Foo: value = 0 class Bar(Foo): @@ -92,7 +93,7 @@ assert cl_dynamic_class(True) == 1 assert cl_dynamic_class(False) == 2 -def test_runtimenew(): +def test_runtime_new(): class Foo: pass class Bar(Foo): @@ -107,12 +108,12 @@ def runtime_new(flag): cls = pick_class(flag) cls() - return 0 + return True cl_runtime_new = make_cl_func(runtime_new, [bool]) - assert cl_runtime_new(True) == 0 - assert cl_runtime_new(False) == 0 + assert cl_runtime_new(True) + assert cl_runtime_new(False) -def test_instance(): +def test_instance_variable(): class Foo: value = 0 class Bar(Foo): From arigo at codespeak.net Sun Apr 30 14:14:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 30 Apr 2006 14:14:20 +0200 (CEST) Subject: [pypy-svn] r26594 - pypy/dist/pypy/doc/discussion Message-ID: <20060430121420.D6FC91007F@code0.codespeak.net> Author: arigo Date: Sun Apr 30 14:14:18 2006 New Revision: 26594 Modified: pypy/dist/pypy/doc/discussion/cli-optimizations.txt pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt Log: Restification. Modified: pypy/dist/pypy/doc/discussion/cli-optimizations.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/cli-optimizations.txt (original) +++ pypy/dist/pypy/doc/discussion/cli-optimizations.txt Sun Apr 30 14:14:18 2006 @@ -118,7 +118,7 @@ return -1 Given that IndexError has no subclasses, we can map it to -IndexOutOfBoundException and directly catch this one: +IndexOutOfBoundException and directly catch this one:: try { Modified: pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt (original) +++ pypy/dist/pypy/doc/discussion/pypy_metaclasses_in_cl.txt Sun Apr 30 14:14:18 2006 @@ -1,134 +1,139 @@ -[09:41] arigo: is it possible to ask the backendoptimizer to completely remove all the oogetfield('meta', obj)? -[09:42] and at the same time to change all the oogetfield('somefield', meta) into oogetfield('somefield', obj) -[09:42] because then we wouldn't need the metaclass hierarchy anymore -[09:42] (at least in common lisp) -[09:42] as far as I know the idea was indeed to be able to do this kind of things -[09:43] but not necessarily in the existing backendopt -[09:44] uhmmm -[09:44] I have no idea how to do this stuff -[09:44] if I understand it correctly, as a first step you can just tweak gencl to recognize oogetfield('meta', obj) -[09:44] I'll think about it on the plane maybe -[09:44] and produce a same_as equivalent instead -[09:44] (do I make any sense at all?) -[09:44] yes -[09:45] same_as(meta, obj) -[09:45] so that the next oogetfield() will still work on meta which in reality is the obj -[09:45] yes -[09:45] thus you obtained the same thing without removing anything -[09:45] cool -[09:46] dialtone: can you explain me better what are you trying to do? -[09:46] it looks kinda simple -[09:46] am I a fool? -[09:46] antocuni: I want to get rid of the metaclass stuff in common lisp -[09:47] since common lisp supports class variables -[09:47] (DEFCLASS foo () ((bar :allocate :class))) -[09:47] cool -[09:47] but to do that I also have to get rid of the opcodes that work on the object model -[09:48] at first I thought about removing the metaclass related operations (or change them) but armin got a great idea about using same_as -[09:48] idnar (i=mithrand at unaffiliated/idnar) left irc: Remote closed the connection -[09:48] there might be a few problems, though -[09:48] and here comes the part I feared -[09:48] I'm not sure if the meta object is used for more than oogetfields -[09:49] and also, let's see if there are name clashes in the fields -[09:49] I can't understand a thing: are you trying to lookup some fields in the obj directly, instead of in the metclass, right? -[09:49] antocuni: yes -[09:50] why an object should have fields that belongs to its metaclass? -[09:50] arigo: uhmmm you can have both a class variable and an instance variable named in the same way? -[09:50] metaclass is not a real metaclass -[09:50] I don't know -[09:50] arigo - r26566 - Support geterrno() from rctypes to genc. -[09:50] dialtone: ah, now I understand -[09:50] I would expect it not to be the case, as the names come from RPython names -[09:51] arigo: indeed -[09:51] but I guess I can set different accessors maybe for class level things and for instance level things -[09:51] let's try -[09:51] no... -[09:52] so a name clash would break stuff -[09:52] but... how do you recognize an access to a class variable and one to an instance variable from RPython? -[09:53] dialtone: I think we don't have name clashes, because there is some mangling anyway -[09:53] cool -[09:53] if I see it correctly, class variable names start with 'pbc' and instance ones with 'o' -[09:53] that's what we've done in gencl yes -[09:54] ? that's what the ootyping is doing -[09:54] yes yes -[09:54] :-) -[09:54] I mean that I see the distinction in gencl :) -[09:54] sooooooo -[09:55] if I have a getfield where the first argument is meta and I simply emit the same code that I emit for the same_as I should be safe removing all the meta stuff... maybe -[09:55] seems like a tiny change in gencl -[09:55] dialtone: in RPython, the annotator says that attributes are instance fields as soon as they are written to instances, otherwise they are class attributes -[09:56] yes, it should work -[09:56] Palats (n=Pierre at izumi.palats.com) left irc: Read error: 104 (Connection reset by peer) -[09:56] unless of course metaclasses are used for something else than class variables -[09:56] ideally, you should not look for the name 'meta' but for some other hint -[09:57] I'm not completely at ease with the various levels of ootype -[09:57] neither am I\ -[09:57] all field names other than those defined by ootype (like "meta") will be mangled, so i guess checking for "meta" is good enough -[09:57] and I also have to ignore the setfield opcode that deals with metaclasses -[09:58] or make it a same_as as well -[09:59] apparently, the meta instances are used as the ootype of RPython classes -[10:00] so they can be manipulated by RPython code that passes classes around -[10:01] I guess you can also pass classes around in CL, read attributes from them, and instantiate them -[10:01] yes -[10:01] so a saner approach might be to try to have gencl use CL classes instead of these meta instances -[10:03] uhmmmmm -[10:03] which means: recognize if an ootype.Instance is actually representing an RPython class (by using a hint) -[10:03] I also have to deal with the Class_ -[10:03] but that can probably be set to standard-class -[10:03] yes, I think it's saner to make, basically, oogetfield('class_') be a same_as -[10:04] cool -[10:04] I think I'll save this irc log to put it in the svn tree for sanxiyn -[10:04] to recognize RPython class represenations: if the ootype.Instance has the superclass ootypesystem.rclass.CLASSTYPE, then it's a "metaclass" -[10:04] he is thinking about this in the plane (at least this is what he told) -[10:05] :-) -[10:05] nikh: yes -[10:05] ootype is indeed rather complicated, level-wise, to support limited languages like Java -[10:05] unfortunately, yes -[10:05] well, in a way it's very convenient for the backends -[10:05] but if you want to use more native constructs, it gets hairy quickly -[10:05] I dunno -[10:05] depends on the backend -[10:06] hum, there is still an information missing that gencl would need here -[10:06] I think if the language of the backend is powerful enough it could use an higher abstraction -[10:07] dialtone: yes, there is also the (hairly to implement) idea of producing slightly different things for different back-ends too -[10:07] using backendopts? -[10:08] would it make sense to have a kind of backend_supports=['metaclasses', 'classvariables', 'first_class_functions'...] -[10:08] maybe, but I was thinking about doing different things in ootypesystem/rclass already -[10:08] yes, such a backend_supports would be great -[10:09] dialtone: there is still an hour left to sprint, so go go go ;) -[10:09] you can do it, if you want it ;) -[10:09] what is missing is the link from the concrete Instance types, and which Instance corresponds to its meta-instance -[10:10] idnar (i=mithrand at unaffiliated/idnar) joined #pypy. -[10:10] dialtone: it's not as simple as making an oogetfield be a same_as -[10:10] KnowledgeUnboundError, Missing documentation in slot brain -[10:10] right now for CL the goal would be to generate for a normal Instance, a DEFCLASS whose :allocate :class attributes are the attributes of the meta-Instance -[10:11] we could optionally have class fields in Instances, and then operations like ooget/setclassfield -[10:11] the reason why I ask is that if we manage to do this then we could also use default Condition as Exception -[10:11] and we could map the Conditions in common lisp to exceptions in python transparently -[10:12] since the object systems will then match (and they are vaguely similar anyway) -[10:12] nice -[10:12] at least I think -[10:18] I'm still rather confused by ootypesystem/rclass -[10:18] although I think that blame would show my name on quite some bits :-) -[10:19] there are no class attributes read through instances -[10:19] they are turned into method calls -[10:19] accessor methods -[10:20] it's a bit organically grown -[10:20] accessor methods were introduced at one point, and the meta-Instance later -[10:21] uhmmm -[10:22] what was the reason for having accessor methods? -[10:22] they seem to be only generated for class vars that are overriden in subclasses. -[10:22] yes -[10:22] before we had the meta-Instance trick, it was the only way to avoid storing the value in all instances -[10:22] aha -[10:23] we could possibly get rid of these accessors -[10:23] now, yes, by storing the values in the meta-Instance -[10:23] they are alway anyway stored in the meta-Instance, I think -[10:23] no, I think that other values are stored in the meta-Instance right now -[10:24] it's the values that are only ever accessed with a syntax 'ClassName.attr', i.e. not through an instance -[10:24] ...more precisely, with 'x = ClassName or OtherClassName; x.attr' -[10:25] hm, i'm still trying to read this out of the code ... -[10:28] it's in ClassRepr._setup_repr() -[10:28] there is no clsfields here, just pbcfields -[10:28] # attributes showing up in getattrs done on the class as a PBC -[10:28] i see \ No newline at end of file +IRC log +======= + +:: + + [09:41] arigo: is it possible to ask the backendoptimizer to completely remove all the oogetfield('meta', obj)? + [09:42] and at the same time to change all the oogetfield('somefield', meta) into oogetfield('somefield', obj) + [09:42] because then we wouldn't need the metaclass hierarchy anymore + [09:42] (at least in common lisp) + [09:42] as far as I know the idea was indeed to be able to do this kind of things + [09:43] but not necessarily in the existing backendopt + [09:44] uhmmm + [09:44] I have no idea how to do this stuff + [09:44] if I understand it correctly, as a first step you can just tweak gencl to recognize oogetfield('meta', obj) + [09:44] I'll think about it on the plane maybe + [09:44] and produce a same_as equivalent instead + [09:44] (do I make any sense at all?) + [09:44] yes + [09:45] same_as(meta, obj) + [09:45] so that the next oogetfield() will still work on meta which in reality is the obj + [09:45] yes + [09:45] thus you obtained the same thing without removing anything + [09:45] cool + [09:46] dialtone: can you explain me better what are you trying to do? + [09:46] it looks kinda simple + [09:46] am I a fool? + [09:46] antocuni: I want to get rid of the metaclass stuff in common lisp + [09:47] since common lisp supports class variables + [09:47] (DEFCLASS foo () ((bar :allocate :class))) + [09:47] cool + [09:47] but to do that I also have to get rid of the opcodes that work on the object model + [09:48] at first I thought about removing the metaclass related operations (or change them) but armin got a great idea about using same_as + [09:48] idnar (i=mithrand at unaffiliated/idnar) left irc: Remote closed the connection + [09:48] there might be a few problems, though + [09:48] and here comes the part I feared + [09:48] I'm not sure if the meta object is used for more than oogetfields + [09:49] and also, let's see if there are name clashes in the fields + [09:49] I can't understand a thing: are you trying to lookup some fields in the obj directly, instead of in the metclass, right? + [09:49] antocuni: yes + [09:50] why an object should have fields that belongs to its metaclass? + [09:50] arigo: uhmmm you can have both a class variable and an instance variable named in the same way? + [09:50] metaclass is not a real metaclass + [09:50] I don't know + [09:50] arigo - r26566 - Support geterrno() from rctypes to genc. + [09:50] dialtone: ah, now I understand + [09:50] I would expect it not to be the case, as the names come from RPython names + [09:51] arigo: indeed + [09:51] but I guess I can set different accessors maybe for class level things and for instance level things + [09:51] let's try + [09:51] no... + [09:52] so a name clash would break stuff + [09:52] but... how do you recognize an access to a class variable and one to an instance variable from RPython? + [09:53] dialtone: I think we don't have name clashes, because there is some mangling anyway + [09:53] cool + [09:53] if I see it correctly, class variable names start with 'pbc' and instance ones with 'o' + [09:53] that's what we've done in gencl yes + [09:54] ? that's what the ootyping is doing + [09:54] yes yes + [09:54] :-) + [09:54] I mean that I see the distinction in gencl :) + [09:54] sooooooo + [09:55] if I have a getfield where the first argument is meta and I simply emit the same code that I emit for the same_as I should be safe removing all the meta stuff... maybe + [09:55] seems like a tiny change in gencl + [09:55] dialtone: in RPython, the annotator says that attributes are instance fields as soon as they are written to instances, otherwise they are class attributes + [09:56] yes, it should work + [09:56] Palats (n=Pierre at izumi.palats.com) left irc: Read error: 104 (Connection reset by peer) + [09:56] unless of course metaclasses are used for something else than class variables + [09:56] ideally, you should not look for the name 'meta' but for some other hint + [09:57] I'm not completely at ease with the various levels of ootype + [09:57] neither am I\ + [09:57] all field names other than those defined by ootype (like "meta") will be mangled, so i guess checking for "meta" is good enough + [09:57] and I also have to ignore the setfield opcode that deals with metaclasses + [09:58] or make it a same_as as well + [09:59] apparently, the meta instances are used as the ootype of RPython classes + [10:00] so they can be manipulated by RPython code that passes classes around + [10:01] I guess you can also pass classes around in CL, read attributes from them, and instantiate them + [10:01] yes + [10:01] so a saner approach might be to try to have gencl use CL classes instead of these meta instances + [10:03] uhmmmmm + [10:03] which means: recognize if an ootype.Instance is actually representing an RPython class (by using a hint) + [10:03] I also have to deal with the Class_ + [10:03] but that can probably be set to standard-class + [10:03] yes, I think it's saner to make, basically, oogetfield('class_') be a same_as + [10:04] cool + [10:04] I think I'll save this irc log to put it in the svn tree for sanxiyn + [10:04] to recognize RPython class represenations: if the ootype.Instance has the superclass ootypesystem.rclass.CLASSTYPE, then it's a "metaclass" + [10:04] he is thinking about this in the plane (at least this is what he told) + [10:05] :-) + [10:05] nikh: yes + [10:05] ootype is indeed rather complicated, level-wise, to support limited languages like Java + [10:05] unfortunately, yes + [10:05] well, in a way it's very convenient for the backends + [10:05] but if you want to use more native constructs, it gets hairy quickly + [10:05] I dunno + [10:05] depends on the backend + [10:06] hum, there is still an information missing that gencl would need here + [10:06] I think if the language of the backend is powerful enough it could use an higher abstraction + [10:07] dialtone: yes, there is also the (hairly to implement) idea of producing slightly different things for different back-ends too + [10:07] using backendopts? + [10:08] would it make sense to have a kind of backend_supports=['metaclasses', 'classvariables', 'first_class_functions'...] + [10:08] maybe, but I was thinking about doing different things in ootypesystem/rclass already + [10:08] yes, such a backend_supports would be great + [10:09] dialtone: there is still an hour left to sprint, so go go go ;) + [10:09] you can do it, if you want it ;) + [10:09] what is missing is the link from the concrete Instance types, and which Instance corresponds to its meta-instance + [10:10] idnar (i=mithrand at unaffiliated/idnar) joined #pypy. + [10:10] dialtone: it's not as simple as making an oogetfield be a same_as + [10:10] KnowledgeUnboundError, Missing documentation in slot brain + [10:10] right now for CL the goal would be to generate for a normal Instance, a DEFCLASS whose :allocate :class attributes are the attributes of the meta-Instance + [10:11] we could optionally have class fields in Instances, and then operations like ooget/setclassfield + [10:11] the reason why I ask is that if we manage to do this then we could also use default Condition as Exception + [10:11] and we could map the Conditions in common lisp to exceptions in python transparently + [10:12] since the object systems will then match (and they are vaguely similar anyway) + [10:12] nice + [10:12] at least I think + [10:18] I'm still rather confused by ootypesystem/rclass + [10:18] although I think that blame would show my name on quite some bits :-) + [10:19] there are no class attributes read through instances + [10:19] they are turned into method calls + [10:19] accessor methods + [10:20] it's a bit organically grown + [10:20] accessor methods were introduced at one point, and the meta-Instance later + [10:21] uhmmm + [10:22] what was the reason for having accessor methods? + [10:22] they seem to be only generated for class vars that are overriden in subclasses. + [10:22] yes + [10:22] before we had the meta-Instance trick, it was the only way to avoid storing the value in all instances + [10:22] aha + [10:23] we could possibly get rid of these accessors + [10:23] now, yes, by storing the values in the meta-Instance + [10:23] they are alway anyway stored in the meta-Instance, I think + [10:23] no, I think that other values are stored in the meta-Instance right now + [10:24] it's the values that are only ever accessed with a syntax 'ClassName.attr', i.e. not through an instance + [10:24] ...more precisely, with 'x = ClassName or OtherClassName; x.attr' + [10:25] hm, i'm still trying to read this out of the code ... + [10:28] it's in ClassRepr._setup_repr() + [10:28] there is no clsfields here, just pbcfields + [10:28] # attributes showing up in getattrs done on the class as a PBC + [10:28] i see From arigo at codespeak.net Sun Apr 30 14:16:38 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 30 Apr 2006 14:16:38 +0200 (CEST) Subject: [pypy-svn] r26595 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060430121638.DB3E11007F@code0.codespeak.net> Author: arigo Date: Sun Apr 30 14:16:37 2006 New Revision: 26595 Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py Log: Fix for objspace/cpy/test/*. Modified: pypy/dist/pypy/rpython/rctypes/rchar_p.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/rchar_p.py (original) +++ pypy/dist/pypy/rpython/rctypes/rchar_p.py Sun Apr 30 14:16:37 2006 @@ -35,7 +35,7 @@ llops.gendirectcall(ll_setstring, v_box, v_str) def convert_const(self, value): - if not isinstance(value, (str, c_char_p)): + if value is not None and not isinstance(value, (str, c_char_p)): # maybe an array of characters? cast to a c_char_p assert type(value)._type_ == c_char value = cast(value, c_char_p) From arigo at codespeak.net Sun Apr 30 14:19:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 30 Apr 2006 14:19:36 +0200 (CEST) Subject: [pypy-svn] r26596 - pypy/dist/pypy/rpython Message-ID: <20060430121936.4D0F11007F@code0.codespeak.net> Author: arigo Date: Sun Apr 30 14:19:35 2006 New Revision: 26596 Modified: pypy/dist/pypy/rpython/llinterp.py Log: All operations must be added to llinterp too (there is a test for that). Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Sun Apr 30 14:19:35 2006 @@ -433,6 +433,9 @@ def op_getslice(self, vargs, start, stop_should_be_None): raise NotImplementedError("getslice") # only for argument parsing + def op_check_self_nonzero(self, fname, vself): + raise NotImplementedError("check_self_nonzero") + def op_setfield(self, obj, fieldname, fieldvalue): # obj should be pointer FIELDTYPE = getattr(lltype.typeOf(obj).TO, fieldname) From sanxiyn at codespeak.net Sun Apr 30 14:51:22 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 14:51:22 +0200 (CEST) Subject: [pypy-svn] r26597 - pypy/dist/pypy/translator/cl Message-ID: <20060430125122.34F411007F@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 14:51:11 2006 New Revision: 26597 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Don't generate empty setf Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Sun Apr 30 14:51:11 2006 @@ -457,7 +457,8 @@ def emit_link(self, link): source = map(self.check_declaration, link.args) target = map(clrepr, link.target.inputargs) - couples = [ "%s %s" % (clrepr(t, True), clrepr(s, True)) for (s, t) in zip(source, target)] - couples = " ".join(couples) - yield "(setf %s)" % (couples,) + couples = ["%s %s" % (t, s) for (s, t) in zip(source, target)] + if couples: + couples = " ".join(couples) + yield "(setf %s)" % (couples,) yield self.format_jump(link.target) From sanxiyn at codespeak.net Sun Apr 30 15:53:46 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 15:53:46 +0200 (CEST) Subject: [pypy-svn] r26598 - pypy/dist/pypy/translator/cl Message-ID: <20060430135346.62F041007F@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 15:53:40 2006 New Revision: 26598 Added: pypy/dist/pypy/translator/cl/opformatter.py - copied, changed from r26597, pypy/dist/pypy/translator/cl/gencl.py Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Split Op class to opformatter.py and rename to OpFormatter Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Sun Apr 30 15:53:40 2006 @@ -3,9 +3,10 @@ from pypy.tool.udir import udir from pypy.objspace.flow.model import Constant, c_last_exception from pypy.translator.translator import graphof -from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, List, Record, Instance, _class, _static_meth, _meth, ROOT +from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, Instance, _class, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import clrepr +from pypy.translator.cl.opformatter import OpFormatter class InsertionOrderedDict(dict): def __init__(self): @@ -20,184 +21,6 @@ def values(self): return [self[key] for key in self.ordered_keys] -class Op: - - def __init__(self, gen, op): - self.gen = gen - self.op = op - self.opname = op.opname - self.args = op.args - self.result = op.result - - def __iter__(self): - method = getattr(self, "op_" + self.opname) - result = clrepr(self.result) - args = map(self.gen.check_declaration, self.args) - for line in method(result, *args): - yield line - - def nop(self, result, arg): - yield "(setf %s %s)" % (clrepr(result, True), clrepr(arg, True)) - - op_same_as = nop - op_ooupcast = nop - op_oodowncast = nop - - def make_unary_op(cl_op): - def _(self, result, arg): - yield "(setf %s (%s %s))" % (clrepr(result, True), - cl_op, - clrepr(arg, True)) - return _ - - op_bool_not = make_unary_op("not") - op_cast_char_to_int = make_unary_op("char-code") - op_cast_int_to_char = make_unary_op("code-char") - op_cast_float_to_int = make_unary_op("truncate") - op_cast_int_to_float = make_unary_op("float") - - def make_binary_op(cl_op): - def _(self, result, arg1, arg2): - yield "(setf %s (%s %s %s))" % (clrepr(result, True), - cl_op, - clrepr(arg1, True), - clrepr(arg2, True)) - return _ - - op_int_add = make_binary_op("+") - op_int_mul = make_binary_op("*") - op_int_eq = make_binary_op("=") - op_int_gt = make_binary_op(">") - op_int_ge = make_binary_op(">=") - op_int_lt = make_binary_op("<") - op_int_le = make_binary_op("<=") - op_int_and = make_binary_op("logand") - op_int_mod = make_binary_op("mod") - op_int_sub = make_binary_op("-") - op_float_sub = make_binary_op("-") - op_float_truediv = make_binary_op("/") - op_char_eq = make_binary_op("char=") - op_char_le = make_binary_op("char<=") - op_char_ne = make_binary_op("char/=") - - def op_int_is_true(self, result, arg): - yield "(setf %s (not (zerop %s)))" % (clrepr(result, True), - clrepr(arg, True)) - - def op_direct_call(self, result, fun, *args): - funobj = self.args[0].value - self.gen.pendinggraphs.append(funobj) - args = " ".join(args) - yield "(setf %s (%s %s))" % (clrepr(result, True), - clrepr(fun, True), - clrepr(args, True)) - - def op_new(self, result, clsname): - cls = self.args[0].value - if isinstance(cls, List): - yield "(setf %s (make-array 0 :adjustable t))" % (clrepr(result, True),) - elif isinstance(cls, Record): - clsname = self.gen.declare_struct(cls) - yield "(setf %s (make-%s))" % (clrepr(result, True), - clrepr(clsname, True)) - elif isinstance(cls, Instance): - clsname = clrepr(cls) - if self.gen.is_exception_instance(cls): - self.gen.declare_exception(cls) - yield "(setf %s (make-condition %s))" % (result, clsname) - else: - self.gen.declare_class(cls) - yield "(setf %s (make-instance %s))" % (result, clsname) - else: - raise NotImplementedError() - - def op_runtimenew(self, result, arg): - yield "(setf %s (make-instance %s))" % (clrepr(result, True), - clrepr(arg, True)) - - def op_instanceof(self, result, arg, clsname): - clsname = clrepr(self.args[1].value) - yield "(setf %s (typep %s %s))" % (clrepr(result, True), - clrepr(arg, True), - clrepr(clsname, True)) - - def op_oosend(self, result, *ignore): - method = self.args[0].value - receiver = self.args[1] - cls = receiver.concretetype - args = self.args[2:] - if isinstance(cls, List): - impl = ListImpl(receiver) - code = getattr(impl, method)(*args) - yield "(setf %s %s)" % (clrepr(result, True), clrepr(code, True)) - elif isinstance(cls, Instance): - name = clrepr(method, symbol=True) - selfvar = clrepr(receiver) - args = map(self.gen.check_declaration, args) - args = " ".join(args) - if args: - yield "(setf %s (%s %s %s))" % (clrepr(result, True), - clrepr(name, True), - clrepr(selfvar, True), - clrepr(args, True)) - else: - yield "(setf %s (%s %s))" % (clrepr(result, True), - clrepr(name, True), - clrepr(selfvar, True)) - - def op_oogetfield(self, result, obj, _): - fieldname = self.args[1].value - if isinstance(self.args[0].concretetype, Record): - yield "(setf %s (slot-value %s '%s))" % (clrepr(result, True), - clrepr(obj, True), - clrepr(fieldname, True)) - else: - yield "(setf %s (%s %s))" % (clrepr(result, True), - clrepr(fieldname, True), - clrepr(obj, True)) - - def op_oosetfield(self, result, obj, _, value): - fieldname = self.args[1].value - if isinstance(self.args[0].concretetype, Record): - yield "(setf (slot-value %s '%s) %s)" % (clrepr(obj, True), - clrepr(fieldname, True), - clrepr(value, True)) - else: - yield "(setf (%s %s) %s)" % (clrepr(fieldname, True), - clrepr(obj, True), - clrepr(value, True)) - - def op_ooidentityhash(self, result, arg): - yield "(setf %s (sxhash %s))" % (clrepr(result, True), - clrepr(arg, True)) - - def op_oononnull(self, result, arg): - yield "(setf %s (not (null %s)))" % (clrepr(result, True), - clrepr(arg, True)) - - -class ListImpl: - - def __init__(self, receiver): - self.obj = clrepr(receiver) - - def ll_length(self): - return "(length %s)" % (self.obj,) - - def ll_getitem_fast(self, index): - index = clrepr(index) - return "(aref %s %s)" % (self.obj, index) - - def ll_setitem_fast(self, index, value): - index = clrepr(index) - value = clrepr(value) - return "(setf (aref %s %s) %s)" % (self.obj, index, value) - - def _ll_resize(self, size): - size = clrepr(size) - return "(adjust-array %s %s)" % (self.obj, size) - - class GenCL: def __init__(self, context, funobj): @@ -387,7 +210,7 @@ if block.exitswitch is c_last_exception: yield "(handler-case (progn" for op in block.operations: - emit_op = Op(self, op) + emit_op = OpFormatter(self, op) for line in emit_op: yield line exits = block.exits From sanxiyn at codespeak.net Sun Apr 30 16:00:16 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 16:00:16 +0200 (CEST) Subject: [pypy-svn] r26599 - pypy/dist/pypy/translator/cl Message-ID: <20060430140016.3C1F910080@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 16:00:12 2006 New Revision: 26599 Modified: pypy/dist/pypy/translator/cl/opformatter.py Log: Refactor opformatter a bit Modified: pypy/dist/pypy/translator/cl/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/cl/opformatter.py (original) +++ pypy/dist/pypy/translator/cl/opformatter.py Sun Apr 30 16:00:12 2006 @@ -18,18 +18,16 @@ yield line def nop(self, result, arg): - yield "(setf %s %s)" % (clrepr(result, True), clrepr(arg, True)) + yield "(setf %s %s)" % (result, arg) op_same_as = nop op_ooupcast = nop op_oodowncast = nop def make_unary_op(cl_op): - def _(self, result, arg): - yield "(setf %s (%s %s))" % (clrepr(result, True), - cl_op, - clrepr(arg, True)) - return _ + def unary_op(self, result, arg): + yield "(setf %s (%s %s))" % (result, cl_op, arg) + return unary_op op_bool_not = make_unary_op("not") op_cast_char_to_int = make_unary_op("char-code") @@ -38,12 +36,9 @@ op_cast_int_to_float = make_unary_op("float") def make_binary_op(cl_op): - def _(self, result, arg1, arg2): - yield "(setf %s (%s %s %s))" % (clrepr(result, True), - cl_op, - clrepr(arg1, True), - clrepr(arg2, True)) - return _ + def binary_op(self, result, arg1, arg2): + yield "(setf %s (%s %s %s))" % (result, cl_op, arg1, arg2) + return binary_op op_int_add = make_binary_op("+") op_int_mul = make_binary_op("*") From arigo at codespeak.net Sun Apr 30 16:00:26 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 30 Apr 2006 16:00:26 +0200 (CEST) Subject: [pypy-svn] r26600 - in pypy/dist/pypy: interpreter objspace/cpy objspace/cpy/test Message-ID: <20060430140026.6B95C10085@code0.codespeak.net> Author: arigo Date: Sun Apr 30 16:00:22 2006 New Revision: 26600 Added: pypy/dist/pypy/objspace/cpy/ctypes_base.py (contents, props changed) Modified: pypy/dist/pypy/interpreter/error.py pypy/dist/pypy/objspace/cpy/ann_policy.py pypy/dist/pypy/objspace/cpy/capi.py pypy/dist/pypy/objspace/cpy/objspace.py pypy/dist/pypy/objspace/cpy/test/test_objspace.py pypy/dist/pypy/objspace/cpy/test/test_wrappable.py pypy/dist/pypy/objspace/cpy/wrappable.py Log: Start supporting exceptions in the CPyObjSpace: the idea is that in RPython there are only a few exception classes, which represent internal errors and shouldn't escape, apart from the OperationError, which represents an app-level exception. - when calling C API functions, wrap any exception into an OperationError back for the RPython caller; - when wrapping an RPython function to expose it to the C API, unwrap OperationErrors generated by RPython. Some more rctypes support seems to be needed for the first point. Modified: pypy/dist/pypy/interpreter/error.py ============================================================================== --- pypy/dist/pypy/interpreter/error.py (original) +++ pypy/dist/pypy/interpreter/error.py Sun Apr 30 16:00:22 2006 @@ -11,8 +11,8 @@ OperationError instances have three public attributes (and no .args), w_type, w_value and application_traceback, which contain the wrapped - type and value describing the exception, and the unwrapped list of - (frame, instruction_position) making the application-level traceback. + type and value describing the exception, and a chained list of + PyTraceback objects making the application-level traceback. """ def __init__(self, w_type, w_value, tb=None): Modified: pypy/dist/pypy/objspace/cpy/ann_policy.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/ann_policy.py (original) +++ pypy/dist/pypy/objspace/cpy/ann_policy.py Sun Apr 30 16:00:22 2006 @@ -1,5 +1,8 @@ from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy from pypy.annotation.pairtype import pair +from pypy.annotation import model as annmodel +from pypy.interpreter.error import OperationError +from pypy.objspace.cpy.ctypes_base import W_Object class CPyAnnotatorPolicy(PyPyAnnotatorPolicy): """Annotation policy to compile CPython extension modules with @@ -8,7 +11,6 @@ def __init__(self, space): PyPyAnnotatorPolicy.__init__(self, single_space=space) - self.spaces = {} def no_more_blocks_to_annotate(self, annotator): PyPyAnnotatorPolicy.no_more_blocks_to_annotate(self, annotator) @@ -26,3 +28,10 @@ w_obj) # restart this loop: for all we know follow_annotations() # could have found new objects + + # force w_type, w_value attributes into the OperationError class + classdef = annotator.bookkeeper.getuniqueclassdef(OperationError) + s_instance = annmodel.SomeInstance(classdef=classdef) + for name in ['w_type', 'w_value']: + s_instance.setattr(annotator.bookkeeper.immutablevalue(name), + annotator.bookkeeper.valueoftype(W_Object)) Modified: pypy/dist/pypy/objspace/cpy/capi.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/capi.py (original) +++ pypy/dist/pypy/objspace/cpy/capi.py Sun Apr 30 16:00:22 2006 @@ -1,16 +1,12 @@ +""" +CTypes declarations for the CPython API. +""" import sys import ctypes from ctypes import * from pypy.rpython.rctypes.tool import ctypes_platform -from pypy.rpython.rctypes import apyobject ##from pypy.rpython.rctypes.implementation import CALLBACK_FUNCTYPE - -class W_Object(py_object): - "A py_object subclass, representing wrapped objects for the CPyObjSpace." - def __repr__(self): - return 'W_Object(%r)' % (self.value,) - -apyobject.register_py_object_subclass(W_Object) +from pypy.objspace.cpy.ctypes_base import W_Object, cpyapi ############################################################### @@ -54,51 +50,63 @@ ########################################################### # ____________________ Object Protocol ____________________ -PyObject_GetAttr = pythonapi.PyObject_GetAttr +PyObject_GetAttr = cpyapi.PyObject_GetAttr PyObject_GetAttr.argtypes = [W_Object, W_Object] PyObject_GetAttr.restype = W_Object -PyObject_GetItem = pythonapi.PyObject_GetItem +PyObject_GetItem = cpyapi.PyObject_GetItem PyObject_GetItem.argtypes = [W_Object, W_Object] PyObject_GetItem.restype = W_Object -PyObject_SetItem = pythonapi.PyObject_SetItem +PyObject_SetItem = cpyapi.PyObject_SetItem PyObject_SetItem.argtypes = [W_Object, W_Object, W_Object] PyObject_SetItem.restype = c_int -PyObject_Call = pythonapi.PyObject_Call +PyObject_Call = cpyapi.PyObject_Call PyObject_Call.argtypes = [W_Object, W_Object, W_Object] PyObject_Call.restype = W_Object -PyObject_CallFunctionObjArgs = pythonapi.PyObject_CallFunctionObjArgs +PyObject_CallFunctionObjArgs = cpyapi.PyObject_CallFunctionObjArgs PyObject_CallFunctionObjArgs.restype = W_Object #PyObject_CallFunctionObjArgs.argtypes = [W_Object, ..., final NULL] -PyObject_RichCompare = pythonapi.PyObject_RichCompare +PyObject_RichCompare = cpyapi.PyObject_RichCompare PyObject_RichCompare.argtypes = [W_Object, W_Object, c_int] PyObject_RichCompare.restype = W_Object -PyObject_RichCompareBool = pythonapi.PyObject_RichCompareBool +PyObject_RichCompareBool = cpyapi.PyObject_RichCompareBool PyObject_RichCompareBool.argtypes = [W_Object, W_Object, c_int] PyObject_RichCompareBool.restype = c_int -PyObject_GetIter = pythonapi.PyObject_GetIter +PyObject_GetIter = cpyapi.PyObject_GetIter PyObject_GetIter.argtypes = [W_Object] PyObject_GetIter.restype = W_Object -PyIter_Next = pythonapi.PyIter_Next +PyIter_Next = cpyapi.PyIter_Next PyIter_Next.argtypes = [W_Object] PyIter_Next.restype = W_Object +########################################################### +# ____________________ Number Protocol ____________________ + +PyNumber_Add = cpyapi.PyNumber_Add +PyNumber_Add.argtypes = [W_Object, W_Object] +PyNumber_Add.restype = W_Object + +PyNumber_Subtract = cpyapi.PyNumber_Subtract +PyNumber_Subtract.argtypes = [W_Object, W_Object] +PyNumber_Subtract.restype = W_Object + + ############################################################# # ____________________ Sequence Protocol ____________________ -PySequence_Tuple = pythonapi.PySequence_Tuple +PySequence_Tuple = cpyapi.PySequence_Tuple PySequence_Tuple.argtypes = [W_Object] PySequence_Tuple.restype = W_Object -PySequence_SetItem = pythonapi.PySequence_SetItem +PySequence_SetItem = cpyapi.PySequence_SetItem PySequence_SetItem.argtypes = [W_Object, Py_ssize_t, W_Object] PySequence_SetItem.restype = c_int @@ -106,11 +114,11 @@ ########################################################### # ____________________ Numeric Objects ____________________ -PyInt_FromLong = pythonapi.PyInt_FromLong +PyInt_FromLong = cpyapi.PyInt_FromLong PyInt_FromLong.argtypes = [c_long] PyInt_FromLong.restype = W_Object -PyInt_AsLong = pythonapi.PyInt_AsLong +PyInt_AsLong = cpyapi.PyInt_AsLong PyInt_AsLong.argtypes = [W_Object] PyInt_AsLong.restype = c_long @@ -118,15 +126,15 @@ ################################################### # ____________________ Strings ____________________ -PyString_FromStringAndSize = pythonapi.PyString_FromStringAndSize +PyString_FromStringAndSize = cpyapi.PyString_FromStringAndSize PyString_FromStringAndSize.argtypes = [c_char_p, Py_ssize_t] PyString_FromStringAndSize.restype = W_Object -PyString_InternInPlace = pythonapi.PyString_InternInPlace +PyString_InternInPlace = cpyapi.PyString_InternInPlace PyString_InternInPlace.argtypes = [POINTER(W_Object)] PyString_InternInPlace.restype = None -PyString_AsString = pythonapi.PyString_AsString +PyString_AsString = cpyapi.PyString_AsString PyString_AsString.argtypes = [W_Object] PyString_AsString.restype = c_char_p @@ -134,7 +142,7 @@ ################################################## # ____________________ Tuples ____________________ -PyTuple_New = pythonapi.PyTuple_New +PyTuple_New = cpyapi.PyTuple_New PyTuple_New.argtypes = [Py_ssize_t] PyTuple_New.restype = W_Object @@ -142,11 +150,11 @@ ################################################# # ____________________ Lists ____________________ -PyList_New = pythonapi.PyList_New +PyList_New = cpyapi.PyList_New PyList_New.argtypes = [Py_ssize_t] PyList_New.restype = W_Object -PyList_Append = pythonapi.PyList_Append +PyList_Append = cpyapi.PyList_Append PyList_Append.argtypes = [W_Object, W_Object] PyList_Append.restype = c_int @@ -154,11 +162,11 @@ ######################################################## # ____________________ Dictionaries ____________________ -PyDict_New = pythonapi.PyDict_New +PyDict_New = cpyapi.PyDict_New PyDict_New.argtypes = [] PyDict_New.restype = W_Object -PyDict_SetItem = pythonapi.PyDict_SetItem +PyDict_SetItem = cpyapi.PyDict_SetItem PyDict_SetItem.argtypes = [W_Object, W_Object, W_Object] PyDict_SetItem.restype = c_int @@ -166,23 +174,30 @@ ##################################################### # ____________________ Utilities ____________________ -PyImport_ImportModule = pythonapi.PyImport_ImportModule +PyImport_ImportModule = cpyapi.PyImport_ImportModule PyImport_ImportModule.argtypes = [c_char_p] PyImport_ImportModule.restype = W_Object +# "RAW" because it comes from pythonapi instead of cpyapi +# which makes it raise the set exception directly instead +# of wrapping it into an OperationError +RAW_PyErr_SetObject = pythonapi.PyErr_SetObject +RAW_PyErr_SetObject.argtypes = [W_Object, W_Object] +RAW_PyErr_SetObject.restype = None + ############################################################## # ____________________ Built-in functions ____________________ -PyArg_ParseTuple = pythonapi.PyArg_ParseTuple +PyArg_ParseTuple = cpyapi.PyArg_ParseTuple PyArg_ParseTuple.restype = c_int #PyArg_ParseTuple.argtypes = [W_Object, c_char_p, ...] -PyArg_ParseTupleAndKeywords = pythonapi.PyArg_ParseTupleAndKeywords +PyArg_ParseTupleAndKeywords = cpyapi.PyArg_ParseTupleAndKeywords PyArg_ParseTupleAndKeywords.restype = c_int #PyArg_ParseTupleAndKeywords.argtypes = [W_Object, W_Object, # c_char_p, POINTER(c_char_p), ...] -##PyCFunction_NewEx = pythonapi.PyCFunction_NewEx +##PyCFunction_NewEx = cpyapi.PyCFunction_NewEx ##PyCFunction_NewEx.argtypes = [POINTER(PyMethodDef), W_Object, W_Object] ##PyCFunction_NewEx.restype = W_Object Added: pypy/dist/pypy/objspace/cpy/ctypes_base.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/cpy/ctypes_base.py Sun Apr 30 16:00:22 2006 @@ -0,0 +1,50 @@ +""" +CTypes base classes to support the particularities of the CPyObjSpace +when it uses the CPython API: the W_Object class and the cpyapi accessor. +""" + +import sys +from ctypes import * +from pypy.rpython.rctypes import apyobject +from pypy.interpreter.error import OperationError +from pypy.tool.sourcetools import func_with_new_name + + +class W_Object(py_object): + """A py_object subclass, representing wrapped objects for the CPyObjSpace. + The reason we don't use py_object directly is that if py_object is + specified as the restype of a function, the function call unwraps it + automatically. With W_Object, however, the function call returns a + W_Object instance. + """ + def __repr__(self): + return 'W_Object(%r)' % (self.value,) + +apyobject.register_py_object_subclass(W_Object) + + +class LevelError(Exception): + pass + +class CPyAPI(PyDLL): + """Class of the singleton 'cpyapi' object, out of which C functions + are getattr'd. It returns C function whose exception behavior matches + the one required for the CPyObjSpace: exceptions are wrapped in + OperationErrors. + """ + class _FuncPtr(PyDLL._FuncPtr): + _flags_ = PyDLL._FuncPtr._flags_ + + def __call__(*args, **kwds): + try: + return PyDLL._FuncPtr.__call__(*args, **kwds) + except OperationError, e: + raise LevelError, "unexpected OperationError: %r" % (e,) + except: + exc, val, tb = sys.exc_info() + raise OperationError(W_Object(exc), + W_Object(val), + W_Object(tb)) + +cpyapi = CPyAPI.__new__(CPyAPI) +cpyapi.__dict__ = pythonapi.__dict__.copy() Modified: pypy/dist/pypy/objspace/cpy/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/objspace.py Sun Apr 30 16:00:22 2006 @@ -5,7 +5,7 @@ class CPyObjSpace(baseobjspace.ObjSpace): - from pypy.objspace.cpy.capi import W_Object + from pypy.objspace.cpy.ctypes_base import W_Object def initialize(self): self.options.geninterp = True @@ -16,6 +16,7 @@ self.w_type = W_Object(type) self.w_Exception = W_Object(Exception) self.w_StopIteration = W_Object(StopIteration) + self.w_TypeError = W_Object(TypeError) self.wrap_cache = {} self.rev_wrap_cache = {} @@ -71,6 +72,9 @@ str_w = staticmethod(PyString_AsString) iter = staticmethod(PyObject_GetIter) + add = staticmethod(PyNumber_Add) + sub = staticmethod(PyNumber_Subtract) + def call_function(self, w_callable, *args_w): args_w += (None,) return PyObject_CallFunctionObjArgs(w_callable, *args_w) Modified: pypy/dist/pypy/objspace/cpy/test/test_objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_objspace.py Sun Apr 30 16:00:22 2006 @@ -1,4 +1,5 @@ from pypy.objspace.cpy.objspace import CPyObjSpace +from pypy.tool.pytest.appsupport import raises_w def test_simple(): space = CPyObjSpace() @@ -15,3 +16,9 @@ w_time = demo.measuretime(space, 10, CPyObjSpace.W_Object(int)) assert isinstance(w_time, CPyObjSpace.W_Object) assert isinstance(w_time.value, int) + +def test_exception(): + space = CPyObjSpace() + w1 = space.wrap('abc') + w2 = space.wrap(11) + raises_w(space, space.w_TypeError, space.sub, w1, w2) Modified: pypy/dist/pypy/objspace/cpy/test/test_wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/test/test_wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/test/test_wrappable.py Sun Apr 30 16:00:22 2006 @@ -1,4 +1,5 @@ from pypy.objspace.cpy.objspace import CPyObjSpace +from pypy.tool.pytest.appsupport import raises_w from pypy.interpreter.function import BuiltinFunction from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root from pypy.interpreter.argument import Arguments @@ -29,3 +30,11 @@ w_result = space.call_args(w_entrypoint, args) result = space.int_w(w_result) assert result == -21 + +def test_exception(): + space = CPyObjSpace() + func = interp2app(entrypoint1).__spacebind__(space) + bltin = BuiltinFunction(func) + w_entrypoint = space.wrap(bltin) + w1 = space.wrap('not an int') + raises_w(space, space.w_TypeError, space.call_function, w_entrypoint, w1) Modified: pypy/dist/pypy/objspace/cpy/wrappable.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/wrappable.py (original) +++ pypy/dist/pypy/objspace/cpy/wrappable.py Sun Apr 30 16:00:22 2006 @@ -7,6 +7,7 @@ from pypy.annotation.pairtype import pair, pairtype from pypy.objspace.cpy.capi import * from pypy.objspace.cpy.objspace import CPyObjSpace +from pypy.interpreter.error import OperationError from pypy.interpreter.function import Function from pypy.interpreter.gateway import BuiltinCode, ObjSpace, W_Root from pypy.interpreter.gateway import UnwrapSpecRecipe, Signature @@ -71,16 +72,25 @@ sourcelines = ['def trampoline(%s):' % (', '.join(tramp.inputargs),)] for line in tramp.wrappings: sourcelines.append(' ' + line) - sourcelines.append(' w_result = ___bltin(%s)' % ( + sourcelines.append(' try:') + sourcelines.append(' w_result = ___bltin(%s)' % ( ', '.join(tramp.passedargs),)) + sourcelines.append(' except ___OperationError, e:') + sourcelines.append(' ___PyErr_SetObject(e.w_type.value,') + sourcelines.append(' e.w_value.value)') + sourcelines.append(' return None') # should never be seen + sourcelines.append(' #except ___Exception, e:') + sourcelines.append(' # raise ___RPythonError(XXX)') sourcelines.append(' return w_result.value') sourcelines.append('') miniglobals = { - '___space': space, - '___W_Object': CPyObjSpace.W_Object, - '___PyInt_AsLong': PyInt_AsLong, - '___bltin': bltin, + '___space': space, + '___W_Object': CPyObjSpace.W_Object, + '___PyInt_AsLong': PyInt_AsLong, + '___bltin': bltin, + '___OperationError': OperationError, + '___PyErr_SetObject': RAW_PyErr_SetObject, } exec py.code.Source('\n'.join(sourcelines)).compile() in miniglobals From arigo at codespeak.net Sun Apr 30 16:05:41 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 30 Apr 2006 16:05:41 +0200 (CEST) Subject: [pypy-svn] r26601 - pypy/dist/pypy/objspace/cpy Message-ID: <20060430140541.E87761007D@code0.codespeak.net> Author: arigo Date: Sun Apr 30 16:05:41 2006 New Revision: 26601 Modified: pypy/dist/pypy/objspace/cpy/objspace.py Log: Fixes enabled by the OperationError support. Modified: pypy/dist/pypy/objspace/cpy/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/cpy/objspace.py (original) +++ pypy/dist/pypy/objspace/cpy/objspace.py Sun Apr 30 16:05:41 2006 @@ -17,6 +17,7 @@ self.w_Exception = W_Object(Exception) self.w_StopIteration = W_Object(StopIteration) self.w_TypeError = W_Object(TypeError) + self.w_KeyError = W_Object(KeyError) self.wrap_cache = {} self.rev_wrap_cache = {} @@ -51,12 +52,6 @@ assert isinstance(w_obj, W_Object) return w_obj.value - def finditem(self, w_obj, w_key): - try: - return self.getitem(w_obj, w_key) - except KeyError: # XXX think about OperationError - return None - def interpclass_w(self, w_obj): try: return self.rev_wrap_cache[id(w_obj)][1] From sanxiyn at codespeak.net Sun Apr 30 16:11:00 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 16:11:00 +0200 (CEST) Subject: [pypy-svn] r26602 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060430141100.C50F41007D@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 16:10:56 2006 New Revision: 26602 Modified: pypy/dist/pypy/translator/cl/opformatter.py pypy/dist/pypy/translator/cl/test/test_cltrans.py Log: op_int_floordiv (maps to CL floor) and test for that Modified: pypy/dist/pypy/translator/cl/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/cl/opformatter.py (original) +++ pypy/dist/pypy/translator/cl/opformatter.py Sun Apr 30 16:10:56 2006 @@ -41,7 +41,9 @@ return binary_op op_int_add = make_binary_op("+") + op_int_sub = make_binary_op("-") op_int_mul = make_binary_op("*") + op_int_floordiv = make_binary_op("floor") op_int_eq = make_binary_op("=") op_int_gt = make_binary_op(">") op_int_ge = make_binary_op(">=") @@ -49,7 +51,6 @@ op_int_le = make_binary_op("<=") op_int_and = make_binary_op("logand") op_int_mod = make_binary_op("mod") - op_int_sub = make_binary_op("-") op_float_sub = make_binary_op("-") op_float_truediv = make_binary_op("/") op_char_eq = make_binary_op("char=") Modified: pypy/dist/pypy/translator/cl/test/test_cltrans.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_cltrans.py (original) +++ pypy/dist/pypy/translator/cl/test/test_cltrans.py Sun Apr 30 16:10:56 2006 @@ -37,6 +37,15 @@ assert cl_cast_int(1.5) == 1 assert cl_cast_int(-1.5) == -1 +def test_int_div(): + def int_div(a, b): + return a / b + cl_int_div = make_cl_func(int_div, [int, int]) + assert cl_int_div(4, 2) == 2 + assert cl_int_div(5, 2) == 2 + assert cl_int_div(4, -2) == -2 + assert cl_int_div(5, -2) == -3 + def test_range(): def get_three(): lst = range(7) From sanxiyn at codespeak.net Sun Apr 30 16:46:59 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 16:46:59 +0200 (CEST) Subject: [pypy-svn] r26603 - pypy/dist/pypy/translator/cl Message-ID: <20060430144659.0555A10080@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 16:46:56 2006 New Revision: 26603 Modified: pypy/dist/pypy/translator/cl/opformatter.py Log: Cleanup op_new Modified: pypy/dist/pypy/translator/cl/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/cl/opformatter.py (original) +++ pypy/dist/pypy/translator/cl/opformatter.py Sun Apr 30 16:46:56 2006 @@ -69,24 +69,22 @@ clrepr(fun, True), clrepr(args, True)) - def op_new(self, result, clsname): + def op_new(self, result, _): cls = self.args[0].value if isinstance(cls, List): - yield "(setf %s (make-array 0 :adjustable t))" % (clrepr(result, True),) + yield "(setf %s (make-array 0 :adjustable t))" % (result,) elif isinstance(cls, Record): clsname = self.gen.declare_struct(cls) - yield "(setf %s (make-%s))" % (clrepr(result, True), - clrepr(clsname, True)) + yield "(setf %s (make-%s))" % (result, clsname) elif isinstance(cls, Instance): - clsname = clrepr(cls) if self.gen.is_exception_instance(cls): - self.gen.declare_exception(cls) - yield "(setf %s (make-condition %s))" % (result, clsname) + clsname = self.gen.declare_exception(cls) + yield "(setf %s (make-condition '%s))" % (result, clsname) else: - self.gen.declare_class(cls) - yield "(setf %s (make-instance %s))" % (result, clsname) + clsname = self.gen.declare_class(cls) + yield "(setf %s (make-instance '%s))" % (result, clsname) else: - raise NotImplementedError() + raise NotImplementedError("op_new on %s" % (cls,)) def op_runtimenew(self, result, arg): yield "(setf %s (make-instance %s))" % (clrepr(result, True), From sanxiyn at codespeak.net Sun Apr 30 16:56:04 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 16:56:04 +0200 (CEST) Subject: [pypy-svn] r26604 - pypy/dist/pypy/translator/cl Message-ID: <20060430145604.08CC310080@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 16:56:01 2006 New Revision: 26604 Modified: pypy/dist/pypy/translator/cl/gencl.py Log: Put asserts and fix a bug exposed Modified: pypy/dist/pypy/translator/cl/gencl.py ============================================================================== --- pypy/dist/pypy/translator/cl/gencl.py (original) +++ pypy/dist/pypy/translator/cl/gencl.py Sun Apr 30 16:56:01 2006 @@ -3,7 +3,7 @@ from pypy.tool.udir import udir from pypy.objspace.flow.model import Constant, c_last_exception from pypy.translator.translator import graphof -from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, Instance, _class, _static_meth, _meth, ROOT +from pypy.rpython.ootypesystem.ootype import dynamicType, oodowncast, Record, Instance, _class, _static_meth, _meth, ROOT from pypy.rpython.ootypesystem.rclass import OBJECT from pypy.translator.cl.clrepr import clrepr from pypy.translator.cl.opformatter import OpFormatter @@ -42,8 +42,18 @@ return self.declare_constant_instance(arg) return clrepr(arg) + def declare_any(self, cls): + if isinstance(cls, Record): + return self.declare_struct(cls) + if isinstance(cls, Instance): + if self.is_exception_instance(cls): + return self.declare_exception(cls) + else: + return self.declare_class(cls) + raise NotImplementedError("cannot declare %s" % (cls,)) + def declare_struct(self, cls): - # cls is Record + assert isinstance(cls, Record) if cls in self.declarations: return self.declarations[cls][0] name = "struct" + str(self.structcount) @@ -55,7 +65,8 @@ return name def declare_class(self, cls): - # cls is Instance + assert isinstance(cls, Instance) + assert not self.is_exception_instance(cls) if cls in self.declarations: return self.declarations[cls][0] name = clrepr(cls._name, symbol=True) @@ -78,7 +89,7 @@ return name def declare_exception(self, cls): - # cls is Instance + assert isinstance(cls, Instance) assert self.is_exception_instance(cls) if cls in self.declarations: return self.declarations[cls][0] @@ -109,7 +120,7 @@ for fieldname in fields: fieldvalue = getattr(inst, fieldname) if isinstance(fieldvalue, _class): - self.declare_class(fieldvalue._INSTANCE) + self.declare_any(fieldvalue._INSTANCE) fieldvaluerepr = clrepr(getattr(inst, fieldname)) ### XXX const_declaration.append("(setf (slot-value %s '%s) %s)" % (clrepr(name, True), From sanxiyn at codespeak.net Sun Apr 30 17:14:15 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 17:14:15 +0200 (CEST) Subject: [pypy-svn] r26605 - pypy/dist/pypy/translator/cl Message-ID: <20060430151415.6489810081@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 17:14:12 2006 New Revision: 26605 Modified: pypy/dist/pypy/translator/cl/opformatter.py Log: Cleanup op_direct_call and op_oosend and simplify ListImpl Modified: pypy/dist/pypy/translator/cl/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/cl/opformatter.py (original) +++ pypy/dist/pypy/translator/cl/opformatter.py Sun Apr 30 17:14:12 2006 @@ -64,10 +64,8 @@ def op_direct_call(self, result, fun, *args): funobj = self.args[0].value self.gen.pendinggraphs.append(funobj) - args = " ".join(args) - yield "(setf %s (%s %s))" % (clrepr(result, True), - clrepr(fun, True), - clrepr(args, True)) + funcall = " ".join((fun,) + args) + yield "(setf %s (%s))" % (result, funcall) def op_new(self, result, _): cls = self.args[0].value @@ -96,29 +94,19 @@ clrepr(arg, True), clrepr(clsname, True)) - def op_oosend(self, result, *ignore): + def op_oosend(self, result, _, selfvar, *args): method = self.args[0].value - receiver = self.args[1] - cls = receiver.concretetype - args = self.args[2:] + cls = self.args[1].concretetype if isinstance(cls, List): - impl = ListImpl(receiver) + impl = ListImpl(selfvar) code = getattr(impl, method)(*args) - yield "(setf %s %s)" % (clrepr(result, True), clrepr(code, True)) + yield "(setf %s %s)" % (result, code) elif isinstance(cls, Instance): name = clrepr(method, symbol=True) - selfvar = clrepr(receiver) - args = map(self.gen.check_declaration, args) - args = " ".join(args) - if args: - yield "(setf %s (%s %s %s))" % (clrepr(result, True), - clrepr(name, True), - clrepr(selfvar, True), - clrepr(args, True)) - else: - yield "(setf %s (%s %s))" % (clrepr(result, True), - clrepr(name, True), - clrepr(selfvar, True)) + funcall = " ".join((name, selfvar) + args) + yield "(setf %s (%s))" % (result, funcall) + else: + raise NotImplementedError("op_oosend on %s" % (cls,)) def op_oogetfield(self, result, obj, _): fieldname = self.args[1].value @@ -152,21 +140,17 @@ class ListImpl: - def __init__(self, receiver): - self.obj = clrepr(receiver) + def __init__(self, obj): + self.obj = obj def ll_length(self): return "(length %s)" % (self.obj,) def ll_getitem_fast(self, index): - index = clrepr(index) return "(aref %s %s)" % (self.obj, index) def ll_setitem_fast(self, index, value): - index = clrepr(index) - value = clrepr(value) return "(setf (aref %s %s) %s)" % (self.obj, index, value) def _ll_resize(self, size): - size = clrepr(size) return "(adjust-array %s %s)" % (self.obj, size) From sanxiyn at codespeak.net Sun Apr 30 17:58:34 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 17:58:34 +0200 (CEST) Subject: [pypy-svn] r26609 - in pypy/dist/pypy/translator/cl: . test Message-ID: <20060430155834.8B9B010082@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 17:58:28 2006 New Revision: 26609 Modified: pypy/dist/pypy/translator/cl/opformatter.py pypy/dist/pypy/translator/cl/test/test_dict.py Log: Beginning of Dict implementation in Lisp backend Modified: pypy/dist/pypy/translator/cl/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/cl/opformatter.py (original) +++ pypy/dist/pypy/translator/cl/opformatter.py Sun Apr 30 17:58:28 2006 @@ -1,4 +1,4 @@ -from pypy.rpython.ootypesystem.ootype import List, Record, Instance +from pypy.rpython.ootypesystem.ootype import List, Dict, Record, Instance from pypy.translator.cl.clrepr import clrepr class OpFormatter: @@ -71,6 +71,8 @@ cls = self.args[0].value if isinstance(cls, List): yield "(setf %s (make-array 0 :adjustable t))" % (result,) + elif isinstance(cls, Dict): + yield "(setf %s (make-hash-table))" % (result,) elif isinstance(cls, Record): clsname = self.gen.declare_struct(cls) yield "(setf %s (make-%s))" % (result, clsname) @@ -101,6 +103,10 @@ impl = ListImpl(selfvar) code = getattr(impl, method)(*args) yield "(setf %s %s)" % (result, code) + elif isinstance(cls, Dict): + impl = DictImpl(selfvar) + code = getattr(impl, method)(*args) + yield "(setf %s %s)" % (result, code) elif isinstance(cls, Instance): name = clrepr(method, symbol=True) funcall = " ".join((name, selfvar) + args) @@ -154,3 +160,14 @@ def _ll_resize(self, size): return "(adjust-array %s %s)" % (self.obj, size) + +class DictImpl: + + def __init__(self, obj): + self.obj = obj + + def ll_length(self): + return "(hash-table-count %s)" % (self.obj,) + + def ll_set(self, key, value): + return "(setf (gethash %s %s) %s)" % (key, self.obj, value) Modified: pypy/dist/pypy/translator/cl/test/test_dict.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_dict.py (original) +++ pypy/dist/pypy/translator/cl/test/test_dict.py Sun Apr 30 17:58:28 2006 @@ -1,6 +1,6 @@ from pypy.translator.cl.buildcl import make_cl_func -def notest_dict_length(): +def test_dict_length(): def dict_length_one(key, val): dic = {key:val} return len(dic) From sanxiyn at codespeak.net Sun Apr 30 18:09:43 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 18:09:43 +0200 (CEST) Subject: [pypy-svn] r26611 - pypy/dist/pypy/translator/cl/test Message-ID: <20060430160943.9189610081@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 18:09:39 2006 New Revision: 26611 Modified: pypy/dist/pypy/translator/cl/test/test_dict.py pypy/dist/pypy/translator/cl/test/test_list.py Log: Rename some tests Modified: pypy/dist/pypy/translator/cl/test/test_dict.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_dict.py (original) +++ pypy/dist/pypy/translator/cl/test/test_dict.py Sun Apr 30 18:09:39 2006 @@ -2,14 +2,14 @@ def test_dict_length(): def dict_length_one(key, val): - dic = {key:val} + dic = {key: val} return len(dic) - cl_dict_length_one = make_cl_func(dict_length_one, [int,int]) - assert cl_dict_length_one(42,42) == 1 + cl_dict_length_one = make_cl_func(dict_length_one, [int, int]) + assert cl_dict_length_one(42, 42) == 1 -def notest_dict_key_access(): - def dict_key(number): - dic = {42:43} +def test_dict_get(): + def dict_get(number): + dic = {42: 43} return dic[number] - cl_dict_key = make_cl_func(dict_key, [int]) - assert cl_dict_key(42) == 43 + cl_dict_get = make_cl_func(dict_get, [int]) + assert cl_dict_get(42) == 43 Modified: pypy/dist/pypy/translator/cl/test/test_list.py ============================================================================== --- pypy/dist/pypy/translator/cl/test/test_list.py (original) +++ pypy/dist/pypy/translator/cl/test/test_list.py Sun Apr 30 18:09:39 2006 @@ -8,8 +8,8 @@ assert cl_list_length_one(0) == 1 def test_list_get(): - def list_and_get(number): + def list_get(number): lst = [number] return lst[0] - cl_list_and_get = make_cl_func(list_and_get, [int]) - assert cl_list_and_get(1985) == 1985 + cl_list_get = make_cl_func(list_get, [int]) + assert cl_list_get(1985) == 1985 From sanxiyn at codespeak.net Sun Apr 30 18:11:36 2006 From: sanxiyn at codespeak.net (sanxiyn at codespeak.net) Date: Sun, 30 Apr 2006 18:11:36 +0200 (CEST) Subject: [pypy-svn] r26612 - pypy/dist/pypy/translator/cl Message-ID: <20060430161136.4BA7110081@code0.codespeak.net> Author: sanxiyn Date: Sun Apr 30 18:11:32 2006 New Revision: 26612 Modified: pypy/dist/pypy/translator/cl/opformatter.py Log: ll_contains and ll_get for Dict Modified: pypy/dist/pypy/translator/cl/opformatter.py ============================================================================== --- pypy/dist/pypy/translator/cl/opformatter.py (original) +++ pypy/dist/pypy/translator/cl/opformatter.py Sun Apr 30 18:11:32 2006 @@ -169,5 +169,11 @@ def ll_length(self): return "(hash-table-count %s)" % (self.obj,) + def ll_contains(self, key): + return "(nth-value 1 (gethash %s %s))" % (key, self.obj) + + def ll_get(self, key): + return "(gethash %s %s)" % (key, self.obj) + def ll_set(self, key, value): return "(setf (gethash %s %s) %s)" % (key, self.obj, value) From jum at codespeak.net Sun Apr 30 18:26:37 2006 From: jum at codespeak.net (jum at codespeak.net) Date: Sun, 30 Apr 2006 18:26:37 +0200 (CEST) Subject: [pypy-svn] r26613 - pypy/dist/pypy/doc Message-ID: <20060430162637.D92681007D@code0.codespeak.net> Author: jum Date: Sun Apr 30 18:26:37 2006 New Revision: 26613 Modified: pypy/dist/pypy/doc/svn-help.txt Log: Updated the local build of subversion for OS X to the latest version. Modified: pypy/dist/pypy/doc/svn-help.txt ============================================================================== --- pypy/dist/pypy/doc/svn-help.txt (original) +++ pypy/dist/pypy/doc/svn-help.txt Sun Apr 30 18:26:37 2006 @@ -146,7 +146,7 @@ .. _website: http://support.microsoft.com/default.aspx?scid=kb%3Ben-us%3B259403 .. _GUI: http://tortoisesvn.tigris.org/servlets/ProjectDocumentList?folderID=616 -.. _MacOS: http://codespeak.net/~jum/svn-1.2.3-darwin-ppc.tar.gz +.. _MacOS: http://codespeak.net/~jum/svn-1.3.1-darwin-ppc.tar.gz .. _versions: http://subversion.tigris.org/project_packages.html .. _Win: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=4B6140F9-2D36-4977-8FA1-6F8A0F5DCA8F .. _guide: http://svnbook.red-bean.com/book.html#svn-ch-1